Natural Language Processing Using Large Language Models and Machine Learning Methods

Abstract

The article analyzes the capabilities of large language models in solving NLP tasks. It describes the features of the Transformer architecture, which serves as the foundation for modern natural language processing models. The individual components of the architecture, their roles, and their significance for working with human language are discussed. A comparative analysis of the Transformer and other existing models in the context of machine translation task is provided.Factors that have enabled the development of models with billions of parameters—known as large language models—are analyzed. The Llama model family from Meta is reviewed as an example of such models. Special attention is given to smaller-scale models, which can be powerful yet accessible tools for natural language processing.Currently, deep machine learning and convolutional neural networks (CNN) hold an important place in the field of natural language processing (NLP). Therefore, the article evaluates the effectiveness of these algorithms, models, and methods for solving key tasks, using the named entity recognition (NER) task as an example.Deep learning methods have revolutionized NER, providing a significantly better understanding of context, capturing dependencies over long distances, and enabling the effective use of large datasets. A classification of Transformer-based models that currently yield the best results is provided. Currently, many models have been developed based on the Transformer architecture.We describe the results of comparing two of the largest BERT models (which have achieved strong results across a wide range of NLP tasks, including question answering, text classification, natural language interference, and context prediction) with GPT-3 (which has demonstrated impressive successes in language modeling, text generation, and question answering). These models are pre-trained on large-scale textual datasets to learn fundamental linguistic representations. Both models leverage fine-tuning to enhance their performance.