How generative artificial intelligence is revolutionizing business intelligence, creative processes, and law

Generative Language Models (GLM) work by predicting the next word in a sequence of words based on previous tokens. In order to perform this word prediction task, they are trained on large amounts of text data. Training involves two key components: supervised pre-training and unsupervised fine-tuning. During the pre-training phase, the models are trained using maximum likelihood estimation to minimize the difference between true and predicted tokens. For fine-tuning, human annotators provide guidance by evaluating model responses to prompts, while reinforcement learning is employed to improve model performance. Supervised pre-training enables the models to leverage vast amounts of unlabeled data, while fine-tuning ensures that user intentions are taken into account. This two-part training procedure is similar to that of animal learning, where the first phase features unsupervised learning to form a general model and the second phase involves reinforcement learning to make the model comply with specific user instructions (Bommasani et al., 2021; Liu et al., 2022; Goel et al., 2023).

To enable GLM capabilities, two major considerations are required. First, training requires massive data and compute. Existing models have 10 to 1,000 times more parameters than earlier models, and training of a single model may be accomplished with 1,000 to 1,000,000 GPU days. The recently proposed Mixture of Expert architecture uses a smaller number of activated parameters during selection than the total number of parameters during inference, leading to efficiency. Second, scaling laws for model size, data size, context length, and compute must be respected. A trained model performs better when the training set is larger; more gradient updates are performed; more parameters are used; more tokens are used from the pre-training dataset during each update at each scale; for larger context lengths; and for denser pre-training on a larger bucketed pre-training dataset (Liu & Jatowt, 2023; McCormack, 2023).