Meta introduces LLaMA, its Large Language Model designed to help researchers advance their AI work, with several sizes ranging from 7 billion to 65 billion parameters and trained on 1.4 trillion tokens, available for noncommercial research use cases
Meta, formerly known as Facebook, is set to enter the AI chatbot race with its state-of-the-art large language model designed to help researchers in the field of artificial intelligence. The Large Language Model Meta AI (LLaMA), like Google's Bard and Microsoft's ChatGPT, is a foundational large language model that can generate creative text, solve mathematical theorems, predict protein structures, answer reading comprehension questions, and more.
However, unlike ChatGPT-driven Bing, LLaMA cannot yet talk to humans but will help researchers. Meta is making LLaMA available at several sizes, including 7 billion, 13 billion, 33 billion, and 65 billion parameters. Large language models, which are natural language processing (NLP) systems with billions of parameters, have shown significant potential benefits to billions of people.
Also Read: Meta Launches Paid Blue Badge For Facebook, Instagram At $11.99 A Month
Smaller models, trained on more tokens, are easier to retrain and fine-tune for specific potential product use cases. Meta has trained LLaMA 65 billion and LLaMA 33 billion on 1.4 trillion tokens. The smallest model, LLaMA 7B, is trained on one trillion tokens.
To train the LLaMA model, Meta chose text from the 20 languages with the most speakers, focusing on those with Latin and Cyrillic alphabets. The model works by taking a sequence of words as an input and predicting a next word to recursively generate text.
In a statement, Meta said, "Smaller, more performant models such as LLaMA enable others in the research community who don't have access to large amounts of infrastructure to study these models, further democratising access in this important, fast-changing field."
The company is releasing the model under a noncommercial license focused on research use cases at the moment to maintain integrity and prevent misuse.
According to Meta, large language models are one of the clearest cases of the substantial potential benefits AI can offer at scale to billions of people. With the release of LLaMA, Meta hopes to empower researchers to advance their work in the field of artificial intelligence, further democratizing access to AI technologies.