Meta open-sources 405 billion parameter large model: Llama 3.1 officially released

Meta quickly mobilized the open-source community to exert competitive pressure on OpenAI.

Here is the English translation of Zuckerberg's open letter:

"Open Source AI Is the Path Forward"

In the early days of high-performance computing, the major tech companies of the time all invested heavily in developing their own closed-source versions of Unix. At the time, it was hard to imagine any other way to develop such advanced software.

Nevertheless, open source Linux eventually became popular - initially because it allowed developers to modify its code as they wanted and was cheaper; over time, it became more advanced, more secure, and had a much broader ecosystem supporting more features than any closed Unix. Today, Linux is the industry standard foundation for cloud computing and the operating systems that run most mobile devices - we all benefit from its superior products.

I believe artificial intelligence will evolve in a similar way. Today, a few tech companies are developing leading closed-source models. But open source is rapidly closing this gap. Last year, Llama 2 could only compete with older generation models that lagged behind the cutting edge. This year, Llama 3 competes with state-of-the-art models and leads in some areas. Starting next year, we expect future Llamas to be the most advanced in the industry. But even before then, Llama is already leading in openness, modifiability and cost efficiency.

Today, we're taking the next step towards making open source AI the industry standard. We're releasing Llama 3.1 405B - the first open source AI model at industry-leading levels - along with new and improved Llama 3.1 70B and 8B models. In addition to having better cost/performance compared to closed-source models, the fact that the 405B model is open source will make it the best choice for fine-tuning and extracting smaller models.

In addition to releasing these models, we're partnering with a range of companies to develop a broader ecosystem. Amazon, Databricks and Nvidia are launching full suites of services to support developers in fine-tuning and refining their own models. Innovative companies like Groq (an AI chip startup) have built low-latency, low-cost inference services for all new models.

These models will be available on all major clouds, including AWS, Azure, Google, Oracle and more. Scale.AI, Dell, Deloitte and others are ready to help enterprises deploy Llama and train custom models using their own data. As the community grows and more companies develop new services, we can collectively make Llama the industry standard and bring the benefits of AI to everyone.

Meta is committed to open source AI. I'll outline why I believe open source is the best development stack for people, why open sourcing Llama is good for Meta, why open source AI is good for the world, and why, because of this, the open source community will persist for the long term.

Why Open Source AI is Good for Developers

When I talk to developers, CEOs and government officials around the world, I usually hear the following themes:

We need to train, fine-tune and refine our own models.

Every organization has different needs, and different sized models can best meet those needs, trained or fine-tuned with specific data. On-device tasks and classification tasks require smaller models, while more complex tasks require larger models.

Now, you'll be able to use state-of-the-art Llama models, continue training them with your own data, and then refine them to your optimal size model - without us or anyone else seeing your data.

We need to control our own destiny and not be bound by a closed-source vendor.

Many organizations don't want to rely on models they can't run and control. They don't want closed-source model providers to be able to change their models, change their terms of use, or even stop serving them altogether. They also don't want to be locked into a single cloud that has exclusive rights to a particular model. Open source provides an ecosystem of tools compatible with many companies that you can easily switch between.

We need to protect our data.

Many organizations deal with sensitive data that needs protection and cannot be transmitted to closed-source models via cloud APIs. Other organizations simply don't trust closed-source model providers with their data. Open source solves these problems by allowing you to run models wherever you want. It's widely accepted that open source software is more secure because development is more transparent.

We need an efficient and affordable operating model.

Developers can run inference on Llama 3.1 405B on their own infrastructure at about 50% of the cost of using closed-source models like GPT-4o for user-facing and offline inference tasks.

We're betting on an ecosystem that can become a long-term standard.

Many people see open source developing faster than closed-source models, and they want to build their systems' architecture in a way that gives them the greatest long-term advantage.

Why Open Source AI is Good for Meta

Meta's business model is to build the best experiences and services for people. To do this, we need to ensure we always have access to the best technology, rather than being locked into competitors' closed ecosystems that would limit what we can build.

One of my formative experiences is how our services are constrained by what Apple allows us to build on their platform. The way they tax developers, the arbitrary rules they apply, and all the product innovations they block from being released make it clear that Meta and many other companies would be free to build better services for people if we could build the best versions of our products without competitors being able to limit what we can build.

Philosophically, this is the main reason I so firmly believe in building open source ecosystems for the next generation of computing in AI and AR/VR.

People often ask me if I'm worried about giving up technological advantage by open sourcing Llama, but I think this ignores some important reasons:

First, to ensure we can access the best technology rather than being locked into a closed ecosystem long-term, Llama needs to evolve into a complete ecosystem including tools, efficiency improvements, silicon optimizations and other integrations. If we were the only company using Llama, this ecosystem wouldn't develop and we'd be no better off than closed Unix variants.

Second, I expect competition to intensify as intelligence grows, which means at that point, open sourcing any particular model won't give up an advantage over the next model with greater advantages. The path for Llama to become the industry standard is through consistently competing, efficiently and open sourcing generation after generation of models.

Third, a key difference between Meta and closed-source model providers is that