Meta introduces Llama 3. What we can expect from it

Meta released Llama 4 in April 2025 — its first model family built on a Mixture-of-Experts architecture, with native multimodality and a 10-million-token context window in its Scout variant. The launch was technically ambitious. The reception was complicated. Independent developers panned the real-world quality, a benchmark-manipulation controversy emerged involving Yann LeCun, and by mid-2026 Meta had reorganized its AI division around a new proprietary model line called Muse Spark. This guide covers what Llama 4 actually is, what went wrong, how to run it today, and where Meta’s open-weight strategy stands heading into late 2026.

What Is Llama 4? A Quick Answer

Llama 4 is Meta’s fourth generation of open-weight large language models. Unlike its predecessors — which used standard dense transformer architectures — Llama 4 uses a Mixture-of-Experts (MoE) design, meaning only a fraction of the model’s total parameters are active during any given inference step. The family shipped in two publicly released variants:

• Llama 4 Scout — 17 billion active parameters, 16 experts, a 10-million-token context window on paper. Designed to run on a single high-end GPU (NVIDIA H100). Best for long-document tasks and local deployment.
• Llama 4 Maverick — 17 billion active parameters, 128 experts, 400 billion total parameters. Higher reasoning and coding capability. Requires multi-GPU or cloud inference.
• Llama 4 Behemoth — Announced as a “teacher model” (288 billion active / ~2 trillion total parameters) used to train Scout and Maverick via codistillation. Not publicly released as of August 2026; community speculation now questions whether it will ship before Muse Spark models reach comparable scale.

The Full Llama Timeline

The Architecture Shift: What MoE Actually Means

The most significant change from Llama 3 to Llama 4 is not a parameter count — it’s the architecture. Llama 4 Maverick uses a Mixture-of-Experts design: 400 billion total parameters, but only 17 billion activate per token across 128 experts. This keeps inference cost closer to a small dense model while approaching the capability of much larger ones.

Llama 4 also introduced native multimodality through early fusion — text and image tokens are processed together within the same backbone from the start, rather than using a separate frozen vision encoder bolted on afterwards (as in Llama 3.2’s vision variants). This theoretically improves cross-modal reasoning, though real-world results have been mixed.

What Can You Actually Do with a 10-Million-Token Context Window?

Llama 4 Scout’s 10-million-token context window is the largest of any publicly available open-weight model at launch — roughly 10× Claude 3’s 1-million-token window, at least on paper. In principle, 10 million tokens is enough to feed a model:

• An entire codebase — a large monorepo of ~500,000 lines fits comfortably in a single prompt
• A full legal contract portfolio — hundreds of documents reviewed simultaneously, with cross-document reasoning
• Multi-day meeting transcripts — weeks of Zoom calls, support tickets, or customer interviews at once
• Book-length content — a 10-million-token window holds roughly 15–20 full novels
• Long-horizon agents — agentic workflows that need to maintain context across many tool calls without truncating earlier steps

The practical caveat is bigger than it first appears. “Needle-in-a-haystack” retrieval performance degrades in the latter portions of very long windows — an issue not unique to Llama 4, but worth testing before committing to an architecture decision that depends on it. In practice, most third-party inference providers initially capped usable context for Scout well below the advertised 10M — typically in the 128K–328K range — due to memory and cost constraints on their infrastructure. Independent long-context evaluations have generally validated strong recall only up to roughly 1M tokens, with retrieval accuracy dropping meaningfully beyond that point. Teams evaluating Scout for genuinely long-context workloads should benchmark on their own documents and chosen provider rather than planning around the 10M headline figure.

Llama 4 vs GPT-4o vs Gemini 2.0 Flash: How They Compare

Note: the table below reflects the April 2025 launch-era competitive landscape. By August 2026, newer releases such as GPT-5.5 and updated Gemini models have shipped with different context windows, pricing, and benchmark standings — treat this as a historical snapshot of the Llama 4 launch, not a current-state comparison.

API cost estimates based on Vellum benchmarking data comparing Llama 3 70B to GPT-4 via cloud providers; Llama 4 Scout costs vary by provider. Check Together.ai, AWS Bedrock, and Cloudflare Workers AI for current pricing.

Independent Reception vs. Benchmark Claims

Meta’s launch material described Llama 4 Maverick as beating GPT-4o and Gemini 2.0 Flash on widely reported benchmarks. The independent developer community was less convinced — and subsequent reporting revealed why.

The Benchmark Controversy

The deeper problem emerged later. Meta had submitted a non-release, internally optimized variant of Llama 4 Maverick to the LM Arena chatbot leaderboard — the platform whose crowdsourced rankings many publications cited as evidence of Llama 4’s superiority. The version users could actually download performed substantially worse on the same tasks.

Yann LeCun, Meta’s Chief AI Scientist, departed Meta in November 2025, with his exit effective at year-end 2025, amid the broader reorganization of Meta’s AI division. In an early-January 2026 interview with the Financial Times, LeCun confirmed that Meta had submitted a specially tuned, non-representative version of Llama 4 to the leaderboard — corroborating the gap between the benchmark ranking and real-world performance that developers had already flagged. The episode crystallized existing concerns about benchmark gaming in the AI industry and damaged trust in Meta’s open-model claims among the developer community that Llama depends on for adoption.

The practical consequence was measurable: Groq, a major inference provider, announced it was deprecating Llama 4 Scout 17B due to low developer uptake — a significant signal that the model had not achieved the community traction Meta needed to sustain the open-weight ecosystem flywheel.

Practical implication for teams evaluating Llama 4: benchmark scores from the April 2025 launch should be treated with skepticism. Use independent evals — MMLU, HumanEval, LMSYS — run against the publicly released weights, not leaderboard-specific variants.

On cost and speed, independent benchmarking firm Vellum found that Llama 3 70B could be up to 50 times cheaper and 10 times faster than GPT-4 when accessed via cloud API providers — a gap that remains one of the strongest arguments for open-weight models in production regardless of which generation is used.

How to Run Llama 4 Today

Option 1: Local Deployment with Ollama (Fastest to Start)

Ollama is the simplest way to run Llama 4 Scout locally. You need an NVIDIA H100, A100, or equivalent GPU with at least 40 GB VRAM for the standard variant (quantized versions can run on consumer hardware with less).

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run Llama 4 Scout
ollama run llama4

# For a quantized version (lower VRAM, ~24 GB)
ollama run llama4:scout-q4_K_M

# Run as a local API server
ollama serve
# Then call via: http://localhost:11434/api/generate

Option 2: Cloud API (No GPU Required)

If you don’t want to manage hardware, Llama 4 is available via several cloud providers. Current access points as of August 2026:

Option 3: Fine-tuning with LoRA

For teams that need domain-specific performance (legal, medical, code), LoRA (Low-Rank Adaptation) is the standard approach for fine-tuning Llama 4 without retraining full weights. Libraries like Hugging Face PEFT, Axolotl, and Unsloth all support Llama 4 Scout fine-tuning. Typical LoRA runs for Scout can be completed on a single 80 GB H100 in a few hours with a modest task-specific dataset.

Licensing and Regulatory Considerations

“Open source” is a loose description of how Llama is actually licensed. In February 2025, the Open Source Initiative (OSI) reviewed the Llama license and concluded it fails on multiple counts: it restricts the freedom to use the software for any purpose, discriminates against users in certain jurisdictions, and restricts fields of endeavor through its Acceptable Use Policy. OSI has publicly accused Meta of promoting Llama as “open source” when it does not meet that bar.

The specific restrictions enterprises should review before building on Llama 4:

• 700 million MAU threshold. If a licensee’s product — including its affiliates — exceeded 700 million monthly active users in the month before a given Llama version’s release, it cannot use the license freely and must request a separate agreement from Meta. This clause has existed since Llama 2 (2023) and was widely read as targeting Meta’s largest rivals.
• EU exclusion for multimodal models. Starting with Llama 3.2, the license does not extend to individuals or companies domiciled in the EU for multimodal capabilities specifically. Companies building core products around a multimodal Llama model from an EU base are not covered.
• Acceptable Use Policy. Meta can revise the AUP unilaterally. It currently bars military and surveillance applications — a field-of-use restriction that the Open Source Definition explicitly prohibits in an open-source license.
• Branding requirements. Derivative models must include “Llama” in their name; distributions must display “Built with Llama.”
• Output-training restriction. Licensees cannot use Llama’s outputs to train a competing model.

Given these terms, most legal analysts describe Llama as “source-available” or “open-weight” rather than open source. Enterprises evaluating Llama for regulated deployments should also track EU AI Act obligations, which apply based on a model’s risk classification and use case, independent of its licensing terms.

Muse Spark vs. Llama: Meta’s Two-Track Strategy

Since mid-2025, Meta has been running two parallel model lines — and understanding the difference matters for teams choosing a foundation model.

The Muse Spark line — promoted on Meta’s developer platform following the creation of Meta Superintelligence Labs under Alexandr Wang — reflects a strategic shift: Meta is no longer committed to releasing its most capable models as open weights. The community reaction has been split. Many developers welcome continued open releases for some tiers while accepting that frontier-level models may stay proprietary. Others see it as the beginning of a retreat that undermines Llama’s foundational value proposition.

That “retreat” framing is complicated by a more recent data point: in August 2026, Meta released Muse Glimmer, a 30-billion-parameter model under the fully permissive Apache 2.0 license — a genuine OSI-approved open-source release, unlike anything in the Llama Community License lineage. Muse Glimmer doesn’t replace Llama’s role in Meta’s lineup, but it is concrete evidence that Meta is still shipping unrestricted open-weight models even while building out the closed Muse Spark line, supporting a hybrid-strategy reading over a simple retreat narrative.

In April 2026, Axios reported that Meta is developing open-source versions of its next models, reportedly derived from internal projects codenamed Avocado (LLM) and Mango (multimedia generation), while confirming some future models will remain proprietary. Meta’s own framing is that open releases exist alongside proprietary ones, not instead of them — a position that is evolving month by month, and one Muse Glimmer’s release now backs with an actual product rather than just messaging.

The Original Llama 3 Launch (Historical Context — April 2024)

For reference, here is what Meta announced at the Llama 3 launch — the basis of this article’s original April 2024 coverage:

Open Weights

Llama models are distributed as open weights, letting developers fine-tune them for specific tasks without per-token licensing costs — a major reason for their fast adoption among startups and enterprises.

Training Dataset

Llama 3 was trained on over 15 trillion tokens — seven times the dataset size of Llama 2, with four times more code data.

Reported Performance

At launch, Meta reported that human evaluators preferred Llama 3 70B over Llama 2, GPT-3.5, Mistral Medium, and Claude Sonnet (the 2024 model) in blind comparisons. These figures came from Meta’s own evaluation set and were not independently replicated at the time — a pattern that would repeat with Llama 4.

Meta’s Distribution Advantage

Meta has a track record of adapting ideas — Stories from Snapchat, Reels from TikTok — into products that succeed at scale. Llama followed a similar pattern: Meta arrived late to foundation models but embedded them directly into WhatsApp, Instagram, and Facebook, reaching an install base no standalone AI product can match. Llama 3 was the first model integrated into Meta.ai as a public assistant — a direct ChatGPT challenger reaching over 3 billion combined users.

References

• Ars Technica — “Meta’s surprise Llama 4 drop exposes the gap between AI ambition and reality” (April 2025)
• Axios — “Scoop: Meta to open source versions of its next AI models” (April 2026)
• The New Stack — “Meta abandons open-source Llama for proprietary Muse Spark” (April 2026)
• Financial Times — Yann LeCun interview confirming Llama 4 benchmark submission (early January 2026)
• SemiAnalysis — reporting on Groq’s deprecation of Llama 4 Scout 17B (2026)
• Box Blog — “Evaluating Meta’s Llama 4 Models for Enterprise Content with Box AI”
• Vellum.ai — “Llama 3 70B vs GPT-4: Comparison Analysis”
• Meta AI — “The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation”
• Open Source Initiative — OSI statement on Meta Llama licensing (February 2025)