Multimodal MoE design
According to the model card, the Llama 4 models are natively multimodal with mixture-of-experts architecture and early fusion.
Open Source Model Profile · meta-llama
Llama-4-Scout-17B-16E-Instruct is a 108.64B-parameter multimodal MoE instruction model from Meta. According to the model card, Scout pairs 17B active parameters with 16 experts for chat and vision.
Llama-4-Scout-17B-16E-Instruct is published by meta-llama as an image-text-to-text instruction model. The captured configuration identifies Llama4ForConditionalGeneration with model type llama4 and about 108.64B Safetensors parameters. According to the model card, it is a 17B-active mixture-of-experts model with 16 experts for chat and visual reasoning, under a custom Llama 4 Community License.
According to the model card, the Llama 4 models are natively multimodal with mixture-of-experts architecture and early fusion.
According to the model card, Llama 4 Scout is a 17-billion-parameter model with 16 experts.
According to the model card, instruction-tuned models target assistant-like chat, visual recognition, image reasoning, captioning, and image question answering.
According to the model card, Scout is released as BF16 weights and can fit a single H100 GPU with on-the-fly int4 quantization.
Source: meta-llama/Llama-4-Scout-17B-16E-Instruct
Captured: Unknown. Processed: 2026-09-07T19:34:51.792065+00:00.
Model Information The Llama 4 collection of models are natively multimodal AI models that enable text and multimodal experiences. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding. These Llama 4 models mark the beginning of a new era for the Llama ecosystem. We are launching two efficient models in the Llama 4 series, Llama 4 Scout, a 17 billion parameter model with 16 experts, and Llama 4 Maverick, a 17 billion parameter model with 128 experts. Model developer : Meta Model Architecture: The Llama 4 models are auto-regressive language models that use a mixtu…
F001F002F003F004F005F006F007F010F011F012F013F018F026F027F029F036