Distilled tiny vision transformer
According to the model card, this distilled DeiT-tiny model was pretrained and fine-tuned with distillation on ImageNet-1k, learning from a CNN teacher through a dedicated distillation token.
Open Source Model Profile · facebook
deit-tiny-distilled-patch16-224
deit-tiny-distilled-patch16-224 is a distilled tiny DeiT image classifier from facebook. Its model card documents ImageNet-1k distillation training and reported 74.5% top-1 accuracy.
- Publisher
- Task
- image-classification
- Model type
- deit
- License
- apache-2.0
- Library
- transformers
- Publication status
- Accepted · not indexed
Model overview
deit-tiny-distilled-patch16-224 is published by facebook as an image-classification model. The captured configuration identifies DeiTForImageClassificationWithTeacher with model type deit, and captured metadata records an Apache-2.0 license. According to the model card, it is a distilled tiny Data-efficient Image Transformer pretrained and fine-tuned on ImageNet-1k at 224x224 resolution.
Recorded capabilities
16x16 patches at 224 resolution
The card says images are presented as linearly embedded 16x16 patches, with inference preprocessing that resizes to 256, center-crops at 224, and normalizes with ImageNet statistics.
Reported ImageNet scores
According to the model card's evaluation table, the DeiT-tiny distilled row reports 74.5% top-1 and 91.9% top-5 accuracy at 6M parameters.
Use cases in the source record
- Image classification of inputs into one of 1,000 ImageNet classes using the card's documented transformers snippet.
- Lightweight vision experiments that follow the card's inference preprocessing of resize, center-crop, and ImageNet normalization.
Limitations and unknowns
- No Safetensors parameter count was extracted; the 6M figure comes from the card's evaluation table.
- Provider state is historical snapshot data, not independently refreshed current availability.
- The card notes it was written by the Hugging Face team rather than the original DeiT team, so card claims are secondhand summaries.
Source and provenance
Source: facebook/deit-tiny-distilled-patch16-224
Captured: Unknown. Processed: 2026-09-07T19:34:44.226222+00:00.
Distilled Data-efficient Image Transformer (tiny-sized model) Distilled data-efficient Image Transformer (DeiT) model pre-trained and fine-tuned on ImageNet-1k (1 million images, 1,000 classes) at resolution 224x224. It was first introduced in the paper Training data-efficient image transformers & distillation through attention by Touvron et al. and first released in this repository . However, the weights were converted from the timm repository by Ross Wightman. Disclaimer: The team releasing DeiT did not write a model card for this model so this model card has been written by the Hugging Face team. Model description This model is a…
F001F002F003F004F005F006F008F009F010F011F012F013F014F015F017F019