distilbert-base-uncased

Model overview

distilbert-base-uncased is published by distilbert as a distilbert fill-mask model. The captured configuration identifies DistilBertForMaskedLM and Safetensors metadata reports 66,985,530 parameters. According to the model card, it distills the BERT base model through self-supervised pretraining and treats uppercase and lowercase English identically.

Recorded capabilities

Use cases in the source record

• Masked-token prediction workflows consistent with the captured fill-mask pipeline tag.
• Fine-tuning for sentence-level decisions such as sequence classification, token classification, or question answering, as described in the model card.

Limitations and unknowns

• According to the model card, the model can reflect bias in its training data, with examples shown, and this bias is stated to affect fine-tuned versions.
• No evaluation results were extracted from this record.
• No pricing, VRAM requirement, or latency figures were extracted.
• Provider state is historical snapshot data, not independently refreshed current availability.

Source and provenance

Source: distilbert/distilbert-base-uncased

Captured: Unknown. Processed: 2026-09-07T19:34:43.566230+00:00.

DistilBERT base model (uncased) This model is a distilled version of the BERT base model . It was introduced in this paper . The code for the distillation process can be found here . This model is uncased: it does not make a difference between english and English. Model description DistilBERT is a transformers model, smaller and faster than BERT, which was pretrained on the same corpus in a self-supervised fashion, using the BERT base model as a teacher. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to g…