Tiny Chinese RoBERTa miniature
According to the model card, this is one of 24 Chinese RoBERTa models spanning layers 2 to 12 and hidden sizes 128 to 768.
Open Source Model Profile · uer
chinese_roberta_L-2_H-128
chinese_roberta_L-2_H-128 is a Chinese RoBERTa Tiny fill-mask model from uer. According to the model card, it is the 2-layer, 128-hidden member of a 24-model miniature set.
- Publisher
- uer
- Task
- fill-mask
- Model type
- bert
- License
- Unknown
- Library
- transformers
- Publication status
- Approved for indexing
Model overview
chinese_roberta_L-2_H-128 is published by uer as a fill-mask model. The captured configuration identifies BertForMaskedLM with model type bert. According to the model card, it is the 2-layer, 128-hidden Tiny member of 24 Chinese RoBERTa miniatures pretrained with UER-py.
Recorded capabilities
Reported six-task table
According to the model card, development results cover book_review, chnsenticorp, lcqmc, tnews, iflytek, and ocnli with Tiny, Mini, Small, Medium, and Base comparisons.
CLUECorpusSmall pretraining
According to the model card, models were pretrained on CLUECorpusSmall with 1,000,000 steps at length 128 followed by 250,000 steps at length 512.
Use cases in the source record
- Chinese fill-mask prediction workflows of the kind illustrated by the card's Beijing-capital ranking examples.
- Lightweight Chinese classification and feature experiments suggested by the card's six-task fine-tuning table and Transformers feature example.
Limitations and unknowns
- No parameter count was extracted from this record.
- No license value was extracted from this record.
- No context-window value was extracted from this record.
- Provider state is historical snapshot data, not independently refreshed current availability.
- Benchmark figures and training comparisons are publisher claims and were not independently verified.
Source and provenance
Source: uer/chinese_roberta_L-2_H-128
Captured: Unknown. Processed: 2026-09-07T19:35:00.278355+00:00.
Chinese RoBERTa Miniatures Model description This is the set of 24 Chinese RoBERTa models pre-trained by UER-py , which is introduced in this paper . Besides, the models could also be pre-trained by TencentPretrain introduced in this paper , which inherits UER-py to support models with parameters above one billion, and extends it to a multimodal pre-training framework. Turc et al. have shown that the standard BERT recipe is effective on a wide range of model sizes. Following their paper, we released the 24 Chinese RoBERTa models. In order to facilitate users in reproducing the results, we used a publicly available corpus and provide…
F001F002F003F004F005F006F008F009F010F012F013F014F018F019F020