Educational-value scoring
According to the model card, the classifier scores web samples from 0 for not educational to 5 for highly educational, trained on 450K Llama3-generated annotations.
Open Source Model Profile · HuggingFaceFW
fineweb-edu-classifier
fineweb-edu-classifier is a 109.48M-parameter BERT classifier from HuggingFaceFW for judging web-page educational value.
- Publisher
- HuggingFaceFW
- Task
- text-classification
- Model type
- bert
- License
- apache-2.0
- Library
- transformers
- Publication status
- Accepted · not indexed
Model overview
fineweb-edu-classifier is published by HuggingFaceFW as a text-classification model. Captured configuration identifies BertForSequenceClassification with a bert model type, and Safetensors metadata reports 109,483,009 parameters under Apache-2.0. According to the model card, it filters educational web content from FineWeb samples annotated by Llama3-70B-Instruct.
Recorded capabilities
Regression-head training setup
The model card describes a single-regression classification head on Snowflake-arctic-embed, trained for 20 epochs at 3e-4 with frozen embedding and encoder layers.
82% binary-classifier F1
According to the model card, the model achieved an 82% F1 score when converted to a binary classifier using score threshold 3.
Documented scope and bias limits
The model card warns of possible distribution shift outside FineWeb, weaker handling of higher-education content, dependence on annotation quality, and possible overfitting to academic-looking text.
Use cases in the source record
- Educational web-data filtering and curation work of the kind used to build the FineWeb-Edu dataset.
- Threshold-based curation experiments using the publisher-recommended integer score of 3 or higher.
Limitations and unknowns
- The publisher notes scope limits for out-of-distribution data and higher-education content, plus possible bias from training data and annotation quality.
- Provider state is historical snapshot data and should be refreshed before being presented as current availability.
- No context-window value was extracted from this record.
Source and provenance
Source: HuggingFaceFW/fineweb-edu-classifier
Captured: Unknown. Processed: 2026-09-07T19:34:31.418333+00:00.
FineWeb-Edu classifier Model summary This is a classifier for judging the educational value of web pages. It was developed to filter and curate educational content from web datasets and was trained on 450k annotations generated by LLama3-70B-instruct for web samples from FineWeb dataset. We used this classifier to build FineWeb-Edu dataset. How to use in transformers To load the FineWeb-Edu classifier, use the following code: from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained( "HuggingFaceTB/fineweb-edu-classifier" ) model = AutoModelForSequenceClassification.from_pre…
F001F002F003F004F005F006F007F010F011F012F015F016F017F018F019