Skip to content

EthenEthenEthen

LoginPlatformTry Ethen

Open Source Model Profile · jackaduma

SecRoBERTa

SecRoBERTa is an 84M-parameter RoBERTa fill-mask model from jackaduma. According to the model card, it is pretrained on cybersecurity text.

Publisher
jackaduma
Task
fill-mask
Model type
roberta
License
apache-2.0
Library
transformers
Publication status
Accepted · not indexed

Model overview

SecRoBERTa is published by jackaduma as a Transformers fill-mask model. The captured configuration identifies RobertaForMaskedLM with model type roberta and about 84M Safetensors parameters. According to the model card, it is pretrained on cybersecurity text from APTnotes, Stucco-Data, CASIE, and SecureNLP sources.

Recorded capabilities

Cybersecurity pretraining

According to the model card, this is a SecRoBERTa model trained on cybersecurity text, with both SecBERT and SecRoBERTa versions described.

Security corpus sources

According to the model card, sources include APTnotes, Stucco-Data, CASIE, and SemEval-2018 SecureNLP reports.

Domain vocabulary

According to the model card, the model uses its own secvocab wordpiece vocabulary matched to the security corpus.

84M RoBERTa masked-LM build

Captured configuration identifies RobertaForMaskedLM with 84095522 Safetensors parameters.

Use cases in the source record

  • Fill-mask analysis of cybersecurity text, including threat-hunting and threat-intelligence terminology.
  • Domain-vocabulary research using secvocab representations built for security corpora.

Limitations and unknowns

  • No evaluation results were extracted from this record.
  • No context-window value was extracted from this record.
  • No training steps, hyperparameters, or hardware details were extracted.
  • Provider state is historical snapshot data, not independently refreshed current availability.

Source and provenance

Source: jackaduma/SecRoBERTa

Captured: Unknown. Processed: 2026-09-07T19:34:47.509276+00:00.

SecRoBERTa This is the pretrained model presented in SecBERT: A Pretrained Language Model for Cyber Security Text , which is a SecRoBERTa model trained on cyber security text. The training corpus was papers taken from APTnotes Stucco-Data: Cyber security data sources CASIE: Extracting Cybersecurity Event Information from Text SemEval-2018 Task 8: Semantic Extraction from CybersecUrity REports using Natural Language Processing (SecureNLP) . SecRoBERTa has its own wordpiece vocabulary (secvocab) that's built to best match the training corpus. We trained SecBERT and SecRoBERTa versions. Available models include: SecBERT SecRoBERTa Fill…

F001F002F003F004F005F006F007F009F010F011F012F013F014