TypeError: ModernBertForSequenceClassification.__init__() got an unexpected keyword argument 'compile'

[amrzv]

Hello.
When running finetune_modernbert_on_glue.ipynb in colab on

hf_model = AutoModelForSequenceClassification.from_pretrained(
    checkpoint, num_labels=n_labels, id2label=id2label, label2id=label2id, compile=False
)

the TypeError raised

TypeError                                 Traceback (most recent call last)
[<ipython-input-16-dd2afb7566fd>](https://localhost:8080/#) in <cell line: 0>()
----> 1 hf_model = AutoModelForSequenceClassification.from_pretrained(
      2     checkpoint, num_labels=n_labels, id2label=id2label, label2id=label2id, compile=False
      3 )

1 frames
[/usr/local/lib/python3.11/dist-packages/transformers/models/auto/auto_factory.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    562         elif type(config) in cls._model_mapping.keys():
    563             model_class = _get_model_class(config, cls._model_mapping)
--> 564             return model_class.from_pretrained(
    565                 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
    566             )

[/usr/local/lib/python3.11/dist-packages/transformers/modeling_utils.py](https://localhost:8080/#) in from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, weights_only, *model_args, **kwargs)
   4088         with ContextManagers(init_contexts):
   4089             # Let's make sure we don't run the init function of buffer modules
-> 4090             model = cls(config, *model_args, **model_kwargs)
   4091 
   4092         # make sure we use the model's config since the __init__ call might have copied it

TypeError: ModernBertForSequenceClassification.__init__() got an unexpected keyword argument 'compile'

[warner-benjamin]

Could you take a look at this @ohmeow?

[ULELHAU]

@warner-benjamin @ohmeow I experienced this issue today as well. Any update?

Also, if I get rid of the compile argument, then I get another error

TypeError Traceback (most recent call last)

in <cell line: 0>()
45 checkpoint = "answerdotai/ModernBERT-base"
46 modernbert_tokenizer = AutoTokenizer.from_pretrained(checkpoint)
---> 47 model = ModernBertModel.from_pretrained(checkpoint)
48
49

5 frames

/usr/local/lib/python3.11/dist-packages/transformers/models/modernbert/modeling_modernbert.py in init(self, dim, base, max_seqlen, device, dtype)
157 the cos_sin_cache wll be recomputed during the forward pass.
158 """
--> 159 super().init(dim=dim, base=base, pos_idx_in_fp32=True, device=device, interleaved=False)
160 self.max_seqlen = max_seqlen
161

TypeError: object.init() takes exactly one argument (the instance to initialize)

[MrRace]

same error