The argument "dim" is gone from LlamaRotaryEmbedding initializer. Intentional?

[jeffhataws]

System Info

4.48.0

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

We see the following error with the latest version 4.48.0 of transformers when initializating LlamaRotaryEmbedding:

2025-01-10 14:08:19,223::DEBUG: [stdout]         cache_ids = cache_ids.view(-1, 1)
2025-01-10 14:08:19,223::DEBUG: [stdout] >       embed = LlamaRotaryEmbedding(dim=d_head, max_position_embeddings=2048, base=10000)
2025-01-10 14:08:19,223::DEBUG: [stdout] E       TypeError: LlamaRotaryEmbedding.__init__() got an unexpected keyword argument 'dim'
2025-01-10 14:08:19,223::DEBUG: [stdout] 
2025-01-10 14:08:19,223::DEBUG: [stdout] transformers_neuronx_test/unit/1_core/test_rotary.py:108: TypeError

To test simply instantiate LlamaRotaryEmbedding with dim equal something:

from transformers.models.llama.modeling_llama import LlamaRotaryEmbedding
embed = LlamaRotaryEmbedding(dim=96, max_position_embeddings=2048, base=10000)

Expected behavior

The dim argument was there in previous version. Is the argument no longer needed?

[pagezyhf]

cc @ArthurZucker

[ArthurZucker]

Hey! We used a deprecation cycle with #32135. We had a warning saying that you should pass the config starting 4.45 (only removed support in 4.48)! 🤗

[jeffhataws]

Thanks. Then it is working as expected. We will update.