Segmentation fault when importing ESMFold and Tokenizers from transformers along with Pyrosetta

[SIAndersson]

System info:

• transformers version: 4.37.1
• Platform: Linux-4.18.0-513.11.1.el8_9.x86_64-x86_64-with-glibc2.28
• Python version: 3.9.16
• Huggingface_hub version: 0.20.2
• Safetensors version: 0.3.3
• Accelerate version: 0.26.1
• Accelerate config: not found
• PyTorch version (GPU?): 2.1.2 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): 0.7.5 (cpu)
• Jax version: 0.4.23
• JaxLib version: 0.4.23
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Code:

import pyrosetta
from transformers import AutoTokenizer, EsmForProteinFolding

Expected behaviour: the module imports without issue.
I can import pyrosetta on its own without issue. I can import the transformers modules without issue and run inference on PDB modules, as described in the protein structure prediction Jupyter notebook. I can do this without issue in a separate script. It is only when I import both that the segmentation fault occurs. The import order does not matter. Given that both work separately, I would expect them to work together as I cannot find any package conflicts.