Welcome to simpletransformers Discussions!

[ThilinaRajapakse]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[Timliuzhaolu]

You have done a great job!

I am wondering if simpletransformers allow loading models fine tuned by native transformers? I have fine tuned LongformerForMaskedLM using transformers and would like to use that do Multilabel Classification using simpletransformers. Is that possible? If so how can I do it. Thank you so much!

Tim.

[Timliuzhaolu]

Kind of like the model_name_or_path option for loading local fine-tuned model from transformers.

[ThilinaRajapakse]

Yes, you can! You can load locally saved models by providing the path to the directory containing the model files as the model_name. E.g.:

model = MultiLabelClassificationModel(
    "longformer", "path_to_model_directory",
)

[Timliuzhaolu]

Great sounds fantastic! Thanks a lot man!

[sofiagpmg]

Hi Thilina,
Thank you for sharing the codes and tutorials. Im quite new to simple transformers and not sure what i did wrongly, but for distilbert the initialize is 0% for my case. Could help to advise what i need to do?

[ThilinaRajapakse]

I think it's just the progress bar not being updated in the notebook. As long as it continues past it, everything should be fine.

[stewartd95]

I am trying to run a model.predict for T5 but it seems to be crashing my notebook even when just testing on one example. Have you come across this before? It was able to train fine but now the predict wont run.

[stewartd95]

FYI I solved it but for future reference, for some reason I had to set 'use_multiprocessed_decoding': False in order for predict to not crash

[GoonjanJain]

Hi,

In named entity recognition, how many categories can we classify the word into? Is there any limitation to it?

I had a data in which there were around 4000 categories into which I have to categorize my words into. It gave me stack overflow error.

Request for your guidance and assistance.

[andrei-ars]

raw_outputs in NER task
Hi. I use NERModel from simpletransformers.ner (pretrained model "roberta", "distilroberta-base").
For input sentence 'Click on Basket', it returns the following raw_outputs
[{'Click': [[0.9166969, 7.8369393, 0.31039014, -0.60283166, -1.2205212, -1.0528294, -0.57920927, -1.8390691, -0.7053572, 0.72872484, -1.2057197, -1.1906811, -0.2462096, -0.73678666]]}, {'on': [[2.864785, 0.81569016, -2.4111693, -1.753845, -1.3591471, -0.45600742, 0.22175379, -0.5122926, -0.43606478, 6.4333878, -1.3908641, -0.48224172, -0.9318897, -1.1649382]]}, {'Basket': [[3.537952, -0.11874729, -0.6143042, 4.150231, -1.6462238, 0.49634176, -1.2271496, -1.7730664, 0.10150815, 0.36229157, -1.1427803, 1.7072755, -0.07331692, -0.8622093], [0.2326435, -0.2758199, -0.67160505, 5.107534, -1.1315546, -0.060734212, -1.6606357, -0.8778804, -0.74439603, -0.7747257, -1.1741363, 3.4446402, -0.66995436, -0.360082]]}]
The question is why the token 'Basket' has two list of logits?

[requitefahrenheit]

I'm trying to use mbart for a clarification task, but it appears I can't set the model type and model location correctly when initializing it
I've tried 'bart','facebook/mbart-large' and variations on it but can't get it to work

Does anyone know how to look up the correct labels?
Thanks!

[dvjohnny]

how can i define input with model ?

i have model trained with data format layoutlm like this

text | labels | x0 | y0 | x1 | y1
Aragorn was the heir of Isildur | 1 | [10, 20, 30, 40, 50, 60] | [10, 10, 10, 10, 20, 20] | [20, 30, 40, 50, 60, 70] | [20, 20, 20, 20, 30, 40]
Frodo was the heir of Isildur | 0 | [15, 20, 30, 40, 50, 60] | [10, 10, 10, 10, 20, 20] | [20, 30, 45, 50, 60, 70] | [20, 20, 20, 20, 30, 40]

model = ClassificationModel(
"layoutlm", pick_model_path,
use_cuda=False
)

predictions, raw_outputs = model.predict(['test abc def'])
// i got error

File "/mnt/data/home/tvkhuong/Data/SOFT/anaconda3/envs/classify/lib/python3.7/site-packages/simpletransformers/classification/classification_model.py", line 1708, in load_and_cache_examples example.label = self.args.labels_map[example.label] AttributeError: 'list' object has no attribute 'label'

[dvjohnny]

When i training

model = ClassificationModel(
"layoutlm", "microsoft/layoutlm-base-uncased",
num_labels=2,
use_cuda=False,
cuda_device = 0,
args = model_args
)

[abrahami]

hi!
I wonder what is the architecture/format that simpletransformers is using when the input arguments contain two strings (rather than a single one). I could not find a place where it is discussed or explained.

thanks!
Abraham