Can BioBERT be used for multilabel classification problem ?

[coolcoder001]

Hi , this is a great project indeed. I was just wondering , can we use BioBERT for multilabel classification problem out of the box ? or do we need to modify the underlying BERT codebase for achieving the same ?

[wonjininfo]

Hi,
We have a multi-class classification problem (Chemprot) as one of our examples, but our example scripts are currently not able to solve multi-label classification problems.
You can use BioBERT model (which is pre-trained weights), but you need to modify a few lines from our example scripts.
Thanks
Wonjin

[joey082497]

Hi, Thanks for making great project, I was able to study a lot.
I also want to do multi-label classification using BioBert.
May I know which part needs to be fixed?

[wonjininfo]

Hi @joey082497,
I appreciate your interest in our work.

We have recently made resources of our participation ( codes and paper) in the DrugProt challenge available online.
In our recent paper on the DrugProt task, we used multi-label classification (relation extraction task) modellings, so I think you will be interested in the following repository.
Please check here: BioRE drugprot KU-AZ
The script utilizes Transformers library by Huggingface with PyTorch library.
Please also be informed that the pre-processed data format is slightly different, and you also need to provide vocab_add.txt and typeDict.json to the model.

Thank you!
Best,
Wonjin