Skip to content

r0han99/word-contextualisation-with-llms

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Word Contextualisation with Fine-Tuned State-of-the-Art Transformers

Building Semantic Surroundings for Advanced Vocabulary

Welcome to the Word Contextualization Generator project! This initiative focuses on fine-tuning large language models (LLMs) such as GPT-2, Llama2, and Gemma-2b to generate contextually relevant and linguistically coherent word usage examples in the field of astronomy. By leveraging a custom dataset of 30 vocabulary words and their usage instances, we aim to enhance the models' ability to produce domain-specific text. The project includes detailed steps for data preparation, model selection, fine-tuning, and evaluation using advanced metrics like chrF++. Our goal is to push the boundaries of automated text generation in specialized domains, providing valuable insights into the customization of LLMs for precise and contextually appropriate applications.

Example Usecase

Instruction: What is the usage of the word 'abound' in the context of astronomy?

pre-trained llm

Response:
'Abundance' refers to the relative amount of a particular element or other property.

'Abundance' is used in the context of astronomy to mean 'a large amount of a specific chemical element or compound'.
Abundance is used to describe the concentration of elements in the universe as a whole, and is a useful measure of the relative amount of a given element or compound in the cosmos.
Abundance of a specific element in the Universe is determined by a number of factors, including the initial conditions of the universe, the rate of chemical reactions and the rate of cosmic ray bombardment.
The abundance of a specific element or compound can be used to infer the age and history of the Universe.
The abundance of a given chemical element can also be used to infer the presence of life on Earth or other planets.
The abundance of a chemical element can be determined by analyzing a sample of the substance or by measuring the ratio of the abundance of a given element to that of a standard element.

Abundance is the amount of a particular element or compound in a sample of a substance.
Abundance can be determined by analyzing a sample of

fine-tuned llm

Response: A plethora of planets has abounded in the Milky Way galaxy.

About

Building Semantic Surroundings for Advanced Vocabulary

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published