Syllogistic Reasoning for Legal Judgment Analysis

Legal judgment assistants are developing fast due to impressive progress of large language models (LLMs). However, people can hardly trust the results generated by a model without reliable analysis of legal judgement. For legal practitioners, it is common practice to utilize syllogistic reasoning to select and evaluate the arguments of the parties as part of the legal decision-making process. We construct and manually correct a syllogistic reasoning dataset for legal judgment analysis in this paper.

Datasets

SLJA datasets are available at SLJA.

Model evaluation

ChatGLM Server

## ChatGLM model
cd tools/ChatGLM
python api.py

Article Retrieval Server

## Retrieval model
cd tools/IR
python server.py

Syllogism

## OpenAI model
cd syllogism
python openai_syllogism.py

## ChatGLM model
cd syllogism
python chatglm_syllogism.py

Plaintext

## OpenAI model
cd plaintext
python openai_plaintext.py

## ChatGLM model
cd plaintext
python chatglm_plaintext.py

Evaluation

cd tools/Evaluate
python evaluate.py