Experiment with OpenFactVerification

[kongzii]

Experiment with https://github.com/Libr-AI/OpenFactVerification to see if it's better than KnownOutcomeAgent, or if it could be used in addition for him, or perhaps as a tool for normal agents

[kongzii]

Could be also nice to create a mech out of it. I'd just double-check with Martin that we can put time into it.

[kongzii]

Except being a tool for traders, seems like this could be also used for market resolution.

Let's say the question was Will Missouri execute death row inmate Brian Dorsey on 15 April 2024?

We can rewrite it as Missouri executed death row inmate Brian Dorsey on 15 April 2024

And then use OFV to verify such claim. I tried it out and it seems to be promising, for this specific case, the result is:

"5_claim_verify": {
        "Missouri executed death row inmate Brian Dorsey on 15 April 2024.": {
            "reasoning": "Based on the provided evidence, it is established that Brian Dorsey was executed by the state of Missouri. However, the specific date of execution is not corroborated by the evidence given. The evidence mentions that the execution took place on a Tuesday, without providing a specific date. Given that 15 April 2024 is a Monday, the execution could not have occurred on 15 April 2024 as stated in the text.",
            "error": "Incorrect execution date.",
            "correction": "Brian Dorsey was executed on a Tuesday, not on 15 April 2024, which is a Monday.",
            "factuality": false,
            "claim": "Missouri executed death row inmate Brian Dorsey on 15 April 2024.",
            "evidence": [
                {
                    "text": "Missouri executed death row inmate Brian Dorsey on 15 April 2024.\nAnswer: The state of Missouri on Tuesday executed Brian Dorsey for the 2006 murders of his cousin, Sarah Bonnie, and her husband, Benjamin Bonnie, after an effort to have his life spared failed in recent days. Dorsey's time of death was recorded as 6:11 p.m, the Missouri Department of Corrections said in a news release.",
                    "url": "Google Answer Box"
                },
                {
                    "text": "Did Missouri execute death row inmate Brian Dorsey?\nAnswer: The state of Missouri on Tuesday executed Brian Dorsey for the 2006 murders of his cousin, Sarah Bonnie, and her husband, Benjamin Bonnie, after an effort to have his life spared failed in recent days. Dorsey's time of death was recorded as 6:11 p.m, the Missouri Department of Corrections said in a news release.",
                    "url": "Google Answer Box"
                },
                {
                    "text": "When was Brian Dorsey executed?\nAnswer: Missouri conducted its first execution of 2024 on Tuesday, putting Brian Dorsey to death by injection at the state prison in Bonne Terre. Dorsey pleaded guilty to murdering his cousin, Sarah Bonnie, and her husband, Benjamin Bonnie, in 2006 at their home in New Bloomfield.",
                    "url": "Google Answer Box"
                }
            ]
        }
    }

[kongzii]

Done, waiting for review in mech: valory-xyz/mech#225