quality of enrichments benchmark (dont close)

[m1ci]

This issue will summarize the results from the benchmark

[m1ci]

Dataset	Language	Experiment type	Micro F1	Micro P	Micro R	Macro F1	Macro P	Macro R	Avg millis/doc	Avg entity/doc
DBpedia Spotlight	English	Recognition, weak match	0.9444	0.9444	0.9444	0.9374	0.9583	0.94	67.02	5.69
DBpedia Spotlight	English	Recognition, strong match	0.8819	0.8819	0.8819	0.872	0.891	0.876	47.9	5.69
KORE50	English	Recognition, weak match	0.9444	0.9444	0.9444	0.9374	0.9583	0.94	45.78	2.86
KORE50	English	Recognition, strong match	0.8819	0.8819	0.8819	0.872	0.891	0.876	45.74	2.86
Reuters-128	English	Recognition, weak match	0.7424	0.7318	0.7534	0.7349	0.6813	0.8474	113.5948	4.85
Reuters-128	English	Recognition, strong match	0.6118	0.6033	0.6205	0.6064	0.5607	0.6986	100.9224	4.85
RSS-500	English	Recognition, weak match	0.6798	0.5289	0.951	0.7299	0.6375	0.951	67.7642	0.99
RSS-500	English	Recognition, strong match	0.5783	0.4499	0.809	0.6258	0.5483	0.809	66.4695	0.99
MSNBC	English	Recognition, weak match	0.825	0.8672	0.7868	0.7925	0.7591	0.8388	734.9444	32.50
MSNBC	English	Recognition, strong match	0.7278	0.765	0.694	0.6924	0.6642	0.7315	894.7778	32.50
News-100	German	Recognition, weak match	0.6438	0.7765	0.5498	0.5872	0.6312	0.5705	369.725	n/a
News-100	German	Recognition, strong match	0.4474	0.5354	0.3843	0.3733	0.3978	0.3646	232.4198	n/a

According to the evaluation results FREME NER in average process one entity in 28 milliseconds or, in other words, 35 entities per second. Please take this statement with some reserve, since we implement caching and documents with frequently occurring entities will be processed faster.

Notes:

• evaluation performed with Gerbil version 1.2.0-SNAPSHOT
• evaluation results FREME NER running at FREME 0.4
• datasets statistics derived from this paper, page 4

[m1ci]

left open for future development