Skip to content
This repository has been archived by the owner on Jan 10, 2024. It is now read-only.

Prediction calibration graph should not be folded (10% ≠ 90%) #103

Open
matthewfallshaw opened this issue Sep 12, 2016 · 1 comment
Open

Comments

@matthewfallshaw
Copy link
Member

Others (Mataculus) have found that low confidence predictions can be better calibrated than high confidence predictions. Pbook should not fold prediction graphs at 50%.

@gwern
Copy link
Contributor

gwern commented Oct 6, 2016

This would also be beneficial from the UX perspective: I've noted a fair number of people are confused by the plot 50-100%, since it doesn't occur to them that PB might be negating <50% predictions and calibrating that way. A straightforward 0-100% graph will be noisier (since it halves the sample size in each bin) but much more easily interpreted.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants