Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the sig_figs default to 8 #1307

Open
avehtari opened this issue Jan 10, 2025 · 0 comments
Open

Change the sig_figs default to 8 #1307

avehtari opened this issue Jan 10, 2025 · 0 comments

Comments

@avehtari
Copy link
Contributor

Summary

CmdStan stores with 6 digits, and Stan checks constraints with 1e-8.

I propose we change the default CmdStan sig_figs to 8, to reduce constraint check failures due to current mismatch.

Description:

After generate_quantities() method was introduced, there have been several reports of problems with constraint checking. CmdStan stores the posterior draws with accuracy defined by sig_figs argument specifies the number of significant digits for all numerical values in the output files (documented at https://mc-stan.org/docs/cmdstan-guide/command_line_options.html#output-control.section, and mentioned also at https://mc-stan.org/docs/cmdstan-guide/stan_csv_apdx.html). The default for sig_figs is 6. When generate_quantities() method is called, the posterior draws are read from the file and constraint checking is made. The constraint checking is made with accuracy CONSTRAINT_TOLERANCE, which by default is 1e-8 as defined in stan/math/prim/err/constraint_tolerance.hpp. The discrepancy between stored 6 figures and constrain check with 1e-8, causes the constraint check to fail sometimes. Earlier this was a problem mostly with simplex as reported, e.g., in stan-dev/cmdstanr#420. Now the zero-sum constraint seem to cause these errors more likely.

I propose we change the default CmdStan sig_figs to 8, to reduce constraint check failures due to current mismatch.

Pros:

  • fewer users encounter confusing constraint check error
    Cons:
  • CmdStan csv files will be 33% bigger

Some users may also see slight changes in the results if they re-run some inference with new CmdStan version, but then bit-by-bit replicability from version to version has not been guaranteed before either.

Eventually binary output files would be a better solution, but that is a much bigger change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant