You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CmdStan stores with 6 digits, and Stan checks constraints with 1e-8.
I propose we change the default CmdStan sig_figs to 8, to reduce constraint check failures due to current mismatch.
Description:
After generate_quantities() method was introduced, there have been several reports of problems with constraint checking. CmdStan stores the posterior draws with accuracy defined by sig_figs argument specifies the number of significant digits for all numerical values in the output files (documented at https://mc-stan.org/docs/cmdstan-guide/command_line_options.html#output-control.section, and mentioned also at https://mc-stan.org/docs/cmdstan-guide/stan_csv_apdx.html). The default for sig_figs is 6. When generate_quantities() method is called, the posterior draws are read from the file and constraint checking is made. The constraint checking is made with accuracy CONSTRAINT_TOLERANCE, which by default is 1e-8 as defined in stan/math/prim/err/constraint_tolerance.hpp. The discrepancy between stored 6 figures and constrain check with 1e-8, causes the constraint check to fail sometimes. Earlier this was a problem mostly with simplex as reported, e.g., in stan-dev/cmdstanr#420. Now the zero-sum constraint seem to cause these errors more likely.
I propose we change the default CmdStan sig_figs to 8, to reduce constraint check failures due to current mismatch.
Some users may also see slight changes in the results if they re-run some inference with new CmdStan version, but then bit-by-bit replicability from version to version has not been guaranteed before either.
Eventually binary output files would be a better solution, but that is a much bigger change.
The text was updated successfully, but these errors were encountered:
Summary
CmdStan stores with 6 digits, and Stan checks constraints with 1e-8.
I propose we change the default CmdStan
sig_figs
to 8, to reduce constraint check failures due to current mismatch.Description:
After
generate_quantities()
method was introduced, there have been several reports of problems with constraint checking. CmdStan stores the posterior draws with accuracy defined bysig_figs
argument specifies the number of significant digits for all numerical values in the output files (documented at https://mc-stan.org/docs/cmdstan-guide/command_line_options.html#output-control.section, and mentioned also at https://mc-stan.org/docs/cmdstan-guide/stan_csv_apdx.html). The default forsig_figs
is 6. Whengenerate_quantities()
method is called, the posterior draws are read from the file and constraint checking is made. The constraint checking is made with accuracyCONSTRAINT_TOLERANCE
, which by default is 1e-8 as defined in stan/math/prim/err/constraint_tolerance.hpp. The discrepancy between stored 6 figures and constrain check with 1e-8, causes the constraint check to fail sometimes. Earlier this was a problem mostly with simplex as reported, e.g., in stan-dev/cmdstanr#420. Now the zero-sum constraint seem to cause these errors more likely.I propose we change the default CmdStan
sig_figs
to 8, to reduce constraint check failures due to current mismatch.Pros:
Cons:
Some users may also see slight changes in the results if they re-run some inference with new CmdStan version, but then bit-by-bit replicability from version to version has not been guaranteed before either.
Eventually binary output files would be a better solution, but that is a much bigger change.
The text was updated successfully, but these errors were encountered: