Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cruncher error logs #484

Open
bezdomniy opened this issue Nov 25, 2024 · 0 comments
Open

Cruncher error logs #484

bezdomniy opened this issue Nov 25, 2024 · 0 comments
Assignees

Comments

@bezdomniy
Copy link

Hi JDemetra+ devs,
We are using cruncher v3 in a data processing application implemented in Java. At a high level we are calling jdplus.cruncher.App.main with a valid workspace and parameters file, then parsing the output files for consumption by downstream services.
One issue with cruncher is that in most cases it does not produce any exceptions or logs when any of the SAItem.process calls fail, so we cannot know when/why a cruncher run failed (other than by seeing if it produces an empty output file).

The cause seems to be as follows:
Cruncher hardcodes the "verbose" param in this SAItem process call to false: https://github.com/jdemetra/jdplus-main/blob/develop/jdplus-main-cli/jdplus-main-cli-bin/src/main/java/jdplus/cruncher/batch/SaBatchProcessor.java#L89
This causes cruncher to use a DummyLog rather than the DefaultProcessingLog class - https://github.com/jdemetra/jdplus-main/blob/develop/jdplus-main-base/jdplus-toolkit-base-parent/jdplus-toolkit-base-api/src/main/java/jdplus/toolkit/base/api/processing/DefaultProcessingLog.java

Feature request
Would it be possible to update cruncher with the following changes:

  1. allow users to parameterise cruncher to use "verbose" logging (i.e. log errors DefaultProcessingLog)
  2. return the content of the ProcessingLog to the user when the cruncher run completes.
@Immurb Immurb self-assigned this Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants