Correct way to filter files for custom plugin?

[zlacelle]

I apologize if this is obvious. I'm working to create a custom discovery and set of tools for a new file type (MATLAB specifically) and can't quite understand from the README.md how this works. My discovery plugin finds all of the *.m files under the directory in which statick runs (e.g. "./my_project") but I don't want to run all of the directories through the analysis tools.

Should these be filtered:

• When I call statick (e.g. "statick good_dir_1 good_dir_2 ")
• Somehow from the discovery plugin (in which case, how do I pass them as arguments using config.yaml)
• Somehow from each tool plugin (in which case, how should I pass them as arguments using config.yaml)

[tdenewiler]

The discovery plugin is supposed to find all of the files of a certain type per package. In this case your package is at ./my_project.

If you want to completely skip running analysis tools against a directory within your package then you can use exceptions. To specify exceptions you need to have your own directory for Statick configuration, such as my_project/statick_config/rsc. Inside that directory is where you would place a file named exceptions.yaml. In that file you would list your exceptions as described in the README. If the README is not clear let me know and I will update it.

https://github.com/sscpac/statick#exceptions

Once you have exceptions listed you have to tell Statick where to find that file using the --user-paths flag. You would do

statick my_project --output-directory /tmp/x --user-paths my_project/statick_config

It will pick up the exceptions.yaml file and overwrite the one that is installed with Statick.

https://github.com/sscpac/statick/blob/main/statick_tool/rsc/exceptions.yaml

Another case is where you want to run some of your tools against all of your discovered files/directories (for example, because they are fast to run), but only some of your tools against a subset of your discovered files/directories (for example, the tools take a long time to run, or some subdirectories contain third-party code that you cannot modify). For that, you can configure exceptions that apply to a file glob (allows you to ignore a single file or an entire subdirectory) and you can specify a subset of tools that the exception should apply for.

Note that the exceptions are applied after discovery but before a tool plugin runs (where possible, to save time), or they are applied after the tool runs but before a report is generated.

[tdenewiler]

I looked into this more closely, and I found that you can filter files early in the discovery plugin by calling exceptions.filter_file_exceptions_early. That is a great way to save time when running Statick against a large package. An example is at:

https://github.com/sscpac/statick/blob/main/statick_tool/plugins/discovery/python_discovery_plugin.py#L37

If your discovery plugin does not implement that filter, then the exceptions you list in exceptions.yaml will still be applied before the reporting plugins run. The entire run will just take longer because you have to wait for tool plugins to run against subdirectories where you are not looking for results.

[zlacelle]

Great, makes sense. I figured out today to use Exceptions, but I wasn't sure where in the process they hit (i.e. I didn't want to run the tools but filter out results...I just didn't want to run the tools at all).

I'll check the python example you show here. Thank you!

[tdenewiler]

Don't hesitate to ask if something is not working as expected for you.

[zlacelle]

Works as expected with the exceptions.filter_file_exceptions_early() call. I don't see a way to accept the answer (clicking the button doesn't let me for some reason) but I think I'm all good.

[tdenewiler]

You are the first Q&A user, so I'm not sure how all that works either. Good to know it's working.