[V - Non-Rule] Rule-specific testing

[Ghesselink]

There are many files at the moment (78 in a newly developed branch) and there is no good integration (as far as I know) between behave in the command line and gherkin.
This functionality allows to test for specific rules (e.g. 'alb001, 'alb002') as well as categories (e.g. 'alb') just by running 'pytest -sv'

[aothms]

Instead of hardcoding this I'd rather look for a way to supply this as a command line argument (python test_main.py alb). See for inspiration: https://github.com/IfcOpenShell/IfcOpenShell/blob/v0.7.0/src/ifcopenshell-python/test/test_rules.py#L15 Also see Jakub's earlier PR #39

[Ghesselink]

It's now possible with a command line argument (e.g. python test_main.py alb, but also multple arguments as in python test_main.py alb002 alb003)

test_for_rules = any(re.match(r"^[a-zA-Z]{3}\d{3}$", arg) for arg in sys.argv[1:])
test_files = [fn for fn in glob.glob(os.path.join(os.path.dirname(__file__), 
                "files/**/*.ifc"), recursive=True) if len(sys.argv) < 2 or any(x in fn.lower() for x in sys.argv[1:]) or not test_for_rules]

[aothms]

I still find it a bit confusing:

• test_for_rules: why is this a boolean, why not do filter() instead of any() to filter out all non-rule code argv and use them below
• the regex is lower or upper case but by doing x in fn.lower() actually an uppercase argument will not work
• some of the rule codes such as alb will arbitrarily match word fragments such as albino which will be rather confusing. Test files are in separate directories so why not filter only on that directory name instead of entire file path?

[Ghesselink]

It's modified in 377f4e6

why is this a boolean, why not do filter() instead of any() to filter out all non-rule code argv and use them below

Done.

some of the rule codes such as alb will arbitrarily match word fragments such as albino which will be rather confusing. Test files are in separate directories so why not filter only on that directory name instead of entire file path?

Done.

the regex is lower or upper case but by doing x in fn.lower() actually an uppercase argument will not work

The aim was to also allow for uppercase input, such as 'Alb001' or'ALB001'instead of 'alb001' It should be good now: the regex match will still allow for this uppercase and it later converts into lowercases to match the folder name (which are always in lowercases, such as in 'alb001').

Furthermore, there is a warning added in the case there is someone wants to test a non-existing rule code (e.g. 'alb005'). This could be either a typo or it could mean that there are no test files added yet (or something else even). Nothing too spectecular, but it might help someone writing rules and testing it locally.

[aothms]

You have rule_code_pattern and rule_codes duplicated in the function body on line 27.

[aothms]

Instead of deciding for running all tests at the end on empty test_files, I would decide for this based on an empty [a for a in sys.argv[1:] if not a.startswith('-')]. That way, running with an non-existant rule code will run nothing instead of everything.

[aothms]

I think we need to test that an argument is either a valid rule code or an existing IFC file. In other cases we should do nothing and throw a big error. Otherwise it's easy to think you have ran the tests while you actually did nothing because you misttyped a filename.