Separating report_target application in interactive mode

[M6AI]

Bug / Requirement Description

The report_target decorator is applied in test_context every time the interactive mode accesses it. This results in an overhead during startup, and also runtime.

Solution description

For now we are moving to the runner method with report_target in interactive mode. Longer term we need to make reapplication dependent on the reload.

Checklist:

• Test
• Example (both test_plan.py and .rst)
• Documentation (API)
• News fragment present for release notes
• MS info leakage check
• For new driver: driver index page
• For new assertion: ui/pdf/std renderers, documentation
• For new cmdline arg: documentation

[M6AI]

Copied over the condition so that possible refactorization catches both occurrences, there should be better ways to test interactivity.

[Pyifan]

I suggest that we address the performance issues more thoroughly, there are 3 of them:

1. we are calling get_test_context again and again under interactive mode --> suggest that we invalidate self._test_context when reload() is called, then next time test_context is needed, it will re-calculate. I think this only needs a few lines of change (set test._test_context to None in reload()).
2. testcase_report_target is default on -- this feature is quite heavy on runtime and report size, consider make it default off?
3. report_target is calculated in get_test_context so in interactive mode, we will do that for all testcases no matter if they are run or not. You have addressed this one by move it to _run_testcases_iter, why not move it to _run_testcase so it is shared by batch & interactive mode?

[M6AI]

I suggest that we address the performance issues more thoroughly, there are 3 of them:

1. we are calling get_test_context again and again under interactive mode --> suggest that we invalidate self._test_context when reload() is called, then next time test_context is needed, it will re-calculate. I think this only needs a few lines of change (set test._test_context to None in reload()).
2. testcase_report_target is default on -- this feature is quite heavy on runtime and report size, consider make it default off?
3. report_target is calculated in get_test_context so in interactive mode, we will do that for all testcases no matter if they are run or not. You have addressed this one by move it to _run_testcases_iter, why not move it to _run_testcase so it is shared by batch & interactive mode?

Except for 2 I agree. The default on came from business requirement, let me reach you offline with that.

[M6AI]

All kinds of unused functions, we can restore from history if needed at some point.

[M6AI]

We checked and this was breaking testcase removal, the reloader would not handle the IndexError that is thrown by lists, we only continued here originally upon KeyError.

[M6AI]

There was a side effect from report_target, it was turning the bound method into a free function, but here we need to access now the func attribute.