conflict between unit tests

[dmlorenzetti]

There is a conflict between the unit tests. Running each test individually, they all pass. When run all at once, only the first one passes.

The conflict may relate to duplicate names on the output tables.

The following steps reproduce the problem, on both Mac and Windows machines, running in branch "1.x" (use changeset 7c57c07 for reference):

(1) Verify that the first three unit tests work when run individually:

py.test applications/utest_applications/utest_daily_summary/test_daily_summary.py
applications/utest_applications/utest_daily_summary/test_daily_summary.py .....

py.test applications/utest_applications/utest_energy_signature/test_energy_signature.py
applications/utest_applications/utest_energy_signature/test_energy_signature.py ..

py.test applications/utest_applications/utest_heat_map/test_heat_map.py
applications/utest_applications/utest_heat_map/test_heat_map.py ....

(2) Verify that all unit tests after the first (daily_summary) fail when run all at once:

py.test --tb=short
applications/utest_applications/utest_daily_summary/test_daily_summary.py .....
applications/utest_applications/utest_energy_signature/test_energy_signature.py FF
applications/utest_applications/utest_heat_map/test_heat_map.py FFFF
etc.

(3) Remove or rename the first unit test, so that it will not execute:

mv applications/utest_applications/utest_daily_summary/test_daily_summary.py hold_tds.py

(4) Verify that what used to be the second unit test (energy_signature) now runs, while all else fail:

py.test --tb=short
applications/utest_applications/utest_energy_signature/test_energy_signature.py ..
applications/utest_applications/utest_heat_map/test_heat_map.py FFFF
etc.

(5) Put things back to rights:

mv hold_tds.py applications/utest_applications/utest_daily_summary/test_daily_summary.py

The failures from lines like the following (this one from energy_signature.py):
self.out.insert_row(LOAD_VS_OAT_TABLE_NAME, {
"oat": x['oat'][0],
"load": x['load'][0]
})
with an error message like the following:
E TypeError: 'load' is an invalid keyword argument for this function

Note that some later tests fail for other reasons. We are aware of, and working on, those applications.

[fstshrk]

Are you clearing the output tables as part of the text fixture before
starting the next test?

The process should be:

1. Set up test
2. Run test
3. Clean up test artifacts
4. Go to step 1 if there are more tests to run.

Thanks

Bora

On 10/20/2014 4:07 PM, David M. Lorenzetti wrote:

There is a conflict between the unit tests. Running each test
individually, they all pass. When run all at once, only the first one
passes.

The conflict /may/ relate to duplicate names on the output tables.

The following steps reproduce the problem, on both Mac and Windows
machines, running in branch "1.x" (use changeset 7c57c07
7c57c07
for reference):

(1) Verify that the first three unit tests work when run individually:

py.test
applications/utest_applications/utest_daily_summary/test_daily_summary.py
applications/utest_applications/utest_daily_summary/test_daily_summary.py
.....

py.test
applications/utest_applications/utest_energy_signature/test_energy_signature.py
applications/utest_applications/utest_energy_signature/test_energy_signature.py
..

py.test
applications/utest_applications/utest_heat_map/test_heat_map.py
applications/utest_applications/utest_heat_map/test_heat_map.py ....

(2) Verify that all unit tests after the first (daily_summary) fail
when run all at once:

py.test --tb=short
applications/utest_applications/utest_daily_summary/test_daily_summary.py
.....
applications/utest_applications/utest_energy_signature/test_energy_signature.py
FF
applications/utest_applications/utest_heat_map/test_heat_map.py FFFF
etc.

(3) Remove or rename the first unit test, so that it will not execute:

mv
applications/utest_applications/utest_daily_summary/test_daily_summary.py
hold_tds.py

(4) Verify that what used to be the second unit test
(energy_signature) now runs, while all else fail:

py.test --tb=short
applications/utest_applications/utest_energy_signature/test_energy_signature.py
..
applications/utest_applications/utest_heat_map/test_heat_map.py FFFF
etc.

(5) Put things back to rights:

mv hold_tds.py
applications/utest_applications/utest_daily_summary/test_daily_summary.py

The failures from lines like the following (this one from
energy_signature.py):
self.out.insert_row(LOAD_VS_OAT_TABLE_NAME, {
"oat": x['oat'][0],
"load": x['load'][0]
})
with an error message like the following:
E TypeError: 'load' is an invalid keyword argument for this function

Note that some later tests fail for other reasons. We are aware of,
and working on, those applications.

—
Reply to this email directly or view it on GitHub
#3.

Help us find a cure for canine cancer.
Please donate at:
http://www.wearethecure.org/friends/skipper

[hashstat]

What I found is that when py.test is run directly the database is not setup and cleared correctly. To fix that, I run it through openeis which, for each test, sets up a temporary database in memory and inserts the fixtures. Try running the tests using openeis pytest.

I also found that to pass files and certain options to pytest, it must be done after a -- option, like so: openeis pytest -- applications/utest_applications/utest_daily_summary/test_daily_summary.py. Please give it a try and let us know how it goes.

Brandon

[dmlorenzetti]

Running openeis pytest gives me the same errors as described above (e.g., daily_summary passes and energy_signature fails, but removing daily_summary allows energy_signature to pass).

-Dave