Support for Qt Test

[joroy]

Add support for Qt Test
Add plugin tests (manual)

Travis integration cannot be implemented because QT lack support for unattended install.
To manually test the plugin, install QT somewhere and update the QT5DIR variable in tests/SConstruct file. Build of the QT5 tests file is possible because the SCons qt5 tool as been installed in the tests directory.

To build and run the tests:

scons -C tests
tox

[nicoddemus]

@joroy, why did you close #22?

Anyway, I will paste my comment below in case you missed it:

[nicoddemus]

Hey @joroy, many thanks for the PR!

I have a couple of questions/issues:

1. I don't know QTest (and I'm short on time to research myself right now, sorry), but does it come bundled with Qt or is a separate tool?
2. In pytest-qt's travis file I use a small script to install PyQt5 on Travis. Can't we use a similar approach to install Qt5/QTest on Travis?
3. I really don't like bundling the scons qt tool with the project. I see two other options:
   1. if we manage to install QTest on Travis, do something similar to install the scons qt tool.
   2. If the above turns out to be impossible, we could ask users to install the scons qt tool manually in the same manner they already would have to do with QTest.

It is really important to me, as a maintainer, to have a reliable CI tool so I can merge contributions from others without having to resort to manually test things, that's why I must insist we find some solution for the test suite to run completely on Travis.

As for the rest of the code I don't have time right now to review it, but I would rather we figure the answer to the questions above first. 😁

Thanks again!

[joroy]

Hi @nicoddemus, for now I just refactored to add a ENABLE_QT_TEST flag in the SConscruct file and the test_pytest_cpp module. At least that way the travis build should not fail.

To anwser your questions:

1. QTest comes with QT
2. PyQ4/5 installer for Windows seems to come with everything they need to run QT application. However, it is not case for Linux install of Qt (not the Python bindings)
3. You're right, I don't like it either. I will remove it from the site_scons directory and ask the user to install the scons qt tool.

I know this pull request is not "master mergeable" material but I wanted to inform you that it exists.
It will live as-is in my fork if somebody needs it.

Regards,

[The-Compiler]

You can install PyQt and Qt on Travis via apt-get quite easily.

[joroy]

Most of the time we need the latest version of Qt (now at 5.7) and we have to use an installer that cannot be run silently. Or it can be with a weird Qt Installer Framework javascript script.

Maybe using an old version of Qt provided by the qt-sdk and qt5-* package is enough to run the plugin tests that I wrote but I never tried it.

[coveralls]

Coverage decreased (-20.1%) to 79.464% when pulling c535bb1 on joroy:qtestlib into 73d6927 on pytest-dev:master.

[joroy]

Regarding coverage, we could push it back to 93% if installing the latest version of Qt on travis was easy as apt-get install {some-package} which is the case for boost and google test...

[The-Compiler]

You're probably looking for the libqt5test5 package. I'm guessing @nicoddemus will want to keep 100% coverage though 😉

[nicoddemus]

I'm guessing @nicoddemus will want to keep 100% coverage though

You got that right. 😉 I would be really surprised if it is not possible to have QTest completely covered on Travis.

Sorry for not participating much, I'm at work right now.

[miurahr]

You can now run unattended install of Qt using python utility 'aqtinstall' which is another Qt CLI instlaler, I hope you become happy with it.

It is still in an alpha status, and there is no wild use case, I always welcome your feedback as usual.