-
Notifications
You must be signed in to change notification settings - Fork 35
Overview
Good quality and bug-free software is always a hard constraint to respect in software development projects. Talking specifically about acceptance tests, the tools to help you develop them are sometimes hard to use and the test scripts can become very difficult to maintain. Writing tests can be a cumbersome, tedious and slow task and sometimes the cost of automating those tests can be so high that you will end up doing manual testing.
Pyccuracy is a Behaviour-Driven-Development-style Acceptance Testing framework that was created to achieve some goals:
We try to use a clear and straightforward language to make it very easy to write and maintain tests, even if you are not a programmer or “tech person”.
Running a test should be simple and fast. We always try to do most of the heavy lifting and performance optimizations to make everything easier and faster.
If you think Pyccuracy built-in actions are not enough, you can easily extend Pyccuracy and create your own. You can also group actions to reuse test code and simplify your tests. If you don’t like Selenium, it’s easy to add support to other browser drivers as well (HTML Unit, Windmill, etc).
This means writing Acceptance Tests prior to the tasks that describe them. This way you take Test-Driven-Development one step further and push yourself towards the right direction when it comes to the application UI (user interface).
We believe it can be very useful to have Acceptance tests that prove your App works. Acceptance tests demonstrate, without a doubt, that your application works as the customer expects. Most of the time people don’t do it and these are some of the reasons:
- Acceptance Tests can be fragile
- Acceptance Tests can be slow
- Acceptance Tests can be hard to write
- They believe Unit Test coverage is good enough
These are real concerns and can really block you from writing such tests.
Pyccuracy is not a silver bullet and will not solve all of these problems, but here is how we try to help:
Fragility is often a problem with how you are writing your tests, other than the actual tests.
Acceptance tests need to test behavior. It’s ok to write acceptance tests that verify that the UI is in accordance to any number of standards you define (Pyccuracy even supports that concept). Just bare in mind that these tests are not how acceptance tests are supposed to be. They are going to be very fragile, because that’s their purpose: breaking if the UI changes. So it’s best to have as few of those as you possibly can.
We believe in writing less-fragile tests and provide you tools to do that. For instance, (1) we allow creating aliases for your page elements so that when they change it’s easier to update the test suite without having to re-write the tests and also (2) we provide flexible CSS selectors instead of XPath only (see “Creating Custom Pages” section).
Given the fact that they are wired tests (meaning they connect to real resources, which are always slower than mocks or stubs), Acceptance tests are always going to be slower than Unit tests.
Even though we rely a lot on the underlying testing framework (i.e. Selenium), we’ve optimised Pyccuracy quite a lot in order to have better performance.
Pyccuracy also offers support to parallel test running (through Selenium Grid), which means that you can distribute your tests over many “workers” (browsers) to reduce total testing time.
With Pyccuracy, the proposition is to describe Acceptance tests with an easier language than Python (or a general programming language). We provide a structured natural language (a DSL – Domain-Specific Language) for writing acceptance tests (see “Quick Start” section). The purpose of this language is to make it easier to write tests.
This enables scenarios such as a wiki that holds all your tests as well-organized pages or using the acceptance tests to enable better discussion with clients on what a story should cover.
Unit and Acceptance tests aim at different targets. While unit tests aim at proving that your methods do what they should do, acceptance tests prove that your whole application does what the client expects.
Aside from that, unit tests are mocked tests, meaning that they can’t depend on any external resources, while acceptance tests are wired tests, meaning they use your application in the same way the client is going to use it.
Even with good unit test coverage, you will still have benefits having acceptance tests.
Pyccuracy currently supports multiple languages. This has been a really important goal for Pyccuracy team since the first release. If you want to write clear tests, what clearer language to use than the one you currently speak?
We currently support only English and Portuguese (the team behind Pyccuracy is composed only of Brazilians), but we are willing to support other languages (please let us know if you want to contribute).
If your language is not covered in Pyccuracy yet, it’s really easy to create a language file. Shouldn’t take more than half an hour. Just contact us at Pyccuracy’s e-mail group, and we’ll point you in the right direction.
A typical Pyccuracy test would be something like:
As a Google User
I want to search Google
So that I can test Pyccuracy
Scenario 1 - Searching for Hello World
Given
I go to "http://www.google.com"
When
I fill "q" textbox with "Hello World"
And I click "btnG" button and wait
Then
I see "Hello World - Google Search" titleAs you can see, that’s pretty clear and yet, very intentful. Without resorting to anything other than the test you could understand what’s happening. Other than that, this test proves an aspect (a simple one) of Google (that the specified search text should be in the title).
With a good test suite you can easily cover all business value of your application and use that as proof that your app does what it should be doing.
Pyccuracy should speed up the creation of Acceptance tests and improve on the way your dev team communicates with the client when related to what business value and behavior the stories contain.