Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test group annotations to riak_test #674

Merged
merged 3 commits into from
Aug 8, 2014

Conversation

bowrocker
Copy link
Contributor

Enable tests to be tagged with a list of attributes that can be called on the riak_test command line to group or aggregate tests (#667)

Test are tagged with the following attribute:

-test_type([bucket_types, http]).

Then, instead of named tests at the command line, groups are specified:

./riak_test -c rtee -g bucket_types,http

This will result in 2 matches against the above test; any match will run the test:

Tests to run: [{http_bucket_types,
                   [{id,-1},
                    {platform,<<"local">>},
                    {version,<<"riak_ee-2.0.0beta6-11-g5a161cf-develop">>},
                    {project,<<"riak_ee">>}]},
               {bucket_types,
                   [{id,-1},
                    {platform,<<"local">>},
                    {version,<<"riak_ee-2.0.0beta6-11-g5a161cf-develop">>},
                    {project,<<"riak_ee">>}]}]

- Tests can be annotated with one or more '-test_type(name)' attributes.
- Tests can be declared to be in multiple groups
- These types can be run in groups via the riak_test command line
  --groups (-g) option
- Multiple groups can be specified by comma delimiting.
…ake lists.

- For example, in a riak_test: -test_type([bucket_types, http]).
Project = list_to_binary(rt_config:get(rt_project, "undefined")),
compose_test_data(DirTests, SpecificTests, ParsedArgs);
load_tests(RawGroupList, ParsedArgs) ->
Groups = lists:foldl(fun(X, Acc) -> string:tokens(X, ", ") ++ Acc end, [], RawGroupList),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize I did the same thing at L248, but this line goes a bit long. Can we pull out the fold funtion definition or put the fold parameters on separate lines to keep within 80 columns?

@kellymclaughlin
Copy link
Contributor

Should you be able to include one or more -t options when specifying -g? Currently, that does not seem to be the case:

13:58:46:riak_test(feature/jra/test-groups *) $ ./riak_test -c riak_ee -g http -t http_security
Tests to run: [{http_bucket_types,
                   [{id,-1},
                    {platform,<<"local">>},
                    {version,<<"riak_ee-2.0.0beta8-4-g51a8e13-develop">>},
                    {project,<<"undefined">>}]}]

@seancribbs
Copy link
Contributor

I also wonder if this should not (also) be metadata in GiddyUp. Then someone could write:

$ riak_test -c riak_ee -r config -g security

@seancribbs
Copy link
Contributor

We could also then potentially filter the matrix to tests in specific groups.

@bowrocker
Copy link
Contributor Author

@seancribbs sounds good -- please let me know if there is anything that would make integration easier or more sensible.

- Options --groups and --tests are mutually exclusive for now.
- fix indentation for overly long lines.
@kellymclaughlin
Copy link
Contributor

+1

bowrocker added a commit that referenced this pull request Aug 8, 2014
Add test group annotations to riak_test
@bowrocker bowrocker merged commit 27b8607 into feature/decouple-cluster-setup Aug 8, 2014
@bowrocker bowrocker deleted the feature/jra/test-groups branch August 8, 2014 21:16
@bowrocker bowrocker mentioned this pull request Aug 8, 2014
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants