v1.10-beta

Release 1.10

This release 1.10 introduces a few new features and fixes a list of bugs. But the more important change of this version is, probably, that we've moved from Codeplex to Github. It's also the opportunityto reveal our new website and url at www.nbi.io This move is important because the documentation needed to be partially rewritten and the software management system has been changed from Mercurial to Git ... more details here under.

This release and the next will be downloadable from there.

What's new ?

New features

• Support the testing of structure for relational database: you can use the assertions exists, contain, equivalent and subsetOf on objects from a relational model including table, views, schemas (More to come in next releases). More info
• Support the testing of sets for the structure of tabular and multidimensional models. More info
• Support the testing of members of sets for tabular and multidimensional models. More info
• Support of a default type (text, numeric, boolean, dateTime) for columns identified as value. More info
• Support of keys with a different type (text, numeric, boolean, dateTime) than text. More info
• Tolerance is now supporting a combination of a percentage and a minimal/maximal absolute value. More info
• Improved performances when loading a test-suite with NUnit GUI.
• Improved user-experience when creating a test-suite with ETL. If some attributes are missing in the definition of the etl, NBi will clearly report the missing attributes.
• Additional partial xsd are also available for the xml elements test, settings, setup and cleanup. See in folder framework, sub-folder schema.

Bug fixes

Framework

• #86: Tolerance and negative values When a value was negative the tolerance was not correctly calculated.
• #77: connectionString is not forwarded to some asserts
• #79: SSIS DTS Variables do not work with dates
• #84: Only "SubsetOf" (and not "Contain" or "EquivalentTo") supports comparaison of members based on another instance

genbi

• #85: xml attribute timeout for element etl not correctly managed by genbi

Major refactoring

• The code supporting the assertions of structure (model) has been refactored to improve maintainability and speed up future development.

Move to github

Why do we leave Codeplex

In a few words: no new feature and no bug fix. To illustrate this, the tab "Source Code" is suffering from a bug since more than six months. Github is really further than codeplex and the gap between the two tools is growing everyday.

Consequences of this move

• The website is hosted http://www.nbi.io
• The downloads section has been moved to GitHub
• This repository (for source code) is now at https://github.com/Seddryck/nbi, Bitbucket will be a mirror and codeplex will be discarded.
• The issue tracker will also be moved to GitHub at https://github.com/seddryck/nbi/issues
• The documentation is now hosted by GitHub at http://www.nbi.io/docs and http://www.nbi.io/automation/ and is built with Jekyll. The source for this documentation is available and editable at https://github.com/Seddryck/NBi/tree/gh-pages

What's next ?

Main ideas for release 1.11 (or next)

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. Work already started
• Support tests on the xml of a report (SSRS). Work already started
• Support assertions about the count of rows added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.

Other ideas probably not for 1.11 but for later

• Support to surround the test with a transaction and rollback the transaction when the test has been executed.
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support transformation of the result-set in the assert (move columns, remove columns, add columns).

I'm (still) working on a Release 2.0. This work is delayed to the lack of GUI for NUnit 3.0. Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in

Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.

v1.9.2

Why have we three releases in a ten days?

Versions 1.9.0.1 and 1.9.1.4 suffered from severe bugs. To be concrete and transparent 1.9.0.1 was more or less a copy of version 1.8: new features were missing. It was a problem of copy/paste on the build server.

In order to fix this, I've created a version 1.9.1.4 Unfortunately, this version had an important and unexpected breaking change in the config file due to a change in a namespace. It was not intended to create this breaking change and I needed to rollback it.

The new version 1.9.2.6, released 2015-02-12, is a fix for these two issues.

I'm sorry for the inconvenient and will try to adapt the process for next releases.

Release 1.9

The release 1.9 includes improvements in different major areas: result-sets assertions, ETL features, genbiL capabilities and test-suite configuration. This release also includes a few bug fixes.

Please report all the bugs to the issues tracker to ensure they are fixed ... even if they are just minor issues.

What's new ?

New features

• ETL: Added support for
  • A timeout different than thirty seconds: previously the timeout for packages running in the SSISDB Catalog had a fixed timeout set to thirty seconds. It's now possible to override this setting.
  • Support of project's parameter: Previously only package's parameters were overridable by NBi, now you can also override the project's parameters.
  • Support of variables: For projects not running in the SSISDB Catalog, you can now override variables. It's done exactly the same way than parameters.
• Test the count of rows in the query's result: This assertion counts rows of a result-set and compares it to an expectation. The possible comparisons are equal, more-than and less-than.
• Improvements for genbiL: I've added a new set of available actions for genbiL (but not available in the UI of Genbi). This new set of features is an evolution and offers new opportunities to automate the process of test-suite generation.
  • Move Column (case): Now, you can directly move a column to the first or last position.
  • Remove Column (case): Now, you can remove more than one column in just one sentence.
  • Hold Column (case): Just the opposite of [Remove Column (case)]. In place of listing the columns to remove, you specify the column that you want to hold.
  • Filter (case): Now, you can filter on column without value (none) or with empty values (empty).
  • Cross (case): If you want to cross a set with another with a unique column, you can do this second set directly from the cross sentence.
  • Replace (case): Ideal if you want to replace values in a column. Eventually you can add a condition that will be checked for each line to decide if you replace the value or not.
  • Substitute (case): At the opposite of replace, just replace a substring of the values by another one.
  • Concatenate (case): Concatenate the values of two columns into a third.
  • Include (suite): Offers the opportunity to add a test directly defined in an external file.
• Define failure's report profile: You can customize the count of items (rows) displayed in the report of failure.
• Defaults and references reports settings: You can create a setting to avoid repetition of attributes path and source in each element report.

Bug fixes

• Setup exe tasks not executing (#48)
• Incorrect list of effective attributes after a failure during a test of structure (property only)

What's cool

• I'll be presenting a session about Test Automation with NBi at the url:SQL Konferenz 2015 - February 4th.
• When a bug is fixed or a feature is ready, I try to put a link to a nightly-build on appVeyor. If this fix/feature is important for you, you can download the build and use it.
• I've a bit delayed the posts about genbiL on the blog. It's mainly due to the new features added to genbiL, during this release. You can decrease by 33% the count of lines needed. Follow this on my url:blog

What's next ?

Main ideas for release 1.10 (and next)

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. (Work already started)
• Offer the opportunity to surround the test with a transaction and rollback the transaction when the test has been executed.
• Support testing of structure for database
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support tests on the xml of a report (SSRS).
• Support transformation of the result-set in the assert (XPath, Xslt, move columns, remove columns, add columns).

I'm (still) working on a Release 2.0. This work is delayed to the lack of releases for NUnit 3.0 (and changes in the API). Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in
  Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.

v1.7

The release 1.7 includes a set of new, tiny but useful, features to be more efficient when creating test-suites. About Genbi, we've added a feature to filter on distinct rows and the support of "if" statements in the template. GenbiL, the macro-language, is now supporting comments. We've also fixed a set of bugs thanks to reporters.

Please report all the bugs to the issues tracker to ensure they are fixed ... even if they are minor issues.

What's new ?

New features

• Setup and Cleanup: Added support for
  • inheritance
  • parallelism
  • run-once
• Support of Shared Datasets when extracting queries from SSRS
• Settings in an external file
• DTD Processing
• Tables to compare result-sets are now in Markdown
• When comparing sets of members or structure's elements, the lists of missing/unexpected elements are now also in Markdown

About genbi:

• Added an option to filter a set of test-cases to only have distinct test-cases.
• Added a way to use the "if" statement in the template.
• genbiL supports the filter distinct instruction and comments

Bug fixes

• Etl runs from the Catalog (SQL Server 2012 and 2014) wasn't supporting parameters
• Setup's command etl-run wasn't reporting the failures of the Etl.
• Details of the exception thrown after an error during a test's execution was removed in some cases.
• The value of defaults in settings was not always correctly applied when they are specified in a genbiL script Bug #25
• Commands were executed in the reverse order for setups
• The root path of the table-load command (Setup/cleanup) was not aligned to the path of the test-suite.
• SSRS parsing (for files) was limited to SQL Server 2008 and 2008R2 (Now suports all versions)

What's cool

• A session about testing BI solutions with NBi will be presented at SQL Saturday at Paris the 13th of September by Cédric L. Charlier.
• Davide Mauri has enlisted NBi as a testing tool during his session at 24hop about Agile Data Warehousing and a blog post.

What's next ?

Main ideas for release 1.8 (and next)

• Execute tests with etls running on a remote server.
• Add a setup command to run sql script or batch.
• Add a setup command to run some executables.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support tests on the xml of a report (SSRS)

I'm (still) working on a Release 2.0. This work is delayed to the lack of releases for NUnit 3.0 (and changes in the API). Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in
  Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.

v1.8

The release 1.8 includes improvements in three major areas: interpretation of CSV files, setup/cleanup features and genbiL capabilities. This release also includes a few bug fixes for Genbi.

Please report all the bugs to the issues tracker to ensure they are fixed ... even if they are minor issues.

What's new ?

New features

• Setup and Cleanup: Added support for
  • File manipulations: copy/delete files
  • Exe or batch: Execute a program or a .bat file
  • SQL batch: Execute a SQL batch on a database
• Define csv-profile: You're not anymore constrained to use semi-columns and carriage return/line feed. This new feature lets you define your favorite field and record separators. The CSV reader has been completely rewritten and also supports UTF-8 and UTF-16 encoding.
• Improvements for genbiL: I've added a new set of available actions for genbiL (but not available in the UI of Genbi). This new set of features is a revolution and offers new opportunities to automate the process of test-suite generation.
  • Add Column (case): Add a column to the test-cases table with a default value
  • Filter (case): Limit the set of test-cases to a subset by applying a filter is now supporting multiple values.
  • Scope (case): Offers the opportunity to have several set of test-cases loaded at the same moment.
  • Copy (case): Duplicate a set of test-cases
  • Merge (case): Merge two sets of test-cases
  • Save (case): Save modification applied to a set of test-cases
  • A silent (quiet) mode is now available for Genbi.
  • Control of parameter "parallelize-queries" in genbiL.
• Trait: a trait is a property (Key/Value) added to a test. Translated to a property of a test in NUnit, this information is rendered in the report of execution.

Bug fixes

• $now$ is not correctly formatted before 10AM (leading 0 missing) and Invalid DateTime generated by GenBI
• Permutation of unexpected and missing qualifiers in the comparison of items
• GenBI and match pattern
• GenBI and member count

What's cool

• Davide Mauri has spoken about NBi during his session at Summit2014
• I'll be presenting a session about Test automation with NBi for the Global French Virtual Chapter. It should be confirmed but probably the 17th of December.
• When a bug is fixed or a feature is ready, I try to put a link to a nightly-build on appVeyor. If this fix/feature is important for you, you can download the build and use it.
• I'll surely blog in the next weeks about the new capabilities of genbiL and how to automate the build of a regression test-suite from scratch in a couple of minutes. Follow this on my blog

What's next ?

Main ideas for release 1.9 (and next)

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. (Work already started)
• Offer the opportunity to surround the test with a transaction and rollback the transaction when the test has been executed.
• Support testing of structure for database
• Support of rows count tests
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support tests on the xml of a report (SSRS)

I'm (still) working on a Release 2.0. This work is delayed to the lack of releases for NUnit 3.0 (and changes in the API). Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in
  Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.