Towards a Test Driven Development Framework in Vala Part 5 - Minimum Viable Product

Things have really been moving quickly since I last posted, with the development branch really starting to take shape. When I sat down to look at the list of requirements, I decided that the best place to start for a first release would be to at least replicate the same feature set of the original. To recap, those were:

• Automatic test discovery like JUnit or .NET testing framework.
• Running tests for all parameters from specific set.
• Utility functions for waiting in a main loop until specified event or timeout occurs.
• Support for asynchronous tests. Method declared async in vala will be automatically run under main loop until completion or configurable timeout.
• Utility functions providing temporary directory to tests.

These have been translated into GitHub issues and the Waffle board as well as a few additional features that I thought should make the first cut, namely:

• Geanie support
• Working examples

These have all been added to the Version 1.0.0 milestone and well, I'm pleased to say that after a little under two weeks of concerted effort, I have (re)implemented almost all of the above features! Based on the level of effort so far, I am now envisaging an initial release as early as the 1st of March.

I'm actually pretty excited about what has come out of the process so far. One of the original itches I set out to scratch was the verbosity of unit tests in Vala and through the voodoo of xml/xslt/json and GModule I believe I have achieved that. While the implementation details are frankly a little scary, the resulting user facing API hides them quite nicely.

With a correctly configured build script, using Valadate is as easy as declaring a subclass of TestCase and adding annotated instance methods like so:

That's it. No main function required, no need to add tests in the TestCase constructor. Clean and simple, the way it should be. The code snippet above is a real live test from the Valadate framework (the actual test of the test, so to speak) and it runs beautifully, producing TAP output both to file and to the terminal -

Astute readers will notice that it is still GLib.Test running things under the hood, although it is sufficiently encapsulated to allow its gradual replacement without affecting the way end users write their tests. It should now be possible to add things like events and notifications without breaking user's code.

The TestRunner class handles test discovery via a Gir file output when the test is compiled. This was a key concept of the original Valadate but I took it a step further, combining it with GModule to create a kind of "poor person's" Introspection module. The test binary needs to be compiled as a Position Independent Executable (PIE) for this to work which is presently only supported on Linux and Mac OSX, although the fundamentals should apply to executable DLLs on Windows as well.

The TestRunner currently supports [Test], [AsyncTest] and [SkipTest] with parameters. Although it is trivial to add new annotations, I am going to keep them to a minimum and move to a plugin based model which will allow plugins to decorate and control how test methods are run.

Of course, if all of this is a little too funky for you, you can still do things the old way by adding each test method in the TestCase's constructor:

and providing your own main entry point like so:

in this case, you don't need to compile it as a PIE binary or add the Method annotations (they won't work anyway). You can still use all of Valadate's other awesome features such as asynchronous tests, you'll just have more redundant code to manage.

With this feature now implemented and on the way to being solidly tested, I feel it's time to merge the development branch and roll a release. That way I can start getting feedback (and bug reports) on what's been done so far before implementing the meatier features like Gherkin integration and a GUI.

It goes without saying that the only way anybody will be able to use Valadate is if there is clear documentation and working examples, so to this end there are now several example projects and a Wiki. I've also added support for building Valadoc and GtkDoc docs from the project source tree. There's still a bit of work to do before the first release, but the infrastructure is now in place (and I can close issue #1!).

So that's all I'm going to go into in this post, so I can get back to documenting the work I've done and getting the release ready for deployment. The next post on Valadate will be about the release, so now's a good a time as any to jump in and let me know what you think, in the comments below or in the "usual" places. Thanks for watching!