Towards a Test Driven Development Framework in Vala Part 2. Requirements and System Architecture

Less than a week ago I posted a call for input on my proposal to build a Test Driven Development Framework for Vala and feedback has been slowly trickling in. You can see a summary here which has also been distilled into a Trello board which will become the Product Backlog and Product Roadmap. The list is looking fairly complete so far, so I figure I'm just about ready to close it off and work on a Release Plan. Then I can finally start writing code! Phew.

The requirements gathered so far are pretty much in line with other testing frameworks, but here's a good time to review our Product Vision again and see if we're heading in the right direction. I've highlighted the parts of the statement which correspond to features so we can compare.

For Vala developers who need to test their code, < insert cool tool name > is a powerful testing framework that provides behavioral, functional and unit testing features to help them write great Open Source software. Unlike other testing frameworks, < insert cool tool name > is designed especially for Vala while integrating seamlessly into existing toolchains.

Let's look at that Requirements we've gathered so far and see if these features would meet this vision:

provides behavioral, functional and unit testing features

• Test discovery
• Async tests
• Test Runner
• Support for Gherkin
• Asserts
• Test protected behavior
• Abstract Tests

designed especially for Vala

• Genie support

integrating seamlessly into existing toolchains

• Output TAP
• Compatible with gstester
• CLI and standalone GUI
• PIE binaries
• Integrate with CI tools like Jenkins
• Tests can compile and run without framework installed

So far so good! Of course, this is an Agile project, so this list is not exhaustive or final and we can expect some features to be added and others modified or removed altogether. The important thing is that our features align with our vision. The result of this prioritization process will be the Product Roadmap and the Product Backlog, which will guide sprints and daily development efforts and inform the release schedule. Before we do that though, we need some guidance on how to break these features up into functional areas which will determine how we structure our code base and where to start writing our tests. To do this we need a System Architecture.

The System Architecture and TDD

One of the misconceptions that newcomers to TDD have is that you don't write any code until you've written a test for it. This leaves many people new to the concept scratching their heads about where to start, as even creating a simple command line application requires a certain amount of boilerplate code to be written before you can even start processing the user's input. At this point, a lot of beginners may inadvertently write reams of redundant tests, start reinventing already well tested wheels or just give up on TDD altogether. There are very few times when your code will be executing without any dependencies (if only libc) so you will almost always be coding within an existing framework, if only loosely. Most of these interactions with other frameworks should be encapsulated in integration tests which are developed in parallel with the unit tests. The tests which inform our system design are those which test its unique features. Our System Architecture defines these interactions and boundaries and gives us a basic skeleton upon which to start laying down a codebase. Once this is in place, we can start writing actual tests.

With a project like this we already have the advantage of several examples of prior art, chief amongst these the xUnit architecture. xUnit is a loose framework which includes JUnit and NUnit which stipulates that any implementation has a common architecture as shown in the diagram below:

From this diagram we can already begin to see how we will structure the code. At a minimum we will be creating separate files and tests for Test, TestRunner, TestSuite, TestCase, TestFixture and TestResult. Yep, tests for tests. I may have said this would get interesting... This will give us the minimum we need to set up a toolchain, create a repository and start pushing to it. Hooray, we're about to start writing code! Except that it still doesn't have a name...

What's in a name? That which we call a rose by any other name would smell as sweet.

William Shakespeare

Thanks Bill. I'm still not 100% sold on Valadate, even though it does reflect the Product Vision of being made especially for Vala and that it's not strictly limited to unit testing. Calling it VUnit would reflect its XUnit origins, but it's not like there's any rigid API to conform to. Technically it doesn't matter at this stage of development, but I would like to avoid having to refactor the code later just to change the name. There's still some more work that can done before laying down any code, so I'll let it percolate for a day or two longer before making a firm decision. Now's a good a time as any to speak up if you feel passionately one way or the other.

But at least it's got a logo! Let me know what you think...

The base was designed by misirlou and I added the nice colors. It's meant to symbolize the eponymous asteroid that gives Vala its name.

That's all for now, tune in again soon when I discuss the Roadmap and Backlog as well as how I set up Jenkins CI on a Raspberry Pi.