Here’s the problem: unit testing can seem tedious. Prohibitively expensive. Tough to justify. Even intimidating! As a result, applications and application components can easily get starved of any form of automated testing.

A colleague let me in on a secret that helps break down these misconceptions. It’s simple, and maybe even obvious, but it’s still a secret worth sharing: a unit is as big as you want it to be. It’s just that once the unit gets big enough, we call it something else: a functional test, or an integration test. This type of testing can be a great strategy if you’re looking to minimize time writing tests yet ensure many high-level functional units are working as expected.

Here’s an example of how I recently used JKI’s VI Tester to develop an integration test for a LabVIEW application I was working on, and turned some low-value “throw-away” code into a permanent, reusable part of my application’s build process.

I received a new requirement that would require significant modification to a stable application. In this project, a .NET FileSystemWatcher monitors a directory where a third-party app creates test result files. This new requirement involved monitoring for an additional file extension, which meant I could no longer rely on the simple filter built-in to the FileSystemWatcher. While working on this modification, two things became evident: I was going to need to write some helper code in order to develop the feature, and I would need a test plan that involved more than just kicking the tires after building.

I started to write some throw-away code to enable development of a regular expression to replace the simple built-in filter. During this process, it hit me: toss this code into a VI Tester Test Case, and the helper code would effectively become my test plan. By making this code part of a Test Case and running those tests automatically at build time, the code became a valuable, functional part of my QA process! (By the way, the winning regex turned out to be “^[^\~].*\.(xls|xlsm)$“)

This VI is the Test Case that is run by VI Tester. It ensures that the application is able to properly monitor the test file directory and filter the filenames of interest.

By using two application functions (File System Watcher and File Call Back) as part of the Test Harness, I ensure functionality of the entire application component.

Functional tests like these help quell misconceptions about writing software tests. Testing does not have to be boring – feel free to whip out some application design patterns to create integration tests! Further, testing does not have to be tedious. Consider functional testing of high-level processes if the application does not warrant unit testing each low-level function.

Download a packaged version of this example below and poke around. There are a few more goodies in there, such as the source of the FileSystemWatcher setup, and a design pattern I dub “The Flaky Delegate” where a .NET callback brokers messages back as LabVIEW User Events.

Download the example as a VI Package. (108 KB, requires the free VI Package Manager Community Edition or higher)

Learn more about VI Tester, JKI’s free unit testing framework for LabVIEW.

A view of VI Tester and the project structure of VIs under test.

This presentation came about after I was working with several other JKI Engineers and it became obvious that many of us were using debugging techniques that others had forgotten or never knew existed. We recorded a live screencast of the session and will upload it once we’ve got it ready, but for those who’d like to review the slides or download the code now, check them out below.

Do you any super-secret debugging techniques of your own?  Share them in the comments below.

Title: Five Clever Debugging Techniques for Every LabVIEW Developer

Abstract: Do you want to be a LabVIEW bug assassin? Do you want to identify, isolate, and fix bugs faster than your coworkers by using clever tricks? Learn several powerful built-in LabVIEW debugging techniques ranging from simple tools like retain wire values to the best LabVIEW debugging feature that no one knows about.

Download Slides & Code: niweek-2011-jki-clever-debugging-techniques.zip (1.0 MB)

Notes:

• Example code requires LabVIEW 2011.
• Example code depends on several free LabVIEW add-on libraries.  To install these libraries, use VI Package Manager 2011 Community Edition to “apply” the included VI Package Configuration file. See Secret #3 in this JKI blog post for more information.
• Example code includes a free trial version of the EasyXML Toolkit for LabVIEW. The full version of EasyXML costs $99 and is a 2011 LabVIEW Add-on of the Year.

LabVIEW Checks for Broken VIs Late in the Build Process

Recently I was working for a JKI customer on a very large piece of LabVIEW software. The development team was trying to optimize the application’s half-day-long build process. One of the problems we faced was that occasionally there were broken VIs in the build but it took hours before the LabVIEW Application Builder returned an error to notify us about them. This feedback was too slow, as often the new build was needed for the next day and starting another build meant several lost hours.

Our solution was to create a Test Case in VI Tester that tests if the application is OK to build (i.e. if any module’s Run Arrow is broken). It provides nearly instant feedback to the development team because it runs as the very first step of the build process.  I’ll show you how we did it.

How to Quickly and Automatically Test Your Build for Broken VIs

How can you create a test case to test broken code, since doing so would cause the test case itself to be broken?

First, create an empty VI Tester test case class. By default, a test case class consists of three VIs: a setUp.vi, a tearDown.vi and a VI template for individual tests. You don’t need to worry about the setUp or tearDown steps in this test case; just use the default templates for these two VIs. Just create the test VI by renaming the template to something that begins with “test.” In this example we will call it testBuildTree.vi.

Second, we need to write the content of the actual unit test. The idea behind this unit test is very simple. We create a VI outside the test case class called BuildTree.vi. On the block diagram of this VI we place all the files defined in the Source Files section of your executable build definition (marked with yellow label in the illustration below).

Wire up all the required inputs of the VIs you dropped on the block diagram to ensure that BuildTree.vi itself is not broken. Don’t worry about whether the data you connect is valid; BuildTree.vi is never actually executed so you don’t need to worry about whether the code would actually do anything if you ran it.

Next open the testBuildTree.vi test from the test case class. Create an empty case structure on the block diagram, and wire a boolean control to the selector input of the case structure as shown below. Drop the just created BuildTree.vi to the True case of the case structure and leave the False case empty. Make sure the default value of the boolean control is false so that the True case of the case structure never executes.

That’s it. Your test case finished. When VI Tester tries to execute the test case, any broken code in BuildTree.vi will cause the test case class to fail to load. VI Tester will simply mark the test case as failed and continue with the next test case.

Verify That It Works

To validate your test case, add the test case class to your project, save the project and open VI Tester from LabVIEW Tools menu (Tools -> VI Tester -> Test VIs…).  VI Tester will automatically find the test case in your project. Execute the test case by pressing the green arrow in the main user interface of the VI Tester. If one of the VIs in your build is broken, the test case will fail and your broken build is properly reported by your unit test.

If you’re a VI Package Manager user, VI Tester 1.1.2 will automatically appear in your package list and you can install or upgrade in just a couple clicks.  If you’re not a VIPM user, you can get started for free with VIPM Community Edition and be up and running with VI Tester in a few short minutes.

The biggest changes in VI Tester 1.1.2 are the following:

• TestSuites can now contain multiple instances of test cases and/or test suites.  This makes reusing tests easier as you can create test properties that get set during the TestSuite initialization step and reuse tests much more efficiently.

• Added support for Test templates. You can create a method called testTemplate.vit and it will never get executed by the test framework, but you can use it as a template to create other real test methods.  We’ve also changed our TestCase template to include a testExample.vit file! (hat tip to Daklu)

• VI Tester is now built using LabVIEW 2009 instead of 8.2.  There were some issues that we just couldn’t fix easily in 8.2 so we decided it was time to upgrade.  This means VI Tester 1.1.2 now requires LabVIEW 2009 or later.

Please download and install VI Tester and keep sending us your feedback; it’s greatly appreciated and really helps improve VI Tester!

Now, go squash some bugs .

You can download & install this package using VI Package Manager.  Just press the “Check the Network for Available Packages” button to refresh your package list, then install the jki_labs_tool_vi_tester-1.1.1.137-1 package.

How can you ensure that important VI Settings are correct for specific VIs, prior to deployment?

That’s easy, just create a unit test (using VI Tester) that validates the required setting, as shown below:

Unit Test for Required Reentrancy

The VI that performs the reentrancy check, passIfReentrant.vi (shown below), is one that you probably haven’t seen before, since it’s not delivered as part of VI Tester.

passIfReentrant.vi

If we look at the block diagram, shown below, we’ll see that it’s not very complicated:

passIfReentrant.vi Block Diagram

This VI simply checks the reentrancy and outputs a descriptive test failure message explaining the details of the failure.  When we run our unit tests, prior to building and deploying our application, VI Tester catches the problem and shows us exactly where to look.

VI Tester showing failure

If you’d like to add reentrancy tests to your LabVIEW application’s unit test suite, you can download passIfReentrant.vi here:

• passifreentrant_lv82.zip

If you have ideas for other common, reusable VI Settings (or other) tests, please share your ideas by leaving us a comment below.

How to get VI Tester

VI Tester is available on the VI Package Network. If you’re already a VI Package Manager user, just click the “Check the VI Package Network for Available Packages” button in the VIPM window. VIPM will notify you that VI Tester is available, and you can simply tell VIPM to download & install it for you.

Check the VI Package Network for Available Packages

If you’re not already a VI Package Manager user, what are you waiting for? VI Package Manager lets you take control of your reusable VIs, and is also the best way to put all the great tools from JKI Labs, OpenG.org, and the whole VI Package Network into your palettes!

For more information about VI Tester, visit jkisoft.com/vi-tester.

• A programmatic API so you can run tests and obtain their results from your own code.

VI Tester's Programmatic API

• Several new toolbar buttons in your LabVIEW Project window so you can access VI Tester’s most common features right from your LabVIEW projects.

VI Tester's Project Toolbar

• The ability to export your test results to a text file and/or print them.

We’re excited by the community’s response to VI Tester.  We hope you find the new features in VI Tester 1.1 as useful as we do!

How to get VI Tester

VI Tester is available on the VI Package Network.  If you’re already a VI Package Manager user, just click the “Check the VI Package Network for Available Packages” button in the VIPM window.  VIPM will notify you that VI Tester is available, and you can simply tell VIPM to download & install it for you.

Check the VI Package Network for Available Packages

If you’re not already a VI Package Manager user, what are you waiting for?  VI Package Manager lets you take control of your reusable VIs, and is also the best way to put all the great tools from JKI Labs, OpenG.org, and the whole VI Package Network into your palettes!

For more information about VI Tester, visit jkisoft.com/vi-tester.

Very soon, you’ll be able to access VI Tester directly from the LabVIEW project toolbar.

By clicking on toolbar buttons you’ll be able to run all your project’s unit tests, as well as add new Test Cases and Test Suites without leaving the project window.

Here’s a short video showing a development version of the VI Tester’s LabVIEW Project Explorer toolbar in action:

All this is made possible by NI’s growing openness to providing LabVIEW developers with ways to plug into and extend LabVIEW.  We are very excited about this trend and are learning more every day about new ways to plug into LabVIEW.  Expect to see better integration with LabVIEW for more JKI tools, such as VIPM, in the near future.

Jim Kring and Omar Mussa will be giving a demo of JKI’s new VI Tester and talking about ways to improve your software quality via unit testing. 

For more details, see the meeting agenda page.

We hope to see you there!

1. Identify a tool that we need for a project, but that doesn’t already exist.
2. Create the tool that we need, so we can work better or more efficiently.
3. Share the tool among our team and use it on multiple projects.
4. Improve the tool based on team/user feedback.
5. Repeat steps 3 and 4 over time.

During this process, we continuously evaluate whether the tool is a candidate for a commercial product or whether it would be more appropriate for a community release (e.g. on JKI Labs).

Now, let’s look more closely at how VI Tester evolved…

Identification:

A few years ago, we identified the need to unit test our code — primarily as a tool for ensuring quality for our JKI Software products like VIPM and EasyXML.  Our customers depend on these products to perform reliably, and they’re too complicated for us to test by hand.  We also wanted to use an xUnit framework because this software testing architecture has proven to be a powerful tool in the software engineering world.  Additionally, xUnit test frameworks have spawned Agile and Test-driven development processes that our company strives to replicate in LabVIEW.  Since xUnit frameworks are object-oriented unit test frameworks, with the release of LVOOP we saw an opportunity to create a powerful tool to make our own lives easier, and our software more robust.

Creation:

We started by reading a lot of unit testing books and articles.  We studied the xUnit architecture and some reference implementations (like PyUnit).  We thought about how the xUnit architecture would best be implemented using LabVIEW’s by-value object-oriented paradigm.  We built a prototype in LVOOP, added a basic GUI, and started using it in projects.

Sharing:

With the basic test designs in place, we had VI Tester working essentially as you see it today.  However, sharing a tool requires hammering it into a form that makes it easy for other developers to use.  We know that there is no better tool than VI Package Manager for LabVIEW tool distribution – so we built VI Tester into a package!  This allowed our whole team to easily use VI Tester on several versions of LabVIEW and across multiple projects.  In fact, we’re even using VI Tester to run unit tests on VI Tester itself, to make sure it works the way we expect it to!  (did we mention the dog food yet?)

Improvement:

Improving VI Tester means incorporating feedback from users to make the tool even better.  As long as a tool remains internal, that feedback is limited to a handful of users.  This has the positive effect of not overwhelming the development team when the product is immature.  However, we really want all of JKI’s tools to make an impact on the entire LabVIEW Ecosystem, so we are constantly looking for ways to help users help us.  In the case of VI Tester, JKI Labs is the best way to let users benefit from VI Tester, and participate in making it better.

Repeating the process:

And now, we arrive at the present day.  In reality, VI Tester has already been through many cycles of sharing / feedback / improvement thanks to our internal team and our private beta testers.  But now, by releasing VI Tester on JKI Labs, we’re making a statement:  we think VI Tester is ready to help you improve the robustness of your software; and if it’s not, we want to know how we need to improve it to make it useful to you.  And in doing this, we want to facilitate the ongoing process of bringing high-quality software engineering practices to the world of LabVIEW development.  Try out VI Tester, join the discussion on our forums, and stay tuned for more exciting blogs posts, examples, templates, and videos that will help you get the most out of VI Tester, and out of LabVIEW.