Saving Test Data In XML Using LabVIEW
by Dan Shangraw, P.E.
Owner, Automated Software Technology

Introduction

Most significant LabVIEW software projects involve acquiring and saving data in some way. After all, what’s the use of all the data we acquire unless we do something with it later? However, in my experience flexible and effective data storage is an afterthought in many projects.  We focus on what sensors to buy, what cool new hardware we get to play with, and which new LabVIEW features we get to use. None of this matters to the client who’s paying us to develop this awesome new measurement system; they care about the DATA!   They need data that solves their measurement problem, and they need that data to be compatible with the analysis tool they are proficient in (Excel, DIAdem, MATLAB, Origin, etc).  They also need the data to be in a format that is well-known, mature, and will still be supported in ten or twenty or more years. A file format that meets all of these requirements is XML.

XML (Extensible Markup Language) is a markup language that defines rules for encoding documents in a format that is both human-readable and machine-readable.  In wide use since 1998, it is governed by an open standard and is supported by virtually every popular analytical program. XML-based formats have even become the default for many programs including Microsoft Office, OpenOffice.org, and Apple’s  iWork. And because XML files are plain ASCII text, your data is future-proof; you’ll still be able to read it 10, 20, or even 50 years from now.

Since this article is about data and LabVIEW, I’ll focus on how to save data acquired by LabVIEW in an XML format. I will also show you how to easily view and analyze that XML data using Microsoft Excel.

The XML Format

In the following example, let’s assume we have a LabVIEW program that monitors temperature and pressure.  We want to save the temperature and pressure data to a file along with some header information.  The header information will be Test Operator, Test Comments, and Sample Rate.

Below is an example of the sample data in an XML format.

<Test Operator="Dan" Comments="Test Passed" Sample_Rate_Hz="10">
   <Data>
     <Temperature_C>1</Temperature_C>
     <Pressure_psi>2</Pressure_psi>
   </Data>
   <Data>
     <Temperature_C>3</Temperature_C>
     <Pressure_psi>4</Pressure_psi>
   </Data>
   <Data>
     <Temperature_C>5</Temperature_C>
     <Pressure_psi>6</Pressure_psi>
   </Data>
 </Test>

If you look closely at the XML above, you can see XML’s simple, self-describing syntax. The first line contains the “root” of the XML structure, in our case <Test>.   This first line also contains attributes of the root: Operator, Comments, and Sample Rate. The next lines describe child elements of the root, in our case data samples which each have a temperature and a pressure element.  The values for an element are between each tag e.g. <Temperature_C>Test1</ Temperature_C >.  A tag is what identifies an element and has the following syntax:

<Name>Value</Name>

Creating XML Files in LabVIEW

The easiest way to create XML files in LabVIEW is to use JKI’s EasyXML add-on for LabVIEW. There are other ways to do the same thing, such as by using LabVIEW’s built-in XML functions; however these built-in functions add data that only pertains to LabVIEW and they generate XML that can’t be easily loaded by tools like Excel.  This defeats the purpose an open file format that can be read with any analysis tool.  EasyXML, by contrast, creates XML data that follows the standard XML format.

EasyXML is also much easier to use than LabVIEW’s native XML functions. All I need to do to generate a properly formatted XML file containing my data is to create a Cluster containing the data I want to save and use EasyXML to turn it into standard XML:

Writing LabVIEW data to disk the easy way...

In this example I am using Easy Generate XML.vi.  This VI converts a LabVIEW Cluster into an XML formatted string.   Then I just save the XML formatted string to a text file. I could make this code even simpler by using Easy Write XML File.vi.

...and the even easier way.

Note: Both these images are VI Snippets. To use this code yourself, just drag the image into a LabVIEW Block Diagram.

The cluster that is wired into Easy Generate XML.vi contains the attributes and acquired data I want to store.  The attributes are in a cluster named “#attributes,” which EasyXML automatically converts into XML attributes.  The data is an array of cluster elements named “Data.”  Each array element contains a single sample for temperature and pressure.

Reading XML Data With Excel

Data is useful only if you can read it with a tool and do something with it. For this example we’ll use Microsoft Excel because almost everybody has Excel, and because Excel makes working with XML really easy. Below is a screen shot of our sample XML file, viewed in Excel.

Importing XML data into Excel.

As you can see, Excel displays the XML data in a very familiar and easy-to-use table. Now we can use Excel’s native functions to analyze the data.

Filtering XML Data With Excel

In the image above, Excel has automatically used our XML tags as column headers .  We can filter data by clicking the column header dropdown as seen below. The built-in filtering tools allow us to easily inspect portions of our data, and we can also use custom filters to build complex drill-downs.

Using Excel's built-in filters with XML data.

Creating a simple numeric filter.

The above 2 screenshots show how I specified a filter to only show data greater than 4.

Our data after applying the filter.

The above screenshot shows what the data looks like filtered.  Our sample file only has 3 datapoints in it, but these same features allow us to inspect vastly larger datasets easily.

Conclusion

Saving data in an XML format can be very powerful because XML is a popular, well-supported, and open standard for data exchange, and what I’ve shown above is just the tip of the iceberg. We’ve seen how easily Excel reads and manipulates XML data, but you can also use many other powerful tools like National Instruments DIAdem.

How do you use XML to store and manipulate your data? Leave a comment and tell us!

If you’re in the Bay Area, don’t miss next Wednesday’s Bay Area LabVIEW User Group meeting!

The Time: Tuesday, February 7th, 2012  5:45 p.m. – 8:30 p.m.
The Place: The Lookout at 605 Macara, Sunnyvale, CA 94085
To Register: Sign up here.

JKI Project Engineer Joe Lee will give a talk called, “Closing the Loop: Generating LabVIEW Data Structures from XML.” He will be discussing how (and why) to use XML with LabVIEW for web application interfacing, configuration management, and flexible data storage. You will learn how to take XML data and start manipulating it right away using EasyXML, and if there’s time Joe will show off a new, unreleased EasyXML feature that analyzes XML code and generates the corresponding LabVIEW cluster automatically.

Joe will be joined at this meeting by NI Field Engineer Logan Herr, and Dmitry Sagatelyan from Arktur Technologies, who will each be sharing presentations on LabVIEW Design Patterns.

So come for the LabVIEW and stay for the food, drinks, and community! We hope to see you there.

JKI's EasyXML Toolkit wins LabVIEW Add-on of the Year award in the VI Library category

Live from NIWeek 2011… we’re delighted to announce that NI has selected JKI’s EasyXML Toolkit as a LabVIEW Add-on of the Year in the VI Library category!

The EasyXML Toolkit for LabVIEW is a simple set of VIs that make generating and parsing XML data in LabVIEW as easy as creating a cluster. It’s NI Certified as Gold Compatible with LabVIEW.

Try it! You can download the free trial directly from within LabVIEW 2011.  Just click on the Find LabVIEW Add-ons… link in the Getting Started window to launch the LabVIEW Tools Network. Search for EasyXML, and click the Install button.  VIPM will automatically download and install EasyXML into LabVIEW, and you’ll find it in your palettes immediately.

Also check out how NI uses the EasyXML toolkit to support the FIRST Robotics championships, in which NI Application Engineer Patrick Corcoran notes that “JKI’s EasyXML made XML easy, to the point that we may have gotten spoiled with it.”

Winners of the 2011 LabVIEW Add-on Awards, surrounded by Jeffs (Meisel on the left, Kodosky on the right)

FRC Team Support System Console

National Instruments engineers Mark Black, Aaron Pena, Patrick Corcoran, and Kyle Smoot are part of NI’s on-the-ground “control system advisors” team for FRC.  They help teams solve technical problems and fix bugs (bugs?! what bugs??) in the robots during the competition.  For the 2011 event they’ve used LabVIEW and JKI’s EasyXML Toolkit to create an FRC Team Support application that allows teams in Saint Louis to use mobile devices to request technical support, and allows the control system advisors to remotely check and handle support tickets.  This means less time spent roaming around looking for people to help, and more time spent helping teams compete!

Read the case study here.



FRC Team Support Mobile Interface



We want to thank you — our users, consulting customers, and the LabVIEW community — for helping make our jobs fun and rewarding. So between now and the end of the year, we’re giving you 20% off any JKI product.

We’re also giving you the chance to help people who really need it. Each time you save 20%, we’ll also donate 10% of the pre-discount price to the Red Cross to provide clean water, medical care, and humanitarian aid to Haiti in the wake of this year’s devastating earthquake and recent cholera outbreak.

Photo courtesy of Flickr user 'mmlolek'

Enter the code JKI-HOLIDAY-2010 at checkout, and we’ll take care of the rest.

Need to purchase or renew a VIPM license? Been dying to get your hands on the EasyXML Toolkit for LabVIEW? Finally ready to simplify your TortoiseSVN workflow? Do it by December 31, 2010 to help the people of Haiti and save a bit for yourself too.

Enjoy your holiday season!

What’s New in EasyXML 2.0?

EasyXML 2.0 includes major performance improvements when parsing XML.  It also fixes several bugs.  See the EasyXML 2.0 Release Notes for full details.

Download EasyXML

You can download & install the demo version of EasyXML using VI Package Manager.  Just press the “Check the Network for Available Packages” button to refresh your package list, and then install the jki_lib_easyxml-2.0-1 package.  The demo version of EasyXML is fully functional, but periodically displays a reminder dialog urging you to upgrade to the paid version.

Purchase EasyXML

If you try EasyXML and like it, you can purchase the full version of EasyXML in our store.

Special Discount for Existing EasyXML Customers

JKI is pleased to offer a free upgrade to EasyXML 2.0 for any customers who purchased EasyXML on or after January 1, 2010.  We also offer a 35% discount on EasyXML 2.0 to all other previous EasyXML customers.  If you are an existing EasyXML customer, you should be receiving an email with instructions on how to redeem these discounts.  If you don’t, please send an email to customer-service@jkisoft.com with your original Order # or the email address you used to place the order, and let us know!