In a recent effort to pull data into an Excel spreadsheet for further analysis, I needed more flexibility than a static report URL could provide. The back end could accept parameterized values (e.g. the desired date range) in the report URLs. I just needed to be able to set these values (initially or through user input) from within the spreadsheet, then request the needed data. Seems generally simple and useful enough, so I’ll share what I did.

Sample Report

Ignoring most of the particularities of the reporting back end, say the report in question consists of an SQL query:

The URL to access it is http://initech.com/reports/total_profits. Using the query above, this would return data from the beginning of 2000 to the end of 2010. To pass in values for the start and end date parameters you would use the URL http://initech.com/reports/total_profits?begin_month=2000-03-01&end_month=2000-06-31. This would limit the data returned to the second quarter of 2000.

Use in Excel

So what if you want to use dynamic parameters like these when pulling data into Excel? Let’s assume you have a spreadsheet already pulling data from the static report URL and displaying results. If you run into trouble, check out my earlier article on the subject. Once that’s working, right-click the resultant data table in Excel, select Edit Query, and add the parameter names and values in the format ["FieldName"]:

FieldName in this case can be whatever you want to call the variable for the user’s convenience. This format is recognized by Excel and you’ll be prompted to enter a value. You can enter it yourself of course, or you can select a cell from which to pull the value. You can also specify whether to use the cell for future data refreshes and whether to refresh automatically. These features are key for our intended use. You can set a parameter value in a single cell and have it effect any number of web queries, across various sheets!

Note: This last feature only partially works in Mac Office 2008. A cell’s value can be selected as a parameter value but after updating it you have to manually refresh all related web queries. In Windows Office 2007, updating the specified cell updates all queries that have this property set, which is great for easing administrative load caused by having to update reports!

Date format annoyance

Your backend reporting system might likely expect dates in proper ISO format (YYYY-MM-DD). Excel, in its infinite wisdom, allows “Date” typed fields to be one of a myriad of formats, not one of which is proper ISO. So you need to make your date fields Text type and enter the dates as ISO format yourself.

This article has additional information on the above approach as well as more screenshots of the various steps.

But what’s even more interesting than the fact that we now have the ability to work at such a scale at all is that datasets of this magnitude, and tools to begin to approach understanding them, are becoming available to the (at least technically-minded) community at large. And more and more instances of attempting to push such approaches to problem solving to the general public are appearing as well. I found this talk by Jeffrey Veen on “Designing for Big Data” to be quite thought-provoking. He discusses the novel shifts in the last few decades from data consumption to data production. This, along with the contemporary developments in storage and processing technology, has produced the unique time in which we live: a time in which people are beginning to be able to quickly and easy utilize large quantities of recent, relevant data to make everyday decisions. Sites such as EveryBlock, services like AWS’ Public Datasets, and countless other endeavors are changing the way we learn and live.

So what’s this Many Eyes business? Many Eyes is an IBM project, from the Collaborative User Experience group. I’ll appropriate one of their more poignant expressions as a description of purpose:

We believe that visualizations gain power when multiple people use them to communicate, and that communication gains power when multiple people can visualize and explore information together.

On the site, anyone can upload new datasets (or use the wide range already available; beware, there are plenty of junk ones, but more complete reliable ones as well), and try out any of a number of visualization styles. Once a style is selected and applied, you customize, name, tag, describe and publish your new creation! Each visualization is permanently available, allowing for embedded uses as well. Here’s a bubble chart of the percentage of population penetration in internet use that I made in two minutes:

Or check out this world map showing life expectancies. The graph interactions are quite interesting in themselves. I encourage you to try your hand at creating and sharing new visualizations. Such new forms of information accessibility will continue to provide benefits to more and more people. Not to mention they are quite fun to make!