As geeky as it sounds - I love data visualization and for me real-time data is all kinds of gravy. So, when I found out about the Tour de Flex Data Visualization contest over at Greg Wilson's Ramblings I was all over it.

As I poured over the data being captured in Tour de Flex... I was surprised to find that there really wasn't much more there than simple counts of stuff. Wow - that's all I could say was 'wow.'

Flashback...

I worked as an audio engineer to pay my way through graduate school and of all the sage advice I received from lead engineers, this tidbit was the most valuable.

"There is no greater sin than pressing the record button late. Punch-in early and you can always clean it up afterwards... if you miss the first few milliseconds - you'll spend an eternity trying to get it back."

-John Holloman

As I passed from grad school to a career in software development I discovered that the principle applied to information as well. In other words, it's always better to capture too much data and not use all of it than to be constantly scrambling for more. Really, having problems with disk space is a good place to be... the alternative is worrying about product viability and data integrity.

Back in my TDF project... I thought that it would be cool to visualize cross-correlations between data captured in TDF and MXNA posts. That would have been nice, but I couldn't find a solid anchor to make that happen. Who owned these posts and examples? Where are they at on the globe?

As I trudged forward, I found that even simple experiments with historical data were far more painful than it should have been. Getting data for a single 24 hour period was taking 30 - 60 seconds (occasionally more). It was a quick hookup to 'talk' to the service... but using it?

The last straws for me were repeated outages... I finally decided to put together a couple 'count' widgets and call it done. If you're interested in the final product (an AIR app) it's called GeoDASH. It doesn't suck too bad.

So... why this post?

I'm aware that Holly Schinsky and Greg Wilson are going to be teaming up for a session at MAX using Tour de Flex as context. I've read the abstract and their "Real-Time Data Visualization" session reads more like an introduction to Flex with BlazeDS.

"From this session you will learn how to do push-subscribe messaging acting in both a publisher and subscriber role as well as how to take real-time data and visualize it in a Flex Charting component."

In filling out my MAX session wish list... I'm having a hard time feeling like this session would be time well spent. Flex + BlazeDS samples abound on the internets and really the messaging aspect (hooking things up) of any data solution is a trivial part of the development process. As soon as you put together or download / compile even one sample you're done (what you have is repeatable / reusable). The focus then turns to creating a performant and architecturally sound data solution. Oh... it needs to be real-time? Time for the gravy... and three key factors that I am definitely interested in ---

Data Richness..
Real-time data visualization can be awesome... the catch is that it has to be rich in data amount and substantiality for that to happen. That is to say that 'value' in a data centric system is directly proportional to the amount and substantiality of data collected. If you aren't sure where to start - CLOB everything and move on. There will always be time for post-production.

Response Time
You've heard of the term 'critical mass' right? If not, when you discover it you'll probably find system performance and response time beginning to sag as well. For the record - no one cares about how much data you have piled away or how difficult it is to maintain it. What they do care about is response time and they will eat your lunch if they don't get it (< 3-4 seconds is ok but still a little high).

Availability
The more valuable your data is... the greater its up-time needs to be. Also, if data is important enough to consume - it's important enough to put hooks into the service that keep people properly notified about maintenance, planned outages, and what not.

Without these three key ingredients... the 'cool factor' of any dashboard, app, or widget is greatly diminished. Perhaps the best way to explain what's rolling around in my head at this point is to link to the latest edition of Drunk on Software (James Ward/Jon Rose) where they discuss http://www.spatialkey.com/.

I know it's not fair to compare the two... but when you are shelling out big bucks to go to a conference there needs to be a proportional exchange. What people really need to know about real-time data visualization (read: pay money to hear about) is already here for free. If this were a fight... I'd have my $5 on SpatialKey.