Visualization Consumerism

Interesting post by Enrico Bertini on the usefulness and 'democratization' of data visualization.  He suggests that we are currently stuck in what he calls 'visualization consumerism' where data visualization is static and prepackaged by experts rather than dynamic and explorable by the general public.

Health and Fitness Metrics

It's been a busy month but I still managed to find some time to devote to my self-tracking investigations.  I decided that I needed a centralized location to store all my personal metric data, and so I created a basic back-end with Google App Engine.  There are plenty of benefits to going with GAE over the traditional LAMP architechure that I've noticed (much easier deployment (with versioning), extremely scalable, free to start, etc.).  Plus, compared to writing your site in PHP, coding in Python is a dream.  The only drawback as far as I can tell is that it's not the same relational database model that we web developers are used to, so there's a bit of learning curve (apparently it works this way for scalability reasons).

Anyway, so far the site is pretty bare bones. It basically just functions as a front-end for the database right now.  The way I imagine the whole tool working is that in addition to the site, the user will have 'data collector' apps, which run on your phone or your PC, and feed data into the site via HTTP POST requests.  Eventually the site will also allow the user to view history, edit logged data, and (most importantly) allow them to chart and analyze the data to learn things about themselves.  But for now the site will just log and serve out data.

I've also started writing some basic data collectors now that the site is semi-functional.  I started with what for me is the most important:  Health and fitness metrics.

Naturally, I looked at weight first.  I wrote an extremely basic Android app for my phone that just takes a decimal number and uploads immediately to the site.  I had never actually written an Android app before and it occurred to me that this one might actually be simple enough that I could do it with Google App Inventor (which, as it turns out, was correct).  So for the last couple weeks I've been weighing myself daily just before my morning shower and recording the result on my phone.  Already I've started noticing a downward trend, which makes me wonder whether maybe there's something to the Hawthorne effect after all

Several weeks work of weight tracking

The second metric that I knew I'd want to track eventually was running.  I was reluctant to add to the millions of smartphone apps that already exist to track your route and record your distance, especially given that I had never written an Android app before (App Inventor does not count!).  But I found that the overhead time in migrating data from these existing apps to my database was too painful and so I wrote it anyway.  It turned out to be surprisingly straightforward. Coding in Java took me back to my university days, and I was right at home in the amazing Eclipse IDE.

The first weeks worth of running data

Finally, I wanted to do something about my eating habits, but I wasn't sure what.  I didn't want to try something crazy like counting calories, since for one thing, I tend to reject the idea that a metric like that is useful at all in the first place, since it's just not possible to be accurate enough for you to really learn anything.  So without any other ideas, I decided that for this one I'd try something kind of experimental. It occurred to me that with the built in camera on the phone, I could easily write an app to record a photo of each meal (and then upload immediately to the site).  I thought that, while this would be pretty much useless for data-mining (unless I wanted to flex some ridiculous image processing muscle) it might add some accountability to my dietary choices.  Maybe I'd be less likely to eat entire bags of Cheetos if I had to take a picture of it and guilt trip over it later.  I decided to try it.

Food diary entries

So now I have several weeks worth of data here, and have stuck with it (despite strange looks from friends/co-workers) but am not totally sold on the idea yet.  One surprising result is that the photos seem to me maybe a bit too personal. If the site ever did go social, I'm not sure I'd want even my friends to know about a) my lackluster cooking skills or b) the 2 AM half-awake snack binge.  But then again, friends would definitely be a great motivator to improve your diet.  The other problem is that I haven't found a really good way to visualize the data yet.  But I'll try to reserve judgement for another couple weeks.

Anyway that's the latest brain-dump of the development on this thing. Progress has been slow, but I've also been getting a lot done. It seems, like most projects, I underestimated how much work was actually involved.  Unfortunately you can only really estimate the amount of work you know needs to be done (rather than all the work that you don't know needs to be done).  Meaning that an honest estimate is more like an approximation of the lower bound if anything.

How to Install RescueTime on Ubuntu 11.04

For anyone else that is interested in using RescueTime with Ubuntu 11.04 Natty Narwhal, the following works for me:

1. Create an account at the RescueTime website
2. Download the RescueTime Linux Uploader (rescuetime-linux-uploader-99.tar.bz2)
3. Extract archive and open terminal at the location
4. Run the following command:
• python setup.py build
5. This should automatically download a file called setuptools-0.6c11-py2.7.egg.  The next command is the following:
• sudo python setup.py install
6. However, this will not work (you will get an error "Unknown distribution option: 'entry_points'").  This is because the version of ez_setup.py that comes with this version of the uploader does not include a reference to the latest version of setuptools.  Download this version and overwrite your local copy of ez_setup.py. Run that command again.  Notice that the warning has disappeared.
7. The other problem is that rescuetime gets installed to /usr/local/bin instead of /usr/bin.  You can fix that problem by making hard links (as suggested by this blog).  Run the following command:
• sudo ln /usr/local/bin/rescue* /usr/bin/
8. Now start rescuetime with the following command:
• rescuetime_linux_uploader
9. Enter your username and password. Rescue time should now be uploading correctly.
10. All that's left now is to get it to start automatically on boot.  Search for "startup" in your unity search bar and click "Startup Applications" (or go to System -> Preferences -> Startup Applications if you aren't using Unity)
11. Click "Add".  Enter "Rescue Time" as the name and "rescuetime_linux_uploader" as the command.
12. Done

Better Living Through Data

Like a lot of people, I've always been interested in self-improvement and being productive (however one decides to define what is productive and what isn't).  However, it wasn't until recently that I realized the potential for self-improvement through data mining.  We live in a time where there is already a ton of data out there about ourselves - data that, if logged and analyzed properly, has the potential to reveal useful things that we could practically apply to our lives.  This blog will contain my thoughts and investigations into these ideas.  In particular, I have a vision for a self-tracking tool that I hope to code and eventually publish for others to benefit from.  So between various ramblings about productivity, self-improvement, and personal metrics, I'll document its progress on this blog.  Hopefully this will add some extra accountability to this goal and give me that last bit of motivation to actually make it there, or failing that, at least share some ideas with some like-minded folks.

The idea of self-metrics has become fairly popular in the last few years, and doesn't take a lot of googling to find communities of other like-minded self-trackers. There are already a lot of tools out there to record and chart data about your health/habits/behaviour.  However, despite my efforts I haven't found an existing solution that covers absolutely everything I would want for such a tool (hence the need for my own).

But first, what am I talking about when I say self-metrics?  Most people already track some number of things about themselves.  Some people just haphazardly jump on the scale every couple weeks.  Weight would be one metric you can use to track health (one of many).  Other people might make an effort to remember how far they can run in a single workout, or how many pushups they are able to do, so that they can try to beat that each time.  These would be metrics for your fitness level.  Even just keeping a basic estimate in your head of your commute time to work (so that you know what time to wake up in the morning) is an example of a metric about your daily habits.  These are examples of what I would call manual metrics - that is, metrics that involve some conscious effort on your part to record the data.

Perhaps even more valuable are automated metrics - that is, metrics that you can record in the background of your day to day life without any real effort on your part.  Some examples of this kind would be the amount of minutes that you use on your cell phone per month (given by your cell phone bill), the amount of money you spend on beer per month (given by your bank statement), the amount of time you spend at work (given by your pay stub), or the amount of emails that you respond to in a given day (given by your email history).  The amount of data that we could track here is actually very large, and as technology progresses it is only going to get larger.  Smart-phones, in particular, contain a wealth of potential here.

We have to be careful, however, to be purposeful in our data tracking.  We don't want to track data purely for the sake of it.  For example, the number of individual grains of rice consumed per month, while vaguely interesting, is not going to be a very valuable piece of data.  Of course, there will be cases where we might not know how useful a given metric will be until we start tracking it, but we should try to keep a practical mindset here.  In particular, when it comes to manual metrics, we want to eliminate as much overhead as possible, because even the smallest amount of overhead would quickly overshadow whatever benefit the metric provides us (and therefore defeat the purpose).  For example, tracking exactly what activity you are doing at any given moment (whether it be cooking, cleaning, going to the bathroom, eating, etc.) would be useful, but the time required to record it would probably outweigh the benefits.

In the coming months, I'll make an effort to ramble some more and document my progress here (as much as is possible working a full-time job). Stay tuned!