So earlier this week, Dave and I were published in a lifetracking/streaming article (thanks to sweet friends Brynn and Chris) in the Washington Post called Bytes of Life. We thought our conference call with reporter Monica Hesse went pretty well, but neither of us expected to get the kind of coverage we did for talking about a little statistics app we’re working on that we’ve tentatively called I Did Stuff, which is basically a combination of every good idea we’ve had in the last year.
The premise for I Did Stuff lies in the belief that we’re tracking so many aspects of our life now that computers need to not only make sense of this data for our own use, but also use it to deliver status on demand. One common example I’ve used is that of the “reverse twitter”…basically to combine Google calendar, IM status, and whole host of other data sources into one remotely queryable interface. And anyone can ask this interface “Where is Dave?” and receive a response like “Well he isn’t in front of IM, but he has class in 10 minutes, so he might be in transit. But his phone is on the charger, so he either forgot his phone or is oversleeping.”
Which was a great idea a year ago, and as far as I know still hasn’t been done. But since then, this idea has grown into so much more. Not only do we want to create an AI who can infer status for others, we want it to learn more about us than we know ourselves.
A few months ago I decided to try a little experiment, which was inspired by a conversation with Tantek in which he stated that RSS feeds aren’t even a part of his personal communication stack…everything he needs to know he hears from his social graph via twitter. Frustrated that I had just declared RSS bankruptcy, I wondered if I was spending too much time keeping up with what other people were doing…time that could instead be invested into bringing my own ideas to fruition. Although all blogs other than those of a few close friends were cut out of my daily reads, what came to be known as the TechCrunch Experiment was born. And although initially liberating, I quickly found that my particular social graph wasn’t feeding me enough of the right kind of data items to keep me motivated; after a few months, I found my new idea generation rate (or at least my idea improvement rate) went almost to zero. My creative juices ran dry when operating in an RSS free vacuum.
This makes sense because I then realized I build my Twitter contacts around keeping up with peoples’ lives, and had built my RSS feeds around keeping up with news. So even though I didn’t end up being able to save time, I’d say it’s still pretty good to know what fuels you, right?
For a more recent example (recent = correlated today), I occasionally have a problem with not eating. More along the lines of forgetting to rather than an eating disorder, but we all have our vices. I’ll not eat well for a few days and get lethargic and depressed and wonder wtf is wrong. Now this sounds like a fairly simple correlation, but with everything else going on in life it can be hard to notice if you’re not paying close attention. So I today I purchased a cheap glucose monitor (along with not-so-cheap test strips, jeez) to track this one dimension of how my diet affects my energy and moods. All I’ve learned over the past 24 hours is that I just need to EAT, but seeing concrete evidence of this reinforces the consequence of my actions, plus gives me a scale to rate how well I’m doing.
Something like this could be tracked in the personal unit testing system, but the value in this data is more than just binary passing or failing. Lifetracking, lifestats, personal informatics, lifestreaming 2.0…call it what you will, these are MBO’s for life.
These examples represent only a few possibilities, just imagine how many factors influence your mood and performance! Diet, weight and exercise, weather, what time you went to bed, where you are, who you’re spending time with, the projects you’re working on, your social connections (and growth rate), what blogs you’re reading, the music you’re listening to, what applications you’re using, the conferences you’re going to, how long it is until payday…or if you’re Brynn and Chris, the last time you had sex.
This data is already being recorded, we just need a smarter way to use it.
Which is, as they say, a bigger algorithm.
Came across your blog as a result of that same article. I’m fascinated by something as full-bore data-crunching as the brief mention of “I did stuff” implies. I was wondering if you wanted any sort of assistance? It sounds like a project I’d really like to help with.
Even if you don’t want/need anyone else working on such an application, it sounds fantastic–here’s to hoping you get it up and running!
I love the idea of lifestreaming and “LifeCasting”. The “I did stuff” idea sounds very cool. I will make sure to check back to see what comes of it!
You’re tracking your blood glucose!? Outstanding! Can I shake your hand? Seriously though, glad to know I’m not the only one who unintentionally starves himself while on the Interwebs. I’ve long pined for a PNA (Portable Nanny Device) which would monitor my water, eating, sleep, exercise, and possibly bowel movements as well as my social interactions (how long has it been since I called my mother) and occasionally nag me with reminders to take care of myself. Of course, everyone thinks I’m an idiot when I tell them that I’d find that to be useful, but that’s just because they don’t understand! When your brain is busy thinking big thoughts, there’s no time for bathroom breaks.
On the subject of your RSS bankruptcy; I feel your pain, but urge you to reconsider. I think you’re correct in observing that RSS feeds can serve more than one purpose. They can keep you informed about your friends, they can keep you abreast of current events, and they can alert you to personal information (like when your web server is down, or when someone left a comment on your blog.) I wrote a rather lengthy article on this subject a while back. It’s not RSS’s fault that the tools to process it are still so primitive.
Finally as one who is prospecting the same tech-wilderness of personal data aggregation and management, let me share something that I’ve learned; finding the data and thinking of how an intelligent system *could* deduce information from that data is the easy part. From looking around at the state of the art, you might be tempted to think that the subsequent implementation would be trivial, but that’s just your brain filling in the blanks. Algorithm’s like the ones that Google uses for Google Suggest, or the Bayesian classifiers that filter spam from your mailbox seem smart and flexible, but they are not, they are specialized.
The real challenge to this puzzle is two-fold; 1) figuring out how to process your data in such a way that machine learning algorithms can consume it, and 2) figuring out exactly what metrics you want them to learn about. It is much harder to imagine these things than it is to imagine the purpose to which they will be applied.
Awesome! Loved the article and look forward to helping you test “I did stuff”!
There’s a handful of things I track about myself on a bunch of different websites… and ultimately.. I’d like to pull all those things together in one place… and have a way to automate things. Sure, its not hard to go to a website and manually enter data to a website… but I’d like to see some interface where I can send messages (via IM or twitter or email) to this system and it can be in a format that’s customizable. Like something simple like.. ‘BloodSugar 88’ and it would record that… or ‘Ran 4.7 miles in 55:20’ or.. ‘ate snickers bar’ or ‘woke up’…
Bonus points if it can take actions by posting data values to a form on a website or otherwise integrate with APIs… unless of course all that functionality (graphing, trending, etc.) can be accomplished within “i did stuff”.
Would love to brainstorm or help implement some of this… or even just beta test! =)
How did I miss your commentary on this until now? I’m glad you decided further to describe your “I did stuff” idea. Monica did a great job with the article, but sensationalized it in some ways (leaving out some details) so that a lot of readers came away with a “pop culture” type of impression (I think) and missed the bigger picture.
To me the bigger picture is, as you mention here, that we are quickly accumulating masses of data about ourselves (whether intentionally or not), and we ourselves could actually benefit from reflecting on it and learning about non-obvious (or unintentional) correlations. Jesse is right, though, that it’s easier said than done.
I think, too, that it may be harder than we think to actually track all the important (relevant?) variables. Consider your glucose tracking: you can capture a glucose score when you take a test and loosely correlate it with how you’re feeling, where you are, what you’re doing—but are you also “scoring” those activities and recording them someplace? There could be larger trends underneath the surface that you can only partially observe with this method. (And same goes with all the (merely) 2-3 things I actively track.)
The new FitBit device has the potential to make tracking of daily behaviors a little simpler (*wink wink* Dave)—facilitating the large-scale capture of data, but not the analysis of it. I’m only mentioning this because I read several reviews of Monica’s Bytes of Life piece that assumed we spent our entire lives tracking our actions and not actually living life! While this is obviously not true, it could get to that point if you wanted to discover hidden or unknown correlations about your life and activities that aren’t even on your radar for tracking (which would require n different variables).
Anyway, glad to find like-minded folks out there. Keep me posted on your project! ;)