I made a thing

A while ago I made a prototype of a simple, high-performance keyword search for bugzilla, using python and Redis.  It was pretty promising, able to do all/any queries against the summaries of all open Firefox bugs in a handful of milliseconds.  The bugzilla maintainers decided to use the database’s full-text engine instead, but I was nonetheless pretty pleased with it.

A month or two ago, I started rewriting it it atop node.js and gave it a simple websocket-based interface.  (It took a while, due to The Unpleasantness rather than the difficulty of the task itself.) It was blazingly fast, easily able to keep up with real-time matching of entered keywords. The queries on the server side were finished in a dozen milliseconds or less, and the remainder of my 200 millisecond request budget was spent on data transfer back to the client. I had to limit the returned result set to the most recent 100 items to keep the transfer time down. I will probably look at compression in JS to improve things there. (Some requests take as much as half a second if I get unlucky on the network and result-set size.)

Building the index takes about 20 seconds on its wimpy VM, though retrieving all the summaries from bugzilla takes a fair bit longer. (It currently indexes open bugs in Firefox, Fennec, Core, Toolkit, NSPR and NSS.) I could do that on a timer and have it be current within 10 minutes, but it would be pretty wasteful: the rate of relevant change is on the order of a few dozen each minute, and updating that many summaries is literally microseconds.

Enter Pulse, Christian Legnitto’s mozilla-wide message broker.  Wired up to Pulse, my index is up to date cheaply every 5 minutes, and once we deploy the bugzilla extension, the index will be updated within fractions of a second.

To support non-websocket browsers, which is currently all of them, I switched to using socket.io for the transport.  The flashsocket transport adds about 100 milliseconds to the round-trip, which is unfortunate but here we are.

In all, it’s about 500 lines of JS on the server side, plus another ~150 lines for the scripts that do the loading, and I think it’s a great example of what tools like pulse and the Bugzilla REST API are going to make possible in 2011. It’s also an example of how holy-crikey excellent Node and Redis are together.

(There’s an installation running here, which might randomly break as I hack on it.)

Update, 2 minutes after posting:

23 Jan 00:16:00 - searching for sex 23 Jan 00:16:00 - undefined: search 1 -> 1 in 0/1 ms
Hi, Internet.

engines half, straight ahead

As I mentioned in my overlong post about my bipolar disorder, the treatment plan informed by my new diagnosis is working quite well. Well enough, in fact, that I was able to give a short talk at the amazing CUSEC conference last week. That success was a major milestone for me, and a proof point that I was doing as well as I had hoped.

So this week, I’m going to be working half-time from the office (mornings), as I build a better structure to some parts of my life, and generally rebuild my brain strength. I still have some days with more anxiety than is usual or healthy, I still cry a little easily (like a bit while writing this post), and my stamina for difficult or frustrating work is about half of where I need it to get back to it. But hot damn, I can work again, and it feels fantastic.

I can’t write about my recovery without mentioning the support that’s made it possible. If someone dear to you is suffering with mental illness, know that your help and support — even just words of well wishing — can literally change their life.

hey web-talker, talk web to me

Mozilla is hiring a technical evangelist to help the world get the most out of the web. It’s a position with a scope as broad as the web itself, bringing Mozilla’s keen sense of the web to people learning, building and pushing the envelope.

If you can write code and prose, listen as well as you explain, and you want to spend your days changing the web for billions of people, peep this link.

or bpd2 to its friends

For almost 20 years I thought I “just” had depression: monopolar, occasional major episodes. I was well-known in my circle of friends for crying at weddings and in movies. (There is video of my convocation address at Seneca a couple of years ago that I still don’t have the courage to post — in spite of very flattering requests — during which I blubbered like a toddler with a skinned knee.) I was on anti-depressants, and they took the edge off well enough I guess. I didn’t realize that I was also suffering from a lot of anxiety, because it was my baseline. In 1993 I was off of school for a couple of months, and in 2000 I was off of work for about 2 months. Winter was the worst, Christmas the worst of the winter.

For the last 3-4 years, I thought I had ADHD. A doctor told me so! Sure had the focus and procrastination issues, as anyone who has spent 15 minutes with me can attest.

Here’s the thing about mental health diagnoses: with a few exceptions, you are a bunch of blind guys feeling up an elephant. There aren’t blood tests; the brain scan stuff is still really immature. The diagnoses are really tautological descriptions of symptoms, and if it impairs your life it’s a Condition. Some of the diagnostic criteria are such that, by definition, you can never get better: if you’ve ever had the episodes it describes, you have the condition. 21st century psychiatry, ladies and gentlemen. If you have symptoms that overlap with the area of focus of your doctor, you are pretty much going to draw that card. Other issues are probably just co-morbid, a delightful medical word for “also”, or you might not even mention them if they’re not solicited in the right way.

I don’t mean to demean the work psychiatrists do — they have done some great work on me! — but it’s clear that there are some pretty significant limitations.

So what do you do in the absence of a clear test for something? Well, when it comes to a diagnosis, I care about it to the extent that it directs us towards effective treatment. (Us, almost always — if I’m not working with a doctor, I’m damned sure working with my wife.) And it should explain a meaningful number of symptoms, like back-testing a model of any other system. I like long walks on the beach and evidence-based medicine.

I have the good fortune to have found a doctor in the Bay Area with very broad clinical range, in both diagnosis and treatment. In her opinion, ADHD was a “soft call”, and I am a pretty textbook case of bipolar disorder, probably type 2.

Bipolar disorder is ups and downs, to the point that they interfere with functioning rather than just being part of the normal ebb and flow of life: severe depression and moderate-to-severe mania. It used to be called “manic depressive”, but apparently that name had a stigma, so they started using a phrase with the word “disorder” in it. Well-played. The main difference between type 1 and type 2 is the severity of the mania. I joke that BPD1 is “why isn’t shaver wearing pants?” rather than BPD2′s “why did shaver stay up for 3 nights writing that code?”, but if you are living with BPD1 in that form it is categorically not a joke. On the other hand, if having a mental illness doesn’t let you make jokes like that, why even bother having one? Anyway, since the Type 1 vs. Type 2 distinction can be related to duration of manic episodes, or their severity, or having mixed episodes, I could probably also be classified as Type 1 (I get mixed episodes, and I’ve had week-long hypomanic periods). Most of the difference in treatment seems to relate to how careful to be about triggering a (hypo)manic episode, and we’re not too worried about that, so Type 2 is the guideline of the day.

BPD2, then, is “bad depression with periods of awesome”. Seriously, the hypomanic (mild) episodes are fantastic, and mine are not hall-of-fame episodes. Apparently there are drugs that can make you feel like that, and I’m glad I didn’t know that when I was growing up. Like “Roger” in Ben Horowitz’s story of “the flake”, I had made some great stuff happen in that state. Being treated effectively for the depression part may mean that I give up the episodes of awesome. That sucks, and it’s a bit scary, and if I weren’t badly depressed I would probably not have been willing to start a course of treatment that could mute or eliminate them. You can have both mania and depression at the same time, called a mixed episode, and I’ve had them, including for part of this little adventure. That is some confusing shit right there: you feel driven to DO SOMETHING, but have no actual energy or motivation; fatigued without being able to sleep much; incredibly distracted and irritable; very likely to keep buying games during Steam sales without ever satisfying the novelty craving. The last one may not be in DSM-IV.

BPD2 + maybe an anxiety disorder (“but just living with BPD2 can cause a lot of anxiety, so let’s wait and see”) would explain not only what I’ve lived with but also how it’s been arranged in my life. But the test is in the treatment effectiveness, because boy is that not a vanity label.

On that front, the diagnosis is a home run in overtime for a 6 from the Russian judge. Though holy-shit-this-can-stop-any-time and so forth, and it has been SEVERAL EARTH WEEKS since I could work, my improvement has been the fastest I’ve ever experienced, and the broadest. For the first time since I was…10? I am sometimes waking up without a tight-band-across-the-chest weight of anxiety. I feel like an idiot saying it, but I thought that was pretty much normal for 20 years. I’m at the point now where, during previous severe episodes, I returned completely to work. That proved to be a horrible idea, but also took at least twice as long as is did this time to reach that “I could play hurt” state.

Adjusting to this knowledge about myself hasn’t been easy. As part of the initial treatment, I’m taking a drug known as an “atypical antipsychotic”. I do not like the sound of that phrase AT ALL. Bipolar disorder is one of the 3 major conditions — along with schizophrenia and ADHD — that modern psychiatry has found really needs medication as part of effective management, so I am probably signing up for a lifetime of “mood stabilizers” — another phrase that has a little too much of both A Brave New World and Arkham Asylum for my tastes.

Here’s the thing, though, about BPD2: I know I can live a full life with it, because I have been. This isn’t something I just contracted, it’s something I’ve just discovered, and that knowledge seems a lot like power right now.

Unless I chicken out, I’m going to keep writing about how it goes, in part because it’s oddly cathartic, in part because it will have me actually keeping track of things, and in part because I wish there’d been something like this for me to read when I was a teenager, and then an adult, trying to dodge the traffic of my brain.

CUSEC books

A quickie to list the books I plugged at the CUSEC conference:

making progress

[I wrote most of this a little while ago, and then remembered/found it yesterday, so the timeline is a little weird.]

Several weeks ago, I stopped working because, put simply, I was not able to work. My depression had gone right through its usual winter intensification, and all the way to basically rendering me non-functional. I was unable to muster the energy, or usually even the motivation, to start to get better, and even once I was no longer able to work it took me several days to go through the process of finding a doctor to work with. Merely digging up my phone and telling people that I wasn’t going to work was such effort that I would literally collapse crying after.

For a couple of weeks, I didn’t enjoy. I didn’t enjoy playing with my daughter. I didn’t enjoy eating good food. I didn’t enjoy reading, playing old video games, playing new video games or chatting with my friends. (As my Steam friends will know, I really tried pretty hard with the new video games.) It was easy to give up alcohol as part of my treatment, because I didn’t really take much pleasure in it anyway. There was never a feeling of accomplishment or progress, even when I intellectually knew that I’d done something, or that I was taking the right steps.

I was ashamed: ashamed that I couldn’t function as well as those around me, that I had let my condition progress this far, that I was letting down the people I work with and the people I love. I was afraid that I would feel like that for the rest of my life, though I could make myself not think that if I got lucky on a given day, and I would cling to the remembrance of the thought of a feeling like the sickeningly cliched liferaft of all depression analogies. It hurt so much that I’m crying a bit as I sit on a plane and write this.

Kinda sucked; I can’t in good conscience recommend it.

Since then, I have been fortunate to learn more about myself and how to take care of myself, with the help of an incredible network of support. I’m the luckiest person ever to feel crushing despair, of that I am certain.

Getting better has been hard, it has hurt, and it I know that there are people reading this who are going through the same pain, or a pain like it only in degree. More still have a friend or loved one who suffers this way, or who they fear might. I know this because people I don’t know — and even people I do, for whom it’s even harder — have been brave and kind enough to share their empathetic encouragement. You have done me the most solid of solids, and I hope to never have the opportunity to repay it in kind. I’ll see being able to write proper thank yous as an important milestone in getting better.

I’m good at what I do: technologist, father, friend, husband, manager, competitor, entertainer. When I am at my best, I can be great. I’m going to be that again, and I can feel (feel, a major upgrade from “think”) it starting to happen. Sometimes when I cycle back a bit, or when I juxtapose how little energy I have against how hard it’s been to get back to this state, I get discouraged. But I have a lot to look forward to, and I don’t plan to squander the blind, stupid luck that has gifted me with such a helpful community.

Right now, I measure my improvement by how long I can sustain energy and “be normal”. It’s stupid and probably a bit self-destructive, but I have a lot of practice at it. (You can’t keep me from doing it, so make the most of the energy you get from me, and try not to judge me too harshly when I turn into an exhausted asshole without much notice.)

I have a mental illness that will be a part of me forever. We believe that it’s called bipolar disorder (type 2), and I still find it a bit scary. I’m still learning about it, but I also have a lifetime of experience with it that I’m mining for lessons and strategies and even hope. I have a lot more to say about BPD2, and I hope I’ll have the strength to say it, because it will help me a lot and maybe help some other people a little. I sure don’t have the energy for it right now.

I’m going to get better. I’m going to be fine. Thank you all for helping me do that, and thank you for your patience.

PSA: If you aren’t sure you’re going to get better, please tell someone. Practice saying “I’m depressed, I need help” in the mirror until you can blurt it out to a loved one or a co-worker. If you’re in the Bay Area, and you don’t know how to find a doctor, use mine, or go to a hospital or walk-in. (My doctor is fantastic, and I am going to miss how fantastic she is when I go back to Toronto.) You can heal; you can actually be better than ever.

another step forward for open video on the web

Today, Google announced that it is joining Mozilla and Opera in exclusively supporting open video codecs — to wit, WebM and Theora — in their Chrome browser.

It’s a great move, and one we at Mozilla are obviously glad to see. It’s been a great first 8 months for WebM: multiple browser implementations, hardware support, an independent implementation from ffmpeg, performance improvements, support from lots of transcoding services, and content growth on the web. Organizations like Google, Mozilla, Opera and others who really believe in the importance of unencumbered video on the web are putting their products where our mouths are, and the web is going to be stronger and more awesome for it.

Congratulations and thanks, Google.