Life on the hedonic treadmill

There’s a theory in psychology called the “hedonic treadmill” which says that most people have a happiness “set point” that they return to pretty much regardless of what happens to them. Things that you might expect to make someone very happy (winning the lottery) or very sad (losing a limb) actually tend to not have a dramatic long term effect on a person’s happiness. In other words, people are pretty resilient. Or, put another way, we’re forgetful. We think how how happy we would be if we got a big raise and how much easier our lives would be with a bit more money in our pocket. And we get the raise and some things do get easier. But pretty quickly we forget how life was before and how happy we are supposed to be now. The new becomes the status quo, we find a new set of things to worry about, and we end up about as happy as we were before.

However most of us don’t get raises or experience any other big life changes often enough to get a real sense of the power of the hedonic treadmill. To get a dramatic demonstration of this phenomenon in action, all you need to do is spend two years of your life working on a book, have it published, and then obsess over the hourly updates to its Amazon Sales Rank. (The Sales Rank is the number Amazon attaches to every book it sells. Better Sales Ranks mean more sales and—more important—a book’s Sales Rank is the only readily accessible, frequently updated data an author has about how a book is selling.)

You finish your book and start paying some attention to its Sales Rank. It’s still in the high 100,000s, or even the millions. Whatever. No one but your parents has even heard of your book yet, let alone bought it. Then you get some word of mouth going on the web which gets you a few pre-orders and your Sales Rank spikes up a bit, maybe into the 2,000s. Woohoo! Party!

Then it starts drifting back down. Oh well, the book’s not even out yet, no big deal. But you dream of maybe one day breaking into the top-1000. That sounds pretty cool. Imagine, one of the top-1000 books on Amazon! They sell essentially every book in print; what’s that ten million books or so? Top-1000 would put you in the top 0.1% of all books. That’d be awesome.

Then some early reviews hit the web. Maybe they get some play on the social networking sites. Wow! Sales Rank goes shooting up—past 1,000, into the 500s. Yippee! And the book isn’t even out yet! Who knows how it’ll do once it’s actually released. Then, once again, the rank starts drifting back down. You feel a bit sad when it starts dropping but pretty soon the days of a 500s-level rank feel like a bit of a dream; anything better than 10,000 is still quite respectable. The decline continues and pretty soon it’s closer to 100,000 than 10,000, but no matter. Hitting the 500s is pretty darn good and the book still isn’t even out yet.

Then a review on a very prominent web site sends the rank rocketing back up. In a few hours it goes from greater than 100,000 to the 500s. And keeps going! Smashes the old high-water mark! Holy cannoli, it’s in the top-200! #178! Wow, that feels awesome. You never thought this could happen. And surely the book won’t have its best rank ever before it’s even out. Maybe it’s not crazy to dream of getting into the top-100, even if just for an hour.

Then the downward drift starts again. But not too fast. It’s hanging out safely in the top-2,000 for a while before dipping down into the 3,000s. And it rallies occasionally into the 1,500 to 2,000 range. Really, if it keeps on like this, this is great. An average sustained Sales Rank better than 10,000 is pretty darn respectable. And that #178 is something you can tell your grand-kids about.

Then the book is released. Whether because of a burst of word of mouth or the publisher’s PR actually working or because there are just a lot of people who wait for the book to come out before ordering, boom! the rank is moving up again. Quick jump back into the top-1,000. 800s. 200s again! Whoa! #173. Beat the old mark! And back down. That may be it. But really, nice that that first foray into the top-200 wasn’t a once-in-a-lifetime event. That’s pretty good. Now, let’s see how long it hangs out in the top-1000.

A few days, it turns out. Life in the top-1000 is pretty nice. Feels like this book might actually be a bit of a success. It would have been nice to crack the top-100 but this is good. Whoops, out of the top-1,000. Oh well. Still a long way to go before it falls out of the top-10,000. And look, it’s rallying again. 600s, 500s, 300s, 200s. Nice! Ah, #239 and then dropping again. Anyway, at least you’re back in the top-1,000 for a while. A dip out of the top-1000 then another rally back to #551.

Then much like before. Drifts down. It takes a couple days to fall out of the top-1,000 and then bounces around a bit between 1,000 and 1,500. Maybe the glory days of #173 and #178 are past but if the book just keeps steadily selling, that’s what’s important.

Then, one night before you go to bed, you see the book has been mentioned on another prominent web site. You wonder if that could possibly pop it back up into the top-200 again. That’d be nice. Maybe, just maybe, you still have a chance to crack the top-100. Something to dream about as you go to sleep.

Get up in the morning. Coffee. Check the ranks your computer has collected overnight. Holy smokes. It cracked the top-100 right after you went to bed! And it’s still there! At #22! In fact it’s been at #22 for three hours. Holy shit! Quick, email your folks! Wait, wait, new rank coming in: #16! Okay, this day is clearly going to be a total loss—you’re not going to be able to do anything until it starts going back down and you can relax. Sales Ranks are supposed to be updated every hour but that doesn’t mean they necessarily change every hour. So, a couple hours at #16. Wow—it’s holding its ground. Then, wait, wait, wait—#9. And #9 again the next hour! #8! #7! You have the #7 ranked book on all of Amazon! Only five books between you and Dan Brown’s new novel.

Finally it drops back down to #8 and hangs out there for a few hours. Then #9. Then #10. Hey, at least you’re still in the top-10. Finally you drop out of that. But you’re still in the top-25 which puts you on the first page of Amazon best sellers. So that’s cool. Anyway, you spent a day and a half in the top-10. A couple days later you drop out of the top-25 and the next day out of the top-50. But what the heck, you made it into the top-10. Who ever figured on that? At the moment you’re still in the top 100, hanging on in the 90s. You know it’ll drift back down. Will it hang out in the top-1000 for a while. Probably. Will you be sad when, inevitably, it falls out of top-1000. Probably not. Life on the hedonic treadmill.

Advertisement

Duct tape context: A tale of two rewrites

Last Thursday Joel Spolsky posted an article on his blog, The Duct Tape Programmer, based on my interview with Jamie Zawinski in Coders at Work. It—as Joel’s posts often do—sparked quite a bit of commentary on the programming web, eliciting responses from Uncle Bob Martin and Tim Bray and hundreds of comments on sites like the programming reddit and hackernews.

It was probably fortunate for me that Spolsky used the title he did—praising someone while calling them a “duct-tape programmer” is provocative and the provocation probably helped drive interest which, no doubt, led to a few people buying the book. So, that’s awesome. Thanks, Joel!

On the other hand, being provocative sometimes leaves little room for nuance. Since I dragged Zawinski into this by asking him to be in the book, I thought maybe I could try to unpack some of the context for folks who haven’t yet read the interview.

The first thing to remember is that the first version of Netscape was written in 1994. When Zawinski talks about the choice to not use C++ and the problems with templates, he’s not talking about C++ and templates in 2009; he’s talking about 15 years ago, four years before the language was officially standardized and a time when different compilers didn’t necessarily implement all the nooks and crannies of the language the same way. And Netscape had to run on Windows, Unix, and Mac, so reliable portability was especially important.

The second thing to keep in mind is that the Netscape team in 1994 was under tremendous, if self-imposed, time pressure. I asked Zawinski how it was that the original Netscape team, many of whom had worked on Mosaic at the NSCA and were thus getting a second crack at the same problem, avoided falling into second-system syndrome:

Zawinski: We were so focused on deadline it was like religion. We were shipping a finished product in six months or we were going to die trying.

Seibel: How did you come up with that deadline?

Zawinski: Well, we looked around at the rest of the world and decided, if we’re not done in six months, someone’s going to beat us to it so we’re going to be done in six months.

Could they have made a different trade-off in the old “fast, good, cheap—pick two” space? Possibly. But history certainly testifies that they produced a product quickly enough that no one beat them to the punch and good enough that they changed the history of the web.

But wait! Didn’t Netscape eventually collapse under the burden of accumulated cruft? Didn’t the Mozilla project have to do a massive, ground-up rewrite because the old code base was so covered in duct tape that it was impossible to work with? Doesn’t that completely disprove Spolsky’s point?

Not clear. I haven’t done an extensive investigation of the history of Netscape’s code but I do have Zawinski’s version of events and some corroboration from Brendan Eich, who was also there at the time.

Certainly they paid a price for their rapid development:

Seibel: After this relentless pace, at some point that has to start to catch up with you in terms of the quality of the code. How did you guys deal with that?

Zawinski: Well, the way we dealt with that was badly. There’s never a time to start over and rewrite it. And it’s never a good idea to start over and rewrite it.

Yet we all know that Netscape famously was rewritten after it was spun out into the Mozilla project. So case closed, right? Not quite. What many people don’t know is that—at least according to Zawinski—the Mozilla rewrite was the second rewrite. Here’s Zawinski’s version, from the part of his Coders interview where we were talking about his work with Terry Weissman on the mail reader that was added to Netscape in version 2.0:

Zawinski: So basically [Netscape] acquired this company, Collabra, and hired this whole management structure above me and Terry. Collabra has a product that they had shipped that was similar to what we had done in a lot of ways except it was Windows-only and it had utterly failed in the marketplace.

Then they won the start-up lottery and they got acquired by Netscape. And, basically, Netscape turned over the reins of the company to this company. So rather than just taking over the mail reader they ended up taking over the entire client division. Terry and I had been working on Netscape 2.1 when the Collabra acquisition happened and then the rewrite started. Then clearly their Netscape 3.0 was going to be extremely late and our 2.1 turned into 3.0 because it was time to ship something and we needed it to be a major version.

So the 3.0 that they had begun working on became 4.0 which, as you know, is one of the biggest software disasters there has ever been. It basically killed the company. It took a long time to die, but that was it: the rewrite helmed by this company we’d acquired, who’d never accomplished much of anything, who disregarded all of our work and all of our success, went straight into second-system syndrome and brought us down.

They thought just by virtue of being here, they were bound for glory doing it their way. But when they were doing it their way, at their company, they failed. So when the people who had been successful said to them, “Look, really, don’t use C++; don’t use threads,” they said, “What are you talking about? You don’t know anything.”

Well, it was decisions like not using C++ and not using threads that made us ship the product on time. The other big thing was we always shipped all platforms simultaneously; that was another thing they thought was just stupid. “Oh, 90 percent of people are using Windows, so we’ll focus on the Windows side of things and then we’ll port it later.” Which is what many other failed companies have done. If you’re trying to ship a cross-platform product, history really shows that’s how you don’t do it. If you want it to really be cross-platform, you have to do them simultaneously. The porting thing results in a crappy product on the second platform.

Seibel: Was the 4.0 rewrite from scratch?

Zawinski: They didn’t start from scratch with a blank disk but they eventually replaced every line of code. And they used C++ from the beginning. Which I fought against so hard and, dammit, I was right. It bloated everything; it introduced all these compatibility problems because when you’re programming C++ no one can ever agree on which ten percent of the language is safe to use. There’s going to be one guy who decides, “I have to use templates.” And then you discover that there are no two compilers that implement templates the same way.

And when your background, your entire background, is writing code where multiplatform means both Windows 3.1 and Windows 95, you have no concept how big a deal that is. So it made the Unix side of things—which thankfully was no longer my problem—a disaster. It made the Mac side of things a disaster. It meant it was no longer possible to ship on low-end Windows boxes like Win16. We had to start cutting platforms out. Maybe it was time to do that, but it was a bad reason. It was unnecessary.

Brendan Eich, who was also at Netscape at this time, confirms at least part of Zawinski’s account:

There was an imperative from Netscape to make the acquisition that waved the Design Patterns book around [i.e. Collabra] feel like they were winners by using their new rendering engine, which was like My First Object-Oriented Rendering Engine. From a high level it sounded good; it used C++ and design patterns. But it had a lot of problems.

He does, however, go on to say:

But the second reason we did the big rewrite—I was in mozilla.org and I really was kind of pissed at Netscape, like Jamie, who was getting ready to quit. I thought, you know, we need to open up homesteading space to new contributors. We can’t do it with this old hairball of student code from 1994. Or my fine Unix kernel-style interpreter code.

So Zawinski and Eich differ a bit on the extent to which the Collabra folks had completely replaced the old code in 4.0. If Zawinski’s recollection is accurate and the 4.0 rewrite had really replaced all the old code, then the fact the Mozilla project felt compelled to do a big rewrite is only evidence of what a mess the first rewrite had made of things and says nothing about the quality of the pre-4.0 code. Or maybe they hadn’t quite replaced everything but had still messed things up enough that it was easier to start over. Or maybe it really was the legacy of the 1994 code—too much duct tape—that was the real problem. Ironically, if Zawinski had had his way it’d be much easier to know what really happened—he tells me in a recent email that he tried, in 1998, to get Netscape to open source both the 4.0 and 3.0 code bases but they wouldn’t go for it.

Enough women in Coders at Work?

When I started work on my book Coders at Work, the first thing I had to do was come up with a list of people I wanted to interview. I came up with a lot of names myself and then put up some web pages where people could suggest names and sort them in various ways in order to show me which folks they were most interested in seeing interviewed. Thanks to some front-page play on the programming Reddit, I received quite a few suggestions and lots of people tried their hand at generating a list of fifteen or sixteen subjects.

Pretty soon the issue arose of whether I would interview any women and, if so, how many? How did it arise? Well, my mom, for one, called me to ask. And a friend pointed me to a blog post by Shelley Powers on burningbird.net criticizing the at-that-time recently released book Beautiful Code, for having only one woman among the thirty-eight authors of its thirty-three essays. (Powers’s post, unfortunately, is no longer online.)

In the end, I ended up interviewing one woman, Fran Allen, and fourteen men. Is one enough? I don’t know. Here are a few things I do know:

The issue of sexism, in the field and in the world at large, is a huge ball of worms. Why are there relatively few women programmers? Is it because all male programmers are irredeemably sexist? Because there are sex linked characteristics that make it more likely that a randomly chosen man will turn out to be a good programmer than that a randomly chosen woman will? Because girls disproportionally drop off the science and math track way back in grammar school? Or because once the field became, for whatever reason, largely male, it became less appealing for women to enter it? Take your pick. Or make up your own reason.

And does it even matter that programmers are disproportionally men? If you read the comments on sites like the programming Reddit whenever the issue comes up, there’s a strong contingent of folks who claim that the field is already an essentially perfect meritocracy and therefore if there aren’t many women programmers it’s simply because women are less interested in being programmers or are somehow less suited to it – no big deal. Obviously if that’s your point of view, you’re not going to be worried about whether there are any women in a book like Coders. However …

I was actively interested in having women in the book. I do believe there’s enough sexism left, both in the world and in the field, that a girl who finds computers fascinating and who wants to be a programmer when she grows up will face obstacles that a similarly interested and talented boy would not. One of those obstacles, I believe, is the lack of women role models. Yeah, yeah, I can hear the “perfect meritocracy” advocates now – why should a woman be a better role model than a man for an aspiring girl programmer when we’re talking about something as purely logical and rational as computer programming. They can ask that if they want. My experience is that people, starting from when we are kids, do notice gender and that the path of least resistance is to do what other people of your gender are doing. So if you’re a girl interested in programming and you see very few other girls and women programming, that’s going to be a obstacle for you to overcome.

Given that, I didn’t want to write a book that was going to be another vector for the implicit message that all programmers are men. But …

A policy of diversity for diversity’s sake is tricky to implement. Assuming you decide you want to achieve a certain level of gender diversity, your problems are just starting. No matter who I selected, man or woman, for a book like Coders, some people are going to ask, “Did they really deserve to be included in that group?” Fair enough. But if people answer that question, “Oh, he just got in because they wanted a Java programmer,” that’s unlikely to cause them to think badly of all Java programmers. But if the answer is, “She was just included because she’s a woman,” it may cause a classic “affirmative action” backlash. Plus, as either Shelly Powers or one of the commenters on her blog pointed out, it can be insulting for a woman to be invited to do something “just because” she’s a woman.

So how did it happen that I ended up with only one of my fifteen subjects being a woman?

At the end of my name gathering, which included posting a comment on the burningbird.net article asking for suggestions of names of women coders and doing my own search for likely women candidates, I had 284 names, of which ten were women. (Or nine if your brand of feminism and/or chauvinism doesn’t accept transgendered people identifying as women as “real” women.) That’s not many: 3.5%. On the other hand, it’s better than the percentage of female Beautiful Code authors (2.6%) and Turing Award winners at that time (2%). (With Barbara Liskov winning the Turing in 2008, that percentage is now 3.6%.)

So if I were to assume that my list of 284 was a good sampling of the kind of programmers I wanted to interview, then in a book of fifteen interviews, I would expect to have .53 women. Since I knew for sure I wanted at least one woman – and since it is certainly possible that despite my best efforts, my list of 284 is not an unbiased sample – I rounded up rather than down and chose Fran Allen. And I’m glad I did. Anyone who wants to claim Allen got in just because she’s a woman can come talk to me after they’ve won the Turing Award. And she provided another important bit of diversity, representing IBM from the heyday of Big Blue.

Do I have any regrets? In retrospect, I wish I had tried to interview Amy Fowler, one of the lead developers of Java’s Swing GUI toolkit. I certainly seriously considered her and she would have added not only a younger woman’s perspective to the mix but also a GUI programmer’s point of view, both of which could have made it an even better book.

Did my own sexism ever lead me astray? Possibly. Early on, Alan Kay recommended that I interview L Peter Deutsch and Dan Ingalls and I signed them up. But what about Adele Goldberg, who was also part of Kay’s group at Xerox PARC? Did some bit of sexism lurking in my heart convince me that she wasn’t really a great hacker like Deutsch and Ingalls? Or was it really, as I told myself, that I just didn’t want to overload the book with Smalltalkers? I really don’t know. I’d like to think it was purely the latter but have to admit it’s possible the former played a part too.

At any rate, I’m pleased with how the book came out and glad that I had a chance to interview Fran Allen, whose historical perspective, technical opinions, and comments on being a woman in the field were all equally interesting and valuable. I hope that if my daughter – to whom the book is dedicated – decides she wants to be a programmer when she grows up, she will find something to inspire her in Allen’s chapter and in all the other chapters of the book.

Not that Skef Wholey!

Whoops! Turns out that Jamie Zawinski suffered a slight brain glitch during our Coders at Work interview when recounting his time as a high-schooler working in Scott Fahlman’s CMU AI Lab. He said:

But there were some really entertaining characters in that group. Like the guy who was sort of our manager—the one keeping an eye on us—Skef Wholey, was this giant blond-haired, barbarian-looking guy. Very intimidating-looking. And he didn’t talk much. I remember a lot of times I’d be sitting there—it was kind of an open-plan cubicle kind of thing—working, doing something, writing some Lisp program. And he’d come shuffling in with his ceramic mug of beer, bare feet, and he’d just stand behind me. I’d say hi. And he’d grunt or say nothing. He’d just stand there watching me type. At some point I’d do something and he’d go, “Ptthh, wrong!” and he’d walk away. So that was kind of getting thrown in the deep end. It was like the Zen approach—the master hit me with a stick, now I must meditate.

However, as Jamie emailed to tell me yesterday, and Skef Wholey emailed me about today, Skef Wholey was not the giant, blond-haired bare-footed hacker who supervised a young Zawinski. Skef, it turns out, is 5’8” and has dark brown hair. Jamie’s giant supervisor was another guy named Rob MacLachlan.

Skef also tells me that Rob did carry a ceramic beer mug but as far as Skef recalls it held coffee, not beer. Skef also described Rob as “substantially more articulate, particularly in matters of coding and in Computer Science in general” than someone who’d just say “Ptthh, wrong!” and walk away. To be fair, Jamie did say, elsewhere in the interview, about Skef/Rob, “Well, he wasn’t completely a cave man. He would actually tell me things, too.”

My apologies to Skef for the case of mistaken identity.

Coders at Work on Kindle?

A number of people have emailed me to ask whether Coders at Work will be available on the Kindle. Short answer: Yes.

Longer answer: for reasons I don’t really understand, it may be a while. My publisher has, they assure me, sent the necessary files to Amazon and now Amazon has to do something – not clear to me what – before the Kindle edition is actually available. However that something can apparently take months. My editor was optimistic that Coders was selling well enough that it might be more like one month than the six months that some titles have spent in the Amazon queue but there’s no way to know.

I really don’t understand what Amazon could be possibly doing that could take 1/4 the time it took me to actually write the book but there it is. If you’re eager to get Coders on your Kindle your best bet is, I guess, to click the “I’d like to read this book on Kindle” link on the Amazon Coders at Work page. Or maybe email Amazon directly since the “I’d like to read this book on my Kindle” is ostensibly for letting the publisher know you want it and the publisher, in this case, is already is happy to oblige.

Yet more Coders at Work reviews

Since I last did a rollup, Coders at Work has received three more reviews on the web, bringing the grand total to nine:

• Brian Carper, briancarper.net
• David Thole, The Dark Trumpet
• Vladimir Sedach, Slashdot
• Chris Hartjes, @TheKeyboard
• Marc Hedlund, O’Reilly’s Radar
• Amit Shaw, Teleported Bits
• Tobias Svensson, return 42;
• Prakash Swaminathan, CloudKnow
• Joseph F. Miklojcik III, jfm3> _

The reviews have continued to be quite positive and the Slashdot review turned out to be quite a coup, shooting Coders‘s Amazon Sales rank, for one glorious hour, up to #176 across all books. While perhaps nothing can complete with the Slashdot Effect for driving an instantaneous spike in interest, I am hoping that it will help to actually get the book into peoples’ hands, as should happen later this week.

Another person who got a sneak peek at the book, Andy Mulholland, the Chief Technology Officer at Capgemini, an $8-billion global consulting firm, sent me this review:

Just had a “sneak peek” at Coders at Work, a book by Peter Seibel due out in September that features interviews with 15 of the industry’s most important programmers.

Through in-depth interviews with the likes of Peter Norvig, formerly VP of Search at Google and now head of Google Labs, and Brad Fitzpatrick, founder of Live Journal, as well as elder statesmen such as Donald Knuth and Ken Thompson, Peter manages to capture the thinking behind these coders’ approach to their work. The book documents an older generation of software – much of it still in use – that coexists with today’s new and different types of applications. It’s as though you have multiple generations of cars on the road at the same time and everything is in flux – the cars themselves, the quality of the roads, the skill of the drivers, and the very purpose for which cars are used.

This book can help programmers sort through all this complexity. It would be a good read for all those studying software today because it provides a fundamental understanding of the world they are about to enter. I highly recommend it.

I guess if everyone–from Slashdot readers to someone advising the highest levels of big-company corporate IT–likes Coders, it’ll probably do okay.