Yesterday I posted a quick link (last entry) to one of the epic Billg emails that somehow became evidence in the Microsoft antitrust trial. The mail was sent in January 2003, when I was working in the marketing group that was responsible for Microsoft.com, which was one of the groups implicated in the email about Bill’s being unable to find, download and install the updated version of Windows Movie Maker.
As someone who spent most of his next 18 months at Microsoft working on some of those challenges, here’s how Bill’s experience matched up to problems with the Microsoft customer experience at that time. (Microsoft.com has completely changed by now, almost five years later, so I feel safe in describing the way it was then):
“The first 5 times I used the site it timed out while trying to bring up the download page. Then after an 8 second delay I got it to come up. This site is so slow it is unusable.” I don’t remember the specific issues here, except to note that capacity management was an ongoing challenge for a part of the site that typically saw between 60 and 80 million unique users a month.
“It wasn’t in the top 5 so I expanded the other 45. These 45 names are totally confusing. These names make stuff like: C:Documents and SettingsbillgMy DocumentsMy Pictures seem clear. They are not filtered by the system … and so many of the things are strange.” The Download Center was something of a battleground and the user experience showed it. The thought process was that search would be the primary way to allow people to get targeted downloads and the default experience would just be ordered by download frequency; the only filter was by which country you accessed the site. The top 5/top 50 list that Bill refers to accordingly mixed downloads aimed at consumers, IT pros, developers, and business users without regard for audience or for operating system.
When the web marketing groups that I worked with did research to figure out how to fix this issue and present more targeted downloads, we found that there was no easy way to “fix it.” You couldn’t do it by OS–if an IT pro were logged in from his XP box and searching for server downloads he wouldn’t find them. You couldn’t even do it by cookie, because business users were consumers when they got home.
And the best part? Some execs who read this part thought that the answer was editorial promotion of “featured downloads.” Never mind that 99% of the users who came to Microsoft.com weren’t looking for Movie Maker; if Billg wants to see it in the top 5, let’s jam it into the top 5!!!!
“I tried scoping to Media stuff. Still no moviemaker.” The product groups owned the keywords used to describe their products, and though we had acres of search data to inform them, very few of them mined the search strings to figure out how to categorize their products. Usually the categories were driven by product group, and so “media” would have meant Windows Media–at that time a separate product group and totally disconnected from the Movie Maker team.
“I typed in movie. Nothing. I typed in movie maker. Nothing.” Ah, Search. I spent so long on problems with Microsoft.com Search that it’s not even funny. At this point in time the search engine behind the 4 million pages of content on Microsoft.com was based on the one that came with Commerce Server. Did Commerce Server scale to cover that much content? Did it do well with dynamically generated content like those download pages? Let’s just say there’s a new engine there now.
“So I gave up and sent mail to Amir saying – where is this Moviemaker download? Does it exist? So they told me that using the download page to download something was not something they anticipated.” Heh. This is my favorite point. Sadly it’s not as insane as it sounds. The product groups had control over their own content areas on Microsoft and so they thought that customers just knew to come to the Windows site to start looking for Windows downloads. This is one of the reasons that the Downloads site was such a ghetto; a lot of marketing groups didn’t understand that it was a destination for a lot of users and thus spent no time on it.
“They told me to go to the main page search button and type movie maker (not moviemaker!). I tried that. The site was pathetically slow but after 6 seconds of waiting up it came.” Search again. There was no internal search engine optimization effort, no one (in the program groups) looking at actual search keyword strings, and the search engine wasn’t smart enough to match moviemakerto movie maker. Since the keyword moviemaker didn’t appear on the page, the search didn’t return the content.
“I thought for sure now I would see a button to just go do the download. In fact it is more like a puzzle that you get to solve. It told me to go to Windows Update and do a bunch of incantations.” The product group had chosen to deploy MovieMaker as an optional download through Windows Update, rather than as a regular software download. Why? Well, the Windows product group had more control over WU than the downloads area. Plus, apparently they thought no one would ever want to download it. How many times do you look for optional downloads through Microsoft Update? Yeah, me either. And from this point the story becomes the familiar one of the nightmare of WU.
It’s really no wonder everyone hated Microsoft at this point. The web experience really showed no understanding of how users actually used the site and what they were trying to do.
So what would the right answer have been? Some of the steps that were taken right away were a dedicated focus on improving Microsoft.com search by providing more scalable indexing and tuning and much better search algorithms. (Unfortunately the guy who headed up the part about “better algorithms” famously was sniped away by Google.) There was a better editorial focus across the entire site starting around this time, based on user behavior data, to improve the user experience. There was significant improvement of the internal BI tooling to help us better understand what people were trying to do on the site (I worked on this part).
I wish I could say that the product groups started working together more closely to figure out an integrated user experience. I don’t know that I can give a fair perspective on what this part of Microsoft’s culture now, since I left in July 2004. But at the time this was the big drawback of Microsoft’s legendary empowerment of their product teams; all the incentives were there for individual product marketers to do everything they could for their particular product or product segment without considering how it played with the rest of what Microsoft did. While the Microsoft.com team that I worked on had this as its charter, we didn’t have the power to change things or override the product groups. In fact, Billg’s email and others like it were critical to Microsoft’s success because there were so few other mechanisms that considered the customer experience as a whole–and had the power to change it.
