Geeks Over Troubled Water
Gates as CES. Microsoft software tanking during his presentation. My ample forehead still is red from the "D'oh!" slapping it got as I watched each embarrassing screw-up. My inner William Shatner is just crying out in anguish...
What. Does it take. To have a. Demo. That actually works AND. Doesn't. crash?
Oy. Where's Microsoft's Montgomery Scott when you truly need him?
I don't know if I'm more embarrassed by the crashes and hangs and crap-outs or over the "yuk-yuk. eh." non-surprised reactions to yet another Microsoft low-quality bit-parade. Are we doing this on purpose to help increase Apple's share?
How about a new quality standard: if your feature crashes during a Microsoft demo, you're fired. Dev. PM. And Test. You are stripped, shaved, have an "L" emblazed on your forehead, and shoved down a gauntlet of angry shareholder-employees who soundly spank you right into Lake Bill to swim across where you can dry off with the provided moth-eaten scratchy wool blankets.
Without consequences for failure, Microsoft product development see crap accepted as the very public norm and continue creating product that meet those expectations. How is it for all this Engineering Excellence that's been inflicted upon us we still achieve this mere modicum of mediocrity?
I'd expect BillG (or any embarrassed executive) to storm back to Redmond riding a roiling swirl of fire and brimstone and start firing such underperformers. And just when such just firings are revealed, you'd bet some folks will pull their fingers back from the keyboard and whisper "Holy Shit!" And rather than checking in their code to let testing find the bugs that round out the feature, they'll decide it's time to do a bit more of their own testing and write a few more automation and unit tests. And to dogfood a bit more intensely.
If pride is absent, I'll take fear.
17 comments:
Haha, that's funny, firing the test or dev team. From my time at Microsoft, I think there is a pretty decent chance he hit a KNOWN problem that some PM somewhere decided to punt. In fact, I'd laugh my ass off if there is a witchhunt and some test guy cites a RAID bug and says, hey, I filed it, and the PM didn't think it was important enough to fix for this release.
Just make sure you fire the right person. Perhaps you should fire the person who set up the presentation.
As I understand, a blue screen occurred in one demo, which typically indicates a hardware driver issue. Fire the person who is in charge of hardware acquisitions. Fire the device driver developers.
haacked
Do you do demos? Sometimes they go wrong, even really important ones. You do as much as you can to ensure they don't but sometimes that doesn't help.
When you are talking about an operating system you are standing on top of a towering abstraction and the foundations (hardware/drivers) are anything but fixed and stable. Its bloody amazing this stuff works at all if you ask me.
Mitch Denny
http://notgartner.com
You forgot to add smallpox to the "moth-eaten scratchy wool
blankets."
"It's, it's... yeah, the driver. Yes. The driver caused it! The driver. Yeah, dat's da ticket!"
Another media executive at CES was wishing something could pull all of the complexity of device inter-op together. A Microsoft person present pointed out it already existed at it was called "Windows." To which the media executive counter-pointed that consumers don't want that blue screen thingy showing up when all they want to do is watch videos.
It's one thing to have a demo tank during an internal tech-talk. You should still be embarrassed. It's another to have it tank during a high-profile global event. If you can't get a demo to run straight for that, you need to find a new business. RR.
Are you talking about firing the team that didn't test Media Center with that exact hardare configuration including a room full of people, many with IR devices? Hmmm. Here's the test case: room with IR interference causing failure. That sounds like it's by design. Too bad nobody thought of that when they were setting up the demo.
Or are you talking about hiring the people that supplied a buggy debug version of an XBox game that hit an assert? It's not released software. It seems likely enough that it was either a new bug or a known bug, but the most recent/stable build of the game available. Again - who you gonna fire? Who is really at fault for showing something that wasn't ready for prime time?
Why not fire Bill? He was the one who tried to run the demos. It doesn't really matter as long as MSFT fires someone, right?
Oh - and dogfooding WHAT exactly would have caught these problems? Use your XBox as a dev box? Use your Media Center as a dev box but only by using the remote control and only somewhere that you'll get IR interference?
I usually tend to agree with you. Or at least I often would like to agree with you. I think you're being a little harsh this time.
Read this for the real explanation of what happened: http://blog.seanalexander.com/default,date,2005-01-07.aspx
Sounds like it was the people who designed the demo who should be fired. I run the product in question all the time (although not with the USB extenders) with no problem.
The problem was with IR interference. The remote couldn't reach the unit, even with the extender on the stage.
Oh, Conan was hilarious.
The problem isn't the failure per se but MSFT's reputation for being buggy which such failures reinforce. Had it been Apple or someone else, many folks would have given them a pass. Instead, it made front page headlines everywhere. In reading through what happened, the team responsible seemed to have taken reasonable precautions and as a result, I'm not sure firing them is the solution. But it does seem that MSFT's technology releases, demos, etc. are disproportionately plagued by problems (3 more IE bugs today) and if pride alone isn't enough to correct that, then perhaps your suggestion of increasing the consequences of failure is a good one.
I think the problem really is systemic. I came to MS having spent years in corporate IT trying to cope with and work around MS software "issues" on a large scale. I thought that given a strong customer perspective, I could really make a difference in the quality of stuff going out the door.
After five years or so in the test organization, i've realized that I really can't. No matter what comprehensive quality assessment and improvement plans i've initiated, schedule shortening and other priorities ensure that those plans will never be executed. Quality gates are instituted by product management, only to be tossed aside when time gets short and Dev/PM are up against the wall. Producing realistic scheduling often get you labelled as an underachiever, when you say no to a pie in the sky plan.
Features need to be foregone so the basics can be nailed. Without this, some other company will (eventually) hand us our ass.
As far as staffing - I agree cuts need to be made, but at the feature/product level. Take the best staff from there and make sure the basics have better coverage. The many vest-and-rest Directors and VP's need to go as well. That alone should cut payroll and improve productivity a bunch.
It was the Phillips Semiconductor CEO who made the knock at Windows: Philips turns knife over Gates' failed CES demos.
Snippet:
However, fellow panellist Frans van Houten, CEO of Philips Semiconductor, countered: "Not everybody wants to put Windows in all boxes. Certainly, when we are sitting on the couch and watching TV, we don't want to see that blue screen in front of us."
I think Steve Jobs's MacWorld demo crashed this year, too. So I guess it's not Apple's share that should increase...
The point of a live demo is to say "This code is going so well we can even play around with it." Crashes completely negate the value of a live demo. If you just want to show what you think it'll look like when it's done, record it to tape under controlled conditions and play it back.
There is no excuse for a failed demo.
If your demo doesn't crash every couple dozen times or so, you're not doing cool enough stuff.
If your demo doesn't crash every couple dozen times or so, you're not doing cool enough stuff.If your demo does crash every couple of dozen times or so... maybe you shouldn't be demoing it yet. Or maybe you shouldn't be pushing bugs into an "end of milestone" bug bash phase - but should be fixing them as you go.
I agree with Richard. Demos are expected to crash at some point. Don't let the chattering nitwits get in the way, keep pushing the limit and keep showing it publically. Most folks really don't care if it crashes or not.
I've seen blue screens on airport information boards, cash machines & registers, interactive booths at museaums & art galleries... hell even 40ft high advertising spaces! It's pretty common unfortunately, and I'd bet a lot more people have seen it in their everyday life than saw Gates plug a dodgy USB device in at a geek show years ago.
People expect demo's to crash, it's when it keeps happening in the 'real world' you get a bad rep.
Post a Comment