When Partners Collide
I'm up to my elbows in recently inherited features, so I'm sharpening my nose on the grindstone vs. spending much time here other than being the moderation monkey. But given the incoming flow of great comments, all I really need to do currently is put up the occasional post and then let a lot of the excellent readers / commenters have their way with this little bit of the blogosphere.
Now, yes, I appreciate the recent comment that most of the commenter identities have to be taken with a grain of salt. Was that really markl dropping by? Dunno. It would have been a lot more certain if he'd logged in via his Blogger account since he has a blogspot web page set up.
When Partners Collide: So, with those few grains of salt, let's enjoy some of the partner comments as of late. First of all, you can only poke Smokey so many times with a sharp stick before he errupts: "I am Partner, hear me roar!" This is a refreshing read since I don't hear so much from our upper management on the issues discussed here and the gripes about the overpaid, underperforming highly over-staffed Partners at Microsoft. In excerpts:
I am a Partner.
[...] I am sick and tired of reading all the whining, and "I deserve to be rich too" comments on this blog. Jeeze, get a life, or better yet, go find another job!
[...] My job now is to manage and to direct strategy for my teams features. This is a very difficult job. Have you ever tried to coordinate and manage a team of hundreds? Spread across multiple continents? Within a broader group of thousands? Yes, I get paid well for this job. I'm sorry, but frankly, I feel that if anything, I am being underpaid! I will be lucky to gross $350k this year and my big stock awards someone mentioned earlier in this post vest over 5 years. That means that for me to cash in on that $5m payday, I have to stick it out for another 5 years! You want this job? Then EARN IT!
[...] I took risks with Vista. Did I do everything I said I was going to do? NO! Did YOU? Should we be fired for this? Thats right. I don't think so and neither do you. I screwed up a little, but my screwup was your screwup to. Afterall, it was YOUR CODE that failed, NOT MINE. It was YOUR SCHEDULE, NOT MINE!
Another self-described Partner kicks in with what I'll describe as a more reasonable perspective:
[I]n 2000 partners were awarded ~400,000 shares, then we had a split and another round of huge grants. Each partner had ~1m options which unfortunately were underwater so Microsoft arrange that infamous underwater option buyback program. Regular worker bees collected a few thousand dollars at best while our illustrious partners collected a cool $1m each. Not the $20m+ that Steve was hoping for when he first issued the huge grants, but still not bad given the situation.
[...] I think that the only thing thats seriously out of whack with the current system is that there is a huge disparity between the raw number of shares for partners and regular folk, and the partner awards seem completely disconnected from company performance.
I know we like to use stock awards as sort of a golden handcuff to retain people, but here, I think it is working against us. Instead of placing our partners on an 8 year program with huge grants, I think we should award them with yearly grants where the raw number of shares is directly related to company performance across a wide range of variables from the previous year. I would include customer satisfaction, industry perception, openness, innovation metrics, schedule prediction proficiency, etc. as the measurement criteria, not just financial results. With the current system, 3 years ago when we thought things were going to rock with Longhorn, before we were afraid of Google, etc. we awarded these guys $5m each. In retrospect, after looking at our broad performance over these past few years, clearly we have overpaid these guys, BUT we have virtually no ability to recall their grants and we certainly don't have the backbone to fire them. In my proposed system, I would dole out their options yearly and would have seen the errors in time to avoid over paying them.
I am a partner, and as a priviledged partner was part of the compensation review committe where I had an opportunity to give feedback on all of the options being discussed. I voiced my concerns then, and gave feedback similar to what I have said here as well as a few other major issues that I thought were wrong with the then proposed compensation plan. While I will likely benefit hugely from the plan, I think the plan is fatally flawed for a number of reasons, and I hope to see it eliminated asap.
Thumbs up! I agree with that and it's wonderful knowing that such a person is working at the Partner level and might effect real cha-
Sorry guys. I'm out of Gas. No one really wants to address the real issues around here so I am planning on leaving in the Spring. I am definitely in the minority around here when it comes to driving change, driving hard core accountability, eliminating schedule chicken from our processes, etc. I love your blog and will keep reading and contributing, and hopefully in my new venture will not make the same mistakes you all have been pointing out. When I startup my new gig, I will be sure to send out an "I'm hiring" notice to this blog.
Oh yeah, and to whoever claimed that partners still get paid, still vest, etc. after they leave? Thats complete BS. You are just fanning the flames. When you leave Microsoft all vesting stops and you have 90 days to excercise remaining unexcercised but vested options. Stock Awards are a little different because those "auto-excercise" on vest so they immediately transfer to your control on the day of termination. Trust me on this. As someone who IS leaving the company on April 1, I would know and have already been thru this with my attorneys and with Microsoft HR.
Dang. Well, an ex-Partner provides this additional perspective:
I am an ex-partner. I left Microsoft a little over a year ago. The number is legit, and if anything is a bit low. Partner level comp of $250k with ~50% bonus is the norm which would result in gross comp of $375k. That coupled with existing options, grants, and future grants is designed to payout a minimum of $1m/yr.
Each quarter there is a secret, partners only meeting where Steve and Bill discuss issues of the day. I remember one meeting in particular, one where I had the priviledge of sitting next to "markl". Steve was talking about cutbacks and accountability. Markl looked at me that Steve doesn't have the balls to do the right thing. He thought that if Steve was going to send this message to the masses, he should first send a crisp message to the partners, that they are going to be held to a higher standard. He should have set up the meeting in advance with 400 pink index cards placed randomly under all of the chairs. He should stop the meeting, ask everyone to reach under the chair and hold their index card up in the air. He should then bring in security and escort all of the partners holding pink cards out of the building, immediately terminating their employment with Microsoft." His point was that we had far too many old timers sitting around doing nothing, and that if we need to make some hard core changes, we should start at the top. He felt that Microsoft would get along just fine with a random 50% culling of the partners, and that the impact this would have on the remaining partners would be a hard core wakeup call and they would likely step up and perform at the level that they should perform at.
After watching our performance over these last few years, and watching our current accountability crisis, I have a feeling that Markl's idea was a good one...
(Excuse me while I replay that scene in my mind... one more time... oh, that's so good.)
Measurable Recommendations: Too much complaining and too little solution brainstorming? The Specific, Actionable, Measurable commenter took care of that by going above and beyond in identifying a whole slew of issues, possible solutions, and how the solution's success can be judged. It just gets me all fired up!
Some interesting follow-ups to that, along with some recent lunch conversations I've had, have to do with compensation not being tied to just individual accomplishment but also to team accomplishments. One commenter suggested that if we're going to have a curve, make it more gentle for excellent teams and more brutal for fumbling teams, along with compensation. And just think if we can align compensation and recognition with something that actually positively affects our products, our partners, our customers, and our shareholders. Anything changes being contemplated that don't have that in mind are waste.
Orange Badges: I just had this strange brain connection between the Microsoft orange badgers (contingent / contractor staff) and the red-shirts that would beam down on Star Trek Classic. You know how this is going to end and, soon, our ranks will be a little smaller. Historically, I've personally always first treated every orange badger as someone to recruit to be fulltime. I'm not so much in the hiring mood anymore and I can't remember the last orange badger's path I crossed. Anyway, as Mr. Bishop noted in a front-page Seattle P-I article, the orange badgers now have a discussion site setup, orangebadges.com. Talisman there makes the following familiar observation:
Microsoft used to be one of the best examples of meritocracy I ever knew. But it has increasinly become, in many quarters, a cesspool of feudal fiefdoms populated with petty princes and petty prince wannabes who won't hesitate a moment to use anybody to advance their own agendas. You just have to be careful what part of the kindgom you occupy and keep an eye on your back.
And A_nomore throws in:
And I thought micro-management was bad in the Navy. I got on as a temp at a little off campus site. I was amazed in my six months there of the constant backstabbing that my boss (BB/good guy) endured.
It continues and the post reminds me a lot of Satan's Process Excellence. The site has some interesting perspectives from people working beside us.
CES: Congratulations on a big ole Microsoft demo that did not crash! There was one online service request hiccup that worked on the second try, but otherwise it went well. I better acknowledge our demo success to adjust my demo karma given my inclination to go on the firing war path when demos blow up. But given that at least the first part looked like the same thing BillG did at the Company Meeting, I think it was all easy for Mr. Gates to just go through the motions of doing it all over again.
As for getting some celebrity personality spice... maybe next time we can do better. Yahoo! had Tom Cruise. Google had Robin Williams. Microsoft... Justin Timberlake? Well, maybe if Justin had forced a wardrobe malfunction on Bill it would have been funny ("ai-yi, Bill's codpiece was supposed to stay put!"), but nothing says safe and boring as Mr. Timberlake.
Given the consumer move to more and more devices I personally feel good about what we're doing. If we can provide easy gadget wrangling then people will see greater value in Vista. I personally will love it when all my doo-dads are wireless and I don't have to do the wire plugging stumble and the vendor specific synchronizing... I just want to come home and let them opportunistically update their content while I'm trying to get visions of new automation violations out of my mind...
(update: fixed a grammar error I actually noticed.)
129 comments:
Question- where does this designation of "Partner" come from? Is this an official designation by HR or something that is less formal?
I'm part of Microsoft's field organization, and I haven't heard of it before. How does one attain this status- is it based on level, tenure, or other factors? The compensation described sounds pretty high, and one of the commenters implies that there are HUNDREDS of these partners. Is that true???
Firing 50% of the Partners at random sure is a nice fantasy, but keep in mind that they probably all have golden parachute contracts that let them vest 100% if they are fired for anything other than "cause". It is sort of unintuitive, but firing them might give them the biggest payday of all. Pretty sick, eh?
The only thing you can do if you don't like it is to leave. It's kind of interesting that the majority of "happy" comments on this blog seem to be from ex-employees. Apparently they realized that while they can't change a broken system, they can remove themselves from it and find a place with a system that works better.
I think this blog is one of the best out there. It has helped me to finally see this for myself. Thanks, Mini.
>Was that really markl dropping by? Dunno
So Mark Lucovsky, one of the highest-profile defections to Google decides to take some time out to bother duking it out with the Scobelizer (of all people) on some public forum when the former architect/DE's own blogspot page was last updated six months ago.
Right.
if there's a team curve, and "bad" teams are punished, why would a superstar ever join one to fix things?
Juat wanted to add a note as a soon-to-be new hire. (I am going in to NEO tomorrow morning... in 10 hours.)
The group I will be working in is very insulated from the rest of the company, and very profitable when you take it on its own. I'm totally confident about my ability to make a great impact on the team, and really help the product(s) that we make succeed.
That said, I'm discouraged about my future with the company before I'm even starting. I don't think of tiny little stock grants will amount to much given the stock price stagnation. My only hope is that our group is very generous to its members because we are so successful with what we do. But with things like Vista getting so little support, even from people within the company, and some really stupid moves lately (Urge? A music service without iPod support? HA ha!) I'm really wishing that the company does a massive and profound restructuring or spins off our group. (Who am I kidding?) Then maybe we can be rewarded for our efforts instead of being dragged down by the rest of the crap around us that we have no impact on.
Am I not a good little worker bee for feeling some resentment?
This is a test by ms management folks .. don't let it fool you.
Blogging has been an interesting media for them to use (jobsblog, scobelizer, channel9 ..etc).
It may a model to allow people to provide untraceable feedback to either improve ms, make people want to leave or who knows what.
Why? OHI and other internal tools are not anonymous nor will they be. Fear of retaliation is high ...
Not sure if i agree with possible (and speculative intent of the blog) but it is certainly serving many purposes and clearly is protecting some from harm.
"Partner" just means level 68 (or 14 in pre-Comp2000 terms) but it is sort-of recognized by HR and via various aliases. I actually think getting to 66 is the hard part these days after that 67 and 68 are more a matter of time + not screwing up too much.
Fwiw, I am partner and no, not everyone gets paid $250K or $1M in stock (me for example). I do get paid plenty so I'm certainly not complaining!
Btw, like quite a few partners these days I'm not "upper management" since I only manage a very few people and really they manage themselves. I guess I'm more "upper technologist"? Still, like everyone I know at my level I hold myself very much accountable for the actions of the people who work for me regardless of the number. None of this "my developers failed to hit their dates so it's not my fault..." crap. Perhaps more significantly so does my manager. When I've done well I've gotten great reviews and big rewards. When I've screwed up (happens to the best of us) I've gotten a 3.0 and not so much on the rewards front :-) I for one happen to like it that way.
Did you see the Business Week article "The Struggle to Measure Performance" in the 1/9 issue? You WILL enjoy it!!
Not so fast. The HD-DVD demo DID crash, or rather, it refused to start up when the disc was inserted. There was a Microsoft guy there, so technically, *a* Microsoft-associated demo did crash. :)
More interesting was Yahoo and the head of Intel (!) mocking Windows when their demo was having hiccups.
>compensation not being tied to just individual accomplishment but also to team accomplishments.
This is a dangerous proposal, IMHO, having seen it implemented at other companies. Couple obvious problems:
1. At what level do you implement this? At the division level, say, Windows Client or MSN? Not granular enough. At a 10 person team level? Maybe too granular.
2. How do you measure success? Shipments? What if your ability to ship was not constrained by your own team, e.g. Windows as a hold was delayed, even though your team's feature was done and ready.
This has the ability to be just as demoralizing as the stack rank. You can easily end up in situations where a team's "performance" (especially product teams) is crippled by an external factor leading to a low bonus.
BTW, Robin Williams being more "exciting" than Justin Timberlake? Puhleeze. At an IMDB glance, the man's last movies were "The Krazees" and "Mrs. Doubtfire 2". Oh yes, nothing says hip and exciting like Mrs. Doubtfire. At least Justin's banging Cameron Diaz.
Orange badges...
I work 5 years in couple of divisions as a developer. What can I say? The experience is the same. What developers are doing most of the time is not a development or design. What they are doing is a fighting to be able to do some development or design. It is not a problem to make a good product, all you need is just choose a couple of developers from the project, assign it to them and do not touch them about a month. Bug it will be never done good, because it is not allowed to do. Sometimes the trivial, but critical bug are not allowed to be fixed, because test is not agree and PMs are not agree. And developer is trying to fight with management, convincing to fix this bug, taking bad review as a result (not a team player, wrong priorities, etc.)
And in the middle of the battle some orange badge coming from outside asking: "why can't we do this or that, why it was done this way". Because noone was responsible for this. This is why. If you will add your piece of code and go away, it will be just worse, not better. Someone will have to understant it, fix it, extend it. Will it be possible? Likely, No. It will be thrown away and rewritten by another person.
Why contractor was hired at the first place? Because manager thought (if we can name it this way), that if we have such a mess in the project, we need more people.
So will I be happy to see a new contractor in our project? Unlikely. I would rather be happy to do not see some existing ones. Not a team player, they are right, sure I am not. I thought we are writing software, not playing career games.
This has the ability to be just as demoralizing as the stack rank. You can easily end up in situations where a team's "performance" (especially product teams) is crippled by an external factor leading to a low bonus.
You can't avoid external factors affecting a product.
This happens on an individual level as well.
For example, a dev lead assigning someone on his team a work item that requires cross-group cooperation where the work item has a lower priority in the other group.
On an individual level, the dev lead could be screwing an employee they don't like. On a team level, they would also be screwing themselves.
Which one's better? I would say the team based approach.
If everyone is affected by a work item with a lower priority in a different group, it is more likely to either not get added to the schedule in the first place or get cut earlier.
"Firing 50% of the Partners at random sure is a nice fantasy, but keep in mind that they probably all have golden parachute contracts that let them vest 100% if they are fired for anything other than "cause". It is sort of unintuitive, but firing them might give them the biggest payday of all. Pretty sick, eh?"
OK, that might be sick. But I don't think it's that bad.
Why was the suggestion made to fire half of the partners? Because (gross generalization) they've become political CYA do-nothings. Firing half of them gets rid of a lot of problems, and tells the remaining half that they'd better shape up, pronto. It seems to me (non-Microsoftie) that doing so is worth almost any amount of money that it would cost.
MSS
if there's a team curve, and "bad" teams are punished, why would a superstar ever join one to fix things?
They wouldn't and the people who can't perform would be more likely to fail and as a result get fired.
The same with lousy managers. If you have a dev lead that consistently agrees with unrealistic schedules of upper management and that affects the team's performance because they can't meet milestone deadlines, IC's are more likely to seek out a manager that has a better track record of meeting deadlines leaving the suck-up manager with no one to lead.
You would need to remove some of the barriers to internal transfers for this to work effectively. For example, a manager should not be allowed to prevent an internal transfer.
"Firing 50% of the Partners at random sure is a nice fantasy, but keep in mind that they probably all have golden parachute contracts that let them vest 100% if they are fired for anything other than "cause". It is sort of unintuitive, but firing them might give them the biggest payday of all. Pretty sick, eh?"
You don't fire them at random, you fire them for cause. Some, perhaps even most, are probably worth every penny they get. But based on performance, too many clearly are not. Will it cost a lot to get rid of those? Of course. So what? Their cost is not just their compensation but the opportunty cost to the company of not having someone more effective in that important position as well as the mixed message on accountability that it sends to regular employees and the street. Time for Ballmer to stop talking about accountability and start enforcing some, starting with a chainsaw at the partner level to weed out the many dead branches so that the tree's health overall improves.
"This is a test by ms management folks .. don't let it fool you."
Doubt it...MS is a very liability averse company. Some comments have been posted on here that clearly would not have been posted if MS management was involved. My 2 cents...
Absolutely agree with the poster about developers spending more time and energy getting around PM/Test than actual design and development.
In our team never has been a feature such that a spec written before code, and the developers find much more issues than testers, and its not this extra effort that they mind. What they mind is not being let do their job in the name of some war room meeting or quality gate or such.
Here is an uncomfortable truth about the corporate world: We're all playing the game; ... A software business is still a busisness, whose sole goal is to make a profit...
Actually I do not see how this two things are related. My thinking is that the goal to create compatitive product and sell it is quite different from the goal to make a career. The career is an employee goal, it is not a company goal. It is supposed to be a motivation in case if there are no other motivations. But some large companies (and teams) creates such an atmosphere, where this goal becomes dominating. IMHO, it is not quite compatible with creativity, so the design quality degrades first and after that everything going down, including stocks.
The atmosphere, where everyone trying to compete with competitor, not with his teammate, much more valuable for company success. And last years I see only track of it in the memory of old Microsoftees.
2. How do you measure success? Shipments? What if your ability to ship was not constrained by your own team, e.g. Windows as a hold was delayed, even though your team's feature was done and ready.
You answered your own question.
1) the team got their work done within the time they estimated for the milestone
2) the team did not have to cut features to meet the deadline
If the team had to cut features or did not meet their deadline, they get a percentage deducted off their bonus (up to 100%) which depends upon how closely they met their goals.
If a feature depends upon work done in another product group and the other product group doesn't place the same value on the feature, you can either convince them that it has the same value you place upon it or cut the feature.
If management is the cause that the team did not meet their goals, deduct a percentage from management's bonus.
For example, we had a reset on the development of a server product because of lack of performance work done in a version of the .NET Framework. The managers who made the decision to use that version were not the ones who were working with the product. In that case, get management's decision in writing (to avoid any revisionist recall) and deduct from their bonus for overriding the decision of the people implementing the product.
Ballmer to stop talking about accountability and start enforcing some, starting with a chainsaw at the partner level to weed out the many dead branches so that the tree's health overall improves.
If Ballmer was serious about accountability, upper management in Windows would be gone. Brianv and Jimall have been in charge "managing" the biggest software disaster this company has ever witnessed. They did not write the code, BUT the oversaw the schedule, set priorities, and in the end watched as their teams failed to deliver over and over again. They had countless opportunities to cut their losses and features early but instead they just waited ond hoped for the best.
Instead of being asked to leave, "with cause", "for failing to deliver on their promises", Jim is sitting here cashing out mega-millions in options, and Brianv is probably doing the same AND waiting for his HUGE partner windfall to begin vesting this spring.
Firing 50% of the Partners at random sure is a nice fantasy, but keep in mind that they probably all have golden parachute contracts that let them vest 100% if they are fired for anything other than "cause". It is sort of unintuitive, but firing them might give them the biggest payday of all. Pretty sick, eh?
You should fire everyone who hasn't shipped a product in the last five years.
Add a clause that specifies that the rights to the stock grants are revoked if a partner does not ship a product within five years of the stock being granted.
That said, I'm discouraged about my future with the company before I'm even starting. I don't think of tiny little stock grants will amount to much given the stock price stagnation. My only hope is that our group is very generous to its members because we are so successful with what we do. But with things like Vista getting so little support, even from people within the company, and some really stupid moves lately (Urge? A music service without iPod support? HA ha!) I'm really wishing that the company does a massive and profound restructuring or spins off our group.
Why are you joining a company on hopes and wishes that it will change?
Let me save you a divorce and tell you not to do the same thing in your personal relationships.
Why not get a job at one that you believe is doing things the right way?
Did you see the Business Week article "The Struggle to Measure Performance" in the 1/9 issue? You WILL enjoy it!!
The Business Week article mentions a lot of what has already been discussed on this blog - how long to apply a forced ranking system.
The Struggle To Measure Performance
"Steve Scullen, an associate professor of management at Drake University in Des Moines, found that forced ranking, including the firing of the bottom 5% or 10%, results in an impressive 16% productivity improvement -- but only over the first couple of years."
"It's a terrific idea for companies in trouble, done over one or two years, but to do it as a long-term solution is not going to work," says Dave Ulrich, a business professor at the University of Michigan at Ann Arbor. "Over time it gets people focused on competing with each other rather than collaborating."
One of the people commenting on the article mentioned a book called Punished by Rewards - The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes.
"Step by step, Kohn marshals research and logic to prove that pay-for-performance plans cannot work; the more an organization relies on incentives, the worse things get."
At a 10 person team level? Maybe too granular.
It would only work on a level where individuals in a team have a large degree of control on what they do depending upon their occupation - write code, write specs, coordinating schedules across product groups, etc.
The problem with team-based compensation, or with making the review model harsh for poor teams, is that it encourages good people to leave the teams. The poor folks stick around and get their 3.0 and their steady paycheck.
On a poor team, the good folks already want to leave even though they're likely to be getting the 4.0s. You can't punish people at review time for the audacity of staying on a poor team.
Finally, punishing the poor teams will no only chase away the good folks, but it will make it hard to get _anyone_ back to replace them. No internal transfers will go to a "bad" team where they'll be punished, and good internal transfers are the key to turning around a team in trouble. Many posters here are fond of saying the problems are at the lead/manager level - if your team is being punished for poor performance, there aren't a whole lot of good leads willing to transfer into that mess.
No, I think per-team ranking changes would be awful.
Changing the review model isn't going to improve anything. The review system isn't the problem.
The problem is that the company has grown so large that it's collapsing under its own politics.
The review model is politics. It is based upon subjective measures easily manipulated by those with an agenda other than measuring performance.
You can't punish people at review time for the audacity of staying on a poor team.
Sure you can. If someone stays on a team that they believe will not deliver, they are wasting company resources including themselves. It shows a lack of sound judgement.
Many posters here are fond of saying the problems are at the lead/manager level - if your team is being punished for poor performance, there aren't a whole lot of good leads willing to transfer into that mess.
As a result, a poorly performing team will be more likely to fail and people who can't deliver product will get fired with cause.
What's wrong with making it more likely people who can't create a quality product are fired?
I thought we are writing software,
not playing career games.
Here is an uncomfortable truth about the corporate world:
We're all playing the game; even if you "refuse to play", you're still playing. You just arent making your own moves.
A software business is still a busisness, whose sole goal is to make a profit, that is: to sell a product or service for more than it costs to produce it.
Yes it is all part of the political game at a company. If you don't make your own moves to control your career, someone else will make it for you.
As for the next comment where he the author states you control your own career and it has nothing to do with the work you do? Take a look at your review. A portion of your merit, bonus, and stock awards are based upon how well you achieved those career goals you previously set. Those goals that your manager never helped you create. The ones your manager never let you take your classes to achieve. Yes I control a bunch of it, but the company controls a bunch as well, and they rate you on what they perceive. In addition, that affects your promotions and income. Therefore, the company does in fact control a portion of that career path while at the company.
This still all leads back to that book that was discussed in December 2005 here. I have now read it, and I am glad it was recommended. (Corporate Confidential by Cynthia Shapiro). The author helps with all kinds of these issues. I suggest anyone who hasn't read the book do so if they are concerned with maximizing the understanding of Corporations, many of their motives, and things you can do to assist yourself in bettering your own career.
I also believe a friend mentioned it is now in the internal library. If so, I wonder who suggested that and how it made it inside?
"If Ballmer was serious about accountability, upper management in Windows would be gone. Brianv and Jimall have been in charge "managing" the biggest software disaster this company has ever witnessed. They did not write the code, BUT the oversaw the schedule, set priorities, and in the end watched as their teams failed to deliver over and over again. They had countless opportunities to cut their losses and features early but instead they just waited ond hoped for the best."
Agree. Which is why Ballmer needs to go first.
Actually I think a team of 3.0s wouldn't be too bad. Superstars are great but I think it's a bad team dynamic. People rely on the superstars, and the superstars are often trying to advance their own careers by imposing their own (not always good) designs and stealing other peoples' work. Having superstars around doesn't force (or even allow) the 3.0s to work together or improve. You also have to ask yourself, if what you're trying to do is so complicated or poorly designed that you need superstars to be able to implement it, is that really the position you want to be in?
Dont expect anything to change because of these comments. If anything, the partners will be paid more. It is the exit of the partners that caused this furore.
What's needed is accountability, plain and simple.
You think Steve Jobs would put up with such flubbed managing at Apple? That guy would boot 'em for daring let OS X stagnate for five years. And that's why they're introducing new Intel computers today while Microsoft's news is more WMF exploits announced. :(
Vista had better be rock solid. It's not a question of engineering talent--the talent is there. It's management. They screwed it all up and haven't stopped yet.
Speaking of change for the better, how has hte new SDET initiative been working out? Are there more SDETs becoming testers or testers becoming SDETs? I'm just curious because I was wondering which the intiative was attempting to accomplish if either.
Perhaps poorly performing teams should be disbanded to make room for a new team that can take over and get some work done.
Penalizing a whole team could be a good thing as long as managers are ready to build new teams to replace failing ones.
Why are you joining a company on hopes and wishes that it will change?
Let me save you a divorce and tell you not to do the same thing in your personal relationships.
Why not get a job at one that you believe is doing things the right way?
Because I'm not going to fool myself into thinking that I can find a company that does everything right. If I am rewarded for my personal performance I have no problem with that. And if my group is never showered with money because of our success, that's also OK. I didn't take this job to get rich off my stock or bonus, I was just giving my thoughts as a new hire.
I'm very very proud of the product I'm working on, and that's more important.
On a slightly seperate note: Here is another thing that is *extremely* broken at Microsoft: The "buddy" system. This does not necessarily happen throughout the company, but does happen enough that it is a serious problem.
Basically I am talking about folks getting promoted to a job which they are not qualified to do and have no business doing. How did they get there? Is it the Peter principle thats getting applied or is it that one of their "friends" is helping them grow even though they should not.
One quick example: One of the GPM's within a group just got promoted to become a Technical Assistant to a very-very senior person in the company. Most people I have talked (and I agree) to think that this person has no business being a technical assistant. How did this happen? Was is that his manager was hyping him more than he is worth?
The bad part about situations like these are now this TA will be giving advice/suggestions to a very-senior person who then can implement these suggestions. We already know that upper management is out of touch with the average developer/pm/tester, and this does not help the situation (when the technical person advising you is anything but technical).
"I'm very very proud of the product I'm working on, and that's more important."
Nice to be young and optimistic / idealistic.
Please advise us of how you feel in 5-6 years (when people you probably consider old timers or "pick your adjective" are gone)
>> Speaking of change for the better, how has hte new SDET initiative been working out? Are there more SDETs becoming testers or testers becoming SDETs? I'm just curious because I was wondering which the intiative was attempting to accomplish if either. <<
This is another sad story. Some moron from high-up, decided to get rid of all the STEs in the hopes of replacing their manual work with test automation. Predictably this initiative has failed. Big Time!
Now we have a test team WITHOUT any testers. Yeah! Don't blink your eyes. You read it right. In the absence of STEs/Testers, we (the SDETs) find ourselves devoting more and more time to Non-Automation related work - manual testing, bug verification, bug closing, lab installations yada yada.
To answer your question - my responsibilities are not clearly sketched. Am I a dev who writes test automation code? Am I a manual tester? I do not know. Go ask my manager.
Speaking of change for the better, how has hte new SDET initiative been working out? Are there more SDETs becoming testers or testers becoming SDETs?
The SDET initiative is to make all testers as SDETs.
Here is some free advice for those with a computer science background and offered an SDET job at MSFT -
Unless you think you have a good shot at becoming a Lead in 4-5 years, it would be unwise (even utterly foolish) to join MSFT as an SDET. SDETs as ICs dont grow beyond 61 in most of the teams. How many products really need level 62 or 63 testers?
The company has you fooled into thinking you can grow on the SDET ladder as an IC. But it is just utter hogwash. The company has its best interests in mind, not your career.
And after 4 or 5 years if you are not a lead, you are in a sticky situation. Hard to change to dev after doing essentially testing for 5 yrs. (yeah, we all know the quality of the automation code:-).
And your value outside of MSFT doesn't amount to much. How many testers get paid more than 100K outside MSFT?
Its best to join as a developer in any other company instead of SDET here, and you are probably doing work that is better for your career in the long run.
I am not saying all SDET jobs are useless for your career, but I would say 80% of them are, unless you are on track to become a Lead. Of course, you should be able to evaluate your chances of becoming a Lead based on your personality, smarts, background etc. Just being an average MSFT SDET hire probably wouldn't cut it.
On the dev ladder at MSFT, no doubt, most jobs add value to your career.
"As for the next comment where he the author states you control your own career and it has nothing to do with the work you do? Take a look at your review. A portion of your merit, bonus, and stock awards are based upon how well you achieved those career goals you previously set. "
That's BS and you know it. Review scores are all calculated before you even write your review. Management "adjusts" their evaluation of you based on the score you are assigned. I seriously doubt any mangers actually look very hard at what you write on the forms.
Most of the "superstars" that I worked with spent a lot of their time cutting other people down trying to climb higher on the curve because they're more aware of what the rewards are for doing so.
There is a lot of stealing other people's work going on. If you think your coworkers are your friends just because they engage in friendly conversation with you, you are very naive.
One time I explained an idea I had to my development lead and then later explained it to a coworker.
Inside of five minutes, that coworker was inside my development lead's office trying to pass off my idea as their own. Hint: Next time, shut the door if you want to steal other people's work without them knowing about it.
People don't get rewarded at Microsoft for having ideas. They get rewarded for implementing them. Managers are often in a position to stop you from implementing an idea.
You can use a digital time-stamping service to prove you came up with the idea.
http://digistamp.com/
If someone takes credit for your idea, sue them. They are costing you compensation and promotion.
If you want to get rid of bad managers, stop sharing your ideas with them if you don't get rewarded for it.
Hi Mini
A lot of comments on this blog seem to relate to the 'older' Microsoft products - Office, Server. Any chance of something about the MBS side. I recently posted an entry on my blog about 2 discussions I had, with a MBS competitor and an MBS channel manager, and their different views on where MBS is going.
You can't punish people at review time for the audacity of staying on a poor team.
Sure you can. If someone stays on a team that they believe will not deliver, they are wasting company resources including themselves. It shows a lack of sound judgement.
It's a self-fulfulling prophecy. If the team has a rocky milestone, their incentive is to jump ship rather than stick around, make the hard choices, and fix things. Now you're short people, things are still broken, and the team is more likely to fail. We need people to stick around and FIX things, not to run away when things get hard.
Many posters here are fond of saying the problems are at the lead/manager level - if your team is being punished for poor performance, there aren't a whole lot of good leads willing to transfer into that mess.
As a result, a poorly performing team will be more likely to fail and people who can't deliver product will get fired with cause.
What's wrong with making it more likely people who can't create a quality product are fired?
If the dev team fails, the superstars on the test and PM team fail. It's not that they "can't deliver product", it's that software development is a team effort and no one is in full control over their own destiny. You seem to be suggesting that when a project is not a success, everyone deserves to be fired.
We absolutely need to hold people accountable for failure, but the solution isn't to inject more risk. If you think we're shipping boring ass shit now, you'll be amazed at the kind of IBMesque crap we'll excrete when fail=fire. No, what we need is LESS risk - less risk of having a good idea get crushed by platform initiatives and cross-org strategy calibration. Let the smart folks put together something cool.
You can see two perfect examples just from Office 2003 - a how-to and a how-not-to. OneNote was born from a brainstorming session from the Word GPM and SteveSi - they identified a problem, threw together a team, and built a fantastic brand-new app. On the other hand, you've got InfoPath - a monstrocity of collaborative process which finally "shipped" an app so well designed that they had to redo the entire UI in SP1.
What's the most risk-adverse organization you can think of? For me it's the Army - shit matters when they get things wrong. People die. Bad! So consequently they're a choking bureaucracy. On the other hand, take my outsider's view of Google - they've got hundreds or thousands of 20% projects going on, and we see a couple new products launch a year. The rest die, and you better believe that doesn't mean you get fired. Yet they're lauded for their innovation. Hmm!
Only by encouraging creativity - and creating an environment where people can BE creative - can we return MSFT to a place where innovation happens and matters. Massive penalties for failure won't do that. We don't even need massive rewards for success - we just need to make it possible again.
> Agree. Which is why Ballmer needs to go first.
What about SQL server? What about all the partners in between?
What about all the HR GMs that were/are touting high OHI across the company? You will get an answer that a consultant designed MSPOLL. Is MSPOLL meaningful? If not why waste everyone's time with idiotic questions. It took an anonymous blogger and a business week article to bring issues to the front.
This is another sad story. Some moron from high-up, decided to get rid of all the STEs in the hopes of replacing their manual work with test automation. Predictably this initiative has failed. Big Time!
Now we have a test team WITHOUT any testers. Yeah! Don't blink your eyes. You read it right. In the absence of STEs/Testers, we (the SDETs) find ourselves devoting more and more time to Non-Automation related work - manual testing, bug verification, bug closing, lab installations yada yada.
To answer your question - my responsibilities are not clearly sketched. Am I a dev who writes test automation code? Am I a manual tester? I do not know. Go ask my manager.
Sometimes I think Redmond makes people lightheaded. It is interesting how VS 2005 was said to be full of bugs and this is AFTER the SDET initiative.
I said months ago that it is not possible to do both jobs. If Vista is depending on not having actual STEs, then Vista will suck. I mean, I tried to be positive about VS but I have to restart it after a few hours because SnapLines stops working, etc. So far the two or three bugs I can easily repro were called no repro.
MS mgmt has lost their damn minds. I guess money can make you crazy and the beneficiaries of the "Greatest Deal Ever" definitely have lots of money.
As many eople are talking baout team accountability I figured I'd comment. Yesy, I do believe that the team level is better to use, since if people want their team to do better they will become mentors rather than informers.
If people know their team will suffer if they don't help the "non-stars." Speaking of which, the word "STAR" is used a lot here and I wonder what people think a star is.
Is a star the person who always has ideas to keep people working on things. Is it a person who always has time to do extra work because they have perfeted their tasks? Is it a person who is always agreeing with the manager and only works on things that they are given?
Here is some free advice for those with a computer science background and offered an SDET job at MSFT -
Unless you think you have a good shot at becoming a Lead in 4-5 years, it would be unwise (even utterly foolish) to join MSFT as an SDET. SDETs as ICs dont grow beyond 61 in most of the teams. How many products really need level 62 or 63 testers?
Every product needs Level 63 testers. Wouldn't they be much better at determining what should be tested the most? Wouldn't they also be better at setting up environments and debugging failures.
That is exactly why I left. QA IS THE MOST IMPORTANT PART OF SOFTWARE DEVELOPMENT and most people considerit to be a "slacker" position where you don't have to know anything. BS. Microsoft should be paying for people to take MCSE\MCSA exams. How could Windows have the fewest numbers of MCSEs?
How can MS expect to elease high quality code if no one is testing it?
I don't think I'll be buying Vista.
Only by encouraging creativity - and creating an environment where people can BE creative - can we return MSFT to a place where innovation happens and matters. Massive penalties for failure won't do that. We don't even need massive rewards for success - we just need to make it possible again.
The "safe" environment that you describe seems to already exist at Microsoft. You have lots of failure without accountability.
It is not working. You have massive delays in shipping product even in doing what is "safe". Then, when it does ship, analysts say that customers should wait for SP1 because it is buggy.
Microsoft is one of the most risk adverse organizations out there. Partners don't want to take risks. They want to do what's safe so they can be there while their stock grants vest.
Creativity is seen as risk and is not something that is encouraged at Microsoft.
Spending 20% of your time on your own projects at Microsoft will not happen, for developers, there's always pressure to spend your time on fixing bugs, writing your own test code, writing your own specs, etc.
In your example of OneNote, you aren't only describing creativity. You are describing a team that wasn't top heavy with PM's and architects each trying to grab a few checklist items for their reviews.
You can create teams that aren't top heavy very easily even without the "risk" of creativity. How often does Microsoft do that since it seemed to work so well?
You allow dysfunctional teams and managers do too much damage.
You have an annual review. If a team can't produce something in a year that let's them survive, they shouldn't.
If you have external factors that totally wiped out a team's ability to produce anything in one year, you have more serious problems in your organization.
Sometimes teams fail because of bad management, sometimes you've got people that just can't work together, sometimes they fail because of too many cooks - If people know things aren't working, they should be allowed to transfer to another project.
Nobody said they have to stay with a sinking ship.
If you have a crap manager who only has his job because he's someone's buddy, you should be allowed to transfer. We had several project resets because of a flurry of buddies each given a chance to run everything.
If you have someone on your team that spends a lot of their time trying to knock you down on the curve because they see you has someone to climb over, you should be allowed to leave that team without your manager's permission; especially if that person is a favorite of your manager and they aren't doing anything about the problem.
If you have a team that is choked on PM's and architects not being able to agree on what the product is because they're all competing for something to put on their reviews, developers and testers should be allowed to transfer to another position in the company.
Re: manticore MBS comment.
Ditto, and please post your blog, manticore. There obviously is a LOT of duplication between the 6 ERP products it sells. Can't somebody cut a product line or 3? (Sadly, the "solution" has consistently been: "We just need 1 more product to clarify the positioning in customers' minds." Yeah, right!)
Agree. Which is why Ballmer needs to go first.
What about David Vaskevitch? I understand that the WinFS fiasco is largely traceable to him. He pushed for the complexity and big bet but is so far removed from reality that none of the hard issues seem like much of a problem. He pushed the WinFS team into impossible features while selling Bill and others on how great things were going. Peter, Quentin care to comment?
I also understand that Ballmer would have him out of the company in a heartbeat, BUT Bill finds him useful and thats the only reason he is still here. How many other Davidv's do we have running around? Sounds like Doug Burgum is one? But wait, wasn't Davidv the guy that pused for us to spend $1.1b on Great Plains?
Exerpt from an interview IBD just did with Ballmer:
IBD: How is the Xbox 360 launch going?
Ballmer: Phenomenal, absolutely phenomenal.
IBD: Except in Japan?
Ballmer: We knew Japan was going to be tough. I'd still call the launch phenomenal.
Phenomenal in Japan? Seriously, how did this guy become CEO and what value does he add? What message does this send to Microsoft employees and customers about who you can trust?
> How many products really need level 62 or 63 testers?
All of them need expert testers - if only one or two - to mentor other testers and architect automation.
Truly effective automation is not trivial. The testers who are currently transitioning to SDET roles are really struggling to learn how to code, write automation AND STILL carry out the manual test needs. They need help from experienced people who don't also have lead responsibilities.
Of course, the foregoing supposes that there is a genuine will to make the STE->SDET transition a success, rather than just fire the STEs and replace them with existing SDETs. A certain Test Manager in bldg. 50 knows what I'm talking about.
QA IS THE MOST IMPORTANT PART OF SOFTWARE DEVELOPMENT and most people considerit to be a "slacker" position where you don't have to know anything.
That's a bit much- you have nothing to perform QA on without code being written.
That being said...people are nuts if they think writing automation code equates to being a good tester.
http://www.joelonsoftware.com/articles/fog0000000067.html
"That is exactly why I left. QA IS THE MOST IMPORTANT PART OF SOFTWARE DEVELOPMENT and most people considerit to be a "slacker" position where you don't have to know anything. BS. Microsoft should be paying for people to take MCSE\MCSA exams. How could Windows have the fewest numbers of MCSEs?
How can MS expect to elease high quality code if no one is testing it?"
There are a bunch of problems here.
1) Somehow a "technical" heirarchy was started that put the most technical people in a dev position, the people who couldn't quite cut it as a dev became testers or if they had decent writing and networking skills, they became a pm. QA and PM need to be just as technical as DEV. Frequently, their jobs are harder and more technically complicated than that of DEV.
2) Test was seen as a stepping point into dev.
3) Test was seen into a cost center, not an aid to development.
4) Some devs (rightly or wrongly) started to view STEs as "keyboard pounding monkeys".
5) "Better together" increased complexity (and as a result test hit) by orders of magnitude but nobody was willing to increase the budget.
6) Someone had the "brilliant" idea that automation could replace those keyboard pounding monkeys.
I agree with you, getting rid of STEs is a HORRIBLE idea. I will run linux on my desktop at home before I buy Vista...and I wrote some of the automation that is testing Vista!
Vista is the ME of the 2k line.
The comment on SDETs got me thinking over what would a good way to change to SDE? Will going for a M.S. or something like that be useful?
What about David Vaskevitch? I understand that the WinFS fiasco is largely traceable to him. He pushed for the complexity and big bet but is so far removed from reality that none of the hard issues seem like much of a problem. He pushed the WinFS team into impossible features while selling Bill and others on how great things were going.
One of the people that worked on the equivalent Oracle product, which shipped several years ago, is working at Microsoft. He told many people that the architecture of WinFS was the wrong approach.
Oracle Internet Filesystem (iFS) FAQ
Oracle Content Management SDK
Oracle Content Management SDK is a robust and scalable run-time and development platform for content-oriented applications. It provides a rich set of Java APIs for folders, versioning, check-in/check-out, security, searching, extensible metadata and other standard content management operations. Oracle Content Management SDK was formerly known as Oracle Internet File System.
Oracle Content Management SDK allows you to access your content with your favorite productivity tools through a wide range of protocol servers: FTP, SMB/NTFS, HTTP/WebDAV, NFS, AFP, IMAP4, and SMTP. Developers can also create custom/proprietary protocol servers.
Every product needs Level 63 testers. Wouldn't they be much better at determining what should be tested the most? Wouldn't they also be better at setting up environments and debugging failures.
That is exactly why I left. QA IS THE MOST IMPORTANT PART OF SOFTWARE DEVELOPMENT and most people considerit to be a "slacker" position where you don't have to know anything.
Testing is considered a "slacker" position at Microsoft.
In our group, management moved to having developers write the automation code. How many testers would write themselves out of a job?
Much of the existing automation code written by testers had to be run several times just to get the tests to pass. Each run took several hours to complete. That code had to be rewritten.
Developers have to write a lot of our own specs as well for various reasons - many PMs aren't technical enough to come up with a decent spec, they don't have time because they work on many projects and/or spend a lot of their time out of office with customers, or it is a low priority item for the PM. Later, a PM would just slap their name on the spec.
If management made better hiring decisions, they wouldn't be in the mess they are in now and it wouldn't be necessary to cull so many people using the review system.
What about all the HR GMs that were/are touting high OHI across the company? You will get an answer that a consultant designed MSPOLL. Is MSPOLL meaningful? If not why waste everyone's time with idiotic questions. It took an anonymous blogger and a business week article to bring issues to the front.
Senior management isn't going to screw up their OHI score by asking honest questions on the MS Poll.
It is designed to eliminate OHI as a source of a low review score.
I remember management telling us every year how wonderful people think the work environment in Redmond is.
If you don't think everything is OK, they say you are a "negative" person or you are not a "team player".
If Ballmer was serious about accountability, upper management in Windows would be gone. Brianv and Jimall have been in charge "managing" the biggest software disaster this company has ever witnessed. They did not write the code, BUT the oversaw the schedule, set priorities, and in the end watched as their teams failed to deliver over and over again. They had countless opportunities to cut their losses and features early but instead they just waited ond hoped for the best.
BrianV was considered a "golden child" for shipping Exchange Server. Chances are he got the job in Windows because the VP at the time wasn't getting the job done.
Did BrianV get the job done? Nope.
Who have they got to replace him?
They didn't do the work they needed up front to make it easier for teams to work without stepping on each other's toes. Only when faced with a massive failure did Windows management give the layering work a higher priority. Why didn't BrianV have this done sooner?
http://blogs.msdn.com/larryosterman/archive/2005/08/23/455193.aspx
"The architectural layering team went through the entire Windows product and identified every single instance of a layering violation. They then went to each of the teams, in turn and asked them to resolve their dependencies (either by changing their code (good) or by explaining why their code matches the plugin pattern (also good), or by explaining the process by which their component will change to remove the dependency (not good, because it means that the dependency is still present)). For this release, they weren't able to deal with all the existing problems, but at least they are preventing new ones from being introduced. And, since there's a roadmap for the future, we can rely on the fact that things will get better in the future."
http://schestowitz.com/Weblog/archives/2005/09/24/vista-re-built-from-scratch/
REDMOND, Wash. – Jim Allchin, a senior Microsoft Corp. executive, walked into Bill Gates’s office here one day in July last year to deliver a bombshell about the next generation of Microsoft Windows.
“It’s not going to work,” Mr. Allchin says he told the Microsoft chairman. The new version, code-named Longhorn, was so complex its writers would never be able to make it run properly.
The news got even worse: Longhorn was irredeemable because Microsoft engineers were building it just as they had always built software. Throughout its history, Microsoft had let thousands of programmers each produce their own piece of computer code, then stitched it together into one sprawling program. Now, Mr. Allchin argued, the jig was up. Microsoft needed to start over.
>>Phenomenal in Japan?
Umm, he's talking about the launch in general.
>Every product needs Level 63 testers. Wouldn't they be much better at determining what should be tested the most? Wouldn't they also be better at setting up environments and debugging failures.
Even that is too limited a view.
At MS, QA is actually a misnomer; what the bulk of testers at MS do (that I've seen) is actually quality control, basically the detection of defects in the compiled product.
Higher level technical SDE/Ts should be able to do true quality assurance, the prevention of defects, by operating at the code and architecture quality level; for example, pointing out where the code is poorly structured/"brittle" or where the design can be changed to prevent security gaps or to make QC less painful.
Truthfully, MS test really is only a QC organization and notion of QA is not valued by the ordinary test leadership nor welcomed by ordinary dev leadership. Hopefully, the new technical track for the test CSPs signals awareness of this problem.
>QA IS THE MOST IMPORTANT PART OF SOFTWARE DEVELOPMENT
Amazing. I thought that kind of hubris was limited to devs.
Test (or QC, rather) does not determine the original quality of the design or code nor does it fix defects found. Test also does not make the final go/no-go decision (or should not, unless we want responsibility for any quality issues that we had no part in causing in the first place!).
All we really can do is evaluate the product, report quality issues and suffer quietly in endless test pass cycles at hands of the failures of the PM and dev teams. (At least, until we can transform test from QC to QA+QC and shoulder part of the responsibility for the creation of the product ourselves.)
"Microsoft should be paying for people to take MCSE\MCSA exams. "
They do pay for them. You can take the exams in Bldg 25 and Sammamish.
The problem is perception of the exams in the real world. Most real IT people know that anyone can read a book and pass a test. The exams really don't deal with real world experiences.
How many people are "Team Members" at Microsoft. From my experience we are all "Individual Contributers".
Big difference between the two.
The testers who are currently transitioning to SDET roles are really struggling to learn how to code, write automation AND STILL carry out the manual test needs. They need help from experienced people who don't also have lead responsibilities.
Of course, the foregoing supposes that there is a genuine will to make the STE->SDET transition a success, rather than just fire the STEs and replace them with existing SDETs.
Too bad the review system pits employees against each other. Otherwise, the STE's would probably get the help they need.
As it stands now, they would need the budget for every STE to take a programming course at a community college or university extension programme.
The answer we got regarding STE's being "laid off" was that Microsoft tried to help them find new positions and then got rid of the STE's that did not have the technical skills to find a different position.
If you aren't technical enough for a company wide policy of replacing STE's with those that can develop software, the company "helping" you look for a position amounts to nothing.
QA and PM need to be just as technical as DEV. Frequently, their jobs are harder and more technically complicated than that of DEV.
PM more technically complicated? No. More politically complicated? Yes.
As it stands now, DEV often has to write specs for PM or hand hold them through the process.
Something you get credit for as a DEV on your review? No.
After PM takes over the spec, it rarely gets updated to reflect how the product was actually implemented. Everybody gets too busy doing something else. Senior management has an effort to have DEV's and PM write more documentation so they can more easily replace DEV's.
Truthfully, MS test really is only a QC organization and notion of QA is not valued by the ordinary test leadership nor welcomed by ordinary dev leadership. Hopefully, the new technical track for the test CSPs signals awareness of this problem.
At other companies, a code review is done by the QA department and the code does not get checked in if it does not pass.
At Microsoft, a code review is done by several developers working on the project. The comments made are often seen as suggestions. However, the code does have to pass the existing pre-checkin tests. You would have to determine how much code coverage you got on the new code with the existing tests to know if that is a good idea. As it stands now, running the tools to get code coverage numbers is a relatively cumbersome process. Later, the tests are updated periodically to get more code coverage on the product.
>In our group, management moved to having developers write the automation code. How many testers would write themselves out of a job?
It never, ever ceases to surprise me how little developers understand about testing.
Test automation can't ever put testers out of a job because the tester's job is never truly done; they merely run out of time to do it. The number of conditionals in a program result in a combinatorial explosion, meaning that it's impossible to test everything. There is always something more to check.
Moreover, automated testing has distinct limitations. It only checks what you thought of to check. Cem Kaner expressed it very well:
"When you perform a test by hand, you bring to bear the entire range of human capabilities, You can improvise new tests or notice things that you did not or could not have anticipated. Test automation is a faint, tinny echo of that rich intellectual process. That's why it's nonsensical to talk about automated tests as if they were automated human testing." - "Lessons Learned in Software Testing", Kaner, et. al.
Kaner also cites a study that found that only about 15 percent of bugs are detected by automated tests.
"Phenomenal in Japan? Seriously, how did this guy become CEO and what value does he add? What message does this send to Microsoft employees and customers about who you can trust?"
Read the interview and had the same reaction. Xbox 360 in Japan sold less units over an Xmas season than the original Xbox did in a non Xmas season. That hardly sounds like success to me. And how about:
"We've competed very effectively for the last four years with open source, gaining share basically in all businesses in which we compete with open source."
Huh? Competed effectively is itself a stretch but gained share in every business? What about the inroads made by Linux Server, Linux in the OEM channel, Apache, Firefox, etc?
or this:
"We've got to have a search that people clearly say is more likely to give good results. Right now, I'd say we're pretty comparable. People do blind taste tests, without brand and knowing in what URL they're typing, and you just look at the results. We're right in there."
Pretty Comparable? About the only comparison is that they're both search engines. MSN still isn't even at YHOO's level far less GOOG's - and this despite mgt's repeated mouthing off that it would be. Here's an idea, shut up and let your actions speak for you.
or:
"I'm not sure we're going to beat Sony in Japan this generation. Maybe. But we're going to beat Sony (overall). We're off to a very good start. We're on track to sell between 4.5 (million) to 5.5 million units by the end of our fiscal year (June 30), which is great."
Beat SONY overall? Sure Steve, just like you were going to ship 3M units in the first 90 days and ended up shipping about 1/3rd of that. Just a small miss there that impacts Xbox's chances strategically as well as no doubt hurting Q2 revenue numbers overall. And of course, no accountability doled out.
or this:
"If we spent more capital, we could grow faster. We could buy businesses and grow faster if people like. Instead, we've been returning capital to our shareholders and are going to make sure that it's being employed in the most productive fashion possible."
Huh? Why not spend that capital then? Why not in '01 when attractive targets like YHOO or SAP were selling for pennies on the dollar and MSFT's war chest was double even it current massive size? Is he growth averse? And as regards "returning" capital to shareholders, we all know what total bs that is. Hey Steve, have shares+equivalents EVER gone down despite $40B in cumulative buybacks since 00? If not, then how did those $ get "returned" to us?
Of course there is a silver lining:
"That was a key of our reorganization. (We wanted) fewer decisions coming to me."
Given the above, that's very wise indeed. Although, since the initial buzz of activity which saw H&E create even more senior jobs, we've heard/seen nothing. Was no one redundant/ineffective? Pllllease. Seriously, Ballmer is either clueless or outright lying and either way, his track record is one of MSFT getting weaker not stronger. The minimum expectation for a company with MSFT's presence/resources, is a CEO who gets it, takes and doles out accountability where required, invests in the business vs hording ridiculous levels of cash, and is honest enough to candidly comment about relative share gains, MSN's still significant gap in functionality vs GOOG and where the capital returned to shareholders has really gone, Xbox launch problems, etc. Time for a change - and no, I don't mean bring back Gates or god forbid give Raikes a shot. An outsider is what is required.
"Test (or QC, rather) does not determine the original quality of the design or code nor does it fix defects found. Test also does not make the final go/no-go decision (or should not, unless we want responsibility for any quality issues that we had no part in causing in the first place!).
"
Perhaps not, but test can yell and scream and file a jillion bugs until a completely broken component is scrapped and a new team is hired to design and build something that works.
...except that doing your job as test means delivering bad news which goes against the good-old-boy system and translates into a short career.
Now, we have test managers who view their job as "shipping on time even if the product sucks" instead of "preventing the rest of the team from shipping something that sucks".
WHAT you ship matters much more than WHEN you ship. If you ship crap, consumers look at you and see crap for the next 10 years no matter what you have done since.
Why are we renaming "help" in Vista? Partly because the help in NT4 was so usless to non-existant that customers still assumed that it wouldn't help them even in W2k3.
Amazing. I thought that kind of hubris was limited to devs.
Test (or QC, rather) does not determine the original quality of the design or code nor does it fix defects found. Test also does not make the final go/no-go decision (or should not, unless we want responsibility for any quality issues that we had no part in causing in the first place!).
So let a bunch of devs create what they say is a perfect architecture and prgram flow, don't send it to QA ( by QA I mean test) and ship it to customers. IF it doesn't blow up, maybe you have a point. Wait, I have seen what happens when QA is not performed by high level STEs.
Besides, if QA says it's not ready and you ship, there will be problems for the users. I have seeen that too.
All in all my statement still stands.
They do pay for them. You can take the exams in Bldg 25 and Sammamish.
Well, in 2000-2002 they WERE NOT PAYING FOR NOR REIMBURSING. At least, that's what my team said.
> Senior management isn't going to screw up their OHI score by asking honest questions on the MS Poll.
Eliminate MSPOLL and fire half the HR Managers employed to craft e-mail and interpret the POLL numbers.
> Instead, we've been returning capital to our shareholders and are going to make sure that it's being employed in the most productive fashion possible
Capital is returned to the partners so they can deploy it most efficiently in their personal bank accounts.
In our group, management moved to having developers write the automation code. How many testers would write themselves out of a job?
Much of the existing automation code written by testers had to be run several times just to get the tests to pass. Each run took several hours to complete. That code had to be rewritten.
It is not possible to automate yourself out of a job. For on, look at the changes that occur every time a new Windows version comes out or everytime DirectX has a new version WMI has changed, someone would need to update any automation code.
STEs should be responsible for environments, not code. They shoul dhandle customer issues with support. They should do End to End usability testing. They should work with SDETs to improve test interfaces an depth of testing. There are plenty of software companies whose employees suffer because of a lack of structured QA, especially Internet companies.
That was one of the things caused the bubble to burst in my opinion.
"...as well as no doubt hurting Q2 revenue numbers overall."
Wrong. The XBox miss is a net positive for Q2 numbers because XBox hardware is sold at a loss. Fewer boxes sold = less money lost.
>IF it doesn't blow up, maybe you have a point.
I've seen plenty of thing blow up after test worked it over. Are you suggesting that the test team is somehow responsible for causing bugs in the code?
>Besides, if QA says it's not ready and you ship, there will be problems for the users.
Well, duh. It's a conscious tradeoff made by PM. Are you suggesting that the resulting problems are somehow the test team's responsibility anyway?
Frankly, I have no idea what your objection is or what point you're attempting to make.
>All in all my statement still stands.
All in all, your statement about QA being more important than development has been thoroughly addressed and refuted by others in this thread. Perhaps you should respond to them instead.
...except that doing your job as test means delivering bad news which goes against the good-old-boy system and translates into a short career.
AMEN!
Microsoft should be paying for people to take MCSE\MCSA exams. How could Windows have the fewest numbers of MCSEs?
MCSE is designed to be a test of your knowledge of how to design and administer a system of machines running Microsoft products. I believe that there is a low correlation between what you need to pass an MCSE and what you need to be a good SDE or SDET at Microsoft. Sure, the courses may help you learn more about the system, which makes you a better tester, but most people Microsoft hires into dev groups can pick that stuff up more quickly than a course. And having the title (taking the exam) is pointless in a dev group.
That said, someone else noted that you can take them for free, I don't know about that, but I have taken several "beta" exams for free, which counted (I'm an MCP and no one knows or cares).
Two comments on testing at MS, one on Quality Assurance, and a joking metaphor:
Caveat: I'm probably a little jaded.
One - I was at a barbecue in 1999, meeting new hires. I talked with two testers in the I.E. team. One tester was very upset - she didn't know what to do and wanted some advice. Her test lead told her to stop filing bugs. Not because they weren't real bugs. But because they were "making the team look bad." Crashes because of URLS nobody would ever use were not a problem.
The i.e. team (probably as far back as when it was created) has always know how hacktacular it was. All these security flaws should never have been a surprise.
Yes, I use firefox.
Two:
The STE -> SDET position at Microsoft is really about shortening the time it takes to tell management how soon we can ship. They don't want to improve quality. I'll say that again. They don't want to improve quality. After 5 years there I realized that my job was to help get to the absolute minimum bar possible to ship the product as fast as humanly possible. Test Automation instead of Testing is about making sure the demo scenario works so the salesmen can make the sale.
Mostly because management has given up on more than the demo working - but will never voice that opinion, instead cheering about our accomplishments.
Qualty Assurance:
If management really wants to improve the quality of the product, it will increase accountability by making everyone work with shipped dependencies instead of this moving dependency train. (The comment above about Windows gives me a little hope.)
This is something that Steve McConnell promoted way back in the day, and could actually help fix the quality problem by confining bugs to their specific teams.
When I left someone was writing a brand new component using a beta C++ compiler and a beta .net runtime talking to a beta SQL server and they ran into bug that made no sense.
It ended up being a "misunderstanding" between the .net runtime team and the C++ team that showed up in the beta SQL server. The change in the Runtime was made three months before and the C++ change was made two months before.
And the guy that made the fix couldn't check in for two weeks because someone in India's checkin suites stopped passing for some completely different reason - and nobody took responsibility for fixing it.
How the hell any developer ever gets work done at Microsoft I have no freaking idea. I was upset because I had one more dependency than my developers - if the developer's code didn't work you couldn't say that my test code was done - and my dependency (the developer's code) was changing even when HE didn't want it to change.
In my opinion this is why they want automated testing at microsoft - any manual testing that proved something worked MUST be redone every checkin because there are so many moving dependencies and nobody knows what the interdependencies are.
What this does however, is move ship cycles out by YEARS and promote the waterfall model. And cause war teams full of managers that don't know the code to start scrutinizing every damn line of code checked in for a year and a half before the product ships, wasting developer time because the team has to make a freaking powerpoint sales pitch just to make a freaking bugfix.
Every added feature is adding N^2 meetings, complexity, testcases, bugs, etc. It is not a surprise to me that MSFT can't ship much of quality on any sort of reasonable schedule with any sort of interesting, new featureset lately.
Of all the things I see where people call out management on this board, I have yet to see someone call out management for this ridiculous mess of moving cross team, cross feature, cross everything dependencies.
Metaphor:
As an SDET at Microsoft my job seemed to be driving over a bridge that is being built from both ends and in the middle across a pool of quicksand during a hurricane.
Telling people they're building in quicksand will get you blacklisted or looked at like you're crazy. Everyone complains about the rain, but nobody talks about how the wind keeps blowing everything over. What they want from you is trendlines on graphs showing that the concrete and the cabling is secure and a lot of entered bugs about how the tensile strengths of the cabling, places where the paint is mismatched, or the concrete is chipped. You alternate between snickering and sobbing when the middle of the bridge sinks another foot. Every once in a while you watch them lower both ends of the bridge to match the middle, and you yell uselessly that you think the ends are pointing in different directions.
The managers on the left end of the bridge sees lots of utility in adding the ability to raise the bridge so that ships can pass under it - and they add all the pieces for the left end, even though the right end doesn't think it's a good idea, and won't do it. The rising star PM on the right end wants to add a traffic circle so that people who enter the bridge but want to turn around can do so and has convinced everyone of his genius. Strangely, competitors have added traffic circles in front of the bridge for years, but it is considered genius to have a traffic circle in the middle of a bridge across the water. Many articles are written about this.
The people making decisions about the bridge can only look at it with a telescope or a microscope, and the only people that are listened to are those that say the bridge is fine.
Every three months they reorganize the people making the gantt charts showing the bridge's progress, and after four years, they redirect all the traffic signs that were leading to the new bridge to the old bridge and paint the old bridge a diffent color. The new bridge is covered with tarp and dirt, and they plant trees on it to cover up that it ever existed.
Everyone declares the bridge to be a wild success, marketing posters are placed everywhere, and all the partners get a two million dollar bonus.
Since everyone needs to go to work, everyone takes the only bridge to downtown, grumbling about the toll.
Thankfully, nobody dies.
"Later, the tests are updated periodically to get more code coverage on the product."
Yeah, and everyone knows that 100% code coverage = 100% bug free.
"The answer we got regarding STE's being "laid off" was that Microsoft tried to help them find new positions and then got rid of the STE's that did not have the technical skills to find a different position."
Sure, they were allowed to interview and look for internal jobs for a few months. According to a dev that was in one of those loops, the interviews required technical programming skills so high that 20% of the current DEVS wouldn't have survived the loop.
I don't know about elsewhere, but inside Windows Server, we had a few people with a STE title who wrote major C++ reskit tools caught up in that layoff because their manager had written their job description to require less than 50% of their time to be spent writing code.
Many were so pissed off that they left the company and went to work for F5, Amazon, or other local companies rather than try to interview for their own job again. I wouldn't be surprised if they start contributing to linux in a year.
Moreover, automated testing has distinct limitations. It only checks what you thought of to check. Cem Kaner expressed it very well:
"When you perform a test by hand, you bring to bear the entire range of human capabilities, You can improvise new tests or notice things that you did not or could not have anticipated. Test automation is a faint, tinny echo of that rich intellectual process. That's why it's nonsensical to talk about automated tests as if they were automated human testing." - "Lessons Learned in Software Testing", Kaner, et. al.
You still need to automate the manual tests you come up with to run again on the next build or before a checkin. Otherwise, after a while, you'll never have time to come up with new tests.
I get the impression senior management is pushing testing on developers and beta testers.
Let's say that developers are forced to stay an extra hour a day to be able to handle all these extra responsibilities.
1 hour testing x 5 days x 4 weeks x 8000 developers / (40 hours per week per developer x 4 weeks) = effectively an extra 1000 FTE's (minus the burn out from all the extra work)
Will it replace full time testing? No.
It is just another manipulation of employees by the company to save money.
The same with the review system. You get trended down on your review score and you are told that if you work harder maybe you will win the review lottery and get a higher score next time. More free work for Microsoft from their employees.
It works even better on people who deserve a higher review score because they've had a taste of the rewards of getting a higher score and the company gets more work out of the employees they consider to be of higher quality.
Management gets a bigger bonus for squeezing more out of you and you get a speech about how you have an opportunity to make an impact on the world by working at Microsoft.
It works better on the young. When you have a family, you get a phone call reminding you about how much of your life you are giving up for the "revolution".
There's one born every minute and a whole new crop to exploit every year.
Have any former Microsoft STE's and SDET's ever formed a company to offer QA services to other companies in the area?
There are a few companies in the area that provide testing services.
Bug hunter persevered through tech's downturn
There has been an uptick in demand for software testing and companies that might have sent the work overseas are now rethinking that strategy, said Rick Carr, who runs DCB Software Testing, a small company in Seattle.
"Wrong. The XBox miss is a net positive for Q2 numbers because XBox hardware is sold at a loss. Fewer boxes sold = less money lost."
Right. Note note the explicit reference to "revenue" numbers not earnings. Selling less Xbox's may help the latter. Analysts are predicting perhaps .01 upside - though I personally think that may get erased due to the likely substantial incremental costs that went into responding to/replacing defective units. However, top line will be affected negatively which isn't good generally and in particularly not in a Q when mgt was forecasting a return (finally)to double-digit growth. Then again, as we're seeing with Xbox 360, forecasts at MSFT - like partner accountability - have a unique and elusive/ethereal meaning.
All in all, your statement about QA being more important than development has been thoroughly addressed and refuted by others in this thread. Perhaps you should respond to them instead.
I have refued their claims. Even Joel On Software agrees that you HAVE TO HAVE 1 tester for every two devs.
Cem Kaner would also agree.
Of course someone has to wite the code, but without test the code is 75% likely to fail.
If your test team is not finding the obvious usability or data loss bugs, the problem is your leads not the concept of QA. I've seen too many leads who have no idea how to structure test plans or instruct testers.
This will not change with using SDETs. Most SDETs are devs and don't have the mentality to test thoroughly.
Making devs do MCSE exams is about the dumbest thing I've heard. All it would achieve is in slipping ship dates. It's like making a Boeing wheel engineer pass a 747 flight exam. The pilot doesn't know (or need to know) the inner workings of the wheel, nor does the engineer need to know how to fly the plane.
My increasing feeling is that TheKhalif is to Mini what Christopher Coulter is to Scoble.
It's all about the signal to noise ratio... Some day we'll get an RSS feed enabling you to blank out the usual suspect comments. But right now, I'd give my lunch for a feed on blogger to blank out comments I've read before. Deja vu is a very common experience here...
"As an SDET at Microsoft my job seemed to be driving over a bridge that is being built from both ends and in the middle across a pool of quicksand during a hurricane."
AMEN!
When I was a STE at MSFT, I described my job as being a building inspector where every day I walked into a skyscraper and checked to see if the faucets produced water, glue, oil, or thumbtacks, checked to see if the floors and walls still existed, etc. and made up a daily list of what wasn't right. Every night, the architect changed the designs, the construction guys re-routed wires, floors, pipes, and even moved buildings around.
I could come in to work and find the city painted blue, upside down, and 220 volts hooked up to every doorknob...and my manager only wanted to know how long it would be until my building matched the blueprint.
I am amazed every time something is shipped. It is truly a miracle that it even ships. After that, who cares about quality.
It always seemed to me that my manager wanted me to write more autiomation to get me to stop filing so many bugs so that his numbers would look better.
"...as well as no doubt hurting Q2 revenue numbers overall."
Wrong. The XBox miss is a net positive for Q2 numbers because XBox hardware is sold at a loss. Fewer boxes sold = less money lost.
Jeez, by that logic, they shouldn't sell any Xboxes at all!
Here is how it works: The more they sell, the less each box costs because of economies of scale. If they ramp up like they're supposed to, they can actually get a profit on each box (don't hold your breath though). If they just dribble them out like they're doing now, the loss per box is *maximized*. Touting less sold=less loss as a shareholder benefit sounds like a desperate attempt to put a good face on a badly bungled launch of one of the company's great pipeline hopes for 2006.
NPD reported today that about 600 thousand 360s have been sold. That means they will massively miss the 90 day target of around 3 million. So much for first mover strategic advantage. Unbelievable that Ballmer said the launch was "phenomenal". The sad fact is that Microsoft's best hope for the 360 is that Sony manages to mess up even worse with PS3.
Either Microsoft needs to sell *a lot more* or sell *zero* and get out of the business. Bottom line: they need to do something drastically different from the current course which is maximizing loss per box while minimizing the benefit of launching first.
>I have refued their claims. Even Joel On Software agrees that you HAVE TO HAVE 1 tester for every two devs.
Whatever. Given that that your statement has nothing to do with what I said, I really have no response to make.
>Of course someone has to wite the code, but without test the code is 75% likely to fail.
There's no need to project your own limitations onto others.
re: XBox 360, you're almost there.
Repeat after me: "attach rate"
When an xbox is sold, it's the attach rate that matters. That is, the number of games that are purchased to go with it. The number of games that are purchased over the lifetime of ownership. Repeat customers. That's where the bloody money is, and so you want to sell as many xboxes as you can because the games will follow.
>You still need to automate the manual tests you come up with to run again on the next build or before a checkin. Otherwise, after a while, you'll never have time to come up with new tests.
Indeed, that is the very point of what I wrote: automated tests free testers to do more testing, it does not replace them, as the developer who posted elsewhere seems to think.
I think that Bill Gates and Steve Ballmer really owe Mini-Microsoft a GIANT THANK YOU for providing a forum to bring these issues to their attention because I don't think they would have ever known about them otherwise.
Bill/Steve, some advice if you will condescend to read it:
(1) Tell your attorneys to shut up. I am sure that your attorneys advised you to implement the stack rank system because it provides you the opportunity to rate an employee without giving any real feedback. However, managers can and do abuse this system, and it reduces morale if members of a great team get penalized for only doing things right. Rather than try to reduce your risk of liability to zero at the advice of your attorneys, understand that it is only by taking risks (if they are the right ones) that you reap rewards.
(2) Spend less time asking HR for advice on how to solve this problem. I am sure that the people in HR are well-intentioned, but the stuff coming out of HR around career development and other "fuzzy" stuff does not help employees whatsoever if they have a tyrant of a manager, can't speak the truth without retaliation, or feel like they are working in survival mode.
(3) Spend more time talking to employees in an open, constructive manner. You really are much too old to act the way that you do sometimes, and you need to understand that your behavior sets the tone for your executives and managers further down the chain. There is an "us" vs. "them" mentality at Microsoft, mostly because you let it happen.
"Making devs do MCSE exams is about the dumbest thing I've heard. All it would achieve is in slipping ship dates. It's like making a Boeing wheel engineer pass a 747 flight exam. The pilot doesn't know (or need to know) the inner workings of the wheel, nor does the engineer need to know how to fly the plane."
I think that the MCSE suggestion was that TEST be encouraged to get certified so that they would know what USERS are expecting.
"I think that the MCSE suggestion was that TEST be encouraged to get certified so that they would know what USERS are expecting."
BS, most of TEST is SDE/T anyway. Only the non-technical staff i.e. PM can derive any value from an MCSE. Since when does TEST meet customers or come up with feature specs anyway.
TEST at MS gets a bad rap because the hiring standards are not uniform. In some groups an SDET means a STE who can code a bit. In other groups an SDET is an SDE who can also test a bit. Big difference between the two.
If the second definition is true and management now thinks that SDETs are what's required, then why are we hiring SDETs? Hire SDEs and send them to a week long course on testing.
"...why are we hiring SDETs? Hire SDEs and send them to a week long course on testing."
1) STEs and SDETs cost between 50% and 80% of SDEs.
2) Testing, like coding, requires a specialized set of skills and you can't go from zero to adequate in either after a week long course.
(external to MSFT) I have watched several developers spend months trying to do testing and still miss the most obvious of show-stopper bugs. I also know a few developers who are excellent at testing even their own code.
The comments are sort of getting into a back-and-forth unproductive state. Just when I was thinking about a new post of dev vs. pm vs. test.
TheKhalif: now that you have your own blog back and running, I recommend you follow-up there regarding your thoughts and continue the conversation in your comment stream. Feel free to post the URL here should you decide to do so.
Or if you link directly to this particular post, it should automagically show up at the bottom.
Beyond the bickering, we might be able to learn something here yet.
There have been good comments about:
Shipping faster vs. increasing integration.
Shipping "on time" vs. shipping "the right product".
"Individual Contributors" vs. "Teams".
There are also good comments around projectmanagement metrics, inconsistent hiring standards, and individual talents.
There are a lot of hard feelings because our system of groups (they are not teams) pits dev against test and pm, pits test against pm and dev, pits pm against dev, test, doc, and other teams, and pits everyone against the clock and against management.
We need a topic discussing how we move from ICs to Team Members, but that discussion will also need to mention the pm vs. dev vs. test problems.
TheKhalif: now that you have your own blog back and running, I recommend you follow-up there regarding your thoughts and continue the conversation in your comment stream. Feel free to post the URL here should you decide to do so.
There is more to my blog than just the problems with MS mgmt. I try to contribute productive input and if "trolling no-birds" attempt to dilute the seriousness of my posts, then perhaps the key is to not instigate by posting things which you already know will cause me to react as I do. Especially since you can post whatever you want and usually have something special to say about my posts. Truthfully, I am just using your blog for my own purposes.
Do with my posts as you will. I'll be around.
This isn't dev vs pm vs test, but I will understand completely if it is moderated away and never appears.
I'm an SDET in the Windows division. Even within our division there are no really consistent rules applied to determine what an STE and an SDET do differently. On some teams there are STEs doing the testing while SDETs write test harnesses. On other teams (like mine) both STEs and SDETs are responsible for both writing tests (yes, often compiled code - somone always has to try to make that distiction for some reason) and for doing the testing. There are still other teams that have SDETs writing the compiled test code and having STEs mostly own scripts for their automation. I'm sure there are other variations elsewhere within the company that I'm unaware of.
When my job title changed from STE to SDET I was doing exactly the same job as I had before. I just had an additional letter in my title.
My point is that any broad generalization about the differences between STEs and SDETs isn't going to hold up to scrutiny. Specific examples, maybe, but not generalizations.
Sure, they were allowed to interview and look for internal jobs for a few months. According to a dev that was in one of those loops, the interviews required technical programming skills so high that 20% of the current DEVS wouldn't have survived the loop.
20% of the current devs at Microsoft are crap, and could be let go without too much impact on the bottom line, aside from the fact that we'd probably end up shipping less buggy software. Hard dev interviews are a GOOD thing, not a bad thing.
I don't know about elsewhere, but inside Windows Server, we had a few people with a STE title who wrote major C++ reskit tools caught up in that layoff because their manager had written their job description to require less than 50% of their time to be spent writing code.
So, presumably they were given a chance to interview as well? Perhaps writing "major C++ reskit tools" didn't make them strong enough developers to join the SDE/T or SDE teams. I don't see the problem here.
"Testing, like coding, requires a specialized set of skills and you can't go from zero to adequate in either after a week long course."
Testing is mostly process mixed with technical skills - there are exactly 3 chapters in my '92 Software Engineering textbook about QA and they cover all of what is done including the things mentioned here like model based, coverage / complexity measurement, formal reviews and what not.
We would be a lot more agile if we only had SDEs who either code or test as the project called for at that time. Sure, you would need separate orgs starting at the project lead(?) level to avoid any conflict of interest, but the technical resources themselves would have uniform skills and be easily replacable.
Wow, this is a serioulsy busy thread :) I wasn't going to post anything and lurk as I normally do but 2 comments in particular had me fired up.
RE: google vs. MSN Search
Pretty Comparable? About the only comparison is that they're both search engines. MSN still isn't even at YHOO's level far less GOOG's - and this despite mgt's repeated mouthing off that it would be. Here's an idea, shut up and let your actions speak for you.
In fact it is true that MSN Search is "pretty" comparable (as vague as that metric is). I am inherently biased against MSN search as I've been using Google for years. I'm a big fan of Google products, despite being a "dedicated" Microsoft and MSN employee. I realized that I have a huge bias against our own search engine, until recently (read: last month) i decided to make MSN search my default search engine and exclusively use it as my first search engine. Results? MSN search does a great job -- still not as good as google, but it is closing the gap quickly. More quickly than people want to admit.
Advice: check your bias at the door and give MSN search its fair shake. And no, just because you tried it 6 months ago doesn't mean it's not better now. They are doing great work over in that team, and if our own employees won't support their effort we're in big trouble.
RE: Xbox360, losses and attach rate
Repeat after me: "attach rate"
When an xbox is sold, it's the attach rate that matters. That is, the number of games that are purchased to go with it. The number of games that are purchased over the lifetime of ownership. Repeat customers. That's where the bloody money is, and so you want to sell as many xboxes as you can because the games will follow.
This is spot on. It's not about the individual losses on the console, it's about the number of games and accessories we sell per-console. The numbers I've seen are a 4-to-1 attach rate for Xbox360 compared to previous 2.4-to-1 with original Xbox. This was just for games. Accesories attach rate are 3-to-1.
I'm not sure how much we make in licensing for each game, but nearly doubling our attach rate is surely a good thing. Isn't the majority of our money made on licensing anyways?
"We would be a lot more agile if we only had SDEs who either code or test as the project called for at that time."
There is at least one team doing exactly that right now. Obviously, it's an experiment. If it works out you'll probably see more teams operating that way.
At the risk of sounding like a troll, there seem to be a lot of commenters who aren't aware of what other teams are doing. I was the same poster warning against generalizations about STE vs SDET. I'm not taking this particular comment to task, but instead pointing out what I see as a trend. I wonder if there is an underlying problem with people working in silos. On the other hand, maybe this is something that management or HR should look into - letting 'softies know operational details about other teams' work. I have a feeling that might be a small step toward tearing down the fiefdoms in the long run, too.
Testing is mostly process mixed with technical skills - there are exactly 3 chapters in my '92 Software Engineering textbook about QA and they cover all of what is done including the things mentioned here like model based, coverage / complexity measurement, formal reviews and what not.
That's not very surprising, considering what the state of QA was in 1992 ("you peons, get back to steerage class with you").
I would think that seeing the horrible problems with security and so forth of the last 14 years might have changed some minds on the importance of QA since then.
We would be a lot more agile if we only had SDEs who either code or test as the project called for at that time. Sure, you would need separate orgs starting at the project lead(?) level to avoid any conflict of interest, but the technical resources themselves would have uniform skills and be easily replacable.
Again, this assumes dev skills are synonymous with test skills. They aren't. Creatively black-hatting code or having empathy with users isn't the same as writing code.
Are technical skills good things to have for a tester? Duh- that should be obvious (a tester who understands code has more ways to approach the task of testing). But I regularly casme up with "Hey, did you think about X" questions to my devs that they had never thought of- because they focused in on small domains in their expertise, where I had to range all over the goddamned OS and userland in test. Some people are better at this than others are.
Sure, they were allowed to interview and look for internal jobs for a few months. According to a dev that was in one of those loops, the interviews required technical programming skills so high that 20% of the current DEVS wouldn't have survived the loop.
20% of the current devs at Microsoft are crap, and could be let go without too much impact on the bottom line, aside from the fact that we'd probably end up shipping less buggy software. Hard dev interviews are a GOOD thing, not a bad thing.
What bothers people is applying standards unevenly.
If these same people left Microsoft and applied as DEVs, the original author is saying that they would pass (using your 20% number as a place holder for the real percentage which is unknown since there is no data to back it up).
As long as all of the current DEVs have to pass the same new standard, there would not be a problem except for people increasing the difficulty of the questions with the purpose of just getting rid of people they don't like or have a personal bias against.
As it stands now, the sands can shift in a series of interviews depending on whether or not the people interviewing you want you or don't want you around for whatever reason (not necessarily technical).
The process would be fair if the questions were set in advance.
In 1999, the extent of education in how to test software in my Computer Science program was a 1 page handout and 30 minute lecture.
In 2000, I spent 9 months performing and learning the ins and outs of globalization, localization, accessibility, scalability, reliability, performance, security, code coverage, deployment, uplevel/downlevel, interoperability, media verification, and a few more "quality areas".
Later, we all learned more about security testing. Later yet, we inherited even more quality gates.
Somehow, standard textbooks don't talk about needing to run polycheck against your resource files so that you can rip out geo-politically sensitive text so that our sales reps in China don't get thrown in prison.
Somehow, standard engineering textbooks don't talk about the US Government accessibility requirements for purchasing.
Somehow, standard engineering textbooks don't talk about techniques to use when trying to reproduce that elusive bug that you saw once and your dev can't fix unless you can repro it on their machine.
Or checking that your app works correctly with a right to left language and that the UI is mirrored correctly. Or checking the various color schemes, high contrast colors, and various font sizes.
I suspect that a lot of devs have little idea what testers do and vice versa. When dev and test and pm orgs go up several levels of management before they meet, there are a lot of ways each can screw the other and result in highly disfunctional groups.
For example, a dev and pm are taking in the hallway about a new feature.
PM: How long will it take to add X?
Dev: Only half a day to code, but I would guess about 3 days to test.
PM: Hmm, we hit code complete tomorrow and this will derail the test schedule. Let's do it anyway.
Equally, test can screw a product release by holding off on submitting critical bugs, or by playing "flip-flop the spec" alternating between conflicting bugs to give devs some busy work.
Or worse, test gets screwed on the schedule and doesn't have time to adequately test, takes shortcuts, and critical bugs go out the door and our reputation goes down even further in the eyes of our customers.
Now, let's talk about the test matrix for one of our previous release:
(free builds + release builds) *
(x86 + x64 + ia64) *
(home + pro + server + advanced server + blade server + datacenter server) *
(joined to a domain + workgroup) *
(long computername + short computername + german codepage chars + japanese UTF8 chars in the computername) *
(guest user + local user + domain user + local admin + special group membership) ... etc.
Already, we are at over a thousand configurations and we haven't even gotten into the features of the network management application. Clearly, there is a need for some automation, but who is going to write the automation when your SDE is busy trying to manually check a reasonable subset of tests so that you can ship?
And sure, you can focus on unit testing, but at some point we need to make a serious investment in adding complete test automation for all of those components that are sitting in maintainance mode.
You can't just automate the new stuff and assume that it will save you money when you have 10 layers of poorly tested dependencies below your new code.
I have a simple solution to the whole mess:
Once code complete is declared, if more than 5 bugs are found in their entire codebase or in the spec, the dev will be fired on the spot.
Once RTM is declared, if more than 5 bugs are found by a customer, the entire team (devs, test, pm, leads, and managers) will be fired.
That way, the whole team will view test as covering their butts and everyone will stop playing schedule-chicken.
In 1999, the extent of education in how to test software in my Computer Science program was a 1 page handout and 30 minute lecture.
In 2000, I spent 9 months performing and learning the ins and outs of globalization, localization, ...
I'm not sure what the point of your post is. If it's that testers have some kind of specialized knowledge that devs can not acquire, then you are mistaken. I'm a dev and I've had to learn about all of these things. If it's that computer science curriculums should change to teach more about, e.g., localization, then I think you're mistaken about that too. The point of a college education is to give you a broad foundation of knowledge. Surely you're not trying to say that college students should have to learn which Microsoft tool to run on which kinds of Microsoft files to produce a localized build of a Microsoft program.
My point is that any broad generalization about the differences between STEs and SDETs isn't going to hold up to scrutiny.
What about black-box vs. white box? Many testers I've met, regardless of ability to code, do not have the CS concepts to be able to narrow down issues beyond a basic level.
>Testing is mostly process mixed with technical skills - there are exactly 3 chapters in my '92 Software Engineering textbook about QA and they cover all of what is done including the things mentioned here like model based, coverage / complexity measurement, formal reviews and what not.
Answer me this: For skills that are so "trivial", the average dev here clearly lacks them and has no interest in getting them. Why? Not a week goes by without another embarrassing MS bug making tech headlines around the world, and here we have devs trumpeting their superiority to test instead of asking to learn how to catch their own mistakes. Why? Testing is so "easy", right?
I'm waiting.
So why don't we start coming up with a list of 68-80+ level folks who should go "spend more time with family"?
My quick list:
#1: Robert Wahabe. What has this guy ever shipped? Has more architects and DEs than anywhere else in the company, but absolutely nothing to show for it. But he was creative enough to convert his teams admins into PMs to up his diversity numbers!
#2 Pieter Knook. Make me a smartphone that actually is smart about making and receiving calls above anything else, then have it run whatever other apps.
#3 Jawad Khaki. I was so glad to escape the networking team. It is super to see that they still go off and do whatever they want. Way to hit Brian's december 31st date for being IN winmain - not on a slow boat towards it. And where is NAP? Take mr. ashida with you on the way out. FJ.
#4 Ya-Qin Zhang. Oh wait, too late for that one...
#5 Lynn Hill. I got this one from a friend, with the comment "pure overhead" and "always looking out for Lynn". We need more folks like this.
#6 Brian && Jim. Sorry, but someone has to be holding the bag for Longesthorn. Probably should add ole Steveo to the group as well, he is responsible for product development. oh wait, our board has no real say in anything.
I do have to say I'm more excited about working at MS now than I have been in the past 4-5 years. I used to make fun of redvest from the comforts of bldg 40 but MSN actually feels like Windows used to in the mid 90s - shipping shipping shipping. If we only had more parking...
Keep it real, mini!
I know this threadis trying to die bu I had to comment on one of the last posts:
have a simple solution to the whole mess:
Once code complete is declared, if more than 5 bugs are found in their entire codebase or in the spec, the dev will be fired on the spot.
Once RTM is declared, if more than 5 bugs are found by a customer, the entire team (devs, test, pm, leads, and managers) will be fired.
That way, the whole team will view test as covering their butts and everyone will stop playing schedule-chicken.
Before you say this you have to determine what a bug is. Is a bug a typo? Is it a data loss problem? Is it an off-by-one? Is it easily seen? You can't just fire people if they have bugs in 10,000 lines of code. According to the normal curve for bugs, 10,000 will have as many as 10 bugs.
Answer me this: For skills that are so "trivial", the average dev here clearly lacks them and has no interest in getting them. Why? Not a week goes by without another embarrassing MS bug making tech headlines ...
Devs can write solid code and test it themselves no problem. You're witnessing something else. Devs are rewarded for working on a lot of different features and finishing them ahead of schedule. There's a big incentive to write something that seems to work, throw it over the wall, and move on. Second, with 2% raises and dwindling benefits, why should _anybody_ do a good job at Microsoft? If somebody's doing a bad job, it doesn't mean he can't do a good job...
1) STEs and SDETs cost between 50% and 80% of SDEs.
Really? I could see that for STEs, but SDETs? I thought SDETs and SDEs were on the same pay scale, although I would expect most SDETs to not increase levels past 61.
I'm not sure how much we make in licensing for each game, but nearly doubling our attach rate is surely a good thing. Isn't the majority of our money made on licensing anyways?
Licensing and peripherals.
For competitors, gamepads and memory cards have great margins (make for < $8, sell for ~$40).
In the 18-month rush to get the first Xbox out the door, MS actually ended up designing a money-losing peripheral program.
On the bright side, Xbox 360's peripheral program is on par with competitors and profits are looking good. Especially considering greater than expected wireless gamepad attach rates.
>Devs can write solid code and test it themselves no problem.
I didn't ask whether they had the ability, I asked why they weren't. Re-read what I wrote.
>Devs are rewarded for working on a lot of different features and finishing them ahead of schedule. There's a big incentive to write something that seems to work, throw it over the wall, and move on.
I am reminded of a Muhammad Ali quote when he was asked about golfing: "I'm the best. I just haven't played yet".
While belittling those lesser than themselves already reflects poorly on the developers that do it, I suggest that such claims would be more credible had they learned the skills necessary and actually applied them in practice before presuming to tell others that their job is not as difficult.
Even better, devs could even stoop to help educate these benighted lesser beings so that they can help test code more effectively. Goodness, imagine that!
>Second, with 2% raises and dwindling benefits, why should _anybody_ do a good job at Microsoft? If somebody's doing a bad job, it doesn't mean he can't do a good job...
This attitude is utterly beyond contempt.
I am a professional. I always do the job I agreed upon to the best of my abilities, meager though they may be. If I do not like the terms of employment, I leave the job.
"#3 Jawad Khaki. I was so glad to escape the networking team. It is super to see that they still go off and do whatever they want. Way to hit Brian's december 31st date for being IN winmain - not on a slow boat towards it. And where is NAP? Take mr. ashida with you on the way out. FJ."
-
THey missed winmain and people have been screaming about broken networking in vista since b1 (and screaming about networking in windows since WFW).
How many DCRs were acccepted and how far did they push out "feature complete" to compensate for the overlapped services of networking? Mind you most of this stuff was around during LH Alpha
Good job
God, half past three in the morning and I'm only just home!
In the canteen this evening there were a bunch of young devs banging on, loud enough so that everyone could hear them, about their home-built systems they were running ripped off copies of OS X on.
Now, I'm not anal enough to worry about ripped off code: it's hacker culture, after all. the only way good software gets great is by people doing uforseen things with it, IMO. However, as a dev I worry about the kind of gungho, loose-canon attitude, that is typical of what these kids were displaying, and which I know their manager encourages (I've even heard it said they have a ripped off copy of OS X in the building to play with).
Are these guys more valable to Microsoft because of what they might do to us if we ever sacked them? I don't know, but because of the team they're on, I can tell you that they're probably at their most productive when they're dicking around with stolen code - and they have a manager that encourages them to take a "you can't touch us" attitude about that... as that was the same as getting anything useful done!
And it's f**ing nearly four AM and I'm still awake! Damn!
I am a professional. I always do the job I agreed upon to the best of my abilities, meager though they may be. If I do not like the terms of employment, I leave the job.
The message from HR is that people need reviews, raises, bonuses, and awards in order to be movtivated to do their jobs.
So when they cut raises, bonuses, and awards, the message must be that we don't have to work as hard.
Maybe employees would be more inclined to be "professionals" if HR didn't treat us like high school students with this review process.
>Maybe employees would be more inclined to be "professionals" if HR didn't treat us like high school students with this review process.
Bravo. Rarely does one see a person so proud of their lack of integrity as to openly argue for it.
May I remind you that one of the important aspects of professionalism lies in your treatment of others, not in their treatment of you. I would almost suggest reviewing the ACM Code of Conduct, but given what occurs when vampires are exposed to sunlight, that may not be the wisest course.
May I remind you that one of the important aspects of professionalism lies in your treatment of others, not in their treatment of you. I would almost suggest reviewing the ACM Code of Conduct, but given what occurs when vampires are exposed to sunlight, that may not be the wisest course
You seem to be excusing the conduct of management and then suggest that employees are supposed to do nothing about it.
I suggest you review employment law.
I'm not sure what the point of your post is. If it's that testers have some kind of specialized knowledge that devs can not acquire, then you are mistaken. I'm a dev and I've had to learn about all of these things.
I believe that the point is a work day only has so many hours and both jobs require a certain amount of knowledge and experience to be done properly.
A DEV can test but then they aren't writing the code needed to complete the product.
If you're going to do both jobs, you're either going to spend twice as much time at the office per day or something is going to suffer.
Since you took a DEV position, chances are you are going to spend more time writing code than testing it.
Testing your own code also sets up a conflict of interest. Are you going to file a bug report for every defect that you find in your code? We had a DEV lead that resolved all of his "defects" as "issues".
You've also only got a certain amount of time to keep your skills up to date.
If you think those who test the code you write are slack, take a test position and show everyone how it is done.
If that is too big of a step for you, tell your test manager that you believe you can do both jobs and have him assign you the test work for a set of features (not yours) in addition to your regular responsibilities for a few weeks.
Bravo. Rarely does one see a person so proud of their lack of integrity as to openly argue for it.
Well, aren't you the model employee, always operating at 100% peak efficiency, regardless of your morale or your enthusiasm or your job satisfaction. Never taking a minute from your work day to chat with a coworker or check a personal web site. Always rigorously testing every line of code you check in. Well, I'll call your manager and tell him to stop wasting morale budget on you. "Professional," whatever.
>You seem to be excusing the conduct of management and then suggest that employees are supposed to do nothing about it.
Quite the contrary. I think that the leadership at this company is floundering badly. However, that fails to excuse any misconduct by individuals. Dissatisfied people should do the honorable thing and end their employment.
(I include the link for the benefit of those who may not be familar with the word.)
>I suggest you review employment law.
Well then, lad, where's your lawsuit?
If there were any valid legal claims, they would already have been made.
>Well, aren't you the model employee, always operating at 100% peak efficiency, regardless of your morale or your enthusiasm or your job satisfaction. Never taking a minute from your work day to chat with a coworker or check a personal web site. Always rigorously testing every line of code you check in.
Except for the last, no, not really. But I don't attempt to justify my failings by blaming others either.
>Well, I'll call your manager and tell him to stop wasting morale budget on you.
No problem. Send it to BlakeI. I'm several levels under him, but he'll know who you're referring to.
>"#3 Jawad Khaki. I was so glad to escape the networking team. It is super to see that they still go off and do whatever they want. Way to hit Brian's december 31st date for being IN winmain - not on a slow boat towards it. And where is NAP? Take mr. ashida with you on the way out. FJ."
Please take the NDT HR generalist who acts like the partner's mistress along.
"Please take the NDT HR generalist who acts like the partner's mistress along."
--
agreed.
oh, there is more than 1 person who got burnted by NDT (t & j) so don't single the above comment to any one candidate ;-)
Sherwood
I suggest you review employment law.
Well then, lad, where's your lawsuit?
As you probably know, managers tend to make their threats behind closed doors.
I did not have a voice recorder with me at the time.
He is no longer a manager though.
There are other ways to effect change besides a lawsuit.
I pursued the most cost effective route and it worked.
"As you probably know, managers tend to make their threats behind closed doors.
I did not have a voice recorder with me at the time.
He is no longer a manager though. "
--
luckily that person got identified as a bad manager.
In my case, the clown is now a GM and still hits me any chance he can get
Still don't understand what Mark Ashida and his management team are doing at Microsoft. NAP is basically unusable,and the Yankee Group marked it "dead on arrival"
http://www.itbusinessedge.com/item/?ci=16031.
Maybe Mr. Khaki needs some tips from Trump!
Post a Comment