Dev Management

Recent experiences with SlickEdit 2009 have reminded me that there are no small changes. Even when you think you understand the nature of a change, severe side-effects can pop up in complex software.

This was the case when we put out our first release of SlickEdit 2009, version 14.0.0.5. We had completed our usual beta cycle before releasing this version, which included a number of drops to correct reported problems. After the last beta drop, we made a few more changes, all of which seemed totally risk-free.

Within two hours of releasing SlickEdit 2009, we had reports of severe problems with the File tabs and the File list. These problems trace back to one of these simple, risk-free changes. We had done our normal testing after these changes, and we hadn’t discovered any problems. But this problem was so severe we had to pull the release until it could be fixed.

So, what did we learn from this?

First, there are no small changes. Even when you think you know the effect a change will have, you could be wrong.

Should we have tested more? Sure, you can always test more, but that is no guarantee that you will find a problem. Testing suffers from a horizon effect: you can only see as far as you test. The problem you need to find could lay just beyond or miles beyond. You don’t know until you find it.

Second, testing is different than using. When you are testing, you click all the buttons and try all the functions, but you are rarely able to perform all combinations of operations. Some problems depend on the order in which operations are invoked. Others require multiple operations before the problem surfaces, as was the case with the problem we missed.

It should be no surprise that the parts of SlickEdit that work the best are the things we use everyday. But there are many languages we don’t use on a daily basis. Even if we did, with a product like SlickEdit, our testing cannot hope to replicate the variety and complexity of our customers’ use patterns.

We’ve determined that prior to shipping any release, we need help validating it. So, we’ll create a Release Candidate when we have finished our testing. We’ll make the Release Candidate available for early adopters to test, and if all goes well, it will be used for the General Availability release. In this way, we hope to catch more problems before they are set free to roam the countryside and terrorize the populace.

Q&A

Does that mean you’ve given up on testing? No, we plan to continue testing just as we always have. We just need help with final validation.

What if I don’t have time to help you catch problems? Then wait until a release is no longer a release candidate. At that point, others will have helped to validate it, and you will have less likelihood of experiencing issues with a new version.

If an unladen swallow flies west from New York…

It’s my favorite time of the year, again: time for annual reviews!

Hopefully, you could hear the sound of sarcasm dripping from each word in that sentence. I don’t know anyone who looks forward to annual reviews. As a manager, I dread this more than others. Not only do I get to have my own review, but I get to do the review for all of those who report to me.

I have to say that doing annual reviews falls somewhere between performing a root canal on myself and suffering through reruns of “Little House on the Prairie”. I know that seems like a broad area of pain, unless you hate that show as much as I do.

The chief reason for this angst is that no one, it seems, is ever happy with their review. Maybe that’s a good sign that I’m taking the process seriously and trying to really evaluate my team members’ strengths and weaknesses. Maybe that means I’m an overly critical [insert epithet here]. Believe me, nothing would be easier than telling everybody that they rock and just send them back to their cubicles.

I can’t bring myself to do that, though. I have to try to make this a meaningful event. To grow, everyone needs to know where they are strong and where they are weak. How many people realistically have no areas they cannot improve on?

Aside from the discomfort of doing an honest assessment of someone, the biggest problem in most review systems is that the scores are arbitrary and meaningless. At most companies, there is no effort to baseline scores or set a reasonable standard for what they mean.

At one company, we used a complex review system with 16 different ratings in 5 areas. Each was assigned a value from 1 to 10 with a 10 being the best score. When I had finished the reviews for my team, most scores ranged from 5 to 10 with a reasonable bell-shaped distribution. So, 7 and 8 were far more common than 5, 6, 9, and 10. I later learned that another manager had simply gone through and given his people all 9’s and 10’s. Since the raises were tied to scores, his people got bigger raises than mine.

I could have saved so much time and so much aggravation by just doing the same thing. Obviously the company didn’t care. They never said anything to me that my scores were unduly low or that his were unreasonably high.

It makes me miss the simple scoring systems used in college. There you are given a course syllabus that outlines the structure of your grade. It shows what percentage of your grade will come from homework assignments and what percentage will come from exams. Where I attended college, scores were posted on the professor’s door. Not only could you look-up your score, but you could see your ranking among your peers. So, even if you got an 87 on a test, you might find that it was one of the highest scores achieved.

In the work world, you rarely have that kind of clarity. Some of your peers succeed on the basis of their social skills rather than their contributions. And even when you are given a rating, you have no idea how it compares to those of your peers. Is it any wonder that employees typically view annual reviews with suspicion and hostility?

So, to simply things this year, I propose a new system based on the Signs of the Chinese Zodiac . I got the idea from a placemat at a Chinese restaurant. There are 12 animals: Rat, Ox, Tiger, Rabbit, Dragon, Snake, Horse, Ram, Monkey, Rooster, Dog, and Pig. Each animal is assigned a set of characteristics both positive and negative.

For example, the rat is said to be “Forthright, disciplined, systematic, meticulous, charismatic, hardworking, industrious, charming, eloquent, sociable, shrewd. Can be manipulative, vindictive, mendacious, venal, selfish, obstinate, critical, over-ambitious, ruthless, intolerant, scheming.” (from Wikipedia).

The Chinese zodiac assigns an animal to each year, month, day, and hour. So, you can interpret your personality based on these animals and how they interact. The interpretation of these signs is very complex and subtle.

To apply this system to annual reviews, I would take the traditional approach of breaking down performance into key areas: communication, teamwork, leadership, initiative, job knowledge, etc. For each area, the manager assigns one of the 12 animals. Along with the reviews, the employees would be given a key, like the placemat, that lists all of the animals and their characteristics. Together the employee and the manager would discuss the interpretation.

“Well, Frank, you got a Monkey for Communication and Rat for Leadership. I’d like to see you bring these two into better harmony next year. You should strive for an Ox in Communication or maybe a Rooster.”

This would also help when employees compare their results. Yes, we know they aren’t supposed to do that, but in my experience those kind of corporate rules are no more effective than speed limit signs. Using the Signs of the Chinese Zodiac for reviews would eliminate any hard feelings over someone getting a better review.

“Jim got a Rabbit in Customer Focus and I got a Horse.” Walks away with a look of utter puzzlement.

This would also head off any arguments due to an employee feeling they got an unfairly low score. “Damn it, Scott! I can’t believe you gave me a Tiger in Judgment. You know I should have gotten a Ram!”

Yes, this would produce a review system that allows for the arbitrary results of most modern reviews while allowing each employee to find their own meaning in their scores. …and peace and harmony rang throughout the kingdom!

The press is abuzz with news items about President-Elect Obama’s search for a CTO and the possibility that he could make this a cabinet-level position.

This is a great idea! Technology is so prevalent in our society and in our government that it’s impossible to conceive of a plan for effective government that is not based on a sound technology roadmap. Certainly, if we need a Secretary of Transportation, then we need a Secretary of Technology.

Making this a cabinet-level position will hopefully give the USCTO the authority to bring uncooperative agencies together. In truth, I think that will be the biggest challenge. Bureaucracies are rife with politics. They don’t like to be told what to do or to work with others. They want to do things their way without someone else suggesting a better way.

A Chief Technology Officer can be instrumental in setting goals and directions for our nation’s IT infrastructure. However, I have seen CTOs wreak havoc on organizations that were doing just fine without them. Here are a few common pitfalls I hope the new US CTO will be able to avoid.

Outsourcing Core Expertise

I have worked for years as a contract programmer, and I have worked for companies that provided outsourcing services. There are times when it truly makes sense to outsource. If you are not a software company, but you need to have software built, then you are generally better off hiring a company to do that for you. A company that develops software full-time will be far more successful than one with no experience.

For example, I worked for a company that developed workflow and routing systems that incorporated high-speed scanners. We had significant knowledge and experience with the hardware and issues involved with these systems. Our customers were companies that processed forms for things like book clubs or medical records. Those companies simply could not have built these systems as effectively as we could.

For them, the systems we built were just a tool. Their core knowledge was related to the business of processing the records. They just needed a system that made it faster and easier to process the information.

However, I have seen companies whose main revenue comes from selling software outsource the development of their products. Commonly, this is done by partnering with an offshore development house.

The problem with this approach isn’t that it’s overseas; it’s that the people working on the product do not work for the company. If you make money by building and selling software, then why would you ever want the knowledge necessary to create and maintain your products to be held by someone with no attachment to your company?!?

Cost Centric Management

Improving an organization is hard. It takes thorough analysis and creative thinking to find ways to do things better. Rather than finding ways to do things better, some CTOs try to make their mark by doing things cheaper.

Don’t get me wrong, I do believe that organizations should be run efficiently and that managers should be looking at costs to make sure they are in line. However, cost management is often taken to an extreme.

Examples of this mentality include hiring cheaper programmers rather than focusing on the knowledge and experience needed. Many companies treat programmers like interchangeable cogs with no appreciation for the difference between individuals. So, if they can hire programmers for $10K or $20K less per year, that’s a big win, right?

This mentality drives the offshoring approach. The lower cost is touted as a big win for the company, but little consideration is given to whether the product can be developed as successfully. This is particularly problematic if the development team is split, as is so often the case, with the business team working here and the developers working overseas. Software development has been shown to be far more successful when there is close interaction between the programmers and the subject matter experts, even more so with direct customer interaction.

Companies also attempt to restrict costs by scrimping on the tools they provide. We see that one a lot at SlickEdit. We often hear from programmers who want to buy SlickEdit but are told by their management that $300 is too expensive.

At SlickEdit, every developer has multiple monitors. We believe this is essential to getting your work done efficiently. I have worked at some companies where I bought my own equipment because they would not approve anything more than a single 17” monitor.

The fundamental mistake that drives these decisions is that they assume a fixed level of output. So, they believe they can cut costs and the output will not change. In truth, we know that the level of output in software development is highly variable and subject to many complex factors. The definitive work in this area is “Peopleware” by DeMarco and Lister.

Over Standardization

Standardization can be a good thing. When done properly, it ensures efficiencies of scale and the ability to dynamically manage resources within an organization. Standardization can be helpful to assist everyone in producing a similar result. If taken too far, it becomes a force to make sure that no one can achieve excellence.

No single tool or technology is good for all situations. The cost of trying to pound a square peg into a round hole can often exceed the cost of buying a round peg. You do have to guard against capricious choices (see Design Fail Patterns), but you should guard against a policy that is too rigid. Standardization should be a gravitational force that pulls towards a center, but it should not become an inescapable black hole

The larger the company, the more diverse their technology should be. Small companies may not be able to afford to dabble in different technologies, but larger companies can often see greater efficiency by allowing teams to select the right tools even when they are different. Further, by having a broader technology portfolio they reduce the risks associated with selecting a single technology, and they are more likely to discover new, more efficient ways of doing things.

“Vision”

I often see the word “vision” used with CTOs. They have “vision”. When I hear this word, I cringe. While having vision is often essential in succeeding, too often this word is used as license for narrow-minded, megalomaniacal tendencies that will brook no discussion or alternatives. Can you tell I feel strongly about this one?

Often the CTO forms this vision before they know anything about the organization. In my experience, the employees already know what needs to be done to fix most problems at the company. What they need is a management team that’s willing to listen. Instead, most employees fear going to upper management with problems and solutions.

Sometimes this “vision” is based on a success at a previous company. They see the new company as an opportunity to do what they did before. However, these successes are rarely repeatable. Organizations are just too different. Often, the reason a project succeeds is because of complex factors, not the least of which is the team that implemented it. Given the choice, I’d rather have the team from that successful project rather than one guy with the “vision”. Be careful how much credit you give managers (or any leaders) for the accomplishments of their teams.

Vision should provide an overall direction, a set of values that are used to balance competing needs. It should not be a totalitarian model to which an organization must conform.

I wish the new CTO of the United States much luck. It will be needed. Confidence in our government is at an all-time low. It will take much more than vision to change that.

« Previous PageNext Page »