It’s my favorite time of the year, again: time for annual reviews!

Hopefully, you could hear the sound of sarcasm dripping from each word in that sentence. I don’t know anyone who looks forward to annual reviews. As a manager, I dread this more than others. Not only do I get to have my own review, but I get to do the review for all of those who report to me.

I have to say that doing annual reviews falls somewhere between performing a root canal on myself and suffering through reruns of “Little House on the Prairie”. I know that seems like a broad area of pain, unless you hate that show as much as I do.

The chief reason for this angst is that no one, it seems, is ever happy with their review. Maybe that’s a good sign that I’m taking the process seriously and trying to really evaluate my team members’ strengths and weaknesses. Maybe that means I’m an overly critical [insert epithet here]. Believe me, nothing would be easier than telling everybody that they rock and just send them back to their cubicles.

I can’t bring myself to do that, though. I have to try to make this a meaningful event. To grow, everyone needs to know where they are strong and where they are weak. How many people realistically have no areas they cannot improve on?

Aside from the discomfort of doing an honest assessment of someone, the biggest problem in most review systems is that the scores are arbitrary and meaningless. At most companies, there is no effort to baseline scores or set a reasonable standard for what they mean.

At one company, we used a complex review system with 16 different ratings in 5 areas. Each was assigned a value from 1 to 10 with a 10 being the best score. When I had finished the reviews for my team, most scores ranged from 5 to 10 with a reasonable bell-shaped distribution. So, 7 and 8 were far more common than 5, 6, 9, and 10. I later learned that another manager had simply gone through and given his people all 9’s and 10’s. Since the raises were tied to scores, his people got bigger raises than mine.

I could have saved so much time and so much aggravation by just doing the same thing. Obviously the company didn’t care. They never said anything to me that my scores were unduly low or that his were unreasonably high.

It makes me miss the simple scoring systems used in college. There you are given a course syllabus that outlines the structure of your grade. It shows what percentage of your grade will come from homework assignments and what percentage will come from exams. Where I attended college, scores were posted on the professor’s door. Not only could you look-up your score, but you could see your ranking among your peers. So, even if you got an 87 on a test, you might find that it was one of the highest scores achieved.

In the work world, you rarely have that kind of clarity. Some of your peers succeed on the basis of their social skills rather than their contributions. And even when you are given a rating, you have no idea how it compares to those of your peers. Is it any wonder that employees typically view annual reviews with suspicion and hostility?

So, to simply things this year, I propose a new system based on the Signs of the Chinese Zodiac . I got the idea from a placemat at a Chinese restaurant. There are 12 animals: Rat, Ox, Tiger, Rabbit, Dragon, Snake, Horse, Ram, Monkey, Rooster, Dog, and Pig. Each animal is assigned a set of characteristics both positive and negative.

For example, the rat is said to be “Forthright, disciplined, systematic, meticulous, charismatic, hardworking, industrious, charming, eloquent, sociable, shrewd. Can be manipulative, vindictive, mendacious, venal, selfish, obstinate, critical, over-ambitious, ruthless, intolerant, scheming.” (from Wikipedia).

The Chinese zodiac assigns an animal to each year, month, day, and hour. So, you can interpret your personality based on these animals and how they interact. The interpretation of these signs is very complex and subtle.

To apply this system to annual reviews, I would take the traditional approach of breaking down performance into key areas: communication, teamwork, leadership, initiative, job knowledge, etc. For each area, the manager assigns one of the 12 animals. Along with the reviews, the employees would be given a key, like the placemat, that lists all of the animals and their characteristics. Together the employee and the manager would discuss the interpretation.

“Well, Frank, you got a Monkey for Communication and Rat for Leadership. I’d like to see you bring these two into better harmony next year. You should strive for an Ox in Communication or maybe a Rooster.”

This would also help when employees compare their results. Yes, we know they aren’t supposed to do that, but in my experience those kind of corporate rules are no more effective than speed limit signs. Using the Signs of the Chinese Zodiac for reviews would eliminate any hard feelings over someone getting a better review.

“Jim got a Rabbit in Customer Focus and I got a Horse.” Walks away with a look of utter puzzlement.

This would also head off any arguments due to an employee feeling they got an unfairly low score. “Damn it, Scott! I can’t believe you gave me a Tiger in Judgment. You know I should have gotten a Ram!”

Yes, this would produce a review system that allows for the arbitrary results of most modern reviews while allowing each employee to find their own meaning in their scores. …and peace and harmony rang throughout the kingdom!

The press is abuzz with news items about President-Elect Obama’s search for a CTO and the possibility that he could make this a cabinet-level position.

This is a great idea! Technology is so prevalent in our society and in our government that it’s impossible to conceive of a plan for effective government that is not based on a sound technology roadmap. Certainly, if we need a Secretary of Transportation, then we need a Secretary of Technology.

Making this a cabinet-level position will hopefully give the USCTO the authority to bring uncooperative agencies together. In truth, I think that will be the biggest challenge. Bureaucracies are rife with politics. They don’t like to be told what to do or to work with others. They want to do things their way without someone else suggesting a better way.

A Chief Technology Officer can be instrumental in setting goals and directions for our nation’s IT infrastructure. However, I have seen CTOs wreak havoc on organizations that were doing just fine without them. Here are a few common pitfalls I hope the new US CTO will be able to avoid.

Outsourcing Core Expertise

I have worked for years as a contract programmer, and I have worked for companies that provided outsourcing services. There are times when it truly makes sense to outsource. If you are not a software company, but you need to have software built, then you are generally better off hiring a company to do that for you. A company that develops software full-time will be far more successful than one with no experience.

For example, I worked for a company that developed workflow and routing systems that incorporated high-speed scanners. We had significant knowledge and experience with the hardware and issues involved with these systems. Our customers were companies that processed forms for things like book clubs or medical records. Those companies simply could not have built these systems as effectively as we could.

For them, the systems we built were just a tool. Their core knowledge was related to the business of processing the records. They just needed a system that made it faster and easier to process the information.

However, I have seen companies whose main revenue comes from selling software outsource the development of their products. Commonly, this is done by partnering with an offshore development house.

The problem with this approach isn’t that it’s overseas; it’s that the people working on the product do not work for the company. If you make money by building and selling software, then why would you ever want the knowledge necessary to create and maintain your products to be held by someone with no attachment to your company?!?

Cost Centric Management

Improving an organization is hard. It takes thorough analysis and creative thinking to find ways to do things better. Rather than finding ways to do things better, some CTOs try to make their mark by doing things cheaper.

Don’t get me wrong, I do believe that organizations should be run efficiently and that managers should be looking at costs to make sure they are in line. However, cost management is often taken to an extreme.

Examples of this mentality include hiring cheaper programmers rather than focusing on the knowledge and experience needed. Many companies treat programmers like interchangeable cogs with no appreciation for the difference between individuals. So, if they can hire programmers for $10K or $20K less per year, that’s a big win, right?

This mentality drives the offshoring approach. The lower cost is touted as a big win for the company, but little consideration is given to whether the product can be developed as successfully. This is particularly problematic if the development team is split, as is so often the case, with the business team working here and the developers working overseas. Software development has been shown to be far more successful when there is close interaction between the programmers and the subject matter experts, even more so with direct customer interaction.

Companies also attempt to restrict costs by scrimping on the tools they provide. We see that one a lot at SlickEdit. We often hear from programmers who want to buy SlickEdit but are told by their management that $300 is too expensive.

At SlickEdit, every developer has multiple monitors. We believe this is essential to getting your work done efficiently. I have worked at some companies where I bought my own equipment because they would not approve anything more than a single 17” monitor.

The fundamental mistake that drives these decisions is that they assume a fixed level of output. So, they believe they can cut costs and the output will not change. In truth, we know that the level of output in software development is highly variable and subject to many complex factors. The definitive work in this area is “Peopleware” by DeMarco and Lister.

Over Standardization

Standardization can be a good thing. When done properly, it ensures efficiencies of scale and the ability to dynamically manage resources within an organization. Standardization can be helpful to assist everyone in producing a similar result. If taken too far, it becomes a force to make sure that no one can achieve excellence.

No single tool or technology is good for all situations. The cost of trying to pound a square peg into a round hole can often exceed the cost of buying a round peg. You do have to guard against capricious choices (see Design Fail Patterns), but you should guard against a policy that is too rigid. Standardization should be a gravitational force that pulls towards a center, but it should not become an inescapable black hole

The larger the company, the more diverse their technology should be. Small companies may not be able to afford to dabble in different technologies, but larger companies can often see greater efficiency by allowing teams to select the right tools even when they are different. Further, by having a broader technology portfolio they reduce the risks associated with selecting a single technology, and they are more likely to discover new, more efficient ways of doing things.

“Vision”

I often see the word “vision” used with CTOs. They have “vision”. When I hear this word, I cringe. While having vision is often essential in succeeding, too often this word is used as license for narrow-minded, megalomaniacal tendencies that will brook no discussion or alternatives. Can you tell I feel strongly about this one?

Often the CTO forms this vision before they know anything about the organization. In my experience, the employees already know what needs to be done to fix most problems at the company. What they need is a management team that’s willing to listen. Instead, most employees fear going to upper management with problems and solutions.

Sometimes this “vision” is based on a success at a previous company. They see the new company as an opportunity to do what they did before. However, these successes are rarely repeatable. Organizations are just too different. Often, the reason a project succeeds is because of complex factors, not the least of which is the team that implemented it. Given the choice, I’d rather have the team from that successful project rather than one guy with the “vision”. Be careful how much credit you give managers (or any leaders) for the accomplishments of their teams.

Vision should provide an overall direction, a set of values that are used to balance competing needs. It should not be a totalitarian model to which an organization must conform.

I wish the new CTO of the United States much luck. It will be needed. Confidence in our government is at an all-time low. It will take much more than vision to change that.

The election season puts me in mind of the many ways I have seen teams decide on which features to implement. Like all human endeavors this process is political, meaning that it is shaped by the relationships and power structure of the organization. Consequently, you can use various systems of government as metaphors for different approaches.

For the sake of this discussion, we’ll treat defects the same as features. This isn’t totally accurate since some defects simply must be fixed as soon as possible. However, most defects are subject to the same decision process as features.

The Dictatorship

With this system, a single, all-powerful voice decides what gets done when. This is a common approach on smaller projects, where these decisions are made by the project manager. That’s not to say that this person doesn’t listen to advice from others. But when it comes time to decide they, alone, make the call.

Ultimately, the other systems typically have a single person who has to make the final decision. In my opinion, this is an essential characteristic of an effective organization. Everything should funnel up to a single person authorized to make decisions. People at different levels should be empowered to make various decisions, but in the end a single person should have the final authority. Otherwise, you flounder endlessly when a consensus cannot be achieved. It becomes a dictatorship when the leader ignores the advice and guidance of others and acts strictly on their own belief.

This approach only appeals to me if I can be the dictator, using my sagely wisdom to guide the product direction. In truth, this system can work well if the person in charge has a keen sense of what it takes to succeed. From accounts I’ve read, this is not unlike Steve Jobs’ approach to new product development. Though you can argue with his approach, it’s hard to question the results.

The Oligarchy

In an oligarchy , power is shared by small, elite group. This approach is often used on larger projects where various constituencies are represented. You might have representatives from project management, sales, development, quality assurance, and product support. In some cases, actual customer representatives might be involved.

The group meets regularly to discuss what is needed and when. Tradeoffs are weighed between features needed to spur new sales and fixes needed to appease existing customers. Typically, the group is lead by someone with final decision authority, but most decisions have a strong consensus.

This approach is certainly more representative, but it is also much more time-consuming. I’ve worked on projects that used this method, but they were very large with complex interactions between subsystems. Still, a light-weight version of this can be helpful to make sure all factors are weighed effectively.

The Democracy

In its purest form, this approach relies on votes from the masses to make the selection. Sometimes, members of the development team do the voting. In other systems, the customers are allowed to vote.

Though I cannot find a reference to it anymore, I recall that Sun used Duke Dollars as a way for developers to vote for specific fixes in the early days of Java. I’ve worked on teams where each developer was given some number of votes that they could apply to the bugs or features of their choosing.

I’m not a big fan of this approach. When the developers do the voting, this tends to favor the interesting, glamorous features over those that are more mundane. Often, these mundane features make a bigger difference to the customers. When the customers vote, they are typically guided by their own way of working. An effective product has to balance the different ways customers will use it.

You could use a hybrid approach, where the developers or customers submit votes but the decisions is ultimately made by a small group or individual. The problem with this is that it can be demotivating to the voters. If a feature with a high number of votes is rejected, they will ask why they even bothered. So, if you use a voting system, you should be willing to act on the results.

The Meritocracy

In the meritocracy , the best idea wins. Instead of suggesting a feature or voting for it, each feature is submitted with a business case. The business case is used to decide which changes make the biggest difference to the product.

This is my preferred approach to prioritizing features. It allows for input from all stakeholders and provides a means to assess priorities. With this system, anyone can challenge a feature and have it replaced in the schedule by demonstrating that another feature has a better business case.

Unfortunately, it is not always evident which idea has the greatest merit. So, this system relies on approaches from the others to make that call. A team or individual weighs the various proposals and picks the ones that best meet the business needs.

This approach also avoids the “squeaky wheel gets the grease” syndrome. Often, it is the customer or developer who is most vocal that gets their way. Great ideas languish because the proponent is not willing or able to be heard above the din of the highly vocal.

Truthfully, there are very few pure examples of any form of government. Most are a combination of these approaches. The most important characteristics of a feature planning system are:

• The ability to gather ideas from all stakeholders.
• The willingness to listen to these ideas.
• A system to evaluate ideas on their true merit.
• A clear vision of the product that helps to determine which ideas truly enhance the product and which just muddy the waters.

Of course, it wouldn’t hurt if we could somehow take humans out of the equation and have features selected by a race of benevolent and powerful robots. But that’s a post for another time.