Dev Management

Though it won’t be flagged as profane in most dictionaries, one word that is often viewed as a dirty word in programming is “management”. Managers are so maligned in engineering that Scott Adams created the “pointy-haired boss” as their representative (http://www.dilbert.com/). It may be true that managers are equally loathed in other fields, but having spent my adult years working in software engineering, this is the only field I feel competent to address. I also have a somewhat unique perspective having been both a developer and a development manager.

I believe that all teams need management. That’s not to say that they all need a manager; some teams do manage themselves. Effective management gets the most out of a team by setting clear goals and minimizing time lost to indecision. A good manager bridges the gap between the business team and the technical team, making sure that both viewpoints are properly considered. A strong development manager can bring a process-centric view to the development team that seeks to improve daily workflows and operations.

So, what’s wrong with “management”?

I think the core reason that software management is so reviled is that it is so rarely done well. In my programming career, I can divide my previous managers into 3 categories:

  1. Benevolent, useless, harmless—this is the category into which fall most of my previous managers. They did a reasonable job of passing along information and managing schedules, but I never really felt they were helping me get my job done. I spent a great deal more time giving them status reports than receiving guidance.
  2. Malevolent, clueless, harmful—fortunately this category contains only a few of my previous managers. These are the ones who made the job much harder than it needed to be. Sometimes it was done to further their individual goals, like the manager who stuck us with an unreachable milestone so he could get a 30% bonus. Others, it was because they had no idea what we were doing and didn’t have the courage to admit it.
  3. Effective, purposeful, supportive—by far the smallest population, this is the category we wish all managers fell into but so few actually do. These are the ones who truly help you get your work done by providing the kind of guidance and environment you need to succeed. Tom R., you were the best!

How did we get here?

There are so many contributory causes for this problem. The first group, above, is composed of good people who just don’t know what to do. The second group shouldn’t be in charge of themselves, let alone others. I can offer little help to the second group, so let’s focus on the first.

There are two kinds of development managers: those with a strong programming background and those without one (henceforth called “programmers” and “non-programmers”). Even if you have some programming experience, it may not be enough to qualify you as a programmer. You have to have done enough to be considered as a peer on your programming team. I divide managers into these groups because they face different challenges. I’ve worked for managers from both backgrounds and neither is a guarantee of success or failure. My best manager learned his craft managing supermarkets before he went into software development.

To be a successful development manager, you need to know how to manage and enough about software engineering to make appropriate choices. Oddly enough, both groups seem to struggle with the first item. Though the non-programmers tend to have better communication skills than the programmers (if they didn’t, what the heck were they hired for since they’re not programmers?), few of them in my career were skilled managers.

Unique Challenges

The unique challenge faced by the non-programmers is their lack of knowledge about software development. This often hinders their decision-making. A common sign of this is an insistence on keeping to the schedule when a minor slippage could make a huge difference later. When you’ve been around software development long enough, you learn that schedules are fluid. What you do today can make your work tomorrow easier, or harder, depending on the quality of the design and implementation. It takes experience and insight to know when to make these tradeoffs.

Another common problem for the non-programmer manager is not being able to call shenanigans when a developer is snowing you. Even as a programmer, you will never know someone’s code like they will, but you will know from your experience when a developer is likely doing something wrong. My background has helped me to sniff out cases of “design by resume”, where developers chose a particular approach in order to get a trendy new technology on their resume. A non-programmer manager would have a hard time asking the appropriate questions to recognize this.

The programmer managers also face unique challenges. The first is the temptation to code and manage at the same time. I read where others actually recommend this as the desired situation. In my experience, the two activities are incompatible. Effective coding is about immersion, getting into a state of flow, and being very productive. Effective management involves being available to your team to tackle their issues when they arise. Even when I wanted to try to keep coding, I’ve never had a position where the management responsibilities didn’t crowd out any time I allocated for coding. As a manager, my favorite means of being involved with the code is to participate in design discussions and code reviews. This keeps things at a high level and still allows you to contribute your knowledge and experience.

Another problem for the programmer manager is trying to be “one of the guys”. This is manifested as a manager who is hesitant or unable to provide negative feedback. Often, these kinds of managers are promoted from within the team, making this even more of an issue. These managers still view themselves or want to be considered to be a member of the development team rather than its leader.

As a manager, your primary allegiance must be with the business team. Your job is to take their goals and translate them into effective plans to accomplish them. Where you think these goals will lead to problems, you need to point that out. However, if you can’t get them to change the goals, then you need to sign up to them and do your best to accomplish them. Programmer managers sometimes have a difficult time making this change.

Shared Challenges

The most common shared challenge is a lack of knowledge about how to manage. Most of the managers I have worked for had no formal training in management. At best they had a seminar or two. Few ever read any books on management or software development management. Since they, too, have had few truly effective managers, they have no guidance on how to manage. They just don’t know what to do.

An example where this shows up is the weekly status meeting. Most managers have one. They have one because their previous managers had one. In its most typical form, the status meeting lasts for an hour or so while the manager has a conversation with each member of the team, discussing what they are doing. The theory is that having this as a team fosters teamwork and helps everyone to know what everyone else is doing. In reality, the rest of the team already knows or doesn’t care. So this ends up being a large waste of time as the manager has a one-on-one conversation with each team member in front of the rest of the team.

Don’t get me wrong, I’m not saying that status meetings are wrong. But you need to choose to have one because of the benefits the team will get from it, because it is right for this team. Instead, most choose to have one because that’s how they’ve been managed in the past, and it is a visible sign that they are in charge. One of the primary goals of management should be to achieve stated goals with as little interruption in the daily work as possible.

The Solution

What we need to solve this is the same thing we need for all leaders: wise, beneficent individuals with a commitment to goals and compassion for their team. In software development, you need to add in enough knowledge to be mindful of the unique challenges in our field. To erase the pointy-haired boss from our minds or at least remove him as the dominant figure in engineering management, we need enough developers crossing over to become committed managers or enough business managers to gain an appreciation for software development.

As discussed in a previous blog, The Perfect Estimate, providing software estimates is one of the most challenging aspects in the software industry. There are many reasons for this, not the least of which is the challenge of estimating the unknown. Unlike many other construction activities, like erecting a skyscraper or building a ship, software projects often involve new combinations of technologies (at least for those on the project) assembled in unique ways.

The more a task resembles a previously completed task, the easier it is to estimate. Ideally, metrics kept about the previous task will include time spent on various activities and the total time overall. When estimating the new task, you could analyze which activities are likely to change based on completing the previous one and derive an estimate for the new task.

In order to best illustrate the challenges of software estimates, I will use the time-proven technique of the parable.

The Parable of the Cave

Two people stand before a cave. One is the sagely manager of a cave exploring company who’s wisdom is only exceeded by his wit, charm, and humility. Let’s call him, oh, “Scott”. The other is a cave explorer of indeterminate gender who bears no resemblance to any programmers past or present that this author may have worked with and whose size may be big or little. Let’s call him/her “Endian”.

“Endian,” said Scott in a sagely voice that was both commanding and compassionate, “I need you to explore this cave. But before you do, I need to know how long you think it will take, so that I may build a schedule and tell the sales team when the cave will be ready.”

“Great Scott,” replied Endian using the title bestowed upon Scott by his admiring employees, “how can I give you an answer when surely you know I have never been in this cave before? The cave may be vast, with deep chasms. It may contain a labyrinth of underwater passages. It may contain fearsome creatures that must first be vanquished. How can I say how long it will take to explore?”
Scott pondered Endian’s words and after a thorough analysis that might have taken days for others but was completed in but a moment for him, he replied, “Surely this is not the first cave you explored. Are there no other caves in this district? Use your knowledge of those caves to form an estimate.”

Endian heard these words and still his doubt prevailed. “Your words are truly wise,’’ said Endian, “but even within a district the caves may vary, one from another. Surely, an estimate based on the size of another cave cannot be deemed accurate.”

“You have spoken truly, good Endian,” replied Scott in a fatherly, supporting tone that lacked any trace of being patronizing as certain cynical readers may think. “Here, take from me this torch and this assortment of cheeses that you may explore the cave briefly. Return ere the morrow and report what you have learned.”

The parable continues like this for pages, as parables are known to do. Let’s see, Endian enters the cave…something about a wretched beast of surpassing foulness…he continues on…hmm, that’s what the assortment of cheeses were for. Ah! Here we go.

Endian returns to Scott, his t-shirt ripped and his jeans covered in mud. Being always concerned with the well-being of his employees, Scott offers Endian a cool drink, then asks, “Endian, what news of the cave? Have you an estimate that I can use for my schedule? What shall I tell the sales team?”

Endian considers all that he has seen and builds a decomposition containing the many tasks necessary to explore the cave based on his earlier reconnoitering. He factors in variables for risk and unknowns, and then he responds, “Two weeks.”

Applying this Wisdom

To truly appreciate the punch line, you do need to check out The Perfect Estimate.

The point I’m trying to illustrate is that estimating software development is like estimating the time needed to explore a cave you’ve never been in. You don’t really know until you get in there. Still, there are techniques that can help.

First, draw on your experience. Even though technologies may be very different, few automation problems are empirically new. They bear resemblance to other things we’ve done in the past.

Expect the unexpected! Make sure you factor time in for the unknowns. If your schedule only contains the things you can think of, then it’s certainly going to fall short when the unexpected arises. There will always be things you don’t know to put into the schedule.

Provide estimates that are ranges rather than just a single duration. If a task can take 2 to 4 weeks, provide it in that manner. If your project planning software cannot accomodate date ranges, enter the task as 2 weeks and then put in a secondary task for another 2 weeks to account for this. I like to name these tasks, “risk factors”. So, if I have a task called, “Integration Testing”, I would add another called, “Integration Testing, Risk Factor”. This way, I can specifically see how much time has been set aside for this variability, and I can see what happens to the schedule if I increase or decrease that amount of time.

Make your estimates as fine-grained as possible. An estimate for a task that only needs a few hours is far more likely to be accurate than an estimate for one that will need a couple weeks. By making your estimates as detailed as possible you not only improve accuracy but the work necessary to create a detailed estimate will often turn up tasks you might otherwise overlook.

Even if your manager only wants an overall estimate, you should do a detailed decomposition to make sure you build the best estimate possible. This decomposition will also serve as a guide during your work, both as a checklist to make sure you haven’t forgotten something and as a status check to see if you are on time. This can help you timebox activities, applying the available time to a task. Projects can run over due to time spent waffling on design decisions. Timeboxing tells you when it’s time to make a decision and move on.

Build your decompositions with tasks that are either complete or incomplete. When tracking progress, don’t fall into the trap of recording activities as some percentage complete. A task is either finished or not finished. Showing partial completion leads to a false sense of security.

Use incremental and iterative processes. As in the parable, sometimes it’s useful to make a quick, preliminary exploration in order to build a more comprehensive estimate. Continue to refine your estimates and the overall plan. The closer you get to the end, the more accurate your schedule should be.

Work on the riskiest part of the system first. Too often projects run over when someone discovers a large problem in the final stages of the work. Assess the risk of each portion of the work and tackle first those parts that are most likely to exceed their estimates or induce changes in the rest of the system.

Lastly, it is absolutely critical that an environment of trust exists between the development team and the rest of the company. Your job should be to deliver the best estimate you can with appropriate risk factors identified. Too often estimates are inflated to make sure that we can hit them. Equally often, due dates are set artificially early to allow for overages. Both of these strategies prevent the company from building an effective plan for this project and developing appropriate contingencies. Worse, these constraints sometimes lead to poor choices and compromises during development that add further risk.

In one of the greatest books ever written about software development, “The Mythical Man Month”, Fred Brooks refutes the premise that effort can be quantified in “man months”, the amount of work done by one person in one month. The central flaw is illustrated as follows: if a woman can have a baby in 9 months, then 9 women should be able to have a baby in 1 month. The core problem is that not all tasks can be reduced into parallel subtasks. Even when they can, breaking work into smaller units often adds overhead.

In a similar vein, I’d like to expose the myth of the so-called “year of experience”. In hiring, this is the basic unit used to measure ability. Look on almost any job posting and you will see years of experience required in something. Some postings use this metric sparingly, requiring a number of years of experience overall or in a single key technology, like 3 years of C++. Some postings carry this to such an extreme that you wonder if anyone could have that magical combination of experience. Thankfully, you don’t see this as much as you used to. Some ask for ridiculous amounts of experience in some areas. A few years back I even saw a posting for a Java position requiring more years of experience than Java had been around!

What companies need is someone with a certain level of expertise. But how do you quantify expertise, particularly in technical fields with broad areas of knowledge? That’s where the mythical year of experience comes in. It may not be accurate, but it is quantifiable and easily verified from a candidate’s work history.

The reliance on Human Resource departments may also explain the popularity of using years of experience, particularly in technical domains. In general, HR representatives aren’t engineers. They don’t truly understand what we do for a living and are unable to tell a good engineer from a bad one. But it’s their job to screen candidates and provide suitable ones to hiring managers. So some criterion must be used to filter out unsuitable candidates.

Experience != Expertise

There is an assumption that a certain amount of experience will produce the desired expertise. However, experience does not produce expertise! Experience is important in developing expertise, but expertise requires that the person has the will and ability to extend their knowledge and refine their abilities. Without this, experience just reinforces your current methods.

On one project, I worked with a person who was responsible for doing the builds, who had been working on this project for 5 years. In that period, the approach used hadn’t changed significantly since the first year. I’m sure he claims 5 years of experience for that work, but it really should be considered 1 year of experience with 4 years of repetition. I’m not saying that the build process needed any changes or improvements. My point is that repeating the same activity over and over should not count as experience. It certainly does nothing to further your expertise.

By focusing on years of experience rather than needed expertise, you may even rule out better candidates. Say that a position has a stated requirement of 3 years of experience in C++. The best programmers learn new languages and technologies quickly. Indeed, a great programmer will have a better understanding of a new language after six months than the average programmer would have after three years!

Most technologies don’t take several years to master. When learning a new technology, the early part of the learning curve is the most intense. By the end of the first year, most people are in the incremental learning cycle where they are picking up knowledge very slowly. From my observations, there is little difference between the knowledge a person has after 1 year of using a technology and after using it for 3 years–unless something motivates them to learn more. Even great programmers are in the incremental learning phase by the end of the first year, but they have attained a higher level of understanding in that period than many others will gain in a lifetime.

In my career, I’ve interviewed many C++ programmer candidates. Most have done little to extend their knowledge past their initial training, often a training class that lasted a week or two. Most have never read any books on the language, other than the book they used to initially learn the language. It’s rare for me to interview programmers who have read even a single book on Object-Oriented Programming. For a person like that, 3 years of experience will not lead to much expertise—unless their project provides an exceptional opportunity to learn from more knowledgeable developers.

The rate at which someone learns a new technology is one of the most important factors in recognizing top programmers. Even if you could write the perfect job description and pin down the exact combination of knowledge and abilities for the current project, that mixture will be out of date when the next project comes along. It’s not uncommon for technologies to change even during the scope of a single project. When it does, what good was that carefully crafted combination of skills?

Yes, Experience Does Matter

This doesn’t mean that experience is unimportant. Experience allows us to learn from mistakes. It provides opportunities for problem solving. Just as important as the knowledge of how to do something is your trouble-shooting ability when things go wrong. This knowledge is often best gained through experience.

Even so, you can’t measure this experience solely in terms of time. It has more to do with the number of lines of code you’ve written and the number of unique technical challenges you have faced than the number of years you’ve done something. Given two programmers with 5 years of experience, the one who has written hundreds of thousands of lines of code will generally be more experienced than the one who has written tens of thousands. Likewise, a programmer who has worked on several projects in that timeframe will likely be more experienced than one who has worked on the same project.

When writing job descriptions, I use years of experience only for the overall number of years as a programmer. This is an effort to factor in the lessons learned from experience. Even then, I’m always watching out for the highly talented person who has come by their knowledge and experience faster than their peers.

What Does This Mean for Me?

In your career, don’t assume that years of experience is enough. Push the learning curve as hard as you can. When you are learning a new technology, seek out the books or other materials in that area that can give you knowledge that would take years to learn on your own (if ever). Use the work you do each day to drive your learning. For example, if you’re writing a new C++ class and you’re unsure whether to overload the copy constructor, use that as a reason to really learn the best approach. Don’t settle for mediocre knowledge.

Having pushed the learning curve, don’t be intimidated if the years of experience don’t match your background when you apply for a position. Stress to the recruiter that you have the knowledge that someone with the specified years of experience would have. Be sure you can back that up in interviews, though. Savvy hiring managers will quickly spot a pretender.

« Previous PageNext Page »