It may be the line of business I am in (consulting), but it seems like finding how to evaluate developers is a common challenge.  There are discussions and even arguments about developer value that impact employees, projects, and even companies.  Although I do not have some divine insight to provide, I do have some food for thought as you consider this obstacle.

Developer Value Varies Geometrically

The most significant challenge in finding the correct mix of skills and adequate compensation is that the growth from low to high developers is not a straight line.  The increase in productivity is geometric.  Thus, a senior developer is often more productive than junior developers splitting the same compensation.  That means that pure math is not going to help you compare resources.

We see this in the way off-shore teams produce when compared to local resources.  Not all off-shore teams are lower skilled than local talent.  However, they often are.  The whole model of off-shore is based on quantity over quality so you can get a team of developers for the price of a local one.  The odds would imply that this will always be a better approach (more is better).  Unfortunately, that is not the case.  Those lower cost resources are not going to bring as much value to the table.  There are a few cases where this will work out better, but those are rare.

Solving Problems Is The Key

The thing to focus on when evaluating software developers is that they are problem-solvers more than skilled technicians.  The ability to write more code in less time has no value if the code does not provide needed solutions.  I often use the terms coder and developer to distinguish between these skills.

Therefore, the real developer value is not in their ability to write code or even understand a language.  The value comes from their ability to analyze and solve problems.  This is why coding tests are rarely useful in evaluating prospects.  On the other hand, interviews and long-form discussions will go further in helping you choose the best candidate for your team.

Hours or Tasks?

Now that we have looked at what gives the most developer value, it is time to consider compensation and bonuses.  This gets far more tricky.  Although you can argue on both sides of this (hours spent or tasks completed), it turns out that both are flawed.  Sometimes it takes many hours to slug through a solution,  at other times your best developer can crank out a solution in a tenth of the time of lesser skilled ones.

I hate to say it, but I find that honesty on both sides is the best answer.  This also requires patience and understanding.  Even the best developers will occasionally lose hours of time on a minor issue or misconfiguration.  If you want to punish them for those “lost” hours, then you need to provide a bonus when they stumble across a solution in far less time than expected.

Ok, Hours Works Best

When you balance out the good and bad luck that impacts developers (and dealing with the mistakes of others) then it seems that hourly pay works best.  That assumes these things balance out.  The good news is that it reduces the level of scrutiny needed.  Hours are easy to track.

When you want to compare developers, you will need to track progress over multiple challenges or projects.  If you have a small number of comparison points, then it is easier to make a mistake.  This is no different from any other form of evaluation.  You need to focus on their overall body of work.  This includes customer reviews and references.

If you only check a reference or two (or see only one or two reviews) then the odds of making a mistake increase considerably.  Take your time and gather enough information to accurately assign a value to your developers.  It will help you build and retain the best team.