Metrics of Success in Development – Part 3

Today we’ll finish the list of ten questions that can give you a quick measure of your development group or department. The purpose is two-fold: to let you see how you measure up compared to other similar departments, and to suggest ways in which you can think about the stresses in your department.

Let’s launch into the final four questions, then we can total them up.

7. Viewed from other departments (outside of Development), how would managers rate your development managers and engineers in each of the following areas?
(a) Cooperativeness (with outside people)
Extremely cooperative – add 3 points
Very cooperative – add 2 points
Cooperative – add 1 point
Uncooperative or unavailable – add 0 points
(b) Flexibility (willing to work with and compromise with others outside the department)
Extremely flexible – add 3 points
Very flexible – add 2 points
Flexible – add 1 point
Inflexible – add 0 points
(c) Team-orientation (beyond the Development department teams)
Extremely team-oriented – add 3 points
Very team-oriented – add 2 points
Team-oriented – add 1 point
Not team-oriented – add 0 points

8. What percentage of the company’s gross revenues in the most recent year were allocated to Development (or R&D)? (If your company has no revenues, or if revenues are less than the Development budget, answer “over 10%”)
(a) over 8% – add 3 points
(b) 4 to 8% – add 2 points
(c) 2 to 4% – add 1 point
(d) less than 2% – add 0 points

9. Comparing this year’s Development budget to last year’s, how did it change?
(a) Increased by 20% or more – add 3 points
(b) Increased by 5 – 20% – add 2 points
(c) Stayed the same or changed by less than 5% – add 1 point
(d) Decreased by more than 5% – add 0 points

10. The number of concurrent development projects now in my department is
(a project is defined as an activity with a timeline and a goal which needs at least 1 full-time person to make progress towards the goal; if you have many small projects, you can count the number of project leaders instead)
(a) over 25 – add 0 points
(b) 15 to 25 – add 1 point
(c) 5 to 15 – add 2 points
(d) 1 to 4 – add 1 point
(e) none – add 0 points

If your total points add up to 52, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 37 to 52 points
Good: 22 to 36 points
Fair: 13 to 21 points
Poor: 12 points or fewer

How did you do? What does this mean? If you think about the stresses on your department, you can see that the point score is not as significant as the individual issues you’re facing. Are you having a lot of turnover? Slipped schedules? Complaints from other departments about your people? These can all be aggravated by declining budgets which are outside of your control.
Later we’ll examine some of these issues and how you can find ways to work around them. In the mean time, click on the Comment button and let us know how you scored.

Metrics of success in development – Part 1

How do you find out if your development organization is functioning well? Naturally, if you are getting products out on time, consistently, and the world around you is happy with the results, you have nothing to worry about.

But what if there ARE complaints? Can you determine whether you’re hearing gripes that have little to do with you? Or whether there really is room for improvement?

I’m working on a self-assessment tool, probably containing about 10 questions, that will help you evaluate whether you are on track. The way to use the tool is to answer the questions and then count the points at the end.

Here are the first three questions in my current working draft:

1. Counting only the past 3 projects and products under your management, how many have been completed on time? For each project/product not completed on time, how much later were they completed?

[4 points] for each project completed on or before the originally scheduled completion date
[3 points] for each project completed within 120% of the originally scheduled duration
[2 points] for each project completed within 150% of the originally scheduled duration
[1 point ] for each project completed after more than 150% of the originally scheduled duration
[0 points] for each project which is never completed

2. During the time that those 3 projects or products were being developed, how much turnover did you experience among your technical staff, including first-level supervisors and managers? Turnover means departed or transferred out from project teams or management without being invited to do so by you or your managers.

[3 points] less than 5%
[2 points] between 5% and 15%
[1 points] between 15% and 30%
[0 points] over 30%

3. In those 3 projects or products, how much of the planned functionality was delivered (at the completion date you used for question 1)?

[add 3 points] for each project in which you delivered all of the planned functionality, plus additional functionality defined after the start of the project.
[add 2 points] for each project in which you delivered all of the planned functionality
[add 1 point] for each project in which you delivered at least half of the planned functionality
[add 0 points] for each project in which you delivered less than half of the planned functionality

If your total points add up to 24, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 18 to 24 points
Good: 12 to 17 points
Fair: 6 to 11 points
Poor: 5 points or fewer

Shipping products on time is the key result that most of your stakeholders want. Shipping the product with the features and functions that they asked for — or expect — is next in line as a measure of your success. But if you are burning out your development crew — and causing turnover as a result — you won’t be able to sustain the results. Therefore, these are the first few measures that tell you whether the organization is successful and sustainable.

Next time we’ll begin looking at other measures that can tell you whether your management practices are helping you build a development organization that is consistent and successful for the long run.