Metrics of Success in Development – Part 3

Today we’ll finish the list of ten questions that can give you a quick measure of your development group or department. The purpose is two-fold: to let you see how you measure up compared to other similar departments, and to suggest ways in which you can think about the stresses in your department.

Let’s launch into the final four questions, then we can total them up.

7. Viewed from other departments (outside of Development), how would managers rate your development managers and engineers in each of the following areas?
(a) Cooperativeness (with outside people)
Extremely cooperative – add 3 points
Very cooperative – add 2 points
Cooperative – add 1 point
Uncooperative or unavailable – add 0 points
(b) Flexibility (willing to work with and compromise with others outside the department)
Extremely flexible – add 3 points
Very flexible – add 2 points
Flexible – add 1 point
Inflexible – add 0 points
(c) Team-orientation (beyond the Development department teams)
Extremely team-oriented – add 3 points
Very team-oriented – add 2 points
Team-oriented – add 1 point
Not team-oriented – add 0 points

8. What percentage of the company’s gross revenues in the most recent year were allocated to Development (or R&D)? (If your company has no revenues, or if revenues are less than the Development budget, answer “over 10%”)
(a) over 8% – add 3 points
(b) 4 to 8% – add 2 points
(c) 2 to 4% – add 1 point
(d) less than 2% – add 0 points

9. Comparing this year’s Development budget to last year’s, how did it change?
(a) Increased by 20% or more – add 3 points
(b) Increased by 5 – 20% – add 2 points
(c) Stayed the same or changed by less than 5% – add 1 point
(d) Decreased by more than 5% – add 0 points

10. The number of concurrent development projects now in my department is
(a project is defined as an activity with a timeline and a goal which needs at least 1 full-time person to make progress towards the goal; if you have many small projects, you can count the number of project leaders instead)
(a) over 25 – add 0 points
(b) 15 to 25 – add 1 point
(c) 5 to 15 – add 2 points
(d) 1 to 4 – add 1 point
(e) none – add 0 points

If your total points add up to 52, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 37 to 52 points
Good: 22 to 36 points
Fair: 13 to 21 points
Poor: 12 points or fewer

How did you do? What does this mean? If you think about the stresses on your department, you can see that the point score is not as significant as the individual issues you’re facing. Are you having a lot of turnover? Slipped schedules? Complaints from other departments about your people? These can all be aggravated by declining budgets which are outside of your control.
Later we’ll examine some of these issues and how you can find ways to work around them. In the mean time, click on the Comment button and let us know how you scored.

Metrics of success in development – Part 1

How do you find out if your development organization is functioning well? Naturally, if you are getting products out on time, consistently, and the world around you is happy with the results, you have nothing to worry about.

But what if there ARE complaints? Can you determine whether you’re hearing gripes that have little to do with you? Or whether there really is room for improvement?

I’m working on a self-assessment tool, probably containing about 10 questions, that will help you evaluate whether you are on track. The way to use the tool is to answer the questions and then count the points at the end.

Here are the first three questions in my current working draft:

1. Counting only the past 3 projects and products under your management, how many have been completed on time? For each project/product not completed on time, how much later were they completed?

[4 points] for each project completed on or before the originally scheduled completion date
[3 points] for each project completed within 120% of the originally scheduled duration
[2 points] for each project completed within 150% of the originally scheduled duration
[1 point ] for each project completed after more than 150% of the originally scheduled duration
[0 points] for each project which is never completed

2. During the time that those 3 projects or products were being developed, how much turnover did you experience among your technical staff, including first-level supervisors and managers? Turnover means departed or transferred out from project teams or management without being invited to do so by you or your managers.

[3 points] less than 5%
[2 points] between 5% and 15%
[1 points] between 15% and 30%
[0 points] over 30%

3. In those 3 projects or products, how much of the planned functionality was delivered (at the completion date you used for question 1)?

[add 3 points] for each project in which you delivered all of the planned functionality, plus additional functionality defined after the start of the project.
[add 2 points] for each project in which you delivered all of the planned functionality
[add 1 point] for each project in which you delivered at least half of the planned functionality
[add 0 points] for each project in which you delivered less than half of the planned functionality

If your total points add up to 24, you have a perfectly-performing development organization and have no need for improvement. For the rest of us, the points are probably in the following ranges:
Excellent: 18 to 24 points
Good: 12 to 17 points
Fair: 6 to 11 points
Poor: 5 points or fewer

Shipping products on time is the key result that most of your stakeholders want. Shipping the product with the features and functions that they asked for — or expect — is next in line as a measure of your success. But if you are burning out your development crew — and causing turnover as a result — you won’t be able to sustain the results. Therefore, these are the first few measures that tell you whether the organization is successful and sustainable.

Next time we’ll begin looking at other measures that can tell you whether your management practices are helping you build a development organization that is consistent and successful for the long run.

Constant Reinvention = Survival

Nothing lasts forever. Even the best-conceived business strategies eventually become constraints on growth.

Consider Dell. “Dell succumbed to complacency in the belief that its business model would always keep it far ahead of the pack.” But the competitors got better while Dell failed “to invest in new business lines, talent, or innovation that could provide another competitive edge.” * [see Business Week citation below]

As a leader in technology or product development, you may think that your entire job is to execute well on the development plans laid out by Marketing or a strategic planning group. But you can do more. You can help the executive staff recognize that the business has other opportunities.

Consider what the Business Week authors went on to say: “Long-term success demands constant reinvention.” This means that while you’re turning out products that meet the current set of goals for functions, price and quality, the viability of the company may depend on your pointing out where innovative products or services could come from, using the brains you already have on your staff. Reinvention means re-thinking the orthodoxies everyone has accepted as the characteristics of the company. Do you have some independent thinkers on your staff who keep coming up with off-the-wall ideas? Maybe some of those ideas are actually your ticket to survival.

“Nurture the next growth platform long before it’s needed.” This means you have to carve out the budget to support the radical ideas from the operating budget you’re supposed to use for mainstream development. If you can’t convince your executive staff to fund a skunkworks operation, then you should look at having some of your key contributors doing some off-the-record investigations. Is this risky in your company? Then maybe the company needs some shaking up.

“Most [companies] don’t [nurture the next ideas]. Distracted by the demands of their current success, they re lulled into a false sense of security.” It’s easy to focus only on the tasks that will satisfy the demands of current customers and current ways of doing business. And while it’s not easy to perform on those tasks at extraordinary levels, you can get lost in gunning for the immediate satisfactions of meeting this quarter’s goals. Can you be a VP or Director of Engineering and still make time for thinking about next year’s products and the businesses that you haven’t entered yet? Consider this: who else is better positioned to view what’s possible, who is out there needing better functions or services, and what can be combined to make a new business?

I suggest taking an advocate’s role as part of your commitment to the long term success of your company. You don’t have to be a marketer or business analyst to know what’s exciting and feasible in the next generation of products and services. Carve out a niche as a visionary and a keeper of the wild ideas that can open up new busineses for your company. Do it regularly, and you’ll be twice as valuable as the person who only meets the usual goals of Development. Besides, it may help your company survive.

———-
* “Where Dell Went Wrong” by Nanette Byrnes and Peter Burrows in Business Week, February 19, 2007, pages 62-63.

http://www.businessweek.com/magazine/content/07_08/b4022074.htm?chan=search

Shifting the focus to longer term

Startup organizations are typically unsustainable and barely stable, because:

1. The pressures to develop and market a first product require taking some expedient shortcuts, such as hiring the most capable, but not necessarily the most team-oriented individuals; placing all priority on getting a workable product out the door, rather than building the product for maintainability and growth; putting in the most features rather than the best-tested features.

2. The top management habitually focuses on the race between funds running out and product delivery, rather than on internal communications, employee satisfaction (except with the potential value of their options), and leadership style. The command-and-control management style is workable for the first few years, but typically fails to inspire the organization to build itself into a self-renewing structure.

3. Having a focus on delivering a product using already-developed technology, the company does not need to invest in longer-term development of underlying technologies, or in the people who will bring in a steady stream of new technology.

The short-term focus of a startup must change soon after the deliver of the first few products. Companies that fail to incorporate longer-term thinking around their third year find themselves living from crisis to crisis. This makes the company unattractive to good managers and good technologists who don’t necessarily get their jolllies from living in a startup environment, where “startup” means short-term thinking.

What sort of changes does your organization need, now that the product has been delivered? A new CEO who actually allows the organization to function as if there were competence at all levels? A seasoned technology executive who knows what to do to make the organization attractive to innovative people? A shift in emphasis to listening to customer feedback and involving existing customers in product decisions? Addition of a Quality department that actually has the teeth to delay a product introduction?

Whatever the changes needed, don’t be surprised by the shift. Two reactions to the shift are typical:
(1) “What happened to my adrenaline rush?” — the people who need crises to keep up their energy should pursue another startup.
(2) “I didn’t know that a company could actually plan and execute with the future (beyond 1 month) in mind!” — the people who are stressed by the company’s failure to plan and execute for the long term grow into steady, reliable contributors.

What is Product Marketing’s role in development?

A colleague asked, “Do you believe Product Marketing could be the bridge between the Engineering and R&D organizations? It seems to me that market requirements are the other piece to incorporate there and Product Marketing could add that to R&D’s specs before working with Engineering to determine what’s feasible and on what time schedule… What do you think?”

It works at the front end to have an Advanced Development group build prototypes and conceptual specs/models, with specs or requirements added by Product Marketing before giving them to Engineering. But the problems (below) don’t come out until the crunch — when you’re waiting for the next milestone in actual product development.

The problem is that Product Marketing and Engineering (the department responsible for developing products delivered to customers) have a natural tension: they have to arbitrate between what’s feasible within the time/dollar/featureset constraints; Product Marketing should have the customer deliveries in mind, and should interpret “what the customer wants,” while Engineering is responsible for determining which (and how many) features can be delivered within the cost and time constraints.

As a product is developed, Engineering will naturally come back now and then to renegotiate features vs. schedule (and sometimes $) as they uncover problems (or opportunities) that impact schedule. Product Marketing cannot act as the arbiter for this negotiation — Engineering must participate as a fully-responsible party, determining what can be delivered when. When Product Marketing has all the power in this negotiation, you either get emasculated Engineering, which won’t take any chances because they’re being second-guessed; or you get promises that can’t be kept, because Engineering isn’t really running the development process.

A + B + C ≠ D (The game changes at the fourth round of financing)

When a startup company reaches a certain size, a number of changes have to occur to allow it to survive. Here are some of them:

1. The founders have to choose new roles for themselves. Having been key idea-people or leaders of a particular part of the business process, they may have trouble envisioning themselves in a role that meshes with a larger company. This is OK — particularly if they are willing to go off and found another company. Where it’s not OK is in a company that desperately needs to establish processes that work for the long term, and a founder is standing in the way of moving in that direction. The impetus to change things may come from the venture investors, from other key executives in the company, or — rarely — from a founder him- or herself. ‘Tis a wise founder who knows his/her own limitations.

2. Product development has to become more predictable. While at this stage of growth a company often is launching multiple development projects at the same time, the need to know when the process will complete becomes more important as the customer base grows and Marketing starts implementing more sophisticated product introductions. In addition, the engineers who have survived the startup environment are often close to burnout, accustomed to an unstructured work environment, and unwilling to consider that in a larger company, risk has to be reduced. What kind of risk? Things like making sure that software backups are made, versions of code are archived, drawings are in fire-safe locations, and that the website is not offering free access to proprietary design information.

3. Knowing how long it will take to implement certain features, whether software or hardware implemented, is key to becoming predictable. Predictablity can be helped by agile development methods, which emphasize frequent demonstrations of working models, making regular estimates of output over the next few weeks and refining one’s ability to predict that output. This gives the developers a lot of say in the process, while also getting realistic “customer” feedback on features and functions on a regular basis.

4. The company management may have to pay attention to issues that aren’t so prominent during the early startup phase, such as infrastructure (development tools, website and equipment maintenance), retention & professional development, trade associations & standards, and intellectual property protection.

5. Scaling up the company is not just a matter of cloning the existing projects and production lines. As a new layer of management is brought in to allow expansion of the operating departments, the top management must examine its way of working, including values, culture, communications, and transparency. Plotting strategy without considering these factors can leave them wondering why the workforce isn’t following management’s lead.