Author: Yogi Schulz

Hallelujah, we can finally measure the systems development process. No more white-knuckle experiences, unmitigated disasters or horrendous cost overruns.

Sounds great. Can measurement deliver on this fervent hope or is it just the latest silver bullet that will turn into a hollow disappointment?

In the late nineties, we’ve created robust methodologies supported by integrated software, learned the value of management and analysis disciplines, built up vast metrics databases of project and industry experience and developed sophisticated management and control software.

Now we can apply measurement techniques with a vengeance. We can perform peer analysis and count function points. We can perform cost benefit analyses. We can check estimated effort against cumulative experience with comparable projects. We can verify the achievement of value through benefits realization. We can ensure we’re picking the high-value projects through portfolio ranking of proposals. We can monitor earned value. We can swirl, summarize and pivot spreadsheets of analytical data from far-flung databases at the speed of light. We can measure it all. Aren’t we carrying all this measurement mania more than a little bit too far?

Think about the killer systems you drool about in envy. Think about the systems that your competitors have built that keep you awake at night. Did these systems get built because of exhaustive and rigorous measurement? Dream on!

The killer systems that stand out as triumphs in the world of business were built with a fanatical focus on the needs and yearnings of customers, the skillful application of technology, great leadership, impressive perseverance and the acceptance of considerable risk. That’s how mammoth markets are conquered.

Three examples of killer systems that have been widely discussed are: Merrill Lynch Cash Management Accounts, Charles Schwab Electronic Stock Trading and American Airlines Sabre Airlines Reservations. They’ve all been cloned. In most cases, the clones have enjoyed only modest commercial success. In each case, the early bird got the worm or better still, the lion’s share of the revenue, profitability and bragging rights.

If the projects that designed and built these killer systems had been subjected to measurement mania, they would have died a premature death by cancellation. How can something like airline reservations, that had never been done before, survive a cost benefit analysis? Who would measure the benefits realization for electronic stock trading when it threatened the livelihood of stockbrokers who are quite content with the status quo thank you very much? What comparative data would you apply to measure the likely cost of implementing cash management accounts? In each case, measurement mania would have killed the golden goose.

The projects, where measurement techniques can be most relevant, have all been done. These are the projects that chop clerical heads in pursuit of productivity gains and cost control. Because they’ve been done so many times before, lots of experience and comparative data exists. These metrics make measurement successful. But who really cares. Such projects won’t create new revenues, new businesses or even fleeting competitive advantage.

The killer systems of tomorrow are emerging in the skyrocketing World Wide Web. These killer systems include shopping sites that sell consumer products, entertainment sites that hawk movies and CD’s, gaming sites that sell advertising and financial services sites for banking, insurance and stock brokerage.

The characteristics of projects that are creating these WWW systems include fuzzy requirements, rapidly changing designs and excessive staff turnover. The technologies include Java, Javascript, CGI, cookies, certificates and back-end databases. The World Wide Web architecture and telecommunications protocols create a technical environment that’s radically different from mainframe and client/server transaction processing. The available development tools are brand new, the experience of the best people is stated in months not years. Accumulated measurement data and techniques will be of limited value in this near chaotic but tantalizing environment.

You can’t get there from here with measurement mania. The projects that will deliver the killer systems of tomorrow will require visionary management and gritty determination to see them to completion. To these projects, measurement mania means death.