Pointing the Finger

By 4 commentsUncategorized

The problem is sales! (well, inheritance, but it’s not as catchy, so I’ll say sales. See this post for more.) Is this because sales people are a bunch of greedy, selfish, blood sucking leeches? Occasionally, but mostly not. Most salespeople worth their frequent flier miles do have some regard for the organizations they work for and the people who will ultimately deliver what they have sold.

I think there are two primary issues. The first is explained by the old adage, “you get what you measure,” which could be modified for salespeople as, “you get what you pay for.” How are these people typically compensated? Usually, their pay is tied to a percentage of the selling price. So they are incented to sell as much work as possible for as high a price as possible. Nothing in there about project success. There are a few places that do compensate based on customer satisfaction, but few look at the long term success of the project. No one hangs around that long. Can you imagine how things would change if a person’s commission was based on client satisfaction, on-time delivery, the final margin, and whether the project met the long term business goals set out in the beginning of the project?

That leads us to the second cause, which I believe is historically long delivery cycles. People are used to long waits in between releases. If it will be a year or more before changes can be made, you better believe I’m going to try and get every feature I can into this release. The irony is that most of these features aren’t used. Jim Johnson of the Standish Group noted that 64 percent of the features in a typical system are rarely or never used. (1) On the one hand, that boggles my mind, but on the other, I’m not surprised. Just think of how much faster the next release would come if you didn’t have to design and build that 64% of features that will hardly see the gentle click of the mouse.

Here’s the rub. Given a fixed time and budget, the more features I want, the less time I have to spend VERIFYING THAT THESE ARE THE RIGHT FEATURES TO BUILD! Let’s follow the logic. The longer it is between releases, the more I need done now. The more I need done now, the less time I have to validate that what I’m doing really solves my users’ problems. I’m starting to wonder if it’s much more than 64% that goes unused. We know intuitively that less is more, but now we attach a business value to it.

The latter is one of those problems that agile/iterative/lean/etc. development is intended to solve. You get less, but at regular intervals.

That begs the question of how can exert influence when we are fighting a poorly conceived compensation structure and the weight of years of experience? Thoughts?

(1) Johnson, “ROI, It’s Your Job.”

, , ,

Share this post

4 comments on “Pointing the Finger”

  1. Drew Simchik Reply

    Maybe I’m dim tonight, or maybe it’s just that the space I work in doesn’t have these specific problems, but I’m having a lot of trouble following your reasoning in this post.

    In my organization the sales reps — the ones compensated by commission — don’t drive product requirements at all. Also, we don’t have especially long delivery cycles, since most of our stuff is web-based. We’ll add most features within a month or so. And our customers pay for subscriptions, so they’re being sold on a relationship as much as on a specific product; the promise of a forthcoming feature is often almost as compelling as the presence of that feature.

    Which brings me to Jim Johnson’s statistic, about which I’d like more details (is “ROI, It’s Your Job” available on the web somewhere? I don’t even know if it’s a magazine article, a seminar, a blog post, or what). The implication is that 64% of the features are wasted work, but that would be true only if they were _never_ used. How often is “rarely”? What percentage of users never touch the 64%, and what percentage have pet features they do use occasionally (and value them highly when they do)? Which of the 64% are truly never used, and which rarely? There’s a big difference between a feature Sales convinced you to shoehorn in because it sounded good but is truly useless, and a feature that’s just infrequently used by a minority of users. In the space I currently design for, customers who use the products more actually generate more revenue, such that a minority of customers is responsible for a majority of revenue, and that minority often has special needs, features that are way beyond the needs and abilities of the numerical majority.

    So I can’t really answer your ultimate question, because a sales-driven product development process is bewildering to me. In a healthy organization it seems to me that sales goals are tied to business goals, and if usability can’t demonstrably serve the business goals, we’re fighting a religious war and not a practical one.

    Does that make sense, or did I totally miss the point?

  2. Jeremy Kriegel Reply

    You’re not dim. You bring up some very good points. I’ll address the easy one now, but want to be more thoughtful on the harder one.

    Who is Jim Johnson and where does the 64% come from? I found this article referenced in “Lean Software Development” by Mary and Tom Poppendieck. I highly recommend the book. Even though it focuses on dev, I found a lot of the techniques interesting and we’re working on integrating them with our evolving design process.

    Unfortunately, this article does not seem to be available anywhere. I’ve searched some library archives and the content has been removed from the wayback machine by the owners.

    However, there is material on the author, Jim Johnson. In the preface to one interview, the writer wrote:

    “Jim Johnson is the founder and chairman of the Standish Group, a globally respected source of independent primary research and analysis of IT project performance. He is best known for his research on why projects fail, as well as on system costs and availability. He is also a pioneer of modern research techniques such as virtual focus groups and case-based analytical technology.

    The Standish Group’s claim to fame is, of course, The CHAOS Chronicles, comprising 12 years of research, done through focus groups, in-depth surveys and executive interviews, on project performance of over 50,000 completed IT projects. The objectives of CHAOS research are to document the scope of application software development project failures, the major factors for failure, and ways to reduce failure. In 1994, The Standish Group made public its first CHAOS Report, documenting the billions of dollars wasted on software development for projects that were never completed. That report is among the most oft-quoted in the industry since then.”

    Unfortunately, I can’t give you more info than that. I may try and reach out to the Standish Group and see if this paper is still available. While any statistic is subject to how it is acquired, and your questions are valid, at least I feel there is some reputable authority behind it.

    As to some of your other points, I think they are important. Not every company is driven in this way, but many are. I’ll compile my thoughts on that and add another comment soon.

  3. Jeremy Kriegel Reply

    I think you are right that perhaps not all companies are subject to this phenomenon, but I don’t think that this is to far from your experience. In response to the post on root causes, you said that proper testing takes time, the business wants to move forward based on the knowledge it has, and there are issues resolving conflicts later on.

    I will posit that this is not too different from what I’ve described, albeit with slightly different rationale. I think it still comes down to feature dev vs. testing and verification. I’m assuming that if you are not involved in user validation, you are developing new features or refining old ones.

    Who drives business priorities? Whomever it is, it seems like developing more features is given a higher priority than testing current designs and researching potential work.

    What would happen if you didn’t add any new features for 4-6 months and focused on refining the current product(s) and researching major pain points you could address? Would you lose customers? Would the acquisition rate slow? If you were able to create some dramatic solutions, how would that effect the business once they launched? How would it change the process internally to have the research needed to make informed decisions?

    You may put out frequent releases, but it sounds like there is a reluctance to substitute one (or more) of those releases for a research/refactoring phase.

    I may very well be wrong. I’m basing this on a few paragraphs that I’m sure tell far less than the full story of your company.

    Out of curiosity, do you have metrics on your site’s features? How many are used very infrequently? Do you think that your customers would have abandoned the platform if you hadn’t included them?

  4. Drew Simchik Reply

    To answer your questions:

    In a way what you describe is happening now, though to explain in more detail than that would be difficult without disclosing more info than I really should. 🙂 What’s interesting is that we’re researching and addressing pain points and in many cases this looks a lot like adding new features, at least for some of our users.

    We do have some metrics, though not as many as I’d like, and they’ve definitely been helping drive the new designs. It turns out that some of the power we designed into the platform has gone largely untapped, though I think we added it with the very best of intentions, and not merely to sell more subscriptions. We might have misread our users’ potential needs, but not because of naked featuritis. In some cases research would have helped, but the nature of some of the features was so novel that researching users’ current needs couldn’t have predicted the potential the features could have had for them. Perhaps we merely failed to design them well enough that they and their potential were discoverable, which might be the real reason for the 64%; it’s tough to foreground every feature. You wouldn’t want to.

    We do have make-or-break features for some users, and I don’t think their demands are all bluff — that is, some will definitely defect, and some have done so!

    So I don’t agree that unused features only show up because some sales guy wanted a commission, or some marketing guy thought they would look good on the side of a box. I agree with you, though, that they often show up due to insufficient research, which would show how cornery the corner cases that necessitate the feature really are.

Leave A Reply

Your email address will not be published. Required fields are marked *