DPM:UK 15 TALK: DID WE REALLY SAY WE WOULD DO THAT?

Technology capability continues to grow at an exponential rate. From Moore’s Law to Gilder’s law, well-known trends provide ample evidence that there will be no plateau in the use of increasingly complex technologies.

By Wgsimon (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons

The social effects are just as astonishing: ten years ago the job of app developer, big data analyst or cloud services engineer didn’t even exist. Any of today’s children destined to work in the tech industry in twenty years time will be working with languages and tools of which we currently have no concept.

This continual journey into the unknown is what makes working on digital projects so exciting; and what makes estimating digital projects so tricky. For most people, knowing how long it’s likely to take to walk to the nearest bus-stop is easy because it is they’ve done it before. However trying to guess how long it might take to walk 50 miles due North proves far trickier as:

  • a) it’s not something you’re likely to have done before
  • b) the margin of error is far greater due to the time and distance involved.

Most software projects of a reasonable size deal continually flirt with the unknown. This makes estimation hard and provokes the question of how to structure a contract when the exact approach  is yet to be worked out.

Traditional client/agency models have centred round a fixed price model, whereby the development team are held accountable to their initial estimates despite not yet possessing all of the information they need. While you can add in contingency to mitigate risk there are still some key flaws with this model:

  1. It’s very hard for client and vendor to avoid an optimism bias at the start of any project and this subsequently places most of the risk on the vendor
  2. If a client expects fixed price and fixed timescales then the only thing that can shift is quality – leading to a product that nobody wants.
  3. This type of contract can lead to an ‘us versus them’ scenario where budgetary pressure gives way to hardened attitudes between client and vendor about ‘what was said’ or what constitutes a change request – essentially it can be a collaboration killer

That’s not to say fixed price can’t be done well. Some companies manage this model extremely well. However, it doesn’t push you into exploring the unknown but is more likely to lead you into seeking out safe ground. This can kill off creative ideas before they’ve been given a chance. Lastly, if you speak to any PM about ‘fixed-price projects’ you’ll occasionally  see an ‘a thousand yard stare’ come across their face as they remember the hardship such shackles can sometimes impose on running an innovative project successfully.

Thousand Yard Stare

 

At the opposite end of the spectrum is working on a time and materials basis. Many agencies do this extremely well. Their reputations and experience encourage clients that continually fund the team till the project has ended despite their being a slight fuzziness as to what the exact bill will be. However, the flaws with this include:

  • The balance of risk is all on the client – They have no guarantees that their budget will be able to deliver them all the features they need to fulfil their business aim.
  • T&M can be erratic – it’s hard to actively manage a backlog when one sprint can suck up a huge amount of time but deliver very little in terms of releasable software; while other sprints deliver a lot more for less dev work. How can you assess the value of a feature when you can’t be sure of its final cost.
  • T&M doesn’t promote efficiency – Any contract where you get paid more for taking longer isn’t really delivering a positive subconscious message about working lean
  • Even if the client has unlimited budget, T&M projects can result in wooly decision making – there’s no fixed end point, so there’s no pressing need to make clear, timely decisions that fit with a lean build.

I’ve had experience working with both sets of models and have found various ways around the problems they can pose. Currently though, I’ve moved to a relatively new model that works well for us and our clients

I call this the Price Per Point model but it’s also know as Fixed Feature or Price per Feature:

  • Functionality is broken down into discrete user stories and we estimate these according to how long/how complex it will be to deliver every aspect of that story: design, UX, development, PM, AM testing, refinements.
  • We estimate these stories in story points. Using points gives us a method of estimating stories in relation to one another rather than fixating on exact times (e.g. it’s easier to estimate that the train station is twice as far away from your house that the supermarket than it is to estimate the exact distances in km).
  • Before starting development as a team we review the initial estimates after a sprint planning session. This gives us a chance to update our estimates based on the very latest knowledge. It can also give the client a chance to add further acceptance criteria, split or simplify a story.
  • Once ready we begin work on the story and progress it all the way through the stages of production until it is at a stage where the client can sign it off before it is pushed to a live site.
  • The client pays a fixed cost for each story point and doesn’t pay a thing until the story has met with their approval and has been fully signed off.

The reaction from clients to structuring billing in this way has been overwhelmingly positive. The key reasons being:

  1. There is a really clear definition of done i.e. Client’s must sign off each story on a staging or live site in accordance with acceptance criteria that they have written
  2. Clients are in control of their backlog and can actively engage with adjusting the scope of their project to meet their budget
  3. The risk on project is balanced between client and agency: the client takes ownership for delivering a functioning product within their points budget; the agency takes ownership for delivering each fixed price component.
  4. It rewards the development team for being efficient but not for cutting corners
  5. It necessitates regular, involved sign-offs from product owners and facilitates a more collaborative environment
  6. It makes it easier to quickly switch directions within a project based on user feedback – a client can adapt and pivot when they’re provided with feedback from real users and are empowered to prioritise user feedback over new features if they wish.

Ultimately, working in this method drives adherence by the agency to an agile working methodology. You simply have to meet your definition of ‘done’ or you won’t get paid.

There are problems. One of the fundamentals of agile is Goodhart’s Law – When a measure becomes a target, it ceases to be a good measure. You can make a pretty strong case that this methodology falls foul of that…however, I feel this relates more to velocity rather than story point though I do accept the criticism.

 

Agile ‘laws’

 

Those in the #noestimates camp might also argue that this still doesn’t solve the overall problem of estimates being inaccurate and unnecessary. However, I think this model can work around such objections, especially in an agency environment. There are a number of things you can do to help do this

  1. Work closely with clients, begin educating them on your estimating process and encourage them and expect them to challenge your estimates. They are  a key point of triangulation in this.
  2. Understand that your estimates will still be inaccurate but over time, a greater project understanding and clear metrics can allow you and the client to make adjustments in order to rebalance any inequality. Invest in consistent and robust estimation techniques and let the numbers provide accuracy over time.
  3. We also have found it a necessity to have a fixed-price or T&M period before the project starts. this covers all the workshopping, server set-up, user story writing etc. The tasks are familiar so we can estimate them fairly accurately but investing in this phase allows us to have relative confidence in setting the price per point for the upcoming project.

Other agencies have tried this model or adaption of it and seem to have also had reasonable success. It’s interesting that I hadn’t heard of this model before but we organically evolved into it. It was only after trying it and doing further research that realised than many other had been using this model for a while:

http://blog.8thlight.com/paul-pagel/2011/12/16/fixed-feature.html

Kurt Haeusler gave a talk on this subject in which he has far more time to expand on these themes and does a far better job than I could. It’s a really interesting video for a number of reasons and I recommend you give it a watch if you can.

Finally, If you what my slides from the event, they are available here:

DPMUK
Or as a google doc:
https://docs.google.com/presentation/d/18R098CKOAxaX9ufjaAOq5YkjdL_l_vT6N1S_P73Cyds/edit?usp=sharing

Please drop me a line if you have any questions or are interested in moving to this model and want some advice. 

 

 

One thought on “DPM:UK 15 TALK: DID WE REALLY SAY WE WOULD DO THAT?

Leave a Reply

Your email address will not be published. Required fields are marked *

*