Shock Therapy: Self-organisation in Scrum, Jeff Sutherland

Jeff started scrum 1993, CTO of 9 companies. Rolled out Scrum in 5. Works 1 week/month with VC group, 12 companies w/investments so far: "each one is a laboratory for how you work with Scrum".

Scrum is not a methodology, a defined process or set of procedures. it's an open dev framework. The rules are constraints on behaviour that cause a complex adaptive system to self-organise into an intelligent state... enabling an average team to self-organise into a super-intelligent team that goes 10 times as fast as normal teams. The goal: high quality, productivity, fun work. In the VC world, a company has to be 10x better to be worth the investment.

Scrum scales linearly across geographies.

Double speed of coding = 5% improvement end-to-end. You need more than a productivity improvement to see end-to-end gains.

The challenge is to understand self-organisation and how to avoid destroying it. Teams entering this state are often surprised at themselves. There is a common strategy for reaching this state. Management frequently systematically destroys most such teams.

The good news: reaching this state is like learning to ride a bicycle. Once you've learned to do it, you can do it again... given the right conditions. People are not "pluggable resources".

Scrum consists of roles (PO, SM, Team), meetings (sprint planning, daily scrum, sprint review, retrospective), and social objects (impediment list, burndown chart, product backlog, sprint backlog).

It's a pull-based system - practically everything you learn about lean manufacturing can be applied to a Scrum team.

PO needs to deliver really valuable stories at the front; otherwise garbage in, garbage out. If teams get to DONE well, velocity doubles and defects go down 40%. Next doubling occurs when the backlog is in a good state and there is zero in-sprint churn. Next doubling occurs with team self-organising in sprint.

Most companies are more like a Dilbert cartoon: drop planning and docs, just write code and whine. This is ScrumButt! Nokia had the habit of writing 300-400 pages of spec before doing anything, they broke through this cycle but still had the right amount of docs.

In a show of hands, half the people in the room doing Scrum admit to passing the first 3 questions of the Nokia Test.

Only 10% of teams meet all criteria. (I'd say only one of our two teams is currently doing this)

Even ScrumButt brought a measured 35% benefit to teams using it.

Challenges for new agile developers:

1. They can't self-organise.
2. They spend weeks arguing about the format of the board and get nothing done.
3. They don't follow priorities - everyone does their own thing and the sprint fails.
4. They can't get DONE at the end of the sprint.
5. They allow not-READY things to enter the sprint.
6. They can take years to work this out. Some don't.

As investors, Jeffs VC group invests only in Scrum+XP companies. Scrum is driven at board level. Many run management team, marketing, client services and support with Scrum.

What's the secret sauce? They want to change the face of investment in the US.

1. Implement basic Scrum practices and pass Nokia tets
2. Involve management, understand velocity

Some companies see 300% improvement in 3 2-week sprints - it used to take years.

Waterfall : 2 function points/dev pcm.
Scrum: 17.8 function points/dev pcm (w/ Mike Cohn test)
SirsiDynix: linearly scalability of a distributed team, 15.3 function points/dev pcm with team split between Moscow and US.
Xebia: half the team in Holland, half India. 15.1 function points/dev pcm. Their definition of "done" is the customer doing acceptance tests and judging complete before the end of a sprint.

Takes MySpace as an example: 1/3 of the company is waterfall, 1/3 ScrumButt, 1/3 Scrum. They have hundreds of developers, owned by Fox News, have founders running development. Management doesn't understand Agile and isn't committed. They brought in 30 project leaders who tried Scrum and thought it was "getting in the way of controlling projects". They've been trying to get rid of it.

They've tried "shock therapy": a set of good practices, but no choice. In agile, we want self-managing teams, but when the team doesn't know how to do this there's a problem.

Leadership changes from directing (telling the team) to coaching (involving the team) to supporting (team takes the lead) to delegating (team does it all). The goal of a Scrummaster is to work themselves out of a job as fast as possible.

Shows Aikido picture.

It's like learning the tango. You need a coach. "There are many of us in agile practice who have worked in martial arts, and there are lots of similarities".

Scott at Myspace takes 2.9 days per team member to improve team velocity by 240%. He's not necessarily popular!

Scrum as a framework gives teams lots of options. In practice, this overwhelms many teams. Just as customers don't know what they want, many agile teams don't know what to do til they're doing it. Scott's rules stay in effect until a team meets their goals for 3 sprints.

• Sprints are 1 week long initially, to give good visibility of progress. Teams tend to stay with 1 week sprints.

• He has a strong definition of done: it's a thorny issue. Done means feature complete, code complete, zero defects, approved by PO, production ready. Testing needs to be done as near as possible to development. Testers need to sign off on "done".

• All estimates are in story points. Waterfall team took 8 hours, scrum team 10 minutes, same output.

• Scott uses physical information radiator. Scott places the board, he moves the cards so that they're not confused.

• Sprint meeting is 4h once a week. Myspace (and many engineers) consider Scrum has too many meetings. So everything is consolidated into a single 4-hour meeting - and every other meeting is eliminated bar standups. Meetings get shorter over time. They have 3h15m of meetings/week.

• Initial sprint meeting is a demo, retrospective, backlog presentation, and the team questioning motives, suggesting alternatives, etc. Improper backlog items aren't allowed to enter the sprint.

• Multi-tasking is forbidden. Work is done in priority order.

• Scott is polite but forceful. The beginning is rocky, things gradually relax.
  

Scott makes the largest, nastiest impediment go away in 2 days. This underscores the fact that he takes the team seriously and is there to make their world better. His impermanent placement with a team gives him the ability to be a bit confrontational. He tends to leave teams with a 500% improvement.

Teams using this quick format approach hit their velocity quicker. It's a culture shock, but "they only hate you for 2-3 weeks". Teams stay in touch :)

Teams estimate sprint backlogs in story points.

Gives examples of teams: one team dispersed in an office move. One got a command-and-control manager who removed the PO and split the team up, they want back from a 300% improvement to 100%. The fourth team lost their PO and have been looking for one ever since, they're not delivering much of value.

Gives an example of a team which did well, disbanded (lost all productivity), and regrouped (got it all back).

It's good to have an experience SM to hand; and a good senior developer with agile experience.

Shows video of Maori Hacka (sp?). "Most IT teams aren't like this".

My question: "As a service company doing work to order. we find that our customers have grown up with the notion that fixed-price fixed-scope contracts are reasonable. In our experience velocity can improve as familiarity with a customer or product develops, and can vary between teams and projects.

So two questions: what alternative models are there for contracts that better fit a Scrum team?

And what strategies are there for fitting a Scrum development process around a fixed-price fixed-scope contract?"

A: The "money for nothing, change for free" model where you have a standard fixed-price contract with an additial clause stating that (a) at sprint boundaries, customers can substitute equivalent-sized features for any not yet begun, with no penalty - encouraging acceptance of change, and (b) customer can walk away before end of project when they've realised the value they wanted, sharing savings with the supplier.

Jeff is seeing consultancies used to 15% margins chasing 60% margins by deliberately making customer aware of ROI cut-off points, and thereby ending projects after 3 months instead of after 20, seeing margins increase as a result.