Why (So Many) Testers (Still) Hate Agile, a presentation from Simon Bennett of EMC and Malcom Beatty, a "career tester", talking about how moving to an agile environment changes the role of testers in a development team... and talking about what this means for them.

Traditionally testing uses the V model - with testers validating specification by looking over documents from the business, deriving test cases and then verifying that a product conforms to these cases. Most communication with developers is done via bug reports, leading to an adversarial relationship with developers and a sequential structure to the project... but hey, at least it's test-first development in one sense.

Why do some testers hate agile? Early XP implementations advocated that developers do their own testing, thereby making the role of tester redundant - you can't blame testers for resenting this, and it's unwise given what else they can bring.

Other classic dislikes are that "there's no point in automating stuff that might change", and "working in sprints is too bursty" (both points that we've had raised at FP - which I find comforting :)).

There was also a point raised about bugs going into a bug tracking system, rather than a backlog. We've been managing both systems over the last couple of years, and I can see why this might cause problems - you end up with 2 sets of priorities, effectively... and there's gaps between these two sets of priorities through which things can fall.

Simon and Malcolm had surveyed testers to ask about their aspirations, likes and dislikes. A good day means raising good bugs, closing them, having a final say in getting product launched, and preventing disasters. A bad day involves trying to grok poor documentation, reported bugs not getting fixed, and "bug noise" - i.e. poor bugs which are typically feature requests raised by a customer.

Testers surveyed typically hated planning poker, testing unfinished work, and having conversations with developers about bugs which had been opened and well reported already.

We then broke into groups to discuss how different aspects of the Agile Manifesto might cause problems for traditional testers. My table (on which I think I was the only non-tester) chose the manifesto point preferring "Working software over documentation". We came out with a few reasons why traditional testers might dislike this:

1. Testing stuff that isn't fully done is frustrating;
2. The definition of done might fall outside a story definition (with e.g. bugs found through destructive testing and button-bashing);
3. Documentation can provide good proof of effort which might be helpful in an adversarial dev/tester environment;
4. This proof of effort might be helpful to show ones contribution to a project, or personal ownership of a section of it;
5. Documentation of bugs opened/closed in a bug tracking system can provide metrics to demonstrate quality of the dev or testing team;
6. Too many bugs in a backlog could make the backlog unmanageable!

Interestingly, lots of testers at the table I sat at were pro using the backlog for all bugs. One girl mentioned the use of STrac for backlog management.

There was then talk of the ideal tester/dev ratio - it seemed to vary between 1:2 and 16:1 depending on project.

Constant testing throughout the project (as opposed to a test phase at the end) means that the test phase can't be "shrunk down" under schedule pressure, to move the live date forwards or to accommodate complexities in implementation. Interesting, I see the same thing in our use of regular Gold Cards for R&D purposes - if we'd set aside a week or so in the future for such activity, it's the first thing to be cut, but if it happens a little at a time it's much easier to justify.

So what's the role of testers in an agile environment?

• Help with requirements definition and acceptance criteria, by taking sometimes vague statements of intention from the product owner ("must be user friendly", "must be high performance") and clarifying them so developers can usefully create code that meets them...
• ...thereby making developers lives easier;
• Use automation to free up tester time, to do exploratory testing;

The last point grated with me. We've really struggled with automation of our testing over the last year and it's been a point of contention within our team for some time - providing questionable value at a disproportionate cost. This is something I want to think about some more, I struggle to believe that automation couldn't deliver some value, but we haven't cracked it yet.

Enjoyable session. Like the distributed teams one, it's comforting to hear that so many of the issues we've had at FP aren't "just us"... and I have a few tweaks to suggest when I get back :)

Ade Miller of Microsoft on Distributed Agile: things the platforms and patterns group has learned running distributed teams.

The slides should be on his blog, but a few points jumped out at me:

• Colocation not only lowers the overhead of communication, but also removes all that thinking you otherwise have to do about how to communicate;
• There are many types of distributed team - from completely distributed (a level playing field, no-one colocates with anyone else) through distributed-by-function (builds walls between disciplines, loses bandwidth, trust and availability), to ad-hoc distribution (e.g. everyone colocated except one guy "who will hate you") and teams distributed across time zones (often doing "follow the sun" development where stories are handed off as working days start and end);
• Were they not distributed, many of the practices of distributed teams (handing over work, communicating by email only) would be considered dysfunctional!
• Trust is easy to lose in such teams, generating a "them and us" atmosphere; dissipate this by relocating teams periodically, humanising remote team members using photography, or actively building a shared sense of humour across teams;
• Many agile practices assume everyone is together - e.g. pairing, story cards as placeholders for conversations, standups. "You need to step back from the practices and look at what they're trying to achieve";
• Conway's Law: your software will start to resemble your organisation, if you let it;
• Bandwidth is important for distributed projects, poor availability of bandwidth will preclude options like videoconferencing;
• Travel at the start of projects builds trust, but also do it before large releases, at the very end of the project (for retrospective and ship party), and maybe on regular rotations;
• Got one remote developer? Use a "buddy system" - assign someone specifically to represent them and look after their interests within the team;
• Tools can make it harder for you to modify your practices; they tend to be set in stone. They're more important when you're distributed, but aren't the be all and end all;
• Continuous Integration is more important with distributed teams, where it's tougher to keep things in sync. Use widgets or system-tray resident tools to replace Big Visible Charts and promote the state of builds to developers;
• DO adapt. DON'T follow stock practices. You're breaking the rules already by not colocating, so you may need to break others to get it working;

Jeepers, the next few months are looking busy. I've just noticed that I'm out and about quite a bit, at quite a few mobile/dev/UX events:

• I was Highly Chuffed to make it onto the lottery allocation of tickets for UXCampLondon on 22 August. Hmm, need to have a think about that one. I will have a rucksack full of handouts and wooden mobiles though...
• ... as the next day I'm off to Agile2009 in Chicago, where Joh and I are running the "Mobile Mountain" workshop we've been prepping (and practicing) for the last few months;
• September 4th brings dConstruct to Brighton, which is taking more of a mobile/ubicomp slant this year. I'll be the guy at the front nodding vigorously and salivating;
• The next two days, we have BarCampBrighton, which - if I can wangle my way in - I'm really looking forward to - last years was fantastic, it's the perfect decompression from dConstruct;
• The following weekend I'll be at EcoMo09, a 24 hour dev camp focused around creating tools to help people minimise their environmental impact. The pressure's off though, instead of competing in the hack day I'll be sat smugly on the judging panel, in full Simon Cowell mode;
• OverTheAir returns on September 25th. I'll be running the "Mobile Mountains" UX workshop once more (by now well-oiled and tweaked after Agile2009);
• At Mobile Web and Apps (20-22 October), I'm very excited to be doing a talk titled "Capitalising on Popular Culture: The Interplay Between Apps and Society", which sounds fantastically interesting and will be utterly amazing once I've worked out what to say and written it all;
• A few of the FP crew, myself included, are heading up to the StackOverflow DevDay in London on 28th October.
• November spawns a monster - on 18th, I'll be at Mobile User Experience, taking part in a panel discussion discussing what sort of mobile services we'll see over the next 5 years;

Pop me an email if you'd like to catch up at any of these...

• Tom deMarco on software engineering - READ THIS.
• The Android Multitouch Conspiracy, or lack thereof.
• My Life Offline: "I felt not just happy, but firmly happy — solid, is the best way I can put it. I felt like I was in control of my life instead of the other way around, like its challenges just bounced off me as I kept doing what I wanted. Normally I feel buffeted by events, a thousand tiny distractions nagging at the back of my head at all times. Offline, I felt in control of my own destiny. I felt, yes, serene."
• Design in the Open, "a community of practice for design & user experience people in Open Source"
• Mozilla Design Challenge, Summer '09: "Reinventing Tabs in the Browser – How can we create, navigate and manage multiple web sites within the same browser instance?"
• The Pushbutton Web: "where any site or application can deliver realtime messages to a web-scale audience, using free and open technologies at low cost and without relying on any single company like Twitter or Facebook"
• Ricardo Semler won't take control: "If you open up everything, including the company’s books and board meetings, you’ll find that the employees are honest, responsible people."
• Webkit has a new web inspector;
• Terrifying password "recovery" (read cracking) speeds;
• You can't move these days without tripping over a zippy Android UI demo...
• Stats on iPhones - interesting to see the breakdown of both hardware and software versions out there. I'm surprised we're not hearing more people point out the obvious: iPhone hasn't avoided fragmentation, just managed it... thus far;
• How 31 year olds consume media: "Will consider getting a Wii when Chucky Egg and Pong are available."
• What Jeff Bezos knows: "The only reason Amazon exists today in any form: we always put customers first. We always obsess over customers."
• Ive on Apple, I loved his comments about taking responsibility for execution of design: "design is about much more than the studio and the drawing board"

Joh and I ran a dry run for our Agile2009 workshop tonight, held at the mighty Skiff in the North Laine, Brighton... and after a frantic day of last-minute prep, I was quietly pleased with how it went.

The session opened with a few slides setting the scene about mobile. These definitely need some work - whilst I know the material quite well, I hadn't prepped them and it showed. I waffled somewhat, but fortunately at such a speed that it was over quickly for the audience, and we could get on with the meat of the workshop: exploring how an iterative design process can help address the kind of constraints that mobile inevitably entails handling.

We split into two groups - with 9 attendees this gave us teams of 4 and 5 respectively. Persona in hand and product concept in mind, one team headed upstairs and the other stayed downstairs, each spending 15 minutes designing a product with the aid of copious amounts of card, post-it notes, the ubiquitous Sharpies and some rather natty 1.5x sized replica mobile phones my dad and his girlfriend had put together for the session. After 15 minutes we halted the groups and swapped one member between them, to act as a test subject for a 5-minute usability test... after which the groups convened to show off their work, and the results of the test, in a short demo session.

Constraints were introduced, with each team having to transpose their product to a new device and form factor. Then rinse, repeat: another 15-minute design session followed by 10 minutes of demos, and a 10-minute run through to discuss learnings.

After the shaky start of my slides, I was pleasantly surprised by how things went: both teams managed to produce a useful design and test it within 25 minutes. One team definitely received worthwhile feedback from their test subject, the other got some but were less sure of its value. In any case, the notion that a group of people could go from zero to a coherent and in some way tested prototype in such a short space of time was both heart-warming and genuinely impressive... and I couldn't point at domain knowledge or mobile experience as being the cause of this success, with both Ribot and Mary (the participants with a background in mobile) located within one team, and no mobile experience in the other.

Learnings from the session displayed a comforting level of consistency between the two groups, too: both felt that moving to a smaller screen lent their design more focus, and that working within tighter constraints contributed to their creativity, rather than detracting from it. Time limits were clearly a problem (how could they not be?), and whilst one group found cross-platform consistency to be a tough trick to pull off, the other observed that their design changed less than they'd have anticipated in the move from iPhone to clamshell.

Both groups then consented to exchange beer for a pile of incredibly useful feedback, which we'll be using to further hone the session before running it in Chicago. And one comment which really chuffed me up was overhearing participants moot that firstly, this format might be an interesting way to open up a Hack Day type of event; and secondly, that he'd really like to see the product he'd worked on *actually get built*.

As always, thanks to Jon Markwell of Inuda and the Skiff for hosting, to Joh for gently stamping on some of my more ridiculous ideas and her rock-steady facilitation, and to everyone who came and participated :) There's a photo-set, yellow-hued thanks to my grotty HTC lens, here.

Update: slides from the warmup are here.