Test Management Forum - 28 October 2015

he 48th Test Management Forum took place on Wednesday 28 October 2015 at the conference centre at Balls's Brothers, Minster Pavement


13:30pm Tea/Coffee
14:00pm Introductions

Session A

Paul Wilford, Exploratory learning - throwing away the slideware

Session B

Graham Thomas, A hierarchy of Software Testing Measures and Metrics – Discuss?

Graham's Slides

Session C

Gordon Baisley, Let’s Do Everything All At Once!

Gordon's Slides


15:30pm Tea/Coffee

Session D

Emily Fielding, You’ve crowdsourced your hotel and your taxi, what about your testing?

Emily's Slides

Session E

Margaret Edney, Keep Your Head When All Around Are Losing Theirs

Margaret's Slides

Session F

Paul Gerrard, What's all the fuss about DevOps?

Paul's Slides

17:15pm Drinks Reception


Margaret Edney, Thompson Reuters, Keep Your Head When All Around Are Losing Theirs

As a Test Manager it so often seems that whenever things go wrong it’s always the Test team’s fault. Why is it that the Test Manager is expected to explain why the release is late and has bugs?

We missed a test. We put the bugs in, didn’t we. We’re told we should automate all our tests.. I wish I had a magic wand that would automate all our tests – instantly.

Then there’s the Project Plan. The PM has scheduled testing account changes, starting on Tuesday. Unfortunately, the page wasn’t ready but something else was ready early. Tester started work on that and upset the Project Manager’s plan.

Everybody (and his dog) has suggestions for how the project should be tested. And, to finish the day, two people from the test team got seconded to another project for 6 months.

My first question is – what can I control? If  I have no control over it, I don’t waste time trying.

  • I can push for the team to be involved in the design meetings. We have the concept of smiley faces – designers, product owners, developers, users and testers. If there’s a face missing from the meeting then there’s a potential problem.
  • I can control our test automation, though I do have to push for enough time to build the framework.
  • I can control the quality gates.
  • I will control the order of testing and what we test.
  • I can’t control losing people.

Gordon Baisley, Independent, Let’s Do Everything All At Once!

As the UK’s biggest government department the Department for Work and Pensions administers the State Pension and a range of working age, disability and ill health benefits to over 22 million citizens. 

Triggered by the cross government ‘Digital by Default’ policy, the model for IT provision at DWP is going through wholesale change. From being very managed service, using major suppliers only, very waterfall, a supplier aligned organisation, and commercial tools only; current direction is towards in-house resourcing or small supplier, agile focussed, with a business unit aligned organisation, and using open source tools. And this is happening all at once and at large scale. In Testing for example, including suppliers, there is around 800 people.

We’d like to talk about how we're tackling this change in the DWP Test community. We hope we’ll share some ideas that will be useful to people making one or more similar changes in direction. Equally, we hope to hear recommendations from others who have been successful by doing things we've not considered.

Graham Thomas, Badgerscroft, A hierarchy of Software Testing Measures and Metrics – Discuss?

This session has been inspired by following the journey of Solar Impulse 2 (SI2), an aeroplane powered only by solar power, as it circumnavigates the globe.

Last June SI2 flew from Japan to Hawaii. It took 6 days, using only solar power. A remarkable feat which smashed all solar powered flight records.

Even more amazing, was that as the plane flew it transmitted real-time data which was published, live and real time, on the internet at solarimpulse.com, giving a real insight.

This got me thinking. Firstly, wouldn’t it be amazing if our software testing projects could broadcast live, real-time metrics and measures just like Solar Impulse? Looking at the information being sent back, the mix of flight control data, route information, and overall journey around the world, there was a clear alignment with Maslow’s Hierarchy of needs. From safety through to self-actualisation. And I wondered if there wasn’t an opportunity for us to revisit our testing measures and metrics, and look at whether they also aligned with Maslow, and what we could learn from that.



Emily Fielding, Testbirds, You’ve crowdsourced your hotel and your taxi, what about your testing?

In the same way that Uber crowdsources its drivers, crowdtesting is a relatively new form of software testing that leverages a filterable online community of many thousands to cover core device and demographic requirements for QA and UX insights. This allows for a greater diversity of devices included in pre- or post- release testing, as well as the opportunity to have the end users themselves to contribute feedback to the software development roadmap. Many crowdtesting companies position themselves as a complement to in-house testing by QA professionals, but what does the TMF think? A valuable tool, a possible threat, too difficult to implement? Emily from Testbirds Ltd will lead a discussion around the pros and cons of crowdtesting as a methodology, explore where in the SDLC it might fit, its limitations and best practices.

And what of the crowdtesters themselves? Crowdsourcing as a form of employment revolves around the ability to provide a variety of solutions in the form of services, ideas or content by soliciting contributions from a group of people, particularly using the internet. However, due to a lack of regulations in this young industry, there are risks of exploitation, poor treatment and a lack of accountability, as has been seen in high-profile Uber lawsuits. What measures are needed to protect crowd sourced testers, while still delivering value to the clients that indirectly engage them?


Paul Wilford, Hiscox, Exploratory learning - throwing away the slideware

Much Agile training seems to be focussed on learning about specific methodologies, all of which need a degree of rigidity about them in order maintain an identity. This presents a rather interesting situation where Agile teams ‘learn’ certain things that could be considered contradictory to some of the fundamental Agile principles.
So what is the solution? In this session we will discuss our experiences of running Agile training courses which have only concepts and principles driving them. Not a single slide in sight.


Paul Gerrard, Gerrard Consulting, What's all the fuss about DevOps?

DevOps is hot news nowadays. Should testers care, and if so, why?

Some years ago, Continuous Integration (testing) emerged and became all the rage as developers began to recognise that testing using tools gave them a more comprehensive safety-net for Test-Driven or Behaviouur-Driven-Development approach. Continuous Delivery (founded on CI) was heralded as the only way to provide the flexibility and speed of delivery to satisfy innovative businesses. CD provides the flexibility for businesses to continuously innovate through experimentation and feedback in the 'digital world'.

DevOps is becoming the preferred approach to build, deploy and release into test and production environments in thecloud. Recent innovations such as 'infrastructure as code' attempt to completely eliminate the human factor in these processes. Does process naturally migrate to code or is DevOps a power-play? Developers have the tools to implement most of the testing; are they now taking over Operations?

The quesion arises yet again (after Agile, CD, CI and DevOps) - where do testers fit?