Category Archives: Methodology

Avoid embarrassment. Perform testing.

This was originally going to be a post about the evolution of the IT department and how if we wish to stay employed and interested in our jobs we must move along with the rest of the industry.

Then an email arrived in the inbox of a colleague.

First some background. As you can see from my linkedin profile (I hate those things too; unfortunately until I write more blog posts it’ll be essential to my ongoing employment). I’ve recently started working at The Royal Society of Chemistry after a few years owning and running a small IT Consultancy.

As a provider of services when I worked for Avanade, IBM & myself I adopted a number of provider <-> client behaviours that I believe should be adopted by all IT departments, one of these was performing managed testing, and handing over both the results and testing to the customer.

I’ll assume that if you’re reading this blog you’re familiar with a software development lifecycle and software testing. I see testing like this:

  • Does this unit do what I want it to do (unit testing);
  • Does this requirement do what we think you want it to do (system testing);
  • Does it do what it used to do (regression testing);
  • Does it do what you want it to do (acceptance testing).

Sometimes we merge these (I’m thinking system and regression) and sometimes we do them informally. But we should do them before moving from one stage to the next. Sometimes we do additional types of testing (load testing, integration and penetration) but we should be doing the four types of testing above.

As a provider of services; when my team (or sometimes just me!) handed something over to a customer we wrote a test plan and executed any tests we thought appropriate. Broadly though, the test plan included manual and automated tests which covered every area of the application. Your requirements help to identify the areas.

Sometimes it was: Click here, do this, install this framework does it still work, uninstall that, does it still work?

Sometimes it was: Does this web service return this result.

It depended on the requirements.

In short though, we had a test plan we created tests and we executed those. We then passed this over to the customer when testing was completed.

Yes. We hand it all over. The tests, what failed, how many bugs, how many cycles. Everything. My main reason for doing this initially was a feeling that the data belonged to the customer. We charged them to create it, we executed it, they owned it.

I must admit, I didn’t think of that as being revolutionary or some extraordinary insight. I still don’t think it is. But if you’re not doing this today for your customers you should be. As I said, they paid for it and it’ll help get them feel a sense of ownership for the product (which they should have!) as well as show how a third party can add value – those tests are a gold mine of useful information for regression, future requirements, and user manuals.

This brings me back to that email that arrived in the inbox of a colleague that prompted this blog post and a slightly ranty tweet.

My colleague (a .NET developer who’s moving into a more TA/BA role, it’s all very woolly) has been working with a third party developer to create a new application which is going onto our website. We’ve spent a fair bit of cash on this, and they have charged us for 30+ days of system testing.

It arrived, and it didn’t work.

Despite the test plan being charged for, and also being someone to execute tests (30+ days!) they state “we only provide unit test” and “you’re the quality gate”

Not acceptable. While I’d never advocate going from supplier to production, I expect the application to hang together and meet the broad set of requirements we’ve provided (or if you’re the supplier the requirements that have been provided to you.)

If you’re an IT department you should be behaving in this way already. If you’re an IT department using a third party supplier you need to make sure these steps are performed before letting your users near any new application. Allow time to do it and allow time to check up on your suppliers.

Now, all I have to do is convince the business that they should be the owners of the test plan.

– Alan Smith