This article was originally published in php|architect in May 2009 and I published it mostly as it appeared in the magazine shy of the odd formatting to fit the blog format better.
—
In the last couple of years, there seems to be a growing trend in the PHP world where application and library developers put out claims that their piece of code is high-quality and thoroughly tested. Indeed, often this is true, but how can you really guarantee quality like this?
Quality Assurance, What Is It All About?
You might wonder, what is Quality Assurance (QA) all about? Is it something you might want to start utilizing for your project?
I don’t want to go too deep into this topic since this is a mere short article but QA can be so many different things depending on who you ask. I’m going to take the plunge and attempt to give my thoughts on QA teams and hopefully enlighten you in one way or another along the way.
The QA Team
So what is the QA team? They are the keepers of quality, those who aid the developers in maintaining the balance between productivity and chaos, these are the people who strive for perfection – The QA Team *cue theme song for “The A team”* Modern day heroes.
From my perspective, the responsibilities that a QA team would take on range from monitoring the ticket system, gathering more accurate information for tickets, verifying tickets, and making sure developers are upholding the coding standards, and any other standards, the project is employing, just to name a few.
One important purpose of QA teams is to think up new ways to make it easier for developers to adhere to the standards, to make it really easy for end users to report issues and to find information they need to reduce duplicate reports or invalid ones. To create a better end-user experience usually requires the QA team to bother the developers to write proper API documentation and harass the documentation team to produce a proper user manual. In many projects, the QA team will also deal with the tests – Writing user stories based on the specifications (if they are so lucky to have such a thing) so that the developers can write accurate functional and unit tests. This will also allow the QA team to do proper verification before releases and sprint closures.
QA teams are an impartial third-party, they have an outside perspective on the application. The problem with developers is that they don’t make good users due to how close they are to the product. They know what can be done and what can’t be done and are thus less likely to be as creative as end users when it comes to using/testing the product, QA is the bridge between the two.
QA will point out things that work but are not as clear or obvious as they could be, e.g. obscure error messages, unintuitive interface (be it user interface or API design) or finding fragile code where something is a little off and ends up blowing up in your face.
This is just the tip of the iceberg and, as you may have noticed, QA teams need to able to touch every aspect of the product, know it inside out, and play nicely with everyone inside the project. They are indeed the most powerful weapon a project manager can have in his/her arsenal, and if used correctly, the entire team will be more productive and produce higher-quality products on time … Well that’s the general idea, at least. ;-)
They Are Around – Even When You Think They Aren’t!
More often than not, projects will have a QA team, or at least QA people, without anyone involved even knowing about it!
Just think for a moment about all of those projects that you have come across that are high-quality and are well-managed, yet have no official QA team nor any mention of QA anywhere.
This scenario tends to happen when you have smaller companies or smaller development teams within companies. They become their own QA team, usually without appointing someone to take care of the bug system, with no one doing proper bug triage sessions and with verification simply happening when someone tries to fix the bug and ends up chasing down the person that filed the report just to get enough information to be able to actually fix the problem. This is all highly inefficient but still displays an effort to maintain a certain amount of quality.
QA and PEAR
A good 7 years ago, the PEAR QA efforts fell into the category: “It is around, even when you think it isn’t.” The project had a QA mailing list where there were some people around that did the occasional poking about and kept the project in decent shape. Quality was by no means bad, but as the project grew in size, the demand on high-quality code rose and since PEAR had some standards in place already, it became evident – A QA team would have to be formed to take on this challenge in a more official and more efficient way.
With that in mind, an RFC was written by Lukas Smith and yours truly (http://pear.php.net/pepr/pepr-proposal-show.php?id=60) on various things in regards to operating the PEAR QA team. The RFC was accepted and became the QA team’s modus operandi which still holds true to this day.
Sadly, as is often the case with open source projects and indeed even companies, resources and man power are scarce, and as such, a QA team may not be in place from day one or it’s just too overwhelming for the team at hand and things start piling up, which was the case for PEAR.
Bug reports as old as 6 or 7 years were still open and unattended, something not acceptable by a project claiming high standards, and so in 2008, after brainstorming, we decided to attempt a bug triage bimonthly in the spirit of the Mozilla-run bug days, where end users and developers come together on a specific day and go through old bugs or a specific part of the system(s) decided upon in advance, verify reports, write test cases and fix code.
I ran the first couple myself with a small group of people attending. Finding myself lacking time, Daniel O’Conner stepped up to take over my role, and he has done it so well that PEAR has never before been in such a good state quality-wise!
This is just one example of how an active QA team can make a difference for a project, another excellent example would be the Test Fest the PHP team put together for 2008, which was all about involving the user groups and communities to help write new test cases and get the coverage up for PHP itself. As a result of the first Test Fest, PHP received over 158 new tests, a truly great effort to combine test writing and community involvement.
The moral of the story is this: A QA team can bring a lot to your project and is an important part of the development process – Do not only think about delivering quickly, think about quality for the sake of your end users.
Helgi Þormar Þorbjörnsson 