Product Bistro: Love Developers, and Trust QA

I was talking with a friend and fellow musician Jason Mendelson last week and the topic of software delivery came up during our conversation. Jason knows that I have a lot of experience leading product teams to bring software products to market. I've learned a ton about shipping products and if there's a mistake you could make, well, I've probably made it at one time or another, and sometimes more than once.

When it comes to shipping software, delivering software releases on time is one of those seemingly elusive concepts that most everyone desires but frequently don't achieve. (I actually believe delivering on time is a learned behavior, but that's for another blog post.) Quality can also be an elusive goal, especially for startups who may not have a lot of experience working together as a team, or are under huge pressures to deliver releases. Dialing in on the winning customer experience and product feature mix is another passion of mine that's vital to creating great products. I could blab on about all these subjects but a philosophy of mine about creating software caught Jason's ear; "Love developers, and trust QA".

These days I'm frequently a coach or mentor to someone who is leading a product development effort, sometimes for the very first time. Frequently they've come up through the developer ranks with a lot great experience that sets them up for new opportunities leading product development efforts. I always try to help them understand a key lesson I've learned; while I love software developers and the innovative work they do, I trust most what QA software testers tell me about the quality and completeness of a software release. I've advised more than one development manager that even though you may not always like what QA has to say (you may even dread hearing it), you need to hear it straight -- they're looking out for your best interest.

Software developers, engineers, artists, or whatever job title you chose to call them, are a loveable, creative, quirky, smart bunch of people who have to try and envision how best to build software that embodies the vision and needs of the business. They are a great group of people and I love hanging around them and being part of the team. Sometimes product developers get great direction and it seems more often than not a lot is left up to their interpretation when creating a new product. Hopefully someone is giving them great input about the target user personas, product requirements, user interface design and help with key decisions about the product architecture and which options help the business most achieve its goals.

But inevitably, releasing software products can become a grind. What's after this release? Another release. And after that, another release, etc., etc. It's hard work and developers are frequently in a situation where they have to be as productive as possible, inevitably making decisions that could make or break the product's quality, feature set and success in the market. Developers have a lot of competing needs to balance and are often asked to make decisions without the time or input that might help make that decision the best one. That's where the development leader, manager and/or product manager comes in, btw, making sure the development team grasps the impact of various options and decisions, and also that business leaders understand consequences of certain demands which could impact the product or the business. But that's why I love developers -- with all these things to consider, the best plow through and get the right things done to create great products.

It's also why asking a developer questions like; Is it done? Is it tested? Does it work? Is the quality better? Did we find all the problems? Is this going to satisfy what our customers asked from us? Is it ready to ship to a customer? I could think of a million questions that while I've asked developers, I know I may not have gotten a completely accurate picture based on their answer. In many ways developers are too close to the problem, head deep into creating the solution we need. One developer is likely only working on one portion of the software release, and more work is required to integrate and get all the elements of the full release working together. That's why everyone knows that to get quality software you shouldn't rely just on developers themselves testing their own code. That's also why I trust developers, but I trust QA engineers more when it comes to understanding the true state of a software release.

QA guys (guys and gals) tend to think very differently. Their job isn't to create stuff, it's to break it -- anyway they can. Because if they don't find a problem, that means a customer will. And some people just have a real knack for it. The best are also very detail oriented, don't assume anything and watch to make sure nothing gets by them. The best QA engineers are also very good about tracking results, creating thorough test plans which get a lot of scrutiny by everyone including the developers, automate testing, and understand the value of metrics and how this can raise the learning and skill of the entire team at releasing quality software on time.

QA engineers are also the first point in the process where someone is actually trying to use the software. Functional testing doesn't always drive this out, but scenario or use case based testing can make a really big difference. Using your own products in your lab or by your IT staff can also be a great way to get real usage feedback, in addition to customer experience testing of course. Testers also have great suggestions about the product. They catch nonsensical error messages, product inconsistencies, or encounter areas of the product that are hard to use. Most importantly, QA engineers know they are the last in line to judge the release, and take their responsibility very seriously when it comes time to say whether they think the release is ready or not. QA engineers aren't just critiques, they want to ship the product too. They just don't want to be embarrassed, taking it personally when bugs or quality issues escape the testing process and are found by customers.

I also believe the effectiveness of any QA team or department is a direct reflection on their leadership, including the visibility, importance and voice given them in the product development process. One of the things I've learned to do as a development manager, VP engineering or CTO is to hold my own weekly meetings with the QA team. Development managers are welcome if they like too. This is our time to talk about release testing, QA processes, metrics, test plans, lab needs, improvements, suggestions, issues escalated from support, and -- most importantly -- give QA a voice directly to the top of the organization to say whatever needs to be said, good news or bad.

I hope this explains some of my learnings around the philosophy, love developers, and trust QA. The fact of the matter is that I love them both for what they bring to product development, and I trust them both too. But I rely on their skills, experiences and judgment differently, and at different times, bringing the best strengths of both to release great software products.