Performance Testing 101: What Managers Need to Know

Performance testing isn’t always required, but knowing when it is can make or break your product. In this episode, Mike Hrycyk and guests Ryan Hobbs(BC Pension Corp.) and Praneeth Eadara(PLATO) explore real criteria for deciding whether to invest in performance testing, what’s at risk if they decide to skip it, how to scope a first project, and what managers should consider before committing time and budget. Whether you’re a QA manager, developer, or business stakeholder, this panel will help you to understand the value (and limits) of performance testing – and how to plan for it long before peak load becomes a problem!

Episode Transcript:

Mike Hrycyk (00:01): Hello everyone. Welcome to another episode of PLATO Panel Talks. I’m your host, Mike Hrycyk, and today we’re going to talk about performance testing. We’ve talked about performance testing in the past, and those topics were interesting, but today we’re going to focus on the idea that you’re a manager, you’re a person in a company, maybe you’re just a QA, and you’re trying to decide, does my project need performance testing? And so, some performance testers might say always. You always need performance testing, but this isn’t true. You don’t always need to because there’s an expense and there’s a cost to it that should be judged. And so, we’re going to talk today about helping you assess and helping you figure out, and maybe a little bit of how to get started. Ryan, can you introduce yourself?

Ryan Hobbs (00:38): Absolutely. My name is Ryan Hobbs. I’m an Assistant Director of IT Engineering at BC Pension Corporation. I have been working in IT for about 25 years. Many years of that were spent in quality assurance and automation, manual testing, and management leadership roles across medium to large-sized quality assurance teams.

Praneeth Eadara (01:01): Hello, Ryan and Mike. And myself, Praneeth, and I’ve been in IT for close to 15 years now, mainly into testing with the specialized skill of performance testing. I’ve been doing that for over 14 years now, and I’m happy to be at PLATO.

Mike Hrycyk (01:17): Thanks for joining, guys. One of the reasons I thought of Ryan for speaking on this topic is that, in his last iteration as a QA manager, he was the guy who had to assess and decide that he needed performance testing and then called us to do the performance testing for him. So, that is a nice big reason.

Ryan Hobbs (01:33): Still a great decision.

Mike Hrycyk (01:36): Awesome. So, everyone, at least in our listenership, is going to have a general idea of what performance testing is. Maybe we’ll just get started. We’ll tell our audience: What are you trying to prove when you do performance testing? And I’ll let you kick it off, Ryan.

Ryan Hobbs (01:49): For me, what I’m trying to prove with performance testing is that the application under test meets the expectations of the customer. And that’s not just the one person in the boardroom – say we’re generating a website for a customer – not just one person in a boardroom clicking it and that it’s performant, but the expectations of the customer base at large, making sure that it meets and exceeds their expectations for their experience in the application.

Praneeth Eadara (02:22): Slight additions to that. Most people think that performance testing is something to test against an application, which should be fast, or stable, or reliable, but in reality, performance testing is something about testing certain conditions, certain expectations that you are discussing with the client, right? And you want to measure the expected loads that you have defined.

Mike Hrycyk (04:00): I think something that’s overlooked when people think about performance testing is when you test the breaking point. So, you should know when that is, and then you can monitor against that and throw up big red flags when you get close or mitigate and try to figure out for it not to break.

Ryan Hobbs (05:33): Interesting question. The reason it’s interesting is that my default answer would be that every type of project or application could benefit from some type of performance testing or investigation. Whether or not you act on that and spend the money is related to how your customers are going to deal with the potential slowness.

Praneeth Eadara (09:03): So, in terms of what type of projects we could test, any project which has a very high traffic, like Amazon, Home Depot, Walmart. So, we can also consider performance testing for banking applications, school websites, and maybe education platforms.

Mike Hrycyk (12:37): So, manual checks, in some cases, to some extent, yes, that’s beneficial. I’ve seen in my experience where we have done it using a stopwatch or just exploratory testing. It is beneficial in some cases, but there are flaws associated with that. We are not testing something which is data-driven, repeatable, or realistic. This isn’t the depth of performance testing that we need to conduct.

Ryan Hobbs (22:18): Really going to simplify this down. I’m going to say there’s two sides to performance testing. One side is a solid understanding of the business, what the expectations are of the workflow, what your patterns are, and general business processes. You have to have a solid understanding of that. The other side that you need experience on is the methodology of the performance testing, the tools, the techniques, the monitoring, and what your actual hands-on implementation site is going to be. Without the two together, you’re seldom successful in performance testing.

Ryan Hobbs (31:49): For me, the normal approach I take when trying to initiate a performance test of a given application is to ask the users or the business folks responsible for the application, what are your three to five most commonly used workflows in your system? What are those key areas that would be seen by the customers the most? Would they have the most impact if they were sped up or if they slowed down for you?