Continuous Development, Continuous Integration (CI/CD) is the new paradigm for creating and improving web applications. Ongoing development and use of incremental changes that can be rolled back easily in case of problems have proven to be an effective way of advancing the attractiveness and usability of websites. The only way for this method to work well, however, lies in the effectiveness of application testing. You might release a perfectly engineered application that has been standalone tested in the development lab. Then it gets hit with a heavy load, or there was a coding error or an error in the way it gets deployed and it blows up. If you don’t understand how the application responds to a load, you don’t know where it will fail, and that’s already a failure.

Best Practice #1

Every time you deploy changes to your application make sure you know how it will react to load. This means making load testing part of your deployment process.

You could do load testing manually, but there are now a lot of tools that simplify automated testing. Many top companies are doing CI/CD using tools like Jenkins, Team City or Amazon Pipeline. Think of load testing as a form of monitoring when it comes to CI because, as we believe here at Apica, “If you don’t monitor it, you can never manage it.”

Best Practice #2

Although it’s better to do some load testing during development than doing none at all, it’s an even better practice to constantly load test. No Application can handle ‘infinite users’, there will always be a breaking point for an application.

Load testing is about finding that point so that:

At what point your application will start having problems, where it will end up not working at all and
When you do have problems, where will they present?
That way when the problem starts to present itself in production you don’t spend time trying to troubleshoot that problem. You know where it’s going to break.

Best Practice #3

Perform load tests against fully-integrated applications. Testing each individual micro-app may show each one passing, but it doesn’t show how they will behave when working as an integrated unit against realistic traffic coming into those applications.

When you’re standalone testing an app against a single user it may run fine, but when 10 or 100 users are in play, the site may go down because your developers are thinking “I don’t have any functional bugs and it works for my (one) user, so we’re good to deploy.” You deploy and you find that with 10 users your response time goes to 6 seconds, you get to 100 and your site goes down because your only information to that point is “well, I don’t have any functional bugs…and it works for my user, so we’re good to deploy.”

Best Practice #4

Continuous Integration and Continuous Development requires Continuous Testing. The smart practice is to assume that there’s always another problem in your code.

We’ve found that when you find and remove a bottleneck you can handle a greater load but you find five more bottlenecks. The bottleneck is no longer the web server, it moves to the database or the application. The more bottlenecks you remove, the more you find. That’s why it’s important to have an iterative process as part of CI. There’s never a point where there is NO bottleneck, it’s a matter of knowing where the bottleneck is and what volume of users you can handle based on the bottleneck you have. Every time you remove a bottleneck you uncover another one, but every time you remove a bottleneck you increase the load your application can handle.

Best Practice #5

Size should never be an issue, even smaller pre-deploy load tests have value. Even if you can’t afford massive load tests, if you do consistent load testing with 100 users over the course of development cycles, you’ll see whether your application has changed its performance profile from run-to-run or code-deploy to code-deploy. So not having enough money or staff time is NOT an excuse to not load test. Even small load tests can have value.

You can even load test against a single leg of your application. If you route your test traffic through your load balancer on a single leg, you can avoid closing down your environment and still find out what goes wrong. You can then scale up from there. With a small-scale test you may not find the breaking point, but you will see where the application changes and make your improvements from there.

Continuous Integration and Continuous Deployment are enhanced by Continuous Testing. These best practices will help you to catch problems that UAT and Automated Testing often miss, such as:

Changes in app performance
Concurrency related bugs
Code interplay from multiple rolled-up commits.
Combining these methodologies are the surest path to applications that run cleanly and provide a quality, enjoyable User Experience. To see a full webinar on this topic.