Performance Testing Live Updates with Atlassian's DC Performance Toolkit

For Easy Agile TeamRhythm’s Server and DC customers, one of the most sought-after features has been Live Updates for Retrospectives. This feature auto-refreshes the Retro board, so team members can see each other interacting with the page. Retrospectives are all about collaboration, so we were stoked to deliver this feature to customers in Easy Agile TeamRhythm 8.2.0!

Why did we do performance testing?

Auto-refreshing involves making frequent network and database requests - something which could potentially slow down internal systems. At Easy Agile, we developed this feature with the customer in mind, and made decisions which would reduce load on our customers' internal systems. For example, we leveraged Atlassian’s Cluster Cache API to optimise database requests and reduce load. In an early experiment, introducing the cache reduced the meany latency of Retrospective Live Update data requests by more than 50%!

Atlassian provides an official DC App Performance Toolkit, which app vendors are required to use annually to demonstrate that their apps don’t slow down customers' Jira instances. The Toolkit simulates an enterprise-scale DC instance with thousands of users and more than 100,000 Jira tickets.

At Easy Agile, one of our core values is “Be the customer”. Before releasing Live Updates, we decided to take it upon ourselves to use Atlassian’s DC App Performance Toolkit to confirm that the Live Updates feature wouldn’t impact the performance of our customers' DC instances.

For the performance test, we created a busy Retrospective board with 100 Retro items. Then we added 100 users to the board, triggering a flurry of Live Updates. Meanwhile, there were also 100 other users performing regular actions around the Jira instance (creating issues, viewing the backlog, etc.). This activity ran for about 45 minutes on a 4 Node DC instance, generating hundreds of thousands of data requests. We compared the results of this setup with a DC instance that didn’t have Retrospective Live Updates.

Results of the test

The chart below displays the 90th percentile time for each user action to complete. When we compare the test “4 Nodes With Retro Polling - Run 2” against “4 Nodes No Retro Polling”, the results are very similar. Fluctuations in results are expected, so some actions executed faster on the instance with Live Updates than they did on the instance without, and vice versa for other actions. Any differences were not significant. On an initial test, there seemed to be an outlier for a front-end action "selenium_search_jql". This was confirmed to be an outlier, as it corrected itself in subsequent tests. Furthermore, the main action used in Live Updates, "GET retro", returned requests in 130 ms, well within the range of other back-end actions which took between 100 and 500 ms.

The results from the test confirmed that Retrospective Live Updates has no significant impact on the performance of the DC instance. All of the user actions succeeded. There also weren’t any noteworthy differences in the speed of actions, when compared to the DC instance without the Live Updates feature.

Be the customer

While we were already confident that the architecture of our app would not put unnecessary load on customers' instances, it was great to verify this with Atlassian’s official performance testing toolkit.

As we continue to add more collaborative features into our apps, customers can rest assured that we are building with their needs in mind.

Retrospectives integrated with your work in Jira

Seeking to improve how your team is working together? Easy Agile TeamRhythm helps you turn insights into action with team retrospectives linked to your scrum or kanban board, to improve your ways of working and make your next release better than the last. Turn an action item into a Jira issue in just a few clicks, then schedule the work to ensure your ideas aren’t lost at the end of the retrospective.