What is deployment frequency and how is it measured?

What is deployment frequency?

Deployment frequency is how often code changes are deployed to production. These changes can include bug fixes, new versions, enhanced capabilities and new features. By better understanding how often changes are shipped, the organization can better predict future delivery timelines for new or enhanced features. By tracking deployment frequency, organizations can gauge a team’s speed and agility. It’s important because it shows how much value teams can deliver to the end users.

Deployment frequency is a metric that has been long used in the software development space - first coming into prominence with the emergence of DevOps, and then being included as a key metric in the DevOps Research and Assessment (DORA) team’s framework, which aimed to help DevOps and engineering leaders improve the way they measured software delivery efficiency - deployment frequency is a ‘velocity’ metric. DORA also raised the bar on what it deemed was acceptable deployment frequency. 

Now, with many teams shifting to platform engineering, deployment frequency is still considered a reliable metric to use, with slowdowns in deployment frequency often correlating with issues in developer experience, developer productivity, and developer tools, all of which are front of mind in the platform engineering space. 

Organizations’ deployment frequency will vary; with some teams deploying code monthly, weekly or even multiple times every day. 

How to measure deployment frequency?

Simply counting the number of deployments - no matter how small (a new feature) or big (a quick bug fix) - made in a particular time frame. This can be measured on a per week or per day basis. That means that if in any given month you had weeks where you deployed 3 times, 7 times, 6 times, and 8 times, then the average per week would be 6. There isn’t a hard or fast rule for calculating deployment frequency, but companies can use analytics tools to measure it. 

What is a good deployment frequency?

The DORA team had studied thousands of software engineering companies in a six-year program to better understand what metrics engineering teams should evaluate themselves with, and to get to grips with the differences between high-performing and low-performing teams. 

The latest benchmarks are as follows:

Elite performers: On-demand or multiple deploys per day

High performers: Once per day to one per week

Medium performers: Once per day to once per month

Low performers: Once per day to once per month

High performers will make multiple deployments per day or be able to do so ‘on-demand’, while lower performing teams may only deploy once per month. 

However, it’s worth noting that different organizations will have their own benchmarks based on their own operational goals, the complexity of projects they work on, and the maturity of their engineering teams. Teams should have a baseline from which to compare with. 

Importance of deployment frequency

Deployment frequency is an easy metric to measure and yet it can provide teams with a lot of power.

Deployment frequency can be a metric you use to evaluate your team’s strength in being agile and responsive to feedback. It can also ensure continuous delivery; keeping your engineering team on track to deliver frequent deployments, and ultimately provide end users with the features, fixes and updates that they need. 

Perhaps most importantly, deployment frequency can indicate whether there are issues with existing systems, processes or even personnel. In contrast, it can indicate when the environment is highly functional. Either way, it provides a basis for engineering leaders to investigate further, and try to either fix issues or ensure that certain methods or systems are always used because they provide good results. 

Strategies to improve deployment frequency

• Create an end-to-end experience for your developers to improve deployment frequency. For example, you could consider how your team currently brings a feature to production and break the process down into smaller steps, optimizing each step.
  In this example, you could enable a user to ‘scaffold a new service’ as a self-service as a start. Then, in order to streamline the entire process, you could provide self-service actions to spin up a developer environment, send a reminder to review a PR, or rollback to a previous version.
  
• Set goals for teams on the deployment frequency per week. This goal should be a realistic target that the team can achieve.
  
• Make small changes by reducing the deployment size. This reduces the likelihood of deploying bugs. Developers are more likely to understand the issue at hand and can roll back any changes.
  
• Establish a clear environment  - a software catalog can enable developers to quickly access the information they need about any given service including versions, owners, dependencies and permissions. This means they can get to work without needing to switch between different tools to find what they need, enabling them to deploy more often. 

Deployment frequency pitfalls

There are some instances where you may see high deploy counts which don’t actually reflect a highly functional team or environment, but are because of other reasons. These could be:

• Gaming the system
  Developers that believe they’re under pressure and are being measured by their deployment frequency, may want to ‘game’ the system’. That means they may more frequently push code.
  
• Inconsistent processes
  Inconsistencies through siloed teams using different practices to break up their code, and therefore in some instances, some teams are shipping an unusually large number of small changes.
  
• Complexity dictates frequency
  Complex projects may require many bug fixes and deploys, but this doesn’t necessarily mean that the code quality is good, and therefore, it doesn’t guarantee that the deployment frequency is down to a well-functioning team.

Deployment frequency is an important metric that many engineering teams currently use to help them to measure developer productivity. But it is just one metric, and engineering teams ought to be using a range of metrics to determine developer productivity and developer experience. Depending on the organization, there may be other objectives that the team wants to accomplish such as better compliance and security, standardization and meeting SLAs/SLOs. These objectives would require measuring and monitoring other types of metrics. 

Find out more about developer experience metrics here.