How to create a developer experience survey: The ultimate guide

Editor’s note: This post was updated on 8 April 2025 to address major changes to Port’s experience ecosystem, including Engineering 360, and add our DevEx Survey template as a new downloadable resource. ‍This article originally featured as a two part series in The New Stack. Read part 1 here and part 2 here. 

Developer productivity & developer experience

Measuring developer productivity can often be a controversial subject. Some believe it’s about counting lines of code, shipping speed, or fixing bugs quickly. Others argue these metrics only tell part of the story. 

How developers feel about their work—known as developer experience (DevEx)—has added yet another layer to developer productivity by shifting the focus away from mere output to an optimized environment. DevEx covers everything from onboarding and tooling to processes, relationships, and software quality. Research shows that better developer experience leads to higher developer productivity. 

Developer experience surveys help organizations identify challenges and shape their processes to their teams’ needs. Understanding how developers feel allows you to prioritize their needs, establish a clear change management process, and create a roadmap for improvement. While surveys are valuable, they should complement other qualitative and quantitative measures of DevEx and productivity.

Surveys also guide the development of your portal MVP and ongoing feature prioritization. Many engineering leaders rely on developer experience surveys—whether using Port or another tool—to gain actionable insights that drive real improvements.

Here’s our rundown of everything you need to know. 

Preparing to run the survey

First and foremost, ask yourself what you want from your survey. There’s no doubt you want to improve the developer experience, and you may want to exploit the features of the portal to achieve that, but perhaps there’s something more specific you’re seeking, such as:

• Improving developer onboarding time
• Increasing speed to ship quality code
• Eliminating bottlenecks in the SDLC
• Reducing MTTR
• Reducing burnout

Focus on questions that will help you act on specific areas. For example, eliminating bottlenecks in the SDLC may be helped by ensuring the team doesn’t have to spend as long searching for answers or solutions to problems, so you can ask:

‍”On an average day, how much time do you typically spend searching for answers or solutions to problems you encounter at work? (This includes time spent searching on your own, asking a colleague and waiting for a response) —with multiple choice answers from 15 minutes a day to over 120 minutes a day.”

Nb: It is important to consider the survey as a way to discover pain points you may have not considered before and evaluate how well specific pain points are being addressed.

Who should be involved?

This may be as simple as “all of our developers,'' but perhaps your survey can be adapted to other personas in the organization. Developers’ day-to-day overlaps with DevOps, SREs, and others and these personas stand to benefit from an internal developer portal, too. The different personas all have different levels of tech knowledge (for eg. using Kubernetes) and they use different technologies and features. Managers may need a quick ability to assess standards compliance, while developers may need self-service.

One of Port’s customers began their portal scoping exercise by surveying one group of developers—the cloud-native developers—as this was where the organization was heading. In a well-known example, organizations like LinkedIn survey all of their developers but break the resulting data down into “developer personas.” 

Time, frequency and engagement

You want to find the right balance between engagement and productivity.

You want the survey to take no longer than 15 minutes for a developer to complete, as this is likely to provide a good sample of data without taking up too much of the developer’s time.

One of Port’s customers uses a platform specializing in developer experience to conduct weekly surveys covering developer experience and other topics. This survey is obligatory for their team to complete, and it yields good engagement—over 90%. However, annual or quarterly surveys are more common. This type of self-reported data has to sustain 80 to 90 percent participation rates for it to be credible within the organization.‍  

Another way of getting developers to engage without forcing them is making the survey anonymous so that developers can answer questions without being concerned about being truthful. 

But in many instances (smaller teams, specific roles or personas), it may be obvious even with an anonymous survey who has answered in a specific way, and this may lead to developers answering in a way that they think they ought to, rather than what they actually believe. One way around this is to save each answer independently and not in a thread to prevent managers from being able to find out the employee just because of one answer. Whether anonymous or not, surveys are not meant to be taken as the whole truth and you should keep that in mind when looking at the data.

To increase participation rates, the survey design and experience must be high-quality, relevant, and useful to engineering organizations, executives, developers, and teams. Efforts to share the data back and communication around organizational learning from the survey are also key. 

It’s important to frame the survey as an exercise that aims to help your developers, not test them or catch them out.

What tool to use for your survey

Several platforms, such as Culture Amp, are explicitly designed for surveys, and general tools like SurveyMonkey, Google Forms, or Qualtrics are also available.

Another option is Port. Unlike traditional survey tools, Port integrates engineering intelligence with your developer portal data, such as those around standards, AppSec, cost or platform adoption, allowing you to identify and address bottlenecks in one place. 

With Port’s Engineering 360, you can:

• Survey engineering, operations, and security teams to uncover frustrations.
• Correlate sentiment data with quantitative metrics using frameworks like DORA.
• Use built-in automations to resolve blockers and streamline workflows.

Most developers spend only 10% of their time writing new code—Port helps you figure out where the other 90% goes and removes obstacles to boost developer satisfaction and productivity (Attribution Source: Gartner, Emerging Tech: AI Developer Tools Must Span SDLC Phases to Deliver Value, 29 January 2025). Using Port’s Experience marketplace, you can quickly connect your data, correlate insights, and launch surveys immediately.

Port isn’t just a survey tool—it’s the only full platform that helps you measure and actively improve developer productivity.

{{cta-demo-baner}}

What should I avoid in my survey?

You want to avoid three big elements: leading questions, validating assumptions, and asking too much.

Often, engineering leaders will ask questions in a way that they hope to validate their own assumptions. This isn’t helpful because they’re not providing a fair and balanced question for a developer to answer. As a result of the poor question, you may work on an area that is not necessarily a big pain point for a developer. This may lead to resentment and distrust from the developers and undermine future surveys and engagement. 

Sometimes, the leading questions you do ask may not yield the response you’re looking for. For example, if you provide a statement that says ‘x task provides a bad experience’ and ask your team to provide a rating, you may get a mixed response. This is partly because it is a leading question that they may either disagree with or have no genuine opinion about, and partly because the framing/format of the question isn’t suitable. Instead, you should ask more open questions such as “rate x task from 1-10, where 10 is a good experience” because these formats lend themselves to better, more informative answers.

Fixing everything at once is tempting, but that approach is rarely practical. Addressing too many issues simultaneously can dilute efforts and lead to minimal impact. Instead, focus on the most pressing problem first. Solving one issue at a time creates momentum, showing developers that their feedback leads to real changes. As a result, they become more engaged and willing to participate in future surveys, strengthening the feedback loop and driving continuous improvement.

What should I ask?

About the importance of tasks

To avoid just confirming your own assumptions, ask the team how important a specific issue is to them and then ask the developers how satisfied they are with their method for remediating it.

Here’s a two-part example question:

• How important is the speed with which your team ships code to developer experience?

• How happy are you with the speed that your team ships quality code?

About pain points

Port’s DevEx survey template (get it by asking us here) helps identify the biggest pain points for developers so you can reduce friction in your internal developer portal. Key questions include:

• Time spent on weekly tasks: Developers rate activities like reviewing PRs, writing features, managing incidents, fixing bugs, handling ops tasks, refactoring, and meetings.
• Top blockers to productivity: Developers rank challenges such as waiting for PR reviews, pending DevOps requests, finding service/API owners, or access issues.
• Estimated onboarding time: Understanding how long onboarding takes helps pinpoint inefficiencies.
• Biggest pain points across workflows: Developers rate frustrations from Jira ticket management to scaffolding new services, toggling feature flags, and troubleshooting outages.

Open-ended questions can also reveal pain points you hadn’t considered, especially if multiple developers share the same concerns.

Here’s how to effectively run our five-question DevEx survey based on insights from top engineering teams:

1. Run the survey: Embed it directly into your Port instance or Backstage portal to collect developer feedback.
2. Analyze the results: Identify the most critical challenges, whether in production workflows, CI pipelines, or access controls.
3. Fix the issue: Prioritize resolving the top problem first. For example, if developers struggle with access permissions, streamline access controls.
4. Repeat the process: Re-run the survey after making improvements to track progress and uncover new bottlenecks.

Focusing on one pain point at a time ensures you make meaningful progress. As developers see real improvements based on their feedback, they’ll be more engaged in future surveys, strengthening the feedback loop and driving continuous optimization.

About the portal’s features

One of our customers uses an alternative approach: they ask developers directly which self-service capabilities they would find most useful from a long list and have them pick one for their top priority, second priority, and third priority. This way, their input leads the work of prioritizing which self-service actions are implemented and developers feel well-served by both the portal and the survey method. 

This customer used the same format to later ask what types of monitoring or management features their developers were most interested in. You could use this format for other portal capabilities, too. 

For feedback

The type of survey you choose to run may change depending on your situation. For example, another Port customer wanted to improve the developer’s release experience in Port. Their survey was, therefore, conducted after implementing their portal, and focused on ranking it from 1 (very easy) to 10 (extremely confusing/difficult). Rather than ask questions, they asked developers to rate their sentiment towards statements such as:

1. The current Port layout is easy to navigate.
2. Releasing an app to production is confusing.
3. I can easily find all my team’s deployments and releases.

4. If a deployment (develop), promotion (stage), or release (production) fails, I am able to quickly identify where it failed.

Then, they asked questions that took a deeper dive into two specific issues:

5. What is a ballpark average I have to wait to receive assistance from the engineering team with resolving a deployment/release problem during business hours?
6. If there’s one thing you’d change or add to Port or in the deployment/release process, what would it be?

The customer in question took the feedback and improved the portal. When asked for feedback again, the developers said they were happy with the improvements.

About satisfaction and well-being

The above examples concern themes, pain points, and features, but developer experience goes beyond this. If you want to know how developers actually feel, ask them! You can use questions like:

• How productive do you feel? (rate 1 as not productive at all, 10 as very productive)
• How satisfied are you with your ability to be productive? (rate 1 as not satisfied at all, 10 as very satisfied)
• What are you most stressed about on a day-to-day basis? (with options, rankings and open fields)
• What is the one thing you would change about your experience?

The answers to these questions and the resulting actions aim to help you retain staff, improve work conditions, and improve collaboration and communication. 

Check out our sample Port survey templates here. 

Tip: It is important to provide open questions for longer answers, as you can get better explanations or alternative reasons, which will help you better understand the situation. It’s a good idea to include one or two open questions per topic or theme (but no more than this).

How to act on your survey results

There are three steps to using survey data to take action:

1. Getting the data into the right hands and in front of the right people
2. Interpreting the data and making decisions on what actions to take
3. Following through on those actions and then re-measuring once those actions or initiatives are delivered to ensure they actually move the needle.

Below is a breakdown of those steps and a few more. 

Use the data to decide on features 

The most important part of a survey is using it to make effective changes. For example, one of our customers used the following format: 

By plotting the results on a chart like this, the team can clearly indicate where they need to improve the developer experience — in this case, improving clarity over cloud costs and security levels. These are both Port use cases, which means they can be resolved natively in the portal (check out our full list of use cases). 

Compare against benchmarks

While the above example is more clear cut because it is about actual pain points and features, there will be some questions that will be more difficult to address. For instance, “how productive do you feel?” and “do you get enough time to focus on coding?” don’t have direct answers.

Here, you should use benchmarks — from previous surveys you’ve conducted, employee satisfaction surveys across the business, or comparisons with other organizations in your industry — to see if there are noticeable changes, and try to get to the root cause of them.

Talk with the team

Sometimes scores warrant further talks — for instance, if the survey provided low results across the board, it may make sense to dig deeper into the issues developers face by having an open team talk or one-on-one chats. This way you may find out where the real issues are.

Regardless, managers should schedule meetings with their team to discuss the results and create an action plan to fix issues. 

Prioritization

Some questions may provide you with clear priorities. But it’s important to focus on important tasks one at a time and iterate according to feedback you receive.

Acknowledgement

One of the hardest things about surveys is to keep engagement going. It’s important to say thank you to the respondents and also provide them with feedback in return of some sort, whether in terms of progress updates for new portal features or follow-up conversations about complex ideas or challenges. If someone had a great idea for a useful self-service action, let them know!

Keep in mind that the more open you are, the better engagement you’re likely to get, so if you’re in a position to provide some details of the actions you’re going to take from the survey results, it’ll help keep the team feeling appreciated and engaged. 

Combine your findings

As we’ve said throughout, developer experience surveys are only one component of a larger framework. Other measures such as developer productivity metrics, employee satisfaction surveys, in-the-moment feedback (while developers are using the portal, similar to NPS scores), and team talks should help you to get a clearer picture of what’s really going on and to build a business case for new features, updates, or changes.

Get feedback on actions you take

You may make changes directly as a result of a survey, but how have those changes affected the developer? Are they happy with the new feature or process put in place? Has it changed things for the better? 

Asking questions promptly after a user has used a new feature can be helpful. This makes feedback immediate and poses little recall issues.

This can look like a series of questions: 

1. A question about the way the new feature works — “How was using this self-service action for spinning up a new service? Was it easy/satisfactory/difficult?”
2. Objective metrics — “How long did it take you to complete that action?”
3. Open feedback — “How could this feature in the developer portal be improved?” 

Conclusion 

Developer experience surveys can act as a feedback loop for your internal developer portal and help you maximize its benefits. This can ultimately impact your developers' feelings about their work and their opinion of the portal, reducing friction points and improving their experience.

Improving DevEx isn’t a one-time project; it’s an ongoing process. Developers are the backbone of any engineering organization, and their experience shapes the quality, reliability, and maintainability of the software they produce. By investing in tools and processes that make their work easier, you’re investing in the success of your organization.

Ready to take action? Install a DevEx survey plugin in your developer portal today. Start the journey toward a more efficient, satisfied, and productive development team. Addressing one pain point at a time will transform your portal into a productivity powerhouse.

{{cta-demo-baner}}

‍