How AMCS Group embraced platform engineering with Port

Background

AMCS Group provides expert business management software for the resources, waste, recycling, transportation and utilities industries in more than 80 countries. Its aim is to help its 4,900 clients to increase customer satisfaction, enhance sustainability and boost margins. 

To deliver on behalf of its clients, the company embarked on a DevOps journey, beginning with the setup of a global DevOps team in 2017 and the adoption of Git, CI/CD, IaC, Azure and Azure DevOps. A member of this centrally positioned team, Pavel Pikat, Technical Product Owner at AMCS Group, explained its functions:

“It was a traditional Ops team, where AMCS internal development teams would come to us with a solution they’ve developed asking for it to be deployed into the cloud. The challenge was the company increased from 200 people to 1000 employees as the business grew rapidly, and despite having more services, developers and teams, it was difficult to scale the Ops team together with the rest of the organization” he said.

“We were challenged with the amount of tickets to do small repetitive tasks, raised by our professional services, product and development teams. It got to a point where we had to explain that if someone wanted their service deployed they had to wait in a queue,” he said.  

In fact, this workload impacted on Pavel's team’s ability to focus on other core activities such as improving processes, quality and standards and implementing new features like a new monitoring stack and driving Kubernetes adoption.

At this stage Pavel realized that something had to change, whether it was the makeup of the team or the way the team worked. 

"Many organizations moved away from traditional Ops teams and adopted DevOps culture. However, the challenge with DevOps teams is the amount of responsibility placed on them. It’s telling developers to go and manage cloud infrastructure, monitor or manage security, and do a lot of the things that create a lot of complexity and cognitive load. And it’s even worse if you have multiple teams in the company who all do things in a different way: different infrastructures, code approaches, naming conventions, security templates - it’s going to be a significant challenge," he said.

‍Embracing platform engineering

It was at this point that Pavel did some research into new approaches for software development and discovered the concept of Platform Engineering. The idea of improving developer experience by using self-service actions was key in persuading Pikat to adopt this approach at AMCS. 

“We wanted to do self-service but not the one where we would give permissions to everyone and they would do their own thing. Instead, we wanted to define templates and standards and offer some abstractions,” he said.

Rather than giving developers direct access to the Azure API, Pavel wanted a way to abstract complexity while adhering to AMCS’s naming conventions, best practice and compliance requirements. Then, his team would provide the developers with guidance on how to perform the action. 

It became clear that to achieve this, AMCS required a dedicated Platform Engineering team and a separate SRE team with different backlogs and processes than they had previously. Therefore, AMCS established a product-oriented Platform Engineering team with its own backlog and roadmap, working within two week iterations. It would incorporate one product owner (Pavel) and four engineers, all of whom are highly skilled with a development background. The team is responsible for enabling developers to build, test, deploy and run their applications in the cloud.

Meanwhile, the separate SRE team has its own TicketOps backlog, and is made up of 15 members with support, DBA and development backgrounds. The team is responsible for production SLA, on-call, monitoring, customer deployments and upgrades. 

In the short-term, the Platform Engineering team wanted to reduce its load by eliminating repetitive tickets coming from development teams with action items such as creating a new git repo, granting temporary access to a development environment, or adding a CI/CD pipeline to a service. 

In addition, Pavel wanted to provide visibility of deployed services, environments, and versions in an automated fashion, and use a software catalog that provides visibility of Git repositories, services, environments and ownership.

It became clear to Pavel that to achieve these goals, AMCS would require an Internal Developer Portal, which would enable developer self-sufficiency through the use of self-service actions with baked-in guardrails. At the same time, the portal would provide an abstraction of AMCS’s software and infrastructure, reducing their cognitive load and enabling them to make sense of anything in the stack and the resulting interdependencies. After a number of proof-of-concepts, AMCS selected Port. 

When asked why Port, Pavel responded: “Port is highly customizable. You can define and model your organization as you see fit. This was crucial in our decision-making because there are no limitations to what we can build." 

“UX and UI were a big part of the decision too — it felt and looked nice and modern,” he added.

‍Starting with self-service actions

Repetitive tasks

The priority for AMCS was to implement self-service actions to reduce tickets so that the Platform Engineering team could focus on its own core tasks. AMCS had already been tracking its tickets with a service delivery manager reporting on delivery time, volume of tickets, requesters and the impact of a ticket. The company found the most repetitive and easy-to-automate tickets and focused on creating self-service actions for these. 

Port enabled Pavel and his team to completely customize the self-service form depending on the abstractions the developers needed. The team has since created self-service actions for the following:

• Whitelist IP
• Refresh database
• Create new Git repository
• Implement CI/CD pipelines
• Create temporary SQL user for debugging development environment

Pavel believes every action executed in Port is a ticket reduced; he said in one week the team saw 100 actions executed, with this likely to grow.

Self-service scaffolding

After focusing on the ‘low hanging fruit’ available for self-service, the Platform Engineering team have now focused their attention on self-service scaffolding. This would allow developers to scaffold new services easily, without having to raise any tickets, or wait for other teams to create CI/CD pipelines and Terraform configurations for them. 

“Today, whenever developers create new services, they come to us asking for a CI/CD pipeline to build and deploy their app. We have it largely automated with Terraform and pipeline templates, but it still requires a human touch to glue different pieces together every time. It would be great to have it all wrapped up in a nice package that developers can consume without being experts.” said Pavel. This is the next area of focus for AMCS. 

Shifting to Kubernetes

AMCS is actively investing in building its next-generation cloud architecture based on Kubernetes, but Pavel said that very few engineers are Kubernetes experts, and this is why AMCS is planning to use Port and Platform Engineering to abstract it away and offer it as a service.

“This approach allows us to standardize and streamline how development teams onboard Kubernetes. We aim to offer them a golden path that makes the transition to containers secure and swift, all while minimizing cognitive load” he said. 

Encouraging adoption

To really benefit from platform engineering, organizations require buy-in from the developers themselves. Pavel and his team are conscious of this, and have made efforts to ensure developers are aware of Port’s functionality and benefits. They are pursuing a portal-as-a-product approach.

Whenever a developer opens a ticket about an action that can be created in Port, they’re directed to Port straight away. This initially enabled the organization to get 100 out of 300 users signed up to Port without any official launch or communication.

The team built a website dedicated to Port, with instructions and best practices for developers, while AMCS has both engineers and developers that are champions of Port. These are advanced users who are also active in Port’s product community. 

Today, AMCS has over 700 internal users in Port, and the Platform Engineering team is actively working on popularizing the platform through monthly product updates and webinars.