Context engineering

What is context engineering?

There’s been a shift in how engineers discuss how to improve what LLMs and AI agents can do. As part of that, the term context engineering has landed in the spotlight.

So what is context engineering? At its core, context engineering is the act of constructing an information environment that can provide the correct amount of directions, tools, and knowledge for LLMs to successfully solve problems. This concept is built on context models, the data that LLMs process to understand a situation and return relevant results. 

These data sets vary across models and functions, which is a challenge context engineering works to solve. It’s a significant shift in mindset from prompt engineering, but it presents opportunities for consistency and ways for AI agents to handle more complex tasks.

Why context engineering matters in AI applications

Providing the right context is a requirement for any engineering task. It’s a challenge for both humans and AI agents; context needs to be curated and correct for all developers to succeed. When SPS Commerce wanted to build an internal developer portal, one of the big questions that led to them choosing Port was, “what context is essential?” even before adopting AI into their regular workflows.

In the age of AI, a domain-integrated context lake — that is, a central knowledge and data source provided to an LLM — can make or break the LLM’s success. If the model is given too broad a range of information that isn't focused specifically on the task the model is being asked to perform, the likelihood of context distraction and task failure increases. If the model isn’t given enough relevant context, it won’t understand how to complete the requested task successfully. In both cases, these unguided agents contribute to agentic chaos.

Context engineering becomes a necessity in this environment because it’s so critical to give AI agents the right amount of the right data to prevent them from taking destructive action. The quality and relevancy of context an agent has to work with directly shapes the agent’s output, and the focused context improves the agent’s overall effectiveness. This ensures more accurate results with less hallucinations, helps overall agility, and can contribute to improving DORA metrics.

You can also optimize the data LLMs work with to improve an AI agent’s consistency and accuracy without creating or retraining a foundational model. By defining what a less specialized LLM can and can’t reference at a granular level, you can influence it to suit your needs without altering its foundational model or dataset. Saving time and resources this way makes LLM context engineering cost-effective for teams looking to implement AI agents.

Key techniques in context engineering

There are several techniques that can be used to provide relevant, useful context to LLMs. When combined, these techniques improve an AI agent’s ability to accurately and consistently take actions.

Provide just enough relevant information in context windows

Picture a literal window with a view of a lake. You can’t see the entire lake through the window, but you can see enough to understand the lake and some of the environment around it. If someone asks you to describe it, you can tell them if the water is still, mention a boat on the lake, or point out a mountain range on the other side.

Like the window in this metaphor, context windows refer to the amount of data, in tokens, that AI agents can see and access at any given time. These windows give agents specific views of the context lake so they can accurately identify and use relevant information with less risk of context distraction. However, the data that's made available to AI agents directly impacts the range of tasks agents are able to successfully complete; if a task exceeds an agent’s context window, the agent won’t be able to complete the task. As such, it’s essential to provide useful, relevant data in an AI agent’s context window.

Write structured prompts

Structured prompts are a key tool for both prompt engineering and AI context engineering, and build paths that lead the AI agent through the context lake. It’s essential to devise clear frameworks that provide organized instructions so an AI agent can consistently act on and return relevant, accurate results. 

Leverage metadata to increase consistency

Metadata can be referenced to improve performance by letting AI agents store, retrieve, compare, and connect semantically similar data. In addition to helping your human developers by collecting and grouping metadata in software catalogs, you can help AI agents consistently take correct actions by using metadata to collect and group segments of data they might not otherwise recognize as being connected.

When using Port, you can save and access all of your metadata from one central location. One way you can help both your human developers and AI agents is by creating a prompt library to build effective prompts, then turning particularly reusable prompts into self-service actions. AI agents with these tools can address a variety of context-specific pain points to improve a team’s agility and scalability, which can directly change the role of the developer.

Platform engineering becomes especially beneficial for teams using MCP servers. For example, the Port MCP Server is fully native and Port’s AI agents have access to it by default, so agents can perform a broad span of context-dependent actions right away. Because Port’s agents don’t have to be trained to access context-relevant data, they can act quickly and accurately — Port focuses the agent on exactly the right information and right context for it to successfully finish a task, and teams can be confident in the agent’s ability to consistently return correct results. 

{{cta_1}}

FAQs

How does context engineering improve AI performance?

The quality and structure of context provided to the LLM directly impacts the model’s reasoning, accuracy, and relevance. With context engineering, the LLM can more readily access accurate, relevant information.

How does retrieval-augmented generation relate to context engineering?

Retrieval-augmented generation is an integral part of context engineering, as it makes LLMs act based on authoritative sources. This ensures LLM responses and actions are consistent and accurate.

Can context engineering reduce AI hallucinations?

Yes. Focused, relevant context improves the effectiveness of AI agents, and the increased ability to produce accurate results reduces the chance of AI hallucinations occurring.

Why is context important in AI systems and human communication?

In the same way humans require the right context to work effectively, AI agents need context to be curated and correct to succeed. If AI agents aren’t given enough contextual data, they will be unable to accurately perform tasks. If they’re given too much data from too broad of a range, the likelihood of context distraction increases.

How do I identify and define the “right” context for a system?

Start by identifying what functions and tasks you want your AI agents to perform, then create a central information repository that’s relevant to those functions and tasks. Use Port’s MCP server and built-in software catalog to provide the right context and maintain relationships across all of your system’s entities. This will reduce risk of context distraction and ensure your agents can handle essential tasks.

How can context engineering be used to reduce ambiguity in AI outputs?

Context engineering can reduce ambiguity in AI outputs by giving LLMs context-relevant data that enables the LLM to return clearer, more accurate results. This can influence the LLM’s output to suit your needs without having to create or retrain a foundational model.

What are good examples of real-world use cases for context engineering?

In his blog on context engineering, our AI Product Manager Matan Grady covers several real-world uses for context engineering, many of which are supported by Port’s AI agents. With the right context, AI agents can:

• Define service ownership (when given access to a software catalog, organizational structure, and relevant team members)
• Outline daily tasks and their priorities for developers (using dynamic context including the developer’s current workload and assigned tickets across different services like Jira and GitHub)
• Recognize critical incidents (by identifying which services are critical, the priority levels of different incidents, and how incidents are mapped to related services)