NIST AI RMF for AI Agents: Govern, Map, Measure and Manage Autonomous AI

The NIST AI Risk Management Framework, or NIST AI RMF, provides a structured approach for managing risks associated with artificial intelligence. Its core functions are:

For organisations deploying autonomous AI agents, these four functions provide a useful structure for turning AI risk management into an operational process.

An agent does more than generate text. It can select tools, interact with applications, access information, make decisions and initiate actions. That means AI risk management needs to consider not only the model but also:

• agent permissions
• tools
• workflows
• missions
• policies
• constraints
• human oversight
• monitoring
• interventions
• outcomes

FirstHelm supports this operational layer by providing a control plane through which organisations can register agents, define missions, apply constraints, require approvals, monitor activity and intervene when necessary.

NIST AI RMF and autonomous AI

The NIST AI RMF is useful for autonomous AI because it treats AI risk as something that should be managed throughout the system lifecycle.

An autonomous agent can create risks that are difficult to capture through model evaluation alone. For example, a model may perform well in testing but the deployed agent could:

• call the wrong tool
• access too much data
• exceed a spending limit
• make an inappropriate external change
• continue operating after circumstances change
• interpret a mission too broadly

Agent governance therefore needs both pre-deployment risk assessment and runtime controls. The NIST functions provide a useful structure for combining these activities.

GOVERN

What does Govern mean for AI agents?

The Govern function establishes organisational structures, policies, accountability and processes for AI risk management. For autonomous agents, this means answering questions such as:

• Who owns the agent?
• Who is accountable for its deployment?
• What policies apply?
• Who can change those policies?
• Who approves high-risk actions?
• What happens when an agent violates a policy?
• How are incidents escalated?
• How is AI risk reported to management?

Governance should not live entirely in a policy document. The controls should be connected to the systems through which AI agents operate.

Establishing AI agent ownership

Every production agent should have an accountable owner. Ownership can cover:

• business purpose
• risk acceptance
• permissions
• constraints
• approval requirements
• monitoring
• incident response

A central agent inventory helps make ownership visible. FirstHelm's agent registration model provides a central record for connected agents and their operating characteristics.

Defining AI agent policies

Governance policies should address the activities that matter to the organisation. Examples include:

• agents must not access systems outside their approved scope
• production changes require approval
• financial transactions above a threshold require human review
• external communications require approval
• agents must operate within defined cost limits
• high-risk actions must be logged

The next step is converting these principles into controls that can be evaluated during execution.

Human accountability

Autonomy does not eliminate accountability. An organisation should be able to identify who is responsible for:

• deploying an agent
• approving high-risk actions
• changing constraints
• reviewing incidents
• deciding whether autonomy should increase or decrease

Approval records and intervention records can help create this accountability trail.

MAP

What does Map mean for AI agents?

The Map function is concerned with understanding the context in which AI is being used and identifying relevant risks. For agents, this starts with knowing what exists. An organisation should map:

• agents
• models
• tools
• data
• users
• workflows
• dependencies
• external systems
• affected stakeholders

Build an AI agent inventory

An AI agent inventory can answer:

The inventory should be maintained as the system changes.

Map the agent's mission

A mission defines what the agent is intended to accomplish. This distinction is important because an agent may have broad technical capabilities but a narrow business purpose. For example, an agent may technically be capable of sending email, but its mission may only require analysing incoming requests.

The governance system should distinguish between:

That distinction reduces unnecessary autonomy.

Map dependencies and tools

An autonomous agent is often a chain of dependencies. For example:

Risk can enter at any stage. Mapping the complete workflow makes it easier to identify where controls should be applied.

MEASURE

What does Measure mean for AI agents?

Measurement provides evidence about how AI systems perform and how their risks behave in practice. For autonomous agents, useful measurements can include:

• successful actions
• failed actions
• constraint violations
• approval requests
• rejected approvals
• intervention frequency
• execution cost
• token consumption
• mission outcomes
• incidents

The exact metrics should reflect the organisation's risk model.

Measure agent behaviour

A useful AI agent monitoring system should answer questions such as:

• How often does this agent violate constraints?
• How often does it require human intervention?
• How much does it cost?
• How frequently are high-risk actions proposed?
• How often are approvals rejected?
• Has behaviour changed over time?

These measurements can inform governance decisions.

Measure control effectiveness

It is not enough to measure the agent. Measure the controls too. For example:

Useful evidence could include:

• number of deployment attempts
• number automatically blocked
• number sent for approval
• approval rate
• rejection rate
• unauthorised attempts
• intervention events

This helps management understand whether a policy is actually functioning.

Measure autonomy

Autonomy should also be treated as a measurable risk variable. An agent that consistently completes low-risk tasks within its constraints may be suitable for more autonomous operation. An agent that repeatedly fails or triggers interventions may need tighter restrictions.

FirstHelm incorporates autonomy levels into its agent-management model so autonomy can be treated as an operational control rather than a fixed assumption.

MANAGE

What does Manage mean for AI agents?

The Manage function turns risk information into action. For autonomous agents, management controls can include:

• constraints
• approval gates
• permissions
• intervention
• suspension
• termination
• mission changes
• autonomy changes

This is where AI risk management becomes operational.

Use constraints to manage agent risk

A constraint is an explicit boundary applied to an agent's behaviour. Examples include:

• maximum spend
• restricted actions
• restricted systems
• rate limits
• time windows
• mandatory approvals

A useful runtime policy engine can return outcomes such as:

This creates a direct connection between AI risk policy and agent execution.

Use approvals for high-risk actions

Not every agent action should require a human. The objective is to apply human involvement proportionately to risk. A typical pattern is:

This approach allows useful autonomy while retaining meaningful human control over consequential actions — see approvals in depth.

Use intervention when conditions change

Risk management does not stop when an agent begins a mission. A human operator may discover that:

• the agent is behaving unexpectedly
• the business situation has changed
• the mission is no longer appropriate
• the agent has entered a risky state
• a new incident has occurred

The ability to intervene provides a runtime response mechanism. Depending on the situation, an operator may pause, resume, redirect or terminate the agent.

Record risk-management decisions

The Manage function becomes much stronger when decisions are recorded. A useful record can connect:

For example:

This creates evidence that risk controls were actually applied.

NIST AI RMF and multi-agent systems

The four NIST functions are especially useful for multi-agent environments. Imagine a workflow involving:

• research agent
• planning agent
• execution agent
• reporting agent

Each agent may have different responsibilities and permissions. Governance should therefore consider:

• which agents can communicate
• which agents can invoke others
• which tools each agent can access
• which agent can cause external effects
• where approval is required
• how the complete workflow can be interrupted

A central control layer can help enforce consistent policies across the multi-agent workflow.

NIST AI RMF implementation checklist

How FirstHelm maps to NIST AI RMF

FirstHelm's capabilities can be viewed through the four NIST functions:

This mapping is useful as an implementation aid. It should not be interpreted as a claim that deploying FirstHelm automatically satisfies every requirement applicable to an organisation under the NIST AI RMF. The framework is deliberately broader than any individual software platform.

NIST AI RMF versus AI agent control planes

The NIST AI RMF describes what organisations should manage. An AI agent control plane helps provide mechanisms for how operational controls can be implemented. The two therefore operate at different levels:

This distinction is important. A governance framework without operational enforcement can become disconnected from runtime behaviour. A technical control plane without organisational governance can enforce rules that were never properly defined. Effective AI governance needs both.

Frequently asked questions

Building a NIST-aligned AI agent operating model

A practical approach is:

1. Govern the organisation and assign accountability.
2. Map agents, missions, systems and risks.
3. Measure activity, performance, violations and controls.
4. Manage risk through constraints, approvals and intervention.
5. Then repeat the cycle.

Autonomous AI systems are not static. The governance system needs to learn from operational evidence and adapt accordingly.

Next steps

The NIST AI RMF provides a practical language for thinking about autonomous AI risk. For agent-based systems, the most important step is connecting the framework's risk-management functions to the actual runtime environment.

FirstHelm provides a central control layer for doing that across autonomous agents, including agents built with different frameworks — supporting AI agent governance and AI agent security. See the docs and compliance overview to get started.