---
title: "How to Secure an AI System on AWS and Stop Made-Up Answers"
description: "Two layers interviewers want to hear: IAM, TLS, CloudTrail and encryption to protect the system, and Bedrock Guardrails grounding checks to block hallucinations."
source: "https://confidentprep.com/blog/how-do-you-secure-an-ai-system-on-aws-and-stop-a-foundation-model-from-making-things-up/"
date: "2026-10-07"
tags: ["aws", "aif-c01", "amazon-bedrock", "ai-security"]
token_estimate: 3030
---

# How do you secure an AI system on AWS and stop a foundation model from making things up?

Interviewers often ask this as one question, and many candidates answer only half of it. Some talk about encryption and access control and never mention hallucinations. Others describe prompt tricks and forget that a chatbot leaking customer data is worse than one that gets a fact wrong. In Amazon Bedrock these are two separate layers. The first layer protects your account and your data, and much of that work is your responsibility under the shared responsibility model. The second layer checks each model response against the source you supplied, using contextual grounding checks in Amazon Bedrock Guardrails. These checks score every response for grounding and relevance and block it when a score falls below the threshold you set. A strong answer names both layers, explains how each one works, and states where each one stops.

## A strong answer separates protecting the system from checking the answers

A strong candidate begins with responsibility. AWS protects the global infrastructure that runs the AWS Cloud. You keep control of your content on that infrastructure and of the security configuration of the services you use.

Next comes what you configure yourself. Give each person an individual identity through AWS IAM Identity Center or AWS Identity and Access Management (IAM), the service that decides who can call which API, so each user gets only the permissions their job needs. On top of that, the guide recommends:

- multi-factor authentication (MFA) on every account
- TLS for all traffic to AWS. TLS 1.2 is required and TLS 1.3 is recommended
- API and user activity logging with AWS CloudTrail
- AWS encryption solutions together with the default security controls of each service

Some workloads need FIPS 140-3 validated cryptographic modules, a United States federal standard for cryptography. Those workloads should use a FIPS endpoint. The Bedrock guide also covers private network access through Amazon VPC and AWS PrivateLink, and it documents data retention as part of data protection.

Then the candidate turns to the model's output. A model can be well secured and still invent facts. Guardrails can detect and filter hallucinations when the application supplies a reference source and a user query. The guardrail is itself a protected resource: AWS encrypts it with an AWS managed key by default, and you can choose your own customer-managed key in AWS Key Management Service (KMS) instead.

Think of it as a bank branch. The vault, the badges and the cameras are the first layer. A second person who checks each statement against the ledger before it reaches the customer is the second layer.

![Two layers: account controls such as IAM, TLS and CloudTrail, and response checks scoring grounding and relevance](visuals/visual-01.svg)

*Account controls protect the system; the grounding check judges each response*

The rest of the interview tests whether the candidate understands each layer in detail. Here are five common follow-ups.

## Follow-up 1: A correct sentence can still fail the check

**Question: What is the difference between grounding and relevance?**

**Model answer.** Grounding asks whether the response is factually accurate according to the source and drawn from it. Any new information in the response counts as ungrounded. Relevance asks whether the response answers the user's query.

Take a source that says "London is the capital of UK. Tokyo is the capital of Japan," and the query "What is the capital of Japan?" The response "The capital of Japan is London" is ungrounded. The response "The capital of UK is London" is grounded but irrelevant, even though it is true.

**The weak answer** treats "true" as "safe." It misses that a response can be accurate and still fail the user. It also misses that "new information" includes facts that are true in the world but absent from the source.

![Four sample responses showing every combination of grounded or ungrounded and relevant or irrelevant](visuals/visual-02.svg)

*Grounding and relevance are scored independently, giving four outcomes*

The source gives a banking version of the same idea. The reference text states a 23.99% charge for late credit card payments. When a user asks about credit card transaction charges and the model answers 23.99%, that answer is relevant but ungrounded, because the source never ties that figure to transactions. In practice, this is the failure that does real damage. The answer sounds right, quotes a real number from the document, and attaches it to the wrong thing.

## Follow-up 2: The threshold sets how strict the filter is

**Question: How do you decide what gets blocked?**

**Model answer.** The check produces a confidence score for grounding and another for relevance on each response. You set a threshold for each, and any response that scores below a threshold is treated as a hallucination and blocked. If both thresholds are 0.7, a response with either score under 0.7 is blocked.

Raising the threshold blocks more ungrounded and irrelevant content, so fewer hallucinations reach users. Valid thresholds run from 0 to 0.99. A threshold of 1 is invalid because it would block all content.

![A response is blocked if either its grounding or relevance score falls below the configured threshold](visuals/visual-03.svg)

*A response passes only when both scores meet their thresholds*

**The weak answer** says "set it as high as possible." A higher threshold has a cost: it also rejects borderline good answers, because it demands high confidence before anything gets through. A sensible default is to start at a moderate value and raise it for workloads where a wrong answer is expensive. Then measure how often good answers are blocked before tightening further.

Think of it as the height of a security gate: higher stops more intruders, and also more of your own staff.

## Follow-up 3: The check needs three inputs and only judges the output

**Question: What does the guardrail need from your application to work?**

**Model answer.** It needs three components:

- a **grounding source**, the context needed to answer the query
- the **query**, the user's question
- the **content to guard**, which for the Invoke and Converse APIs is the model response

The check runs only on output, never on the prompt, because it needs the model's response. If a request contains several grounding sources, the guardrail evaluates them together as one combined source, and it treats several query tags the same way. There are also size limits: up to 100,000 characters of grounding source, 1,000 characters of query and 5,000 characters of response.

You can also use the check without calling a model. The ApplyGuardrail API evaluates text you already have. Because no model is invoked, you add a third content block holding the text to guard, which takes the place of the model response.

![The app sends a query and source, gets a response, and the guardrail scores that response before release](visuals/visual-04.svg)

*The guardrail scores the model's response against the source and query the app supplied*

**The weak answer** says "the guardrail checks the model's knowledge." It does not. It checks the response against the source text you supply, so a missing or wrong source makes the check meaningless. Think of it as an open-book exam graded against the book you handed over, not against the world.

## Follow-up 4: Marking text as a source removes it from other filters

**Question: If retrieved documents contain sensitive data, will your other guardrail policies catch it?**

**Model answer.** Not by default. Content marked as a grounding source or query is evaluated only by the contextual grounding check. It is skipped by word filters, topic filters, content filters, sensitive information detection and prompt attack detection.

To have that text screened by the other policies too, add `guard_content` alongside the grounding qualifier, for example `["grounding_source", "guard_content"]`. With the Invoke APIs, the same effect comes from nesting the grounding tag inside a `guardContent` tag. For the other policies to run at all, at least one content block must be unqualified or explicitly qualified with `guard_content`.

![Table showing that grounding_source and query blocks skip other policies unless guard_content is also set](visuals/visual-05.svg)

*Only blocks qualified as guard_content, or left unqualified, reach the other policies*

**The weak answer** assumes everything passing through a guardrail is filtered. That assumption is how sensitive text in a retrieved document can slip past a sensitive information filter that the team believed was protecting them.

A related data-hygiene point from the security guide: do not put confidential information such as customer email addresses into tags or free-form fields like a Name field. That data may be used for billing or diagnostic logs.

## Follow-up 5: The supported use cases and streaming both have limits

**Question: Can you turn this on for a customer chatbot and consider hallucinations solved?**

**Model answer.** No, for two reasons. First, the supported use cases are summarization, paraphrasing and question answering. Conversational question answering and chatbot use cases are not supported.

Second, relevance is checked for each chunk processed. If any one chunk is relevant, the whole response is considered relevant. With a streaming API, this means an irrelevant response can reach the user in full and only be marked irrelevant after streaming ends.

**The weak answer** treats the guardrail as a universal truth filter. In practice it is a strong control for single-turn tasks with a clear source. Multi-turn conversations and streamed output need additional design around it.

## Follow-up 6: Model providers cannot see your prompts

**Question: When you call a third-party model through Bedrock, does the model's provider see your data?**

**Model answer.** No. In each AWS Region where Bedrock is available, there is one Model Deployment Account per model provider. The Bedrock service team owns and operates these accounts. After a provider delivers a model, Bedrock makes a deep copy of the provider's inference and training software into those accounts. Providers have no access to the accounts, so they cannot see Bedrock logs or customer prompts and completions.

**The weak answer** gets this wrong in one of two directions. Either it assumes providers receive customer data, or it concludes that because providers cannot see it, the customer has nothing left to secure. The shared responsibility model still leaves identity, logging and configuration with you.

Think of it as a vendor's machine installed inside a sealed room you rent from the building owner: the vendor built the machine but holds no key to the room.

![Customer prompts go to an AWS-operated deployment account that model providers cannot access](visuals/visual-06.svg)

*Provider software runs in AWS-operated accounts the provider cannot access*

## Key takeaways

- Answer in two layers: account and data controls (IAM least privilege, MFA, TLS, CloudTrail, encryption) and response checks (contextual grounding).
- Grounding and relevance are separate scores. A true answer can fail on relevance, and a relevant answer can fail on grounding.
- Thresholds run from 0 to 0.99. Higher values block more hallucinations and also more good answers.
- The check needs a grounding source, a query and content to guard, and it judges only the output.
- Text marked as a grounding source or query skips every other guardrail policy unless you also qualify it as `guard_content`.
- Chatbot and conversational use cases are not supported, and streaming can deliver an irrelevant answer before it is flagged.

These sources do not settle how to choose a threshold value for a given workload, how the checks perform at large scale, or how to guard multi-turn conversations, which the grounding check does not support. Those questions require testing against your own data.