Free live cohort on Google Meet — register your interest →
aws aif-c01

How would you make sure an AI system is responsible?

Based on Chapter 13 of the free AWS AI Practitioner track

Using this with an AI assistant? View as Markdown ~3,221 tokens

In short

  • Responsible AI here means six observable features: bias, fairness, inclusivity, robustness, safety and veracity. Learn each one by the failure that reveals it.
  • Bias is the measurement and fairness is the standard. Harm is safety, truth is veracity, and stability under unexpected input is robustness.
  • Guardrails check input and output, and show evidence only for safety and veracity. The contextual grounding check is the veracity policy. Guardrails do not measure bias or change the model.
  • Sustainability is a real criterion for model selection. Apply it after capability and cost, and write down the rationale.
  • The five legal risks are business exposures. Hallucination and end user risk are two ends of one incident, and loss of customer trust is where all the risks accumulate.

An interviewer asks how you would make sure an AI system is responsible. Most candidates answer with values, but the credit goes to answers built on properties.

This article argues that responsible AI, as a widely used AI practitioner exam guide frames it, is a short list of things you can check. It names six features, and each one fails in a way you can see. A guardrail service can show evidence for two of them. Model selection has its own responsible practices. Five legal risks describe what the business pays when the features are missing. Learn the features and the risks as one list read from two ends, and you can answer both the interview question and the follow-ups. Those follow-ups are written to catch anyone who answers from good intentions.

The strong answer names properties, not good intentions

A weak answer says the system was built responsibly and points to a policy document. A strong answer runs in five beats, in this order:

  1. Name the properties you claim: bias, fairness, inclusivity, robustness, safety and veracity.
  2. Say what you would observe if each one failed.
  3. Name how you enforce them and say plainly what that tooling cannot see.
  4. Bring responsibility into model selection, not only into operation.
  5. State the exposure in business terms, so a decision-maker can act on it.

The second beat separates a designed answer from a sincere one. A property you cannot detect is a value, not a control. Think of it as the difference between a smoke alarm and a fire-safety poster. Both say they care about fire, but only one tells you when there is a fire.

A failure scenario branches to six features, each tied to the observation that reveals it

Each responsible AI feature is identified by the failure you would observe

Every option in a question on this topic describes something undesirable. That is why common sense fails here. Credit goes to the option that names the property the guide uses.

Follow-up: bias is the measurement, fairness is the standard

“Aren’t bias and fairness the same thing?”

Model answer: No. Bias is the measurable skew: outcomes differ across groups in a way the individual records do not explain. You can compute it. Fairness is the standard you use to decide whether that skew is acceptable. You cannot compute fairness. You have to decide it. So a system can be biased before anyone has agreed it is unfair. A system can also be judged unfair over a small skew.

The weak candidate’s mistake: treating the two words as synonyms, so that either one seems to fit. Use a simple test. A scenario that says “outcomes differ by group” describes bias. A scenario that asks “is this difference acceptable?” is asking about fairness.

For example, picture a lending assistant that gives more cautious eligibility guidance to customers who write in one language than to customers who write in another. The gap in outcomes is the bias. Whether the organisation accepts that gap is the fairness question.

Follow-up: safety, veracity and robustness fail in different ways

“The model gave a wrong answer. Is that a safety problem?”

Model answer: Not necessarily. Each of the three properties asks a different question:

  • Safety is about harm. The output is abusive, dangerous, or in a category the application should refuse.
  • Veracity is about truth. The output is wrong.
  • Robustness is about stability. The system behaves well on clean input and erratically on misspellings, mixed languages, odd formatting or deliberate attack.

Take a model that confidently states a drug interaction that does not exist. That is a veracity failure. If a user acts on it and is harmed, the harm is a separate consequence, and the guide gives it its own name: end user risk, which is one of the legal risks covered below.

The weak candidate’s mistake: reasoning “it was wrong, so it was unsafe,” or calling erratic output on messy input an accuracy failure. Instability under unexpected input is robustness, even when the answers are also wrong.

Inclusivity completes the six. It asks whether everyone the system is meant for can actually use it. A model that works well in only one language, or that assumes literacy or connectivity its users lack, has an inclusivity problem. That is true even if it is perfectly accurate for the users it does serve.

Table contrasting safety, veracity and robustness by their question and their observable failure

Three bad-output properties, separated by the question each one asks

Follow-up: guardrails check two points and show evidence for only two features

“Which tool would you use to show these properties?”

Model answer: The guide’s named example is Amazon Bedrock Guardrails. It evaluates what goes into a model and what comes out against policies you configure, and blocks or filters anything a policy prohibits. It acts at two points:

  • On input, before the model is called. A prohibited request can be stopped before it ever reaches the model.
  • On output, before the user sees it. A response can be blocked or masked.

It has five policy types:

  • Content filters cover hate, insults, sexual content, violence and misconduct, at a strength you set.
  • Denied topics define subjects the application will not discuss at all.
  • Word filters block specific words and phrases.
  • Sensitive information filters detect personally identifiable information and mask or block it.
  • The contextual grounding check confirms that a response is supported by the source material provided and is relevant to the query.

The first four provide evidence for safety, and the sensitive information filters also cover privacy. The grounding check is the only policy that addresses truth rather than harm, which makes it the veracity policy.

Input is checked before the model is called and output is checked before reaching the user

Guardrails evaluate input before the model runs and output before the user sees it

The weak candidate’s mistake: claiming more than the tool does. Guardrails do not measure bias across demographic groups, because no policy computes differences in outcomes between groups. They do not assess inclusivity or robustness. They do not retrain the model. They sit around the model and evaluate its traffic, and the model’s weights are untouched. They do not document the model either. Recording intended uses and limitations is the job of a model card, a separate document.

Saying these limits without being asked is the third beat of the strong answer. It is also what makes the rest of the answer believable. One more point is worth adding. Retrieval-augmented generation supplies source material to the model, which makes a grounded answer possible. The grounding check is what verifies that the model actually used that material.

Follow-up: sustainability counts when you choose a model

“How does responsibility affect which model you pick?”

Model answer: The guide names environmental considerations and sustainability as responsible practices in model selection. In practice there are four:

  • Prefer the smaller model when it is sufficient. Energy and compute grow with model size, so a larger model that does the same job spends more for no gain.
  • Adapt an existing model rather than train a new one. Training is the largest energy cost in a model’s life.
  • Consider the provider’s position, because infrastructure efficiency and energy sourcing differ.
  • Write down the rationale. A choice you cannot explain is not a responsible practice, whatever you chose.

The order matters. The responsible screen runs after capability and cost, and it chooses only among models that already meet the requirement. Efficiency never rescues a model that cannot do the job.

Capability and cost come first, then size, adaptation and provider, ending in a documented choice

The responsible screen chooses among models that already meet capability and cost

The weak candidate’s mistake: dismissing sustainability as a soft, non-technical concern, or defending the bigger model as “headroom for the future.” When a smaller model is enough, the larger one is the less responsible choice. One way to remember this: the usual ordering of customisation options, from prompting through retrieval and fine-tuning up to full pre-training, is ordered by cost, and it is ordered by energy too. “Don’t fine-tune when prompting would do” is also the sustainable answer.

“What legal risks does generative AI create?”

Model answer: The guide names five: intellectual property infringement claims, biased model outputs, hallucinations, end user risk, and loss of customer trust. The way to frame them matters more than the list. These are exposures for the organisation, not bugs in the model.

It helps to sort them by where the exposure enters:

  • What went in. Infringement claims arise when training data or generated output reproduces protected work. One example is an assistant that copies sentences word for word from a competitor’s brochure.
  • What came out. Biased outputs disadvantage a group. Hallucinations are confident, fluent and untrue.
  • What someone did with it. End user risk arises when a user relies on output in a setting where being wrong has consequences, such as medical, legal, financial or safety-critical decisions. Loss of customer trust is where all of these accumulate.

Table grouping five legal risks into inputs, outputs and user actions, with a signal for each

The five legal risks sorted by where the exposure enters

The weak candidate’s mistake: offering a technical fault such as “overfitting” or “a robustness failure” as a legal risk. A second mistake is waving hallucination away as an evaluation issue. Hallucination is on the list on purpose. The technical event is a wrong answer. The legal event is that someone relied on it. That is why hallucinations and end user risk so often appear together: they are the two ends of a single incident.

Loss of customer trust differs from the other four. It is not a separate kind of incident. It is the consequence that outlasts the incident. You can fix a single hallucination, but fixing it does not repair the belief that the product cannot be trusted.

Follow-up: features and risks are the same list read from two ends

“How do the features relate to the risks?”

Model answer: The features name the properties a system should have. The risks name what it costs when they are missing. A missing veracity property shows up as hallucinations, which can lead to end user risk. Gaps in fairness and bias show up as biased model outputs. A safety failure becomes end user risk when someone acts on harmful output, and so does a robustness failure on unexpected input. An inclusivity gap becomes lost trust among the users who cannot use the system. Every one of these ends in loss of customer trust.

Missing features map to hallucinations, biased outputs and end user risk, all reaching lost trust

Each missing feature becomes a named legal risk, and all of them end in lost trust

The weak candidate’s mistake: treating the features and the risks as two topics to memorise separately. Once you learn the mapping, a question that describes a harm and asks for the feature, and a question that describes a gap and asks for the risk, become the same question. In our view, this mapping is the single most useful thing to take into either an exam room or an interview.

Key takeaways

  • Responsible AI here means six observable features: bias, fairness, inclusivity, robustness, safety and veracity. Learn each one by the failure that reveals it.
  • Bias is the measurement and fairness is the standard. Harm is safety, truth is veracity, and stability under unexpected input is robustness.
  • Guardrails check input and output, and show evidence only for safety and veracity. The contextual grounding check is the veracity policy. Guardrails do not measure bias or change the model.
  • Sustainability is a real criterion for model selection. Apply it after capability and cost, and write down the rationale.
  • The five legal risks are business exposures. Hallucination and end user risk are two ends of one incident, and loss of customer trust is where all the risks accumulate.

This article settles how to name what is wrong with a system. It does not settle how to prove it. Showing that bias exists takes measurement, such as dataset analysis, subgroup comparisons and human audits, and none of that is covered here.

Want more like this?

Free newsletter. Real projects, delivered fortnightly.

No spam. Unsubscribe anytime. Replies go to a real person.

← Back to all articles