Before You Deploy AI in Safeguarding: Seven Questions Public Services Must Ask

The safeguarding case no one talks about

Generative AI tools are spreading through public service organisations faster than many risk and safeguarding teams can evaluate them. A social worker drafting a case summary. A housing officer reviewing complaint patterns. A charity triaging referrals. Each scenario feels like a natural fit for productivity gains. Each one carries the weight of decisions that affect vulnerable people.

Yet many organisations deploying these tools have not asked the right questions first. Not because they are reckless, but because the conversation between operational teams and safeguarding teams often does not happen until after a tool is live. And by then, asking hard questions feels like admitting a mistake.

It does not have to be this way. The organisations getting this right are the ones that pause before adoption and ask uncomfortable questions together.

What makes this different from other operational decisions

AI adoption in most roles is reversible. If a scheduling tool creates friction, you change tools or go back to spreadsheets. If a document summariser misses nuance, you adjust how people use it. Risk is contained.

In safeguarding and sensitive work, the consequences of an AI error are not operational inconvenience. They are human harm. A missed pattern in case data. A bias in risk assessment. Sensitive information retained in a third-party system designed for general use. These are not hypothetical risks: they are the reason safeguarding frameworks exist.

This is not an argument against using AI in these settings. It is an argument for doing it deliberately, with safeguarding expertise at the table before the tool arrives.

Seven questions to ask before deployment

1. Where exactly is the sensitive data going? If a tool processes case notes, risk assessments or personal information, where does that data live? Is it held on your servers, on the vendor's servers, used for training, logged for improvement? Many generative AI systems designed for general business use are not built for data security standards public services need. Asking this question should feel obvious. Often it does not happen until implementation has started.

2. Who can see the output, and how will you control that? A generative AI summary of a case might be accurate but could be shared inappropriately, forwarded insecurely, or copied into systems where access controls are weaker. If the tool improves efficiency by creating outputs people use casually, you have created a new safeguarding risk. How will you govern that?

3. What does bias look like in this context, and how will you spot it? Generative AI systems can reflect bias in their training data in ways that are not obvious at first. A risk assessment tool might weight certain factors differently for different demographics. A tool summarising case notes might miss patterns it has not learned to recognise. You need a specific way to monitor for bias in your context, not a general promise that the tool is fair.

4. Can a human reasonably review and correct the output? If a tool generates a safeguarding summary, case plan or risk assessment, is it realistic for a caseworker to review it properly, or is the output presented with enough authority that review becomes a rubber-stamp? Oversight only works if it is genuinely possible. If timescale, training or workload make thorough review impossible, the tool creates risk, not relief.

5. What decision is the AI making, and what is the human decision? This matters more in safeguarding than almost anywhere else. If an AI tool flags a case as high risk, is that a recommendation to a caseworker, or has the caseworker abdicated the judgment? If it sorts referrals by priority, does a human review the ranking? Be specific about where the human judgment sits and ensure that person has the time and training to make it properly.

6. What happens when the tool gets it wrong? Not if: when. An AI system will produce outputs that are inaccurate, biased, incomplete or inappropriate for your context. How will you know? What is your process for identifying failures, correcting them, and preventing harm? This is not about perfection: it is about having a way to catch problems quickly and learn from them.

7. Do your team and your governance structures understand what this tool does and why? Staff need training, not just access. Safeguarding boards, audit committees and senior leadership need to understand the risk and oversight. If frontline staff are using an AI tool but management is not aware, or if awareness exists but understanding does not, governance breaks down. Adoption without informed consent from the people using it and the people overseeing it is a safeguarding risk in itself.

This is not paralysis

Asking these questions might reveal that a generative AI tool is genuinely safe for a specific task: flagging non-sensitive administrative patterns, drafting templates for human completion, supporting research that does not involve individual case data. These are legitimate uses. But they only become legitimate after proper assessment.

The organisations getting this right are treating AI adoption in sensitive work as a governance question first and an efficiency question second. They are building time for safeguarding teams to assess tools before they reach users. They are training people not just on how to use a tool, but on the risks it creates and how to oversee it properly. They are treating the first weeks of deployment as a high-scrutiny period, not a rollout to be completed.

This approach takes longer upfront. It is also the approach that prevents an efficiency gain from becoming a safeguarding failure.

If your organisation is planning to deploy generative AI in safeguarding, case management or sensitive work, these questions deserve answers before you start. They are not obstacles to adoption: they are the foundation that makes adoption safe enough to proceed. VAxAI works with public services and charities to identify where AI and automation could genuinely help, assess the safeguarding and governance questions this creates, and prepare your organisation, your data and your people for safe adoption. If you are navigating this decision, we can help you ask the right questions together. Start a conversation with us about what safe AI adoption looks like in your context.