AI Governance · Field Notes
Why I Stopped Talking About AI Governance and Started Building Instead
An auditor's case for getting your hands dirty.
Over the past couple of months, I've noticed that my LinkedIn feed is full of people talking AI in risk and compliance. Whether it's the first line, second line, or third line (Internal Audit FTW, woo!), everyone is talking about AI governance, but very few people have had to make an AI system actually behave.
With so many tools out there, ChatGPT, Claude, Gemini, Grok, what are we really looking at? Is prompt engineering all a non-tech compliance fella or auditor is supposed to do? Or do we push ourselves from being the traditional Clark Kent wearing a suit and tie to an actual techy Clark Kent who knows what we're talking about?
But let's be honest, let's take a step back. Most operational and IT auditors aren't tech people. Most of us came from a traditional accounting or finance background and were fortunate enough to go through the public accounting rigor to learn a thing or two about technology, applying all of that knowledge on end clients. Then when we went in-house to an audit shop, applying that knowledge is another thing entirely, since politics plays an important role in implementing new technologies and actually getting your work done.
I've been fortunate enough to be in scenarios and companies where I worked with the brightest (and not-so-brightest), but I was able to learn what a good control looks like and, more importantly, what counts as evidence that the control works. When generative AI arrived, I watched my last organization rushing to deploy it, execs talking about applying it everywhere, and the CAE rambling about the importance of AI and data analytics. Like most organizations, they wanted to rush to deploy it without really understanding what the heck we were supposed to do. Was Internal Audit responsible for this malarkey, or was IT? So instead of writing another think-piece about AI risk, I built two working AI tools and then did the harder thing: I tried to govern them.
This is what that taught me, and why I think it makes the middle folks, us auditors, more useful people to have in the room when a company is deciding how to adopt AI "safely."
The tools
The first is a SOX Control Triage Tool (nextrend-sox-triage.streamlit.app). You paste in an internal control description and it returns a structured audit analysis: control type, framework alignment, a risk rating, a plain-English summary, and recommended test procedures.
The second, and the one I want to talk about here, is an AI Risk Assessment Tool (nextrend-ai-risk.streamlit.app). You describe an AI use case, set its context (deployment scope, data sensitivity, decision impact, level of human oversight), and it returns a structured risk assessment grounded in the NIST AI Risk Management Framework (AI RMF 1.0): an overall risk rating, which of the four NIST functions the risk sits in, the specific risk categories, a plain-English summary, and recommended controls.
Both are live. Both are built with Python, Streamlit, and the Claude API. Neither is complicated. That's the point. The value was never in the code. The whole point was in what building it forced me to understand.
What I learned
Before this, I could have described an AI risk assessment in the abstract. Building one made me confront the parts I didn't see coming.
An ungoverned model gives you a different answer shape every time. My first instinct was to ask the model to "assess the risk and recommend controls." The output was inconsistent, hard to compare across use cases, and impossible to build a workflow around. The fix was to constrain the model to return strict, structured output with named fields, and to tell it explicitly never to invent regulation numbers or control identifiers it wasn't certain about. That single constraint, forcing structure and prohibiting fabricated citations, is the difference between a demo and something you could actually put in front of a risk committee. A hallucinated citation (we've all seen the headlines) in a risk report is worse than no citation.
The prompt is a control, and it deserves versioning. My first prompt was vague and produced vague results. My second was specific: it defined the four NIST functions (Govern, Map, Measure, and Manage, or GM cubed for short), told the model how to weight the context inputs, and pinned down the exact output format. I kept both versions on purpose, because the gap between them is the lesson. You don't get a governed system by accident. You get it by deciding what "good" looks like and constraining the system toward it.
Deploying safely is its own design decision. A public tool that calls a paid model on every click is an open door: to cost, to abuse, and to prompt injection. So I defaulted the public version to a mode that returns pre-written sample assessments with no live model call. The public sees a safe, deterministic demo. The live model only runs in a controlled setting. That's an input and output control applied to my own tool, and it's exactly the kind of decision I'd expect a client to make before exposing an AI system to the public.
Credentials never belong in the code. The tool reads its API key from the environment, never from the source. That's a small thing that beginners get wrong all the time (the audacity of Claude to make me change my API keys a billion times, but I get it now). Jokes aside, with all the token cost dollars in the news lately, this is where it matters for everyone.
Why this matters for AI governance work
Here's the connection that really brought it home for me.
A lot of AI governance advice is written by people who've never had to make a model do what they wanted. And a lot of the engineers who can make models behave have never had to produce evidence that would survive an audit (no pun intended). The useful person sits in the overlap: someone who understands how these systems actually fail, and also understands what a regulator, an audit committee, or a client's second line of defense will accept as proof that the risk is managed.
That overlap is where I wanted to stand, and where I want the middle people, the auditors, to stand. Our audit background means we don't just ask "does the model work?" We ask the questions that governance actually turns on: For whom does it work? What data did it use? How was it evaluated? How will we know when it drifts? Who is accountable when it's wrong? And crucially, where is the evidence?
Building these tools didn't make me an AI engineer. It made me something more useful: someone who can sit with a data science team in the morning and an audit committee in the afternoon, and translate honestly between them.
What's next
The tools work, but governance is never "done," and I want to be straight about where these stand.
The honest next step is measurement. Right now the AI Risk Assessment Tool produces structured, reasonable output, but "reasonable" is not a metric. So I'm building an evaluation set: a library of use cases with expert-labeled expected outcomes, drawn from my own risk judgment, so I can measure where the tool agrees with a trained assessor and, more importantly, where it doesn't. Finding the failure cases is the whole point. A tool you haven't tried to break is not a governed tool. It's a hopeful one.
I'll write that up when it's done, failure cases and all. That's what building in public should mean: not a highlight reel, but an honest account of a system getting more trustworthy over time.
If you're working through how to adopt AI responsibly in a mid-market organization, or you just want to compare notes, I'd genuinely like to talk.