The Shadow AI You Cannot See

A few weeks ago I wrote about the model Chrome had quietly downloaded onto my machine, and the small retrieval system I built over a weekend to put it to work. This is a field note about the next thing I built with the same on-device model. It is a more serious tool than the last one, because it answers a question that regulated organisations are increasingly being asked, and mostly cannot answer: which AI are your people actually using?

The tool is called ShadowGlass. It reads a firewall or proxy log export, tells you which AI providers and models your organisation is calling, flags the ones that are not on your approved list, and never sends a single byte anywhere. It is a Chrome extension with two permissions and no network access. The code is on GitHub.

This is a note about why I built it, how it works, and why the local-first shape of it is the whole point rather than a technical detail.

The problem you cannot see.

Every AI governance framework I have read assumes a fact that is usually not true. It assumes the organisation knows which models it is calling.

The board asks whether the company is exposed to AI risk. The framework says to inventory your AI systems, classify them by risk, and put controls around the high-risk ones. All of that is sound. All of it depends on a first step that quietly gets skipped: knowing what is actually being used. Not what was procured. Not what the AI policy permits. What is running, right now, in the hands of real people getting real work done.

The gap between those two things has a name. Shadow AI is the AI usage that nobody sanctioned: the analyst pasting client data into a consumer chatbot, the engineering team wiring a side project to an unapproved provider, the marketing tool that added a generative feature in its last update and now calls a model your security team has never assessed. It is not usually malicious. It is people reaching for the fastest tool to hand. But it is, by definition, the AI you are not governing, because it is the AI you cannot see.

Here is the part that should be more widely known. The evidence is already in your building. Every one of those calls leaves a trace in your firewall or web proxy logs. A request to api.openai.com, to api.anthropic.com, to a Bedrock or Azure OpenAI endpoint, is sitting in an export that your security team could pull this afternoon. The information exists. It is just not being read for this purpose.

When I looked at the tools that do read it, they nearly all shared one property that makes them unusable for the people who most need them. They want you to upload your firewall logs to their cloud.

What I built.

ShadowGlass is a Chrome side-panel extension. You drop an xlsx, CSV or JSONL log export onto it, from zScaler, Palo Alto, or a generic proxy. It parses the file in the browser, matches every request against a registry of known AI provider endpoints, and produces an inventory: which providers, which models, how many calls, from which internal source addresses. It flags anything not on a local approved-providers list as shadow AI. Then, if you want it, it writes a plain-English summary of the findings.

The whole thing runs on your machine. There is no account, no server, no upload. The extension requests exactly two permissions, side panel and local storage, and has no ability to make a network request or read the pages you browse. You do not have to take that on trust. The manifest is nine lines, and the source is public.

That last point is not a marketing line. It is the design constraint that everything else follows from.

How it works.

Three decisions did most of the work, and each one was a decision about what to leave out.

Detection is deterministic, not AI. The part that decides “this request went to Anthropic” is plain regular-expression matching against a registry of provider endpoints. There is no model in that loop. This is deliberate, and for a security tool it is not a compromise, it is the requirement. A tool that tells a security team they are running unapproved AI has to be exactly right and has to be able to show its working. Regular expressions are reproducible and auditable. A language model's guess about each URL is neither. So the registry is the backbone, and it decomposes the wrapper routes too: a call to AWS Bedrock or Azure OpenAI is resolved back to the underlying model owner rather than left as an opaque platform hit.

The on-device model does the reading, not the deciding. Chrome's built-in Gemini Nano, the same local model from the last field note, has two jobs here. It writes the narrative, turning the deterministic inventory into the executive summary a human actually wants to read. And it does the one analytical thing regular expressions cannot: it looks at the traffic the registry did not recognise, the unfamiliar endpoints, and flags the ones that look like AI inference APIs. That is the model earning its place, finding the unknown rather than re-checking the known. Its suggestions are kept strictly separate from the confirmed inventory and never raise an alert on their own, because a model's guess is a lead for a human to verify, not a finding.

Nothing leaves the browser. Nano runs on the device. The parsing happens in the browser. The model never even sees a raw log row, only a small aggregate summary with all source identity stripped out. There is no telemetry and no remote call anywhere in the bundle. The privacy claim is not a policy you have to trust. It is a property of the code you can audit.

Why it has to be local.

This is the part that matters for the kind of advisory work I do, and it is the same lesson the retrieval prototype taught, in a sharper form.

A security team investigating shadow AI is handling some of the most sensitive data in the organisation. Firewall logs reveal who talked to what, when, from where. They are exactly the sort of data that a security team cannot casually hand to a third-party SaaS, and exactly the sort of data that most shadow-AI-discovery products ask them to upload. The result is a category of tool that is aimed at security teams and is quietly unusable by the ones with the strictest posture, which are the ones with the most to lose.

A tool that runs entirely on the analyst's laptop does not have that problem, and not because it is more compliant by nature. It is because the pilot data path and the production data path are the same path. There is no moment where proving the idea on a sample export becomes running it on real logs, and that transition requires a data processing agreement, a residency negotiation, and a security review that stalls the whole thing. The analyst who ran a sample file in the trial runs the real file in production, on the same machine, with the same posture. Nothing changes between the two.

I have watched enough cloud pilots die in exactly that gap to treat it as the default outcome rather than the exception. Local-first does not beat cloud for every case. It is simply a different operating model, and for high-trust, security-shaped work it is often the only one that survives contact with the compliance team.

What this means for governing AI.

If you are accountable for AI risk in a regulated organisation, the useful reframing is this. Visibility is not a reporting nicety that comes after governance. It is the precondition for it. You cannot classify, control, or attest to AI usage you cannot see, and the AI you cannot see is precisely the AI most likely to be a problem.

The reassuring finding is that the first step is cheaper than the frameworks imply. You do not need a platform, a procurement cycle, or a data-sharing agreement to answer “are we exposed to shadow AI”. You need a log export you already have and a way to read it that your security team is allowed to use. The answer is often available in an afternoon, from evidence already sitting in your estate.

That is a smaller, more tractable first move than “implement an AI governance programme”, and it is the one that makes the rest of the programme real, because it replaces an assumption about your AI usage with a measurement of it.

Limits worth naming.

Every honest write-up should name its own limits before someone else does.

Detection is based on the destination of a request, so where a provider names the specific model only in the request body rather than the URL, and most do, ShadowGlass can prove the provider is in use but not always which model. Firewall logs rarely capture request bodies, so this is a limit of the data, not the tool, and no method escapes it from logs alone.

Detection of self-hosted or novel endpoints is best-effort and deliberately low-confidence, because an unfamiliar host with an AI-shaped path could be a genuine private inference endpoint or an ordinary internal service. That is exactly why those matches are surfaced for review and never treated as confirmed.

The narrative and classification features depend on Chrome's built-in Nano being available, which varies by browser version, device capability, and enterprise policy. Every other part of the tool works without it.

And the endpoint registry is only ever as current as the patterns in it. New providers appear constantly. That is why the registry is the open, community-maintainable part of the project, and the part I would most like other people to add to.

You cannot govern the AI you cannot see.

This is not a product. It is a working tool that makes a point. The point is that the visibility gap at the centre of AI governance is smaller than it looks, because the evidence is already in the organisation, and the technology to read it privately is already in the browser on the analyst's desk. The model is on the device. The logs are in the estate. The only thing missing was a way to bring them together without the data leaving the room.

You cannot govern the AI you cannot see. As it turns out, seeing it does not require sending it anywhere.

ShadowGlass is open source and MIT-licensed on GitHub. It is a working tool, not a supported product. The endpoint registry welcomes contributions.