How to audit an MCP server before you connect it. Seven checks, and what 55 official servers scored.

Connecting an MCP server is a one-line change. It is also a permission grant: you are handing an agent a set of tools, a credential, and a remote party that gets to shape what the model sees. The one-line change is the reason nobody audits it.

We built a harness to score official MCP servers — hosted and local — against the OWASP MCP Top 10, the OAuth 2.1 family of specifications, and the Model Context Protocol conformance suite. The results are published as the MCP Safety Index. This post is the method behind it, written so you can run the same checks yourself on a server we have not scored.

Before the checks, one number from the aggregate, because it should change the order you do things in.

Protocol fidelity predicts almost nothing about security.

Across the 55 servers in the index, the correlation between a server’s protocol-fidelity score and its authentication score is 0.04. That is noise. How faithfully a server implements the spec tells you essentially nothing about how carefully it handles your credential. Security and authentication, by contrast, correlate at 0.75 — teams that think about one tend to think about the other. Protocol conformance sits in its own world.

The divergence is visible in individual servers. Three score 88 or higher on protocol fidelity while scoring below 70 on authentication: GitHub (91 protocol, 59 auth), Superhuman (91 / 67) and Monday.com (88 / 67). At the far end, the lowest-scoring server in the whole index still implements the protocol above average — 85 on protocol fidelity, and 0 on authentication, because it requires none.

This matters practically because protocol conformance is the easy thing to check, so it is the thing people check. A server that speaks the spec cleanly feels well built. Forty-two of the 55 servers score higher on protocol than they do on authentication, which means that feeling is usually wrong in the same direction.

What 55 servers look like when you score them.

Fifty hosted, five local. As the index stands today: no server scored an A. Ten scored a B, forty a C, three a D and two an F. The top score is 87 and the median is 76.

By dimension, protocol fidelity averages 80.2, security 72.7 and authentication 71.7. Authentication is both the weakest dimension and by far the widest spread — scores run from 88 down to 0, while protocol fidelity never drops below 58. The floor is what should worry you: nine servers score below 70 on authentication, and the two that failed outright are the two hosted servers that do not use OAuth at all — one on a static bearer token, one with no authentication whatsoever.

A word on what these numbers are and are not. They are point-in-time measurements against a published rubric, re-run by the harness rather than collected by hand. A low authentication score is not an allegation that a server has a vulnerability; it is a measure of how much of the modern OAuth surface the server implements and how much of your credential exposure it removes. Scores move when servers ship.

1. Establish what authentication the server actually requires.

Start by connecting as an unauthenticated client and seeing how far you get. You are separating three cases: OAuth 2.1 with a real authorization flow, a static bearer token you paste into a config, and nothing at all.

The distinction is not academic. A static bearer token is a long-lived credential that lives wherever your agent config lives, cannot be scoped per agent, and does not expire on its own. Both of the failing grades in the index are servers in this category. If the answer to this first check is anything other than OAuth, you have learned most of what the rest of the audit would have told you.

2. Walk the OAuth metadata endpoints yourself.

Do not read the documentation for this. Fetch the protected-resource metadata described in RFC 9728, follow it to the authorization server metadata, and see what is actually advertised. Then check two things: whether the server supports dynamic client registration (RFC 7591), and whether PKCE (RFC 7636) is required rather than merely permitted.

Dynamic client registration matters more than it sounds. Without it, every team that wants to connect has to go through a manual onboarding with the vendor, and the usual result is that one shared client gets registered once and reused by everyone forever. The specification detail becomes an operational failure two quarters later.

3. Check that the token is bound to an audience.

Look for resource indicators (RFC 8707) on the authorization request and issuer identification (RFC 9207) on the response. Together they answer one question: is the token you receive usable only against this server, or is it a general-purpose credential that happens to have been issued here?

An unbound token is the failure mode that turns one compromised MCP server into access to everything else that trusts the same authorization server. It is invisible from the outside, it costs nothing to check, and it is the check most often skipped.

4. Enumerate the tool surface and read every description.

List every tool the server exposes. That list is the permission grant — not the OAuth scope you approved, the tools. A server with forty tools has handed your agent forty capabilities, and the agent will find the ones you did not think about.

Then read the descriptions, in full, as text. Tool descriptions are model-facing instructions written by a third party that land in your agent context on every turn. They are the natural home for both accidental ambiguity and deliberate injection, and almost nobody reads them. Note which tools mutate state, which reach other systems, and which return free text you cannot predict the shape of.

5. Test what the server returns, not just what it accepts.

Call a low-risk read tool and look at the entire response body, not the part the tool was for. Over-sharing on the response path is the most commonly missed failure in the whole audit: an issue-lookup tool that also returns the reporter’s email address, a customer-record tool that returns the full record because filtering it server-side was extra work.

Everything in that response reaches the model, gets summarised into context, and may end up in a downstream call to a different tool entirely. Request-side controls do not help here, which is why this check has to be run against real responses rather than reasoned about from the schema.

6. For local servers, find the secret and measure the blast radius.

Local servers are scored on a different question, because they fail differently. Two things decide it. Where does the credential live — an OS keychain, or a plaintext environment variable in a config file that is one careless commit away from a repository? And what can the process reach once it is running under your user account?

A local MCP server is not sandboxed. It inherits your filesystem, your network position and your ambient credentials. The package-integrity question — who publishes it, what the install pulls in, whether it is pinned — is a supply-chain question you would ask of any dependency, and this one runs next to your agent with your keys.

7. Run the conformance suite, then judge security separately.

Run it — protocol conformance is worth knowing, and a server that mishandles capabilities, tool schemas or errors will cost you reliability. Just do not let the result colour the other six checks. That is what the 0.04 correlation means in practice: the conformance result and the security result are independent measurements, and the conformance one is the one that will arrive first and feel most reassuring.

Score the two separately, write them down separately, and make the connect-or-not decision on the security number.

What to do with a server that fails.

Most of the time the answer is not “do not connect it”. The median server in the index scores 76, which is to say: usable, with gaps. The realistic response to a gap is to close it somewhere you control — put the credential somewhere the agent never sees it, scope the tool list down to what the agent actually calls, constrain the arguments it can send, and filter the response before it reaches the model.

That is the same argument as our playbook for governing tool access in an MCP hub, arriving from the other direction. The audit tells you what a server will not do for you. The gateway is where you do it instead.

The full scores for all 55 servers, by security, authentication and protocol fidelity, are in the MCP Safety Index. If you run a server you would like measured, you can request a score there.