How a Meeting Got Killed by an AI Agent — and What We Built to Prevent It

On March 18th, a user in our Discord reported that their Microsoft Teams meeting detection stopped working at exactly 3:00 PM — while they were still in the call. Their "in meeting" indicator vanished. The meeting record in screenpipe was closed prematurely.

They were running a custom pipe — a scheduled AI agent that summarizes Teams meetings every hour. At 3:00 PM, the pipe's hourly cron fired. The agent queried the screenpipe API, read meeting data, and somewhere in its chain of tool calls, hit POST /meetings/stop.

The meeting was still going. The agent didn't mean to end anything. But our API didn't distinguish between "the user is stopping a meeting" and "an automated agent is stopping a meeting." The pipe had full access to every endpoint. There was nothing preventing it.

This is our fault, not the user's.

We had a blocklist. It had five entries.

Our server middleware had a hardcoded deny list for pipe requests:

const MUTATION_PATHS: &[&str] = &[
    "/data/delete-range",
    "/data/delete-device",
    "/audio/retranscribe",
    "/speakers/merge",
    "/speakers/reassign",
];

Five paths. That was it. Every other endpoint — including /meetings/start, /meetings/stop, /meetings/bulk-delete, and everything we'd add in the future — was wide open.

We wrote this blocklist months ago when we first added pipe permissions for data access. At the time, we didn't have meeting endpoints. When we shipped meeting management, nobody thought to update the blocklist. And that's exactly the problem with blocklists: they require someone to remember. Nobody ever does.

This is the default-allow anti-pattern — the same mistake firewalls made in the 1990s before the industry converged on default-deny. We were repeating it.

Every agent framework has this gap

Before building a fix, we looked at how the rest of the industry handles agent permissions. The answer, broadly, is: they don't.

LangChain: agents have access to whatever tools you register. You can limit the tool set, but once a tool is available, the agent can call it with any arguments. No way to say "use this HTTP tool, but only for GET requests."

CrewAI: a "researcher" agent is told in its system prompt to only read. Nothing enforces it. A hallucinated tool call sails through.

OpenAI Assistants: if you register a function, the model can call it. Authorization logic lives in your application code — which means you're building a bespoke permission system from scratch every time.

Claude Code was the closest to what we needed. Anthropic uses Tool(specifier) patterns with glob matching and deny-before-allow evaluation:

permissions:
  allow:
    - Bash(npm run *)
    - Read(src/**)
  deny:
    - Bash(rm *)

This is the right mental model. But Claude Code is designed for interactive IDE sessions with a human watching. We needed the same rigor for autonomous agents running on a cron schedule at 3 AM with nobody watching.

Three lessons from building the fix

1. The permission boundary must be the API, not the prompt

The most common "solution" to agent permissions is prompt engineering: tell the agent not to call dangerous endpoints. This is what CrewAI does with role descriptions. It doesn't work.

LLMs hallucinate tool calls. They misinterpret instructions. They reason their way into "I need to call this endpoint to complete the task" despite being told not to. Prompt-level restrictions are suggestions, not enforcement.

The enforcement boundary has to be somewhere the model can't talk its way past. For us, that's two layers:

Client-side: before the agent executes a curl command, a Pi extension parses the URL and method and checks it against permission rules. If denied, the agent sees "Permission denied" in the tool output and adjusts its approach. This gives fast feedback.

Server-side: every restricted pipe gets a unique session token (sp_pipe_*). The server middleware resolves the token and checks is_endpoint_allowed(method, path) before forwarding. Even if the agent bypasses the client (uses Python requests instead of curl), the server rejects it.

Agent → curl localhost:3030/meetings/stop
         ↓
  [Client] parse URL, check rules → BLOCK
         ↓ (if passed)
  [Server] resolve token → is_endpoint_allowed() → 403 or forward

The model never gets to argue with the middleware.

2. Allowlists beat blocklists — but you can't force them on everyone

The secure default would be to restrict every pipe and require explicit opt-in. We chose the opposite: no permissions block means full access. Here's why.

Screenpipe pipes are personal automation. They run on your machine, on your data, written by you. The threat model isn't "malicious agent from untrusted source" — it's "my own agent accidentally did something I didn't intend." Forcing every pipe author to declare permissions for a problem they haven't hit yet adds friction for zero benefit.

Instead, we provide opt-in presets. One line of YAML:

permissions: reader

This activates an explicit allowlist — only safe read endpoints pass through. Everything not on the list is denied. New endpoints we ship tomorrow are blocked by default. The five-entry blocklist is dead.

For fine-grained control, we borrowed Anthropic's syntax and adapted it for HTTP:

permissions:
  deny:
    - Api(* /meetings/stop)
    - Api(* /meetings/start)
    - Api(DELETE /meetings/*)

Api(METHOD /path) with glob matching. * in the method position matches GET, POST, PUT, DELETE. * in the path matches any character sequence. Evaluation order: deny → allow → default → reject. Deny always wins — same as every firewall rule since the 1970s.

The meeting pipe that caused the incident can now add three lines of YAML and be permanently safe.

3. The syntax should match what happens, not what it means

We considered OAuth-style scopes (meetings:write, search:read), capability tokens, and resource-level grants. We rejected all of them for the same reason: abstraction mismatch.

When the server middleware checks a request, it sees an HTTP method and a path. That's it. meetings:write is an abstraction that maps to POST /meetings/start and PUT /meetings/* and POST /meetings/merge — but which of those does meetings:write actually cover? You'd have to look it up. And when we add POST /meetings/bulk-archive next month, does meetings:write include it? Who decides?

Api(POST /meetings/start) has no ambiguity. It means exactly what the middleware checks. Glob patterns like Api(GET /meetings/*) handle evolution — new sub-endpoints are automatically covered. Developers already understand this syntax from nginx configs, API gateway rules, and .gitignore patterns.

The best abstraction is the one you don't have to look up.

What we got wrong, and what we'd change

We should have shipped this before meeting endpoints. The blocklist was a known liability. We treated permissions as a nice-to-have instead of a prerequisite for new API surface. Going forward, new endpoint categories ship with a permissions review.

The token system adds overhead for unrestricted pipes. Currently, pipes without a permissions block run without tokens — zero overhead. But it means the server can't distinguish "pipe request" from "user request" for unrestricted pipes. When we add audit logging, we'll likely generate tokens for all pipes.

We need store-level permissions. Right now, permissions are self-declared by the pipe author. When we ship the pipe store, installed pipes should declare required permissions in their listing, and users approve on install — like mobile app permissions. A pipe that requests admin access should get more scrutiny than one requesting reader.

The fix in one line

If you build screenpipe pipes and want the safety net:

permissions: reader

Your pipe gets full read access to search, meetings, activity summaries, and notifications. It cannot start or stop meetings, delete data, run raw SQL, or call any mutation endpoint. If we add new endpoints next week, they're blocked by default until we explicitly add them to the allowlist.

The user whose meeting got killed? They added permissions: reader to their pipe. Problem solved — permanently.

Full reference: pipe permissions docs →

screenpipe is open source and runs locally on your machine. The code for this permission system is in crates/screenpipe-core/src/pipes/permissions.rs. Get screenpipe →