Vulnerability Assessment for AI Security Testing: Mapping the Attack Surface of LLM-Integrated Applications

The application security industry spent the last two decades refining a playbook for HTTP, SQL, and the browser. Burp Suite, OWASP ZAP, Nuclei, Semgrep — the tools we depend on today are built on a set of assumptions about how software behaves: inputs are structured, outputs are predictable, state lives in databases, and an attacker talks to the application through a finite set of well-defined endpoints. For everything built before 2022, that playbook mostly works.

For everything built after, it does not.

LLM-integrated applications break almost every assumption the traditional DAST stack relies on. The attacker's input is natural language. The model's output is stochastic. The application's state is smeared across a vector store, a system prompt, and a chain of tool calls that can read from email, write to production databases, and make outbound API requests on behalf of a user who never asked for any of it. The attack surface is no longer a set of endpoints; it is a graph of intentions, and the nodes are language.

What traditional scanners miss

Run Burp Suite against a modern LLM-integrated application and you will find the same classes of bugs you always found: an exposed .env, a missing cookie flag, a reflected XSS in a debug route. You will not find the prompt injection that exfiltrates the system prompt. You will not find the tool call that lets an attacker write to a different customer's row because the agent was convinced by a sentence in a PDF. You will not find the retrieval path that silently poisoned the model's reasoning because someone uploaded a document with the right 200 tokens in it.

The bugs the old playbook finds are still real. They are just no longer the most dangerous thing in the application.

The four classes worth mapping

At Xtrinel we have spent the better part of a year building a vulnerability catalog organized around four classes that together describe most of what actually goes wrong in production LLM systems. None of these classes are new in the academic sense, but the engineering practice of testing for them, reproducing them, and shipping findings that a developer can act on is still in its infancy.

Prompt injection. The direct variant — a user pasting "ignore previous instructions" into a chat box — is well understood at this point and largely mitigated by any team paying attention. The interesting variants are the indirect ones: instructions embedded in a retrieved document, in a calendar invite that the agent summarizes, in the alt text of an image the model is asked to describe. We catalog direct, indirect, and multi-turn variants, and we care particularly about multi-turn attacks where the malicious intent is built up across several benign-looking messages.

Tool-call abuse. Every modern agent framework exposes a set of callable tools — read files, send email, query the database, make HTTP requests. Each of those tools is an API in the traditional sense, with parameters that can be fuzzed, auth boundaries that can be crossed, and state changes that can be coerced. The novelty is not the API; the novelty is that the API is now being called by a non-deterministic planner that can be socially engineered. VAAST enumerates the tool graph and probes parameter injection and unauthorized state changes across it.

RAG poisoning. Retrieval-augmented generation treats the corpus as trusted context. For most ingestion pipelines, "trusted" really means "whatever ended up in the vector store." If an attacker can write to a Confluence page, a Notion database, or a customer-support ticket that eventually gets ingested, they can plant instructions that will surface later during retrieval and shift the model's reasoning in ways that never touch the user interface. We walk ingestion paths, check for prompt-level poisoning, and verify retrieval behavior under adversarial queries.

Agentic pipeline abuse. Plan-and-execute loops, long-horizon autonomous workflows, multi-agent orchestration — these systems create failure modes that single-shot testing never sees. The fourth class in the catalog is everything that only goes wrong after the third or fourth step of an autonomous run, when the agent is far enough from the original user intent that a nudge in the middle can carry it somewhere no one anticipated.

Reproducible findings, not black boxes

The worst thing we could do in this space is ship another scanner that emits paragraphs of severity-weighted prose and calls it a day. The security industry already has enough of those, and LLM findings are already hard enough to trust without adding a second layer of opacity on top of them.

Every VAAST finding exports as a minimal, self-contained script. If the catalog says your agent is vulnerable to indirect prompt injection through the retrieval path, the output is a script that sets up the retrieval path, plants the payload, runs the query, and asserts the failure. Your engineering team runs it in CI. They fix the bug. They re-run it. They ship.

We chose this shape deliberately. Research-led security only works if the research is legible. If a finding cannot be reproduced on demand, it cannot be challenged, regressed, or ultimately fixed. Every detection VAAST ships with is paired with a public writeup in the Xtrinel research log, and the repro script is the executable version of that writeup.

Authorized testing, always

VAAST will refuse to run against a target that has not been explicitly registered with written authorization. This is not a marketing claim; it is a check enforced at the platform level, and it is the single most important design decision we have made. The offensive-security community has spent decades fighting to establish authorized testing as a professional norm. We do not intend to be the product that erodes that norm for convenience.

If you are reading this and you are responsible for an LLM-integrated application, the most useful next step is probably to map your own attack surface before someone less friendly does it for you. VAAST is live in public beta, the vulnerability catalog is open to read, and the free tier is enough to do a meaningful first pass on a real target. We would rather you find your own bugs than learn about them from a disclosure email.