Assessing Vector and Embedding Weaknesses

Authorized use only: These tests interact with vector stores and embedding models in RAG systems you own or are authorized to assess. Embedding inversion and cross-tenant probing against systems you do not control may expose third-party data and is prohibited without authorization.

Overview

Retrieval-Augmented Generation (RAG) systems convert documents into embedding vectors stored in a vector database (Pinecone, Qdrant, Weaviate, Chroma, pgvector, FAISS) and retrieve the nearest vectors to ground LLM responses. OWASP LLM08:2025 Vector and Embedding Weaknesses covers the security risks unique to this layer:

• Embedding inversion — embeddings are not one-way. A trained inversion model (or a black-box reconstruction attack) can recover substantial portions of the original text from its vector, leaking source documents (maps to MITRE ATLAS AML.T0024.001 Invert ML Model).
• Membership inference — querying whether a specific record contributed to the corpus (AML.T0024.000).
• Cross-tenant / multi-tenant leakage — when one namespace/collection is shared or filter isolation is missing, a tenant retrieves another tenant's chunks.
• Knowledge-base poisoning — an attacker who can write to the corpus inserts crafted chunks that dominate retrieval (high cosine similarity to expected queries) and carry indirect prompt-injection payloads.
• Retrieval manipulation — adversarial documents tuned to be retrieved for many unrelated queries ("retrieval hijacking").

The parent technique is AML.T0024 — Exfiltration via ML Inference API: an attacker uses legitimate inference/query access to exfiltrate data (source text via inversion, membership, or model extraction). This skill provides a repeatable assessment of all five weakness classes.

When to Use

• During a security assessment of any RAG / vector-search application (OWASP LLM08 coverage).
• When a vector store is multi-tenant and you must prove namespace/metadata isolation.
• When the corpus accepts user-supplied or third-party documents (poisoning surface).
• When the embedding endpoint is externally reachable (inversion/membership surface).
• When validating retrieval-filtering controls before go-live.

Prerequisites

• Authorization and scope covering the target embedding endpoint and vector store.
• Python 3.10+.
• Read (and, for poisoning tests, write) access to a test collection — never the production corpus.

# Vector DB clients + embeddings + similarity tooling
python -m pip install numpy scikit-learn sentence-transformers
python -m pip install qdrant-client chromadb pinecone-client weaviate-client
# (optional) text-embedding inversion research baseline
python -m pip install vec2text

Objectives

• Measure embedding-inversion exposure on the target embedding model.
• Run a membership-inference probe against the corpus.
• Test multi-tenant isolation (namespace, metadata filter, RBAC) for cross-tenant leakage.
• Inject benign poisoned chunks into a test collection and measure retrieval dominance.
• Detect indirect prompt-injection content surviving in retrieved chunks.
• Recommend controls: tenant-scoped filters, content validation, embedding-access limits.

MITRE ATT&CK Mapping

Workflow

Step 1: Inventory the RAG pipeline

Document the embedding model + dimensions, the vector store and its tenancy model, the chunking strategy, retrieval top_k and similarity metric (cosine/dot/L2), and any metadata filters applied at query time.

# Example: inspect a Qdrant collection
from qdrant_client import QdrantClient
client = QdrantClient(url="http://localhost:6333")
info = client.get_collection("docs")

…