Security

Log safety: what data leaves your environment when you use AI analysis tools

Nov 14, 2024 · 7 min read · By Security

A transparent walkthrough of ErrorLens's data handling — what we process, what we store, and what never leaves your environment.

The question you should ask any AI log analysis tool

Before using any tool that processes your production logs, you should know exactly what data leaves your environment, what gets stored, where it's stored, and who can see it. This is especially true for AI tools, which process your data through external APIs and may use it for model training.

We want to answer this question directly and completely for ErrorLens. No marketing language. No vague "we take security seriously" statements. Specific answers about what happens to your data.

What you submit and what we see

When you submit an error log or stack trace to ErrorLens, the content travels from your browser to the Anthropic API via an HTTPS-encrypted connection. ErrorLens calls the Anthropic Claude API to perform the analysis.

This means the log content you submit is processed by Anthropic's infrastructure. You should review Anthropic's privacy policy and API terms before submitting logs. In particular, be aware that Anthropic's commercial API does not use your data to train their models by default (as of the date of this post), but Anthropic's terms govern this, not ErrorLens's.

What ErrorLens stores — and what it does not

What we store after an analysis:

What we do not store:

This is a deliberate design choice. We call it privacy by default. The log content itself is the most sensitive part of a production log — it may contain user IDs, email addresses, partial PII, internal hostnames, service names, and other details that reveal your system architecture. We chose not to store it so that even in the event of a breach of the ErrorLens database, none of that content is exposed.

The analysis results — error types, root causes, code suggestions — are much less sensitive. They describe general patterns in your code, not the specific data flowing through your system.

The training data question

This is the question we get most often from security-conscious teams: does ErrorLens use my logs to train AI models?

ErrorLens does not train its own AI models. We use the Anthropic Claude API, which provides the AI analysis capability. Whether Anthropic uses API traffic for their own model training is governed by Anthropic's terms — you should review those directly.

We do offer a "Do Not Train" setting in Settings → Privacy. When enabled, this setting adds a do-not-train signal to the API request. Anthropic honors this signal based on their API terms. We cannot guarantee what Anthropic does with the signal, but we pass it through every request when the setting is active.

If data residency and training exclusion are hard requirements for your organization, the appropriate solution is to use the ErrorLens API behind your own backend proxy that routes through Anthropic's zero-data-retention API endpoint (available on Anthropic's enterprise tier). The ErrorLens hosting guide covers this setup in Section 2.5.

What lives in the browser

ErrorLens is a browser-based application. The following data lives in your browser's local storage and is never sent to our servers:

If you use the ErrorLens web app at errorlens.ai, your API key is held only in your browser's local storage for that session. Clearing your browser's site data removes it. ErrorLens servers never see your Anthropic API key.

The log content you should and should not submit

Even with the privacy protections above, you should be thoughtful about what you submit to any AI analysis tool. Our recommendation:

Safe to submit: Stack traces, error messages, error counts, system component names, service names, database error codes, HTTP status codes, performance metrics, generic application log lines.

Scrub before submitting: Log lines containing email addresses, user IDs that map to real people, full names, payment card numbers, authentication tokens, private IP addresses, or internal hostnames that reveal sensitive network topology.

The good news: for root cause analysis, you almost never need the sensitive parts. The error type, stack trace, and error message are almost always sufficient to identify the root cause. You can safely redact or omit log lines that contain user-level data without losing diagnostic value.

ErrorLens's disclaimer on the New Analysis page reminds you of this every time: do not include confidential data, secrets, API keys, passwords, or personal information in your error logs before submitting them.

Audit logs and access controls

On Pro and above plans, every analysis run is logged in your organization's audit trail: who ran it, when, and what filename was submitted. Admin users can see all analyses run by all members of the organization. On the Business and Enterprise plans, admins can also set access controls to restrict which team members can view analysis history.

On the Enterprise plan, data residency options allow you to specify that your analysis metadata is stored in a specific geographic region. Contact sales for details on available regions.

Our security posture

ErrorLens is currently undergoing SOC 2 Type II certification. In the meantime, our security practices include:

We publish our current security status and any incidents at errorlens.ai/security. If you find a vulnerability, please report it to security@errorlens.ai before public disclosure — we aim to respond within 24 hours.

More articles