Uploading log files to Agent Analytics

File upload is the simplest way to get data into Agent Analytics, best for one-time or historical analysis. Upload your web server's access logs and OtterlyAI extracts every AI agent visit from them, no live connection required.

This article covers the file limits, which log formats are supported, and how to prepare each one.

We do not store personal data

OtterlyAI will NOT store any personal identifiable information (PII) like IP addresses from your logfiles or your tracking events. As soon as the logfiles are uploaded, we are removing the IP addresses from the data before storing the data in our infrastructure. No personal data will be stored in the OtterlyAI infrastructure. The same security measurement is taken if you use the Webhook, Cloudflare or other tracking methods.

Upload limits

• Up to 20 files per upload, maximum of 1 GB in file size
• Accepted file extensions: .log, .txt, .csv, .json, .ndjson, .gz, .zip

You don't need to select a format, OtterlyAI detects it automatically from the file's contents.

Supported formats at a glance

Apache / nginx — Combined Log Format

The standard combined access log format, used by default on most web servers:

192.0.2.10 - - [18/Jun/2026:10:28:02 +0000] 
"GET /pricing?utm_source=chatgpt.com HTTP/1.1" 200 1234 "https://www.google.com/" "Mozilla/5.0 (compatible; GPTBot/1.1; +https://openai.com/gptbot)"

A few things to know:

• The two quoted fields at the end — referer and user-agent — must be present. Plain Common Log Format (without the user-agent) isn't accepted, since the user-agent is what identifies which AI agent made the request.
• A leading virtual host is supported and used as the visited domain (e.g. shop.example.com 192.0.2.10 - - [...]).
• Extra trailing fields (request time, cache status, upstream) are ignored, so extended nginx log_format variants work as long as they start with the combined fields.

Cloudflare Logpush

Newline-delimited JSON from the http_requests dataset — one JSON object per line:

{"ClientIP":"192.0.2.10","ClientRequestHost":"example.com",
"ClientRequestURI":"/pricing?utm_source=chatgpt.com",
"ClientRequestReferer":"https://www.google.com/",
"ClientRequestUserAgent":"Mozilla/5.0 (compatible; ClaudeBot/1.0)","EdgeStartTimestamp":"2026-06-18T10:28:02Z","EdgeResponseStatus":200}

EdgeStartTimestamp and ClientRequestURI are required on each line. ClientRequestHost, ClientRequestUserAgent, ClientRequestReferer, and ClientIP are used when present. Timestamps may be RFC 3339 or Unix epoch (seconds, milli-, micro-, or nanoseconds).

💡 For continuous data, use the Cloudflare Worker integration instead — file upload is meant for historical exports.

Amazon CloudFront

Both shapes of CloudFront standard logging are supported:

• Standard logging v2 — JSON, one request per line.
• Legacy standard logging — tab-separated W3C files starting with #Version: / #Fields: header lines. Keep those header lines in the file: without #Fields:, the columns can't be resolved.

💡 Include the x-host-header field in your log configuration if you can — it lets OtterlyAI attribute visits to your website's domain rather than the *.cloudfront.net distribution domain. Without it, visits are attributed to the domain you upload under.

CSV

If your logs come from somewhere else, export them as CSV with a header row. Column order doesn't matter, and unknown columns are ignored:

Example:

timestamp,host,url,user_agent,referer,ip 2026-06-18T10:28:02Z,example.com,/blog/post?utm_source=chatgpt.com,GPTBot/2.0,https://www.google.com/,192.0.2.10

Values containing commas must be wrapped in double quotes (standard CSV quoting).

Compressed files

Files can be uploaded as plain text, gzip (.gz), or zip (.zip). Inside a zip, each file is processed independently — you can mix formats, and each CSV needs its own header row.

Limits per archive: encrypted/password-protected zips aren't supported, at most 10,000 files, and at most 8 GB total uncompressed.

What isn't supported

• Common Log Format without the user-agent field (see the Apache/nginx section above).
• JSON arrays or pretty-printed JSON — JSON must be newline-delimited (one object per line), using Cloudflare or CloudFront field names. For your own JSON events, use the Webhook instead, or convert to CSV.
• W3C-style logs from other servers (e.g. Microsoft IIS) — only CloudFront's W3C files are parsed.

All timestamps are converted to UTC during processing. Lines that can't be parsed are skipped, and a file whose format isn't recognized at all is dropped without failing the rest of the upload.