Varnish Logo

Log Exports

Archive your access and WAF logs to your own S3 or S3-compatible bucket. Learn how log exports work, the log format, the file layout in the bucket, and how to set up retention.

Log exports deliver the access logs and WAF logs of your services to an S3 bucket that you own. Any S3-compatible object storage works, not only AWS S3.

Setting up log exports takes two steps:

  1. Connect a bucket to your Organisation. A connected bucket can be used by every service in the Organisation, but nothing is stored in it until a service opts in.
  2. Choose a bucket per service, separately for access logs and WAF logs. Each service can send each log type to one bucket, and one bucket can receive logs from many services.

Users with the Admin or Developer role can change log export settings. Finance and Viewer users can only view them.

How it works

Every edge node writes a log event for each request it serves. The events are collected centrally and then uploaded to your bucket in batches.

Logs are buffered on disk at every step, so short outages of your bucket do not lose data. Keep the following in mind:

  • Exactly-once delivery is not guaranteed. In rare cases, such as retries after a failed upload, the same event can appear more than once. A duplicate is an exact copy of the original line, so remove identical lines if this matters to you. Do not deduplicate on vxid or request_id alone: each edge node numbers its requests separately, so different requests can share the same ID.
  • If your bucket stays unreachable or the credentials stop working for a long time, the buffers eventually fill up and new events are dropped.
  • When you enable log exports for a service, the first minutes of logs may be missing while the configuration is rolled out to the whole CDN.

Prepare your bucket

Create the bucket and an access key with your storage provider before connecting it. The access key needs:

  • s3:PutObject on the bucket (or on the key prefix you plan to use). This is required.
  • s3:DeleteObject. This is optional and only used to remove the test file written when you connect the bucket (see below).

The CDN does not set any ACL, storage class or server-side encryption on uploaded files, so the defaults configured on your bucket apply.

An example AWS IAM policy that allows writes under the cdn-logs/ prefix:

policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::my-log-bucket/cdn-logs/*"
    }
  ]
}

Connecting a bucket

  1. Log in to the dashboard.
  2. Open the Organisation dropdown in the top-right corner and select "Settings".
  3. Go to the "Log Storage" section.
  4. Click "Connect bucket".
  5. Fill in the form:
    • Name: a name for the connection, e.g. log-archive. Use lowercase letters, digits and dashes, at most 63 characters. The name cannot be changed later.
    • Bucket name: the name of your bucket.
    • Region: the bucket's region, e.g. eu-north-1 or fra1.
    • Endpoint: leave blank for AWS S3. For other S3-compatible storage, enter the provider's endpoint URL, e.g. https://s3.example.com.
    • Key prefix (optional): a folder path inside the bucket, e.g. cdn-logs/, with no leading slash.
    • Access key ID and Secret access key: the credentials of your access key. Use the key's ID, not the name you gave the key.
  6. Optionally click "Test connection" to check the settings.
  7. Click "Connect bucket".

When you save, the CDN uploads a small test file named 00-cdn-logexport-writetest-<random>.txt and then tries to delete it. If the upload fails, the bucket is not connected and the error is shown. If the access key does not allow s3:DeleteObject, the test file stays in the bucket. It is safe to delete.

"Test connection" writes its test file to the root of the bucket and ignores the key prefix. If your access key is limited to a prefix, the test may fail even though saving the connection works.

Stored credentials are never shown again. When editing a connection, leave the access key fields blank to keep the current values.

Removing a bucket connection stops log exports for every service that uses it.

Enabling log exports for a service

  1. Navigate to the service in the dashboard.
  2. Go to "Security" → "Log storage".
  3. In the "Access Logs storage" and/or "WAF storage" card, select a bucket.
  4. Click "Update Storage Location".

To stop exporting, click "Stop storing" in the same card. Files already in your bucket are not touched.

File structure in the bucket

Files are grouped by service and by day:

<key prefix>/<service name>/<YYYY>/<MM>/<DD>/<log type>-<timestamp>-<uuid>.log.gz

For example:

cdn-logs/my-service/2026/11/06/access-2026-11-06T12:05:00.123+00:00-0b7c1e4a-6f2d-4e0b-9a51-3c8d2f7e1a90.log.gz
cdn-logs/my-service/2026/11/06/waf-2026-11-06T12:05:00.456+00:00-5d1f9b2c-8a3e-4c7d-b6e0-9f4a2c1d7e35.log.gz
  • <key prefix> is the optional prefix of the bucket connection. Without a prefix, the path starts with the service name.
  • <service name> is the name of your service.
  • <log type> is access or waf.
  • Each file is gzip-compressed and holds many log events.

Log format

Each file contains newline-delimited JSON (NDJSON): one JSON object per line, one line per log event. Fields without a value are null.

Access logs

Each request served by the CDN produces one access log event.

FieldTypeDescription
log_typestringAlways access.
service_namestringName of the service.
timestampstringTime of the request (RFC 3339, millisecond precision).
vxidintegerVarnish transaction ID of the request. Same value as the X-Varnish response header.
hoststringHost header of the request.
methodstringHTTP method.
urlstringRequest path, without the query string.
protocolstringHTTP protocol version, e.g. HTTP/2.0.
statusintegerFor waf_block, the status code the request was blocked with, e.g. 403. Always 200 for waf_detect.
bytes_bodyintegerSize of the response body in bytes.
bytes_inintegerTotal bytes received from the client.
bytes_outintegerTotal bytes sent to the client.
ttfb_sfloatTime to first byte, in seconds.
handlingstringHow the request was handled, e.g. hit, miss or pass.
remote_ipstringIP address of the connecting peer.
x_forwarded_forstringX-Forwarded-For request header as seen by the CDN: the value sent by the client, if any, followed by the client's IP address.
client_ipstringIP address of the end user.
user_agentstringUser-Agent request header.
refererstringReferer request header.
content_typestringContent-Type response header.
cache_controlstringCache-Control response header.
agestringAge response header.

Example (formatted over several lines for readability, in the file it is a single line):

{
  "log_type": "access",
  "service_name": "my-service",
  "timestamp": "2026-11-06T12:03:41.512Z",
  "vxid": 32770,
  "host": "www.example.com",
  "method": "GET",
  "url": "/images/logo.png",
  "protocol": "HTTP/2.0",
  "status": 200,
  "bytes_body": 15320,
  "bytes_in": 412,
  "bytes_out": 15780,
  "ttfb_s": 0.000213,
  "handling": "hit",
  "remote_ip": "203.0.113.24",
  "x_forwarded_for": "203.0.113.24",
  "client_ip": "203.0.113.24",
  "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0 Safari/537.36",
  "referer": "https://www.example.com/",
  "content_type": "image/png",
  "cache_control": "public, max-age=86400",
  "age": "1520"
}

WAF logs

WAF logs are written for requests that match a WAF rule. The events depend on the WAF mode of the service:

  • Blocking mode: a blocked request produces a single waf_block event.
  • Detect mode: nothing is blocked, and a request produces one waf_detect event for each rule that matched.

Requests that match no rule produce no WAF events.

FieldTypeDescription
log_typestringAlways waf.
service_namestringName of the service.
event_typestringwaf_detect or waf_block.
timestampstringTime of the event.
request_idintegerVarnish transaction ID of the WAF check.
parent_request_idintegerVarnish transaction ID of the client request. Matches vxid in the access log.
hostnamestringHost of the request.
methodstringHTTP method.
pathstringRequest path, including the query string.
http_versionstringHTTP protocol version of the WAF check. This can differ from the protocol the client used, so use protocol in the access log for that.
client_ipstringIP address of the end user.
statusintegerFor waf_block, the status code the request was blocked with, e.g. 403. Always 200 for waf_detect.
wafobjectDetails of the matched rule, see below.

The waf object contains:

FieldDescription
rule_idID of the matched rule.
messageDescription of the rule.
file, lineRule file and line the rule is defined in.
severitySeverity of the rule.
uriRequest path the rule was evaluated against, without the query string.
unique_idUnique ID of the transaction.
refReference to the matched data.
accuracyAccuracy rating of the rule.
tagsTags of the rule, e.g. attack categories.

Log volume

Use these approximate figures to estimate storage needs. Actual sizes depend on your traffic: long URLs, long User-Agent or Referer headers and bot traffic increase them.

Log typeUncompressed per eventCompressed per event
Access log, typical~700 bytes~100–150 bytes
Access log, long headers~1.2 KB~250 bytes
WAF log, typical~700 bytes~70–100 bytes
WAF log, long attack payload~1.6 KB~150–250 bytes

As a rule of thumb:

  • Access logs: roughly 1 KB per request uncompressed and 100–250 bytes compressed, or about 0.1–0.25 GB stored per million requests.
  • WAF logs: in blocking mode, each blocked request produces one event, about 0.1–0.25 GB stored per million blocked requests. In detect mode, a flagged request usually produces 3–5 events, about 0.2–0.5 GB stored per million flagged requests. WAF log volume follows the amount of attack traffic, not your total traffic.

Retention

The CDN only writes to your bucket. It never deletes or expires log files, so the files are kept until you remove them. To keep storage costs under control, set up a lifecycle rule on the bucket that deletes or archives old logs.

For example, with the AWS CLI, to delete logs under cdn-logs/ after 90 days:

terminal
aws s3api put-bucket-lifecycle-configuration \
  --bucket my-log-bucket \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "expire-cdn-logs",
      "Status": "Enabled",
      "Filter": { "Prefix": "cdn-logs/" },
      "Expiration": { "Days": 90 }
    }]
  }'

Most S3-compatible providers support the same lifecycle API, or offer the setting in their console.

Check your own retention requirements before choosing a period. Logs contain IP addresses, which are personal data in many jurisdictions.