Get bot traffic pages
The pages crawlers request most, the discovery files they read (robots.txt, llms.txt, sitemaps, .md pages) and the pages that returned 404. Pro and Team plans.
GET https://usedatasense.com/api/v1/bot-traffic/pages
Permission: analytics:read. MCP tool: datasense_bot_traffic_pages.
Request
| Parameter | In | Type | Required | Description |
|---|---|---|---|---|
websiteId |
query | string | With an account token | The website to query. Required with an account token or OAuth connection that can reach several websites; omit with a ds_sk_ website key. |
startAt |
query | string | No | First day of the range, YYYY-MM-DD. Must be sent with endAt. Omit both for all time. |
endAt |
query | string | No | Last day of the range (inclusive), YYYY-MM-DD. Must be sent with startAt. |
timezone |
query | string | No | IANA timezone for the days and buckets, such as America/New_York. Defaults to the website's timezone. |
category |
query | string | No | One of answer, index, training, other. |
provider |
query | string | No | |
agent |
query | string | No |
sh
curl "https://usedatasense.com/api/v1/bot-traffic/pages?startAt=2026-10-01&endAt=2026-10-26" \
-H "Authorization: Bearer ds_sk_xxx"With an account token, add websiteId:
sh
curl "https://usedatasense.com/api/v1/bot-traffic/pages?websiteId=YOUR_SITE_ID" \
-H "Authorization: Bearer ds_at_xxx"Response
json
{
"status": "success",
"data": { … }
}Errors use the standard error format.