# Agent and Automation Integration Guide Canonical URL: https://api.waterdata.usgs.gov/docs/agents/ > Machine-readable entry points for automated consumers of this site: the llms.txt index, Markdown page copies, the apis.json catalog, and API keys. # Agent integration guide This site documents the USGS water data APIs, and it publishes a small set of machine-readable surfaces alongside that documentation so an automated consumer does not have to reverse-engineer the HTML. Every surface described here is produced by the static build, so a plain HTTP GET is enough to read all of it. Nothing on this page requires you to execute JavaScript. Two things are worth knowing before you start. Documentation lives on this site; data lives on the API endpoints this site documents. And bulk retrieval belongs on those endpoints with an API key, not on these documentation pages. ## Start with the llms.txt index The orientation index is published at [`/llms.txt`](/llms.txt). It is a small UTF-8 Markdown document that names the site, summarizes it in one blockquote, and then lists every resource worth fetching under five headings, in this order: `Start here`, `Documentation`, `API endpoints`, `Machine-readable indexes`, and `Optional`. Each entry is a single line holding a Markdown link with an absolute `https` URL, then a colon, then a short description. Where a page has a Markdown copy, the entry links to that copy rather than to the HTML. Fetching this one file tells you what else is worth fetching, which makes it the cheapest first request against the site. ## Markdown copies of documentation pages Most documentation pages are published twice: once as HTML for people, and once as Markdown for automated readers. The naming rule is to append `.md` to the rendered page URL. Because rendered page URLs end in a slash, the Markdown copy of a page at `/docs/agents/` is published at [`/docs/agents/index.html.md`](/docs/agents/index.html.md), and the general shape is `index.html.md`. Each copy opens with a three-line preamble followed by one blank line: the page title as a level-one Markdown heading, then a line reading `Canonical URL: ` followed by the absolute canonical URL of the HTML page, then the page description as a blockquote. The authored body of the page follows, with the YAML front matter and the site's curly-brace attribute annotations removed and everything else left as authored. There is no modification-date line and no other metadata. Every HTML page that has a Markdown copy advertises it in the document head as ``, so you can discover the copy from the page itself. A few pages have no Markdown copy, because their bodies are embedded scripts rather than prose; the index links to those at their canonical HTML URLs instead. Every page also carries `` pointing at `/llms.txt`, so any single page you land on tells you where the index is. ## The API catalog The machine-readable catalog of the APIs documented here is published at [`/apis.json`](/apis.json). It is a JSON object rather than a bare array, carrying a feed `version`, the site `name` and `description`, an entry `count`, and a `services` array in ascending publication order. Each service entry carries an `id` of lowercase letters, digits, and hyphens, a `name`, a `description`, a `documentationUrl`, and a `links` array whose entries each carry a `label` and an `href`. Every `href` is an absolute `https` URL, and `documentationUrl` repeats the one link a reader should follow first. The feed `version` is an integer, and it increments only when a property is renamed or removed, so you can detect a contract change without diffing the whole document. ## Structured data on every page Every rendered page carries `schema.org` structured data as one or more `application/ld+json` script blocks, each holding a single top-level object rather than a graph. - The primary block is a `TechArticle` on documentation pages, meaning any page under `/docs/`, and a `WebPage` everywhere else. It declares `name`, `headline`, `description`, `url`, `inLanguage`, and a `GovernmentOrganization` publisher. - Documentation pages carry a second `BreadcrumbList` block whose `itemListElement` entries run from the site root to the current page, numbered consecutively from 1, with every `item` an absolute URL on this site. - The landing page carries an `ItemList` block of `WebAPI` entries built from the same catalog that produces `/apis.json`, so the two agree by construction. The primary block's `url` is always the page's canonical URL, which is also its `og:url` value and the `href` of its `link rel="canonical"`. A page declares `dateModified` only where the source file records a modification date, so the absence of that property means the date is unknown rather than recent. ## The sitemap The sitemap is published at [`/sitemap.xml`](/sitemap.xml) and is also announced by the `Sitemap:` directive in [`/robots.txt`](/robots.txt). It lists the rendered HTML pages of the site and nothing else: the Markdown copies, `/llms.txt`, and `/apis.json` are deliberately absent from it, because they are alternate representations and indexes rather than separate pages. Use `/llms.txt` to enumerate the machine-readable surfaces and the sitemap to enumerate the pages. ## Getting and sending an API key API keys are managed by [api.data.gov](https://api.data.gov/), and a key raises your rate limits well above the anonymous allowance. Request one from the [signup page](/signup/); the key is emailed to the address you provide. Keys are free and no approval step is involved. There are two ways to send a key with a request. Pass it as the `api_key` query parameter: ``` https://api.waterdata.usgs.gov/ogcapi/v0/collections/daily/items?limit=1&api_key=YOUR_API_KEY ``` Or send it in the `X-Api-Key` request header, which keeps it out of logged URLs: ``` curl -H 'X-Api-Key: YOUR_API_KEY' 'https://api.waterdata.usgs.gov/ogcapi/v0/collections/daily/items?limit=1' ``` `YOUR_API_KEY` is a placeholder in both examples. Substitute the key you were sent; this page publishes no real credential. For rate-limit details and error responses, see [Using API keys with USGS water data APIs](/docs/ogcapi/keys/). ## Paths this site asks automated consumers not to retrieve [`/robots.txt`](/robots.txt) carries a single `User-agent: *` stanza. Four of its directives cover the service endpoints rather than documentation, and a compliant consumer does not retrieve the paths they name: - `/ogcapi/*/collections/*/items*` - `/samples-data/*` - `/squid/*` - `/statistics/*` These are the unbounded query surfaces, where a crawler following links can generate a very large number of expensive requests. The documentation for those same services is explicitly permitted by the companion `Allow: /samples-data/docs*`, `Allow: /squid/docs*`, and `Allow: /statistics/docs*` directives, so you can read how a service works without crawling its data. None of this makes the data closed. Query those endpoints deliberately, with an API key and the filters your task needs, as described in the [OGC API documentation](/docs/ogcapi/). What the policy asks you to avoid is reaching them by crawling. ## The two applications that require JavaScript Two pages on this site are client-rendered applications rather than documents. The [download tool](/download/) builds and runs queries against the OGC API collections, and the [STAC Explorer](/stac-explorer/) browses STAC collections and items. Both fetch their content in the browser, so a consumer that does not execute JavaScript receives a page shell with no data in it. Retrieve data from the documented API endpoints instead. The download tool is a query builder over the endpoints described in the [OGC API documentation](/docs/ogcapi/), and the STAC Explorer is a browser view over the service described in the [STAC API documentation](/docs/stac/). Anything either application shows you can be requested directly. No-JavaScript fallbacks for these two applications are tracked in GitLab issues #18-#21. They are not part of this release, so do not expect a server-rendered version of either page today. Both applications do keep their state in the URL, though, and both URL grammars are documented below. That makes a configured view shareable, and it lets you read a URL someone sends you without running the application that produced it. ### The download tool's query parameters The download tool holds its whole form selection in the query string of `/download/`. It owns exactly seven parameters, reads only those on load, and rewrites only those as you change the form; any other parameter in the URL is preserved in place and ignored. - `collections` is a comma-separated list of the collections to download from, using the same identifiers the OGC API uses, such as `continuous`, `daily`, `field-measurements`, `channel-measurements`, and `peaks`. At least one is needed before the form will run a query. - `monitoring_location_id` is a comma-separated list of monitoring location identifiers, each an agency prefix and a location number joined by a hyphen, as in `USGS-01435000`. The form accepts up to 100 of them; past that it asks you to query the endpoints directly, which is also the better choice for a large batch. - `parameter_code` is a comma-separated list of five-digit USGS parameter codes, as in `00060` for discharge. Leave it empty to request every parameter available at the selected locations. - `temporal_argument` selects which of the two temporal forms applies, and is either `interval` or `duration`. Any other value is discarded and the form's own default stands. - `start_date` and `end_date` express an interval and are read when the temporal argument is `interval`. Both are calendar dates in `YYYY-MM-DD` form; a value in any other form, including the `MM/DD/YYYY` form the form fields display, is discarded in favor of the default. Omitting `end_date` leaves the interval open at the recent end. - `duration` expresses a period counted back from today and is read when the temporal argument is `duration`. Its form is `P`, then a whole number, then a unit of `D`, `W`, `M`, or `Y`: `P30D` is the last 30 days and `P1Y` the last year. A value that does not match that shape is discarded. A configured interval query reads like this: ``` /download/?collections=daily&monitoring_location_id=USGS-01435000¶meter_code=00060&temporal_argument=interval&start_date=2024-01-01&end_date=2024-01-31 ``` And the same selection over the most recent 30 days: ``` /download/?collections=continuous&monitoring_location_id=USGS-01435000¶meter_code=00060&temporal_argument=duration&duration=P30D ``` Both URLs configure the form rather than returning data. The tool turns a selection into an OGC API request in the browser, so if you want the data itself, compose the equivalent endpoint request described in the [OGC API documentation](/docs/ogcapi/) instead. ### The STAC Explorer's hash routes The STAC Explorer carries its location in the fragment of `/stac-explorer/`, and it has five route shapes. They are written here with the parameter names the application's router uses: - `#/` is the entry view. It is also the fallback: a hash matching no route redirects here. - `#/collections` is the list of collections the STAC API publishes. - `#/collections/:id` is one collection, where `:id` is a collection identifier such as `ratings`. - `#/collections/:id/items` is that collection's item list. - `#/collections/:collectionId/items/:itemId` is a single item, where `:itemId` is an item identifier such as `USGS-16238500.exsa.rdb`. The item list is paginated 20 items at a time, and the page is carried as a query parameter appended to the hash path, so `#/collections/ratings/items?page=3` is the third page of that collection's items. Page 1 is written without the parameter, and a `page=` value that is not a whole number of 1 or greater falls back to page 1. Every view above is a rendering of a STAC API response. The [STAC API documentation](/docs/stac/) describes the equivalent requests, which return the same collections and items without a browser. ## Content types and cache lifetime The agent-facing artifacts are served as text with a cache lifetime of 86,400 seconds, which is 24 hours: - Markdown copies, published at `index.html.md`, are served as `text/markdown`. - The orientation index at `/llms.txt` is served as a `text/` type. - The API catalog at `/apis.json` is served as `application/json`. A day-long cache lifetime is appropriate because these are documentation surfaces rather than real-time data sources. Treat them as a description of what the APIs offer, refreshed daily at most. Current observations, current monitoring location metadata, and everything else that changes come from the documented API endpoints, which carry their own caching behavior.