# Web Data API URL: https://www.deepinfo.com/data-api/apis/webdata-lookup/ ## A Single Response Carries Everything a Web Page Reveals. The Web Data API takes a URL, loads the page in a real browser and returns its full profile: technologies, metadata, source and visible text with their hashes, links, trackers, favicons, `robots.txt`, headers, cookies and redirects. [Talk to us about access](https://www.deepinfo.com/contact/?topic=data-api) [Web Data reference](https://docs.deepinfo.com/reference/lookup/web-data/) ### The Whole Page Comes Back From One URL. GET `https://api.deepinfo.com/v1/lookup/webdata` [Reference](https://docs.deepinfo.com/reference/lookup/web-data/) | Parameter | Type | | Description | | --- | --- | --- | --- | | `url` | string | Required | The page to analyze, as a full URL with its scheme. | | `screenshot` | boolean | Optional | Also capture a JPEG of the rendered page, returned as a `screenshot` object. Default `false`. | | `wait_until` | string | Optional | Load events to wait for before the page is parsed, comma-separated, such as `load` or `networkidle2`. | | `max_file_size` | integer | Optional | Largest page source to return, in bytes. Default `8388608`. | Request curl: ```bash curl "https://api.deepinfo.com/v1/lookup/webdata?url=https://www.deepinfo.com&screenshot=true" \ -H "apikey: YOUR_API_KEY" ``` Python: ```python import requests r = requests.get( "https://api.deepinfo.com/v1/lookup/webdata", params={"url": "https://www.deepinfo.com", "screenshot": "true"}, headers={"apikey": "YOUR_API_KEY"}, ) page = r.json() print(page["html"]["meta"]["title"]) print([f["hash"] for f in page["favicon"]]) ``` JavaScript: ```javascript const url = encodeURIComponent('https://www.deepinfo.com') const r = await fetch( `https://api.deepinfo.com/v1/lookup/webdata?url=${url}&screenshot=true`, { headers: { apikey: 'YOUR_API_KEY' } } ) const { html, http } = await r.json() console.log(html.meta.title, http.redirection_history) ``` Response 200 OK · application/json ```json { "url": "https://www.deepinfo.com", "technology": { "stacks": [ { "slug": "cloudflare", "name": "Cloudflare", "categories": ["CDN"], "cpe": "cpe:/a:deepvendor:cloudflare", "…": "…" } ] }, "html": { "meta": { "title": "Deepinfo | Continuous Threat Exposure Management Platform", "language": "en", "encoding": "utf-8", "canonical_url": "https://www.deepinfo.com/", "noindex_status": false, "og": ["…"], "json_ld": ["…"], "…": "name, description, keywords, language_alternatives" }, "internal_links_fqdns": ["www.deepinfo.com", "docs.deepinfo.com", "…"], "external_links_domains": ["linkedin.com"], "script_links": ["…"], "iframe_links": [], "trackers": [], "emails": [], "source_code_hash": "…", "content_hash": "…", "content_keywords": ["data", "platform", "management", "deepinfo", "…"], "…": "source_code, content, external_links, favicon_links, inspect_disabled" }, "favicon": [ { "url": "…/favicon.svg", "hash": "…" } ], "robots_txt": { "content": "…", "hash": "…", "disallowed_links": ["…"] }, "http": { "redirection_history": [ { "url": "https://www.deepinfo.com", "status_code": 200 } ], "headers": [ { "name": "content-type", "value": "text/html; charset=utf-8" }, { "name": "server", "value": "cloudflare" }, "…" ], "cookies": [ { "name": "…", "value": "", "secure": false, "http_only": false, "…": "…" } ] }, "screenshot": { "screenshot_url": "https://…/.jpeg", "connection_status": "success", "…": "url, redirected_url, check_date" }, "connection_status": "success", "version": 1, "check_date": "2026-09-23T14:45:32.606Z" } ``` ### The Blocks in the Response. | Block | Contents | | --- | --- | | `technology.stacks[]` | Detected technologies with the same fields as the [Technology API](https://www.deepinfo.com/data-api/apis/technology-lookup/), CPE names included. | | `html.meta` | Title, description, keywords, language, encoding, canonical URL, noindex status, Open Graph tags and JSON-LD blocks. | | `html.source_code`, `html.content` | The HTML and the visible text, each with a SHA-256 hash, plus the page’s most frequent words. | | `html` links | Host names on the site’s own domain, external links with their host names and registered domains, and the scripts, iframes and icons the page loads. | | `html.trackers[]` | Tracking IDs found in the page, such as Google Analytics, AdSense and Tag Manager IDs. | | `html.emails` | E-mail addresses that appear in the page. | | `favicon[]` | Each icon’s URL and hash. | | `robots_txt` | The file, its hash and the paths it disallows. | | `http` | Response headers, cookies with their flags, and every redirect on the way with its status code. | | `screenshot` | With `screenshot=true`: a signed link to a JPEG that expires after seven days. | ### Hashes and IDs That Link Sites Together. A favicon hash, an analytics ID and a content hash are fingerprints. Two sites that share them were often built by one team, or one was copied from the other. Web Data returns all three in one call, so comparing a lookalike page with yours means comparing fields instead of reading the pages side by side. The fingerprints ```json { "favicon": [ { "url": "…", "hash": "…" } ], "html": { "trackers": [ { "name": "…", "values": ["…"] } ], "source_code_hash": "…", "content_hash": "…" }, "robots_txt": { "hash": "…" } } ``` ### Questions One Page Profile Answers. #### [Impersonation Review](https://www.deepinfo.com/solutions/use-cases/brand-impersonation-protection/) Compare a lookalike’s title, favicon hash and content hash with your page’s before you escalate it. #### [Phishing Triage](https://www.deepinfo.com/solutions/use-cases/phishing-detection-and-prevention/) See what a suspicious domain serves, where its links and scripts point and where its redirects end. #### [Threat Hunting](https://www.deepinfo.com/solutions/use-cases/threat-hunting/) Group sites by the analytics IDs, favicon hashes and e-mail addresses they have in common. ### Large Pages and Busy Browsers. - **Loaded live**: The page is rendered in a real browser at request time, so a call takes longer than a record lookup. `503` means the browser is busy: retry after the seconds in the `Retry-After` header. - **Large pages**: When a page is larger than `max_file_size`, `source_code` holds a placeholder message instead of the HTML. Raise the limit, up to 32 MiB, for very large pages. - **Targets not allowed**: A `400` that carries only `details`, without `parameters`, means the target cannot be analyzed. - **Rate limits**: Plan-dependent. The default is one request per second per key and endpoint. See [rate limits](https://docs.deepinfo.com/getting-started/rate-limits/). ### Narrower Calls on One Page. - [Technology](https://www.deepinfo.com/data-api/apis/technology-lookup/) (Lookup): Only the technologies, with categories, version and CPE. - [Screenshot](https://www.deepinfo.com/data-api/apis/screenshot-lookup/) (Lookup): Only the image, with full control over the browser. - [SSL](https://www.deepinfo.com/data-api/apis/ssl-lookup/) (Lookup): The certificate the host serves, checked live. ### See This Data at Work in the Deepinfo Platform External Attack Surface Management runs these lookups continuously over your assets and raises what changes as issues, and Deep Search & Insights opens the whole index to your analysts. - [External Attack Surface Management](https://www.deepinfo.com/platform/easm/) (EASM) - [Deep Search & Insights](https://www.deepinfo.com/platform/dsi/) (DSI) - [How the Platform Works](https://www.deepinfo.com/platform/how-it-works/) ### Questions About Web Data API **Can we find other sites that share a page or an analytics ID?** Web Data profiles one page per call, so compare the IDs and hashes across the pages you profile. Across the index, [Domain Search](https://www.deepinfo.com/data-api/apis/domain-search/) matches hosts on `webdata.html.source_code_hash`, which identical pages share. **Which e-mail addresses does the response list?** `html.emails` holds every address found in the page, and `html.emails_internal` the ones on the site’s own domain. **Can it tell when a page tries to block inspection?** Yes. `html.inspect_disabled` reports whether the page tries to block inspection of its content. **Which load events does Web Data wait for?** By default, `networkidle2`, `load` and `domcontentloaded`, and `wait_until` changes the list. With `screenshot=true` the list is ignored: the page is always parsed after the `load` event. **How is the Web Data API priced?** As part of an API services plan, by volume and frequency. Each call is a lookup and counts against your plan’s quota at one unit per request. See [Data & API pricing](https://www.deepinfo.com/pricing/#data-pricing). **Can analysts profile a page without writing code?** Yes. [Instant Lookups](https://www.deepinfo.com/platform/dsi/instant-lookups/) includes Web Data in the Deepinfo Platform, next to Screenshot and Technology. ### Profile a Page You Know Well. Run Web Data against your own site first, so you know what a clean result looks like before you compare a lookalike with it. We set up access on a short call. [Talk to us about access](https://www.deepinfo.com/contact/?topic=data-api) [All API services](https://www.deepinfo.com/data-api/apis/)