Request demo

Web Data API

A Single Response Carries Everything a Web Page Reveals.

The Web Data API takes a URL, loads the page in a real browser and returns its full profile: technologies, metadata, source and visible text with their hashes, links, trackers, favicons, robots.txt, headers, cookies and redirects.

Talk to us about access Web Data reference

Request

GET /lookup/webdata?url=https://www.deepinfo.com

Response · 200 OK

  1. technology.stacks

    Cloudflare · CDN

  2. html.meta.title

    Deepinfo | Continuous Threat Exposure Management Platform

  3. links

    internal www.deepinfo.com, docs.deepinfo.com · external linkedin.com

  4. favicon

    …/favicon.svg · hash …

  5. robots_txt

    content · hash · disallowed_links

  6. http

    status 200 · server cloudflare · text/html; charset=utf-8

    • source_code_hash …
    • content_hash …

  8. screenshot

    optional · signed link, expires after 7 days

The Whole Page Comes Back From One URL.

GET https://api.deepinfo.com/v1/lookup/webdata Reference
ParameterTypeDescription
urlstringRequiredThe page to analyze, as a full URL with its scheme.
screenshotbooleanOptionalAlso capture a JPEG of the rendered page, returned as a screenshot object. Default false.
wait_untilstringOptionalLoad events to wait for before the page is parsed, comma-separated, such as load or networkidle2.
max_file_sizeintegerOptionalLargest page source to return, in bytes. Default 8388608.

Request

curl "https://api.deepinfo.com/v1/lookup/webdata?url=https://www.deepinfo.com&screenshot=true" \
  -H "apikey: YOUR_API_KEY"

Response

200 OK · application/json 
{
  "url": "https://www.deepinfo.com",
  "technology": {
    "stacks": [
      { "slug": "cloudflare", "name": "Cloudflare", "categories": ["CDN"], "cpe": "cpe:/a:deepvendor:cloudflare", "…": "…" }
    ]
  },
  "html": {
    "meta": {
      "title": "Deepinfo | Continuous Threat Exposure Management Platform",
      "language": "en",
      "encoding": "utf-8",
      "canonical_url": "https://www.deepinfo.com/",
      "noindex_status": false,
      "og": ["…"],
      "json_ld": ["…"],
      "…": "name, description, keywords, language_alternatives"
    },
    "internal_links_fqdns": ["www.deepinfo.com", "docs.deepinfo.com", "…"],
    "external_links_domains": ["linkedin.com"],
    "script_links": ["…"],
    "iframe_links": [],
    "trackers": [],
    "emails": [],
    "source_code_hash": "…",
    "content_hash": "…",
    "content_keywords": ["data", "platform", "management", "deepinfo", "…"],
    "…": "source_code, content, external_links, favicon_links, inspect_disabled"
  },
  "favicon": [ { "url": "…/favicon.svg", "hash": "…" } ],
  "robots_txt": { "content": "…", "hash": "…", "disallowed_links": ["…"] },
  "http": {
    "redirection_history": [ { "url": "https://www.deepinfo.com", "status_code": 200 } ],
    "headers": [
      { "name": "content-type", "value": "text/html; charset=utf-8" },
      { "name": "server", "value": "cloudflare" },
      "…"
    ],
    "cookies": [ { "name": "…", "value": "<cookie value>", "secure": false, "http_only": false, "…": "…" } ]
  },
  "screenshot": {
    "screenshot_url": "https://…/<signed link, expires after 7 days>.jpeg",
    "connection_status": "success",
    "…": "url, redirected_url, check_date"
  },
  "connection_status": "success",
  "version": 1,
  "check_date": "2026-09-23T14:45:32.606Z"
}

The Blocks in the Response.

BlockContents
technology.stacks[]Detected technologies with the same fields as the Technology API, CPE names included.
html.metaTitle, description, keywords, language, encoding, canonical URL, noindex status, Open Graph tags and JSON-LD blocks.
html.source_code, html.contentThe HTML and the visible text, each with a SHA-256 hash, plus the page’s most frequent words.
html linksHost names on the site’s own domain, external links with their host names and registered domains, and the scripts, iframes and icons the page loads.
html.trackers[]Tracking IDs found in the page, such as Google Analytics, AdSense and Tag Manager IDs.
html.emailsE-mail addresses that appear in the page.
favicon[]Each icon’s URL and hash.
robots_txtThe file, its hash and the paths it disallows.
httpResponse headers, cookies with their flags, and every redirect on the way with its status code.
screenshotWith screenshot=true: a signed link to a JPEG that expires after seven days.

Hashes and IDs That Link Sites Together.

A favicon hash, an analytics ID and a content hash are fingerprints. Two sites that share them were often built by one team, or one was copied from the other.

Web Data returns all three in one call, so comparing a lookalike page with yours means comparing fields instead of reading the pages side by side.

The fingerprints 
{
  "favicon": [ { "url": "…", "hash": "…" } ],
  "html": {
    "trackers": [ { "name": "…", "values": ["…"] } ],
    "source_code_hash": "…",
    "content_hash": "…"
  },
  "robots_txt": { "hash": "…" }
}

Questions One Page Profile Answers.

Impersonation Review

Compare a lookalike’s title, favicon hash and content hash with your page’s before you escalate it.

Use case

Phishing Triage

See what a suspicious domain serves, where its links and scripts point and where its redirects end.

Use case

Threat Hunting

Group sites by the analytics IDs, favicon hashes and e-mail addresses they have in common.

Use case

Large Pages and Busy Browsers.

Loaded liveThe page is rendered in a real browser at request time, so a call takes longer than a record lookup. 503 means the browser is busy: retry after the seconds in the Retry-After header.
Large pagesWhen a page is larger than max_file_size, source_code holds a placeholder message instead of the HTML. Raise the limit, up to 32 MiB, for very large pages.
Targets not allowedA 400 that carries only details, without parameters, means the target cannot be analyzed.
Rate limitsPlan-dependent. The default is one request per second per key and endpoint. See rate limits.

Narrower Calls on One Page.

Lookup Technology Only the technologies, with categories, version and CPE. Read Lookup Screenshot Only the image, with full control over the browser. Read Lookup SSL The certificate the host serves, checked live. Read

See This Data at Work in the Deepinfo Platform

External Attack Surface Management runs these lookups continuously over your assets and raises what changes as issues, and Deep Search & Insights opens the whole index to your analysts.

Questions About Web Data API

Can we find other sites that share a page or an analytics ID?

Web Data profiles one page per call, so compare the IDs and hashes across the pages you profile. Across the index, Domain Search matches hosts on webdata.html.source_code_hash, which identical pages share.

Which e-mail addresses does the response list?

html.emails holds every address found in the page, and html.emails_internal the ones on the site’s own domain.

Can it tell when a page tries to block inspection?

Yes. html.inspect_disabled reports whether the page tries to block inspection of its content.

Which load events does Web Data wait for?

By default, networkidle2, load and domcontentloaded, and wait_until changes the list. With screenshot=true the list is ignored: the page is always parsed after the load event.

How is the Web Data API priced?

As part of an API services plan, by volume and frequency. Each call is a lookup and counts against your plan’s quota at one unit per request. See Data & API pricing.

Can analysts profile a page without writing code?

Yes. Instant Lookups includes Web Data in the Deepinfo Platform, next to Screenshot and Technology.

Profile a Page You Know Well.

Run Web Data against your own site first, so you know what a clean result looks like before you compare a lookalike with it. We set up access on a short call.

Talk to us about access All API services