HKEx Filing Scraper

An open-source scraper for 25+ years of HKEx regulatory filings — into any of nine databases, with full-text extraction, graph linking, and a read-only MCP server for AI agents.

The US has EDGAR full-text search. Japan has EDINET. Hong Kong has a search form that returns
one page at a time. There is no bulk, machine-readable, full-text corpus of HKEx filings. This
builds one.

An open-source Python tool that scrapes 25+ years of Hong Kong Stock Exchange (HKEx)
regulatory filings and ingests them into any combination of nine databases — with
full-text and table extraction, chunk-level coverage, optional graph linking, and a
read-only MCP server so AI agents can query the corpus or the live site.

It speaks the undocumented HKEx JSON API directly, which is faster and more resilient than
driving a browser.

Vendors & Integrations

Databases — nine first-class destinations, in documented popularity order (see the
support matrix):

• PostgreSQL — production-grade open-source relational
• MySQL / MariaDB — GPL relational servers, one driver
• SQLite — zero-server file database, no install needed
• MongoDB — document database
• Neo4j — property-graph database
• ClickHouse — columnar analytics engine
• DuckDB — in-process analytical engine
• SurrealDB — multi-model graph + document database

AI clients — any MCP-capable agent; ready-made configuration for
Claude, ChatGPT, Cursor, VS Code/Copilot, Gemini CLI, opencode, Manus, and Perplexity.

Available on — PyPI ·
Glama ·
MCP Registry ·
hosted gateway.

Two Ways to Use It

Use the Hosted MCP Gateway

POST, Streamable HTTP, no API key:

https://hkex-listco-updates.ascent-partners.com/api/mcp

Four read-only tools: get_server_info, search_filings (a window of at most 31 days, with
optional stock-code, title, document-type, category, and stock-name filters),
list_filing_facets (browse what a window contains), and get_filing (downloads one
document and extracts its text and tables).

Point a client at it — for example opencode:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "hkex-live": {
      "type": "remote",
      "url": "https://hkex-listco-updates.ascent-partners.com/api/mcp"
    }
  }
}

Then ask:

Use hkex-live to list the filings published between 2026-09-01 and 2026-09-18,
then summarise the interim report.

Ready-made configuration for Claude, ChatGPT, Cursor, VS Code/Copilot, Gemini CLI, opencode,
Manus, and Perplexity is in AI agent support — and for a stored corpus,
the stdio MCP server exposes a wider tool catalog and is published on
Glama. The gateway is listed
in the official MCP Registry as
io.github.simonmak-ascent/hkex-filings.

Featured on Glama — the read-only stdio MCP server is also published on
Glama, where Glama scans the
built server and scores tool-definition quality (currently 4.7/5).

Quick Start (Local)

pip install hkex-filing-scraper        # core; SQLite needs no server
pip install "hkex-filing-scraper[all]" # Excel + dotenv + every driver + the MCP server
cp .env.example .env                   # then set DATABASE_TARGET (below)
hkex-scraper --metadata-only --limit 100

Optional extras: excel, postgres, mysql, duckdb, mongodb, clickhouse, neo4j,
mcp, pdf, all, dev.

DATABASE_TARGET is an ordered, comma-separated list of sink ids; the order decides which
sink serves reads. To start with no server:

DATABASE_TARGET=sqlite
SQLITE_PATH=hkex.db

hkex-scraper runs the full pipeline (metadata + documents + graph); hkex-scraper --full-history covers everything since April 1999. The schema is created automatically.
Full install options and per-sink settings are in Getting started.

Database Support

Every sink is a first-class destination; rows are in documented popularity order. The full
matrix — licenses, capability differences, per-engine notes — is in
Database sinks.

¹ Source-available, not OSI-approved — labeled exceptions per
ADR 0003.

Valid sink ids, in documented order: postgres, mysql, sqlite, mongodb, mariadb, neo4j, clickhouse, duckdb, surrealdb. Set one variable and the same run feeds every sink:

# Order sets read precedence.
DATABASE_TARGET=postgres,sqlite
POSTGRES_DSN=postgresql://user:password@localhost:5432/hkex
SQLITE_PATH=hkex.db

How This Compares

Four ways to get HKEx filings, and what each one costs you.

If you need licensed, redistributable, SLA-backed data, buy the feed. If you need a complete
local corpus for research, compliance, or RAG, this replaces the pipeline you would otherwise
write yourself.

How It Works

flowchart LR
    A[HKEx JSON API] --> B[Phase 1: metadata]
    B --> C[Canonical record]
    C --> D{DATABASE_TARGET}
    D --> E[(PostgreSQL)]
    D --> F[(MySQL / MariaDB)]
    D --> G[(SQLite)]
    D --> H[(MongoDB)]
    D --> I[(Neo4j)]
    D --> J[(ClickHouse)]
    D --> K[(DuckDB)]
    D --> L[(SurrealDB)]
    B --> M[Graph linking]
    M --> D
    B --> N[Phase 2: download and extract]
    N --> C

• Phase 1 scrapes filing metadata through a JSF session, splitting the range into monthly
  chunks and deduplicating on a 16-character MD5 filingId.
• Phase 2 downloads each filing's PDF/HTML/Excel document, extracts text and tables to
  Markdown, and writes the payload.
• Graph linking (optional) writes has_filing and references_filing edges when
  COMPANY_TABLE is set.
• Failure isolation — a failure on one sink is logged and counted but never blocks
  another; the run exits non-zero if any configured sink failed.

Deeper detail: Architecture · ADR 0002.

Features

• Fast API scraping — direct HKEx JSON API; no browser or Selenium.
• Full history — every filing from April 1999 to today, with chunk-level coverage checks.
• Document processing — PDF/HTML/Excel text and structured tables, extracted to Markdown.
• Multi-sink — any ordered combination of nine databases, each with native idempotent upserts.
• AI-ready — a hosted live MCP gateway plus a local stdio MCP server.
• Resumable and observable — batching, parallel downloads, stalled-job detection, per-sink
  counters, and --coverage-report / --parity-report / --verify.
• Optional dependencies — the core is requests + beautifulsoup4; drivers and document
  extraction are extras with graceful fallbacks.

Documentation

• Getting started · Configuration · CLI
• Database sinks (matrix) — PostgreSQL, MySQL/MariaDB, SQLite, MongoDB, Neo4j, ClickHouse, DuckDB, SurrealDB
• Live MCP gateway · AI agent support · MCP server
• Architecture · Troubleshooting · Testing
• Roadmap · De-risking register · Upgrading
• What's new · Releasing · Legal & Terms of Use · Changelog
• Docs site: https://hkex-listco-updates.ascent-partners.com/ · Try it locally (examples/)

Development

pip install -e ".[dev,all]"
ruff check           # lint (py310, line-length 100)
ruff format --check  # formatting
pytest               # unit tests (no DB or network required)

Tests are pure unit tests; SQLite and DuckDB contract tests run in-process, and integration
tests that need a server are skipped unless that sink is configured. See
Testing.

Contributing

See CONTRIBUTING.md; report security issues per SECURITY.md.
Ideas and questions are welcome in
Discussions.

Built by Ascent Partners.

If this saves you time, a ⭐ on GitHub helps others find it.

Use with Context7

Up-to-date HKEx Filing Scraper documentation is indexed on Context7, so coding agents can pull it into context on demand. With the Context7 MCP server or ctx7 CLI installed, name the library in your prompt:

use library /simonmak-ascent/hkex-filing-scraper for API and docs

License

MIT — see LICENSE. That covers this project's code only; optional dependencies
carry their own licenses, notably the pdf extra (PyMuPDF / pymupdf4llm), which is
AGPL-3.0 and deliberately excluded from .[all]. See
docs/legal.md.

Data & Terms of Use: this is a research tool for the undocumented HKEx JSON API, and it is
not affiliated with or endorsed by HKEx. Commercial redistribution of HKEx data may require a
licensed HKEx feed; see docs/legal.md.