Subgraph & Query Layer
Status: Requirements / model. This is the detailed model that builds on the Map-Reduce Framework. Nothing here is built yet; the first concrete instance is the Algebra subgraph in the dark-pool work (pocs/services/docs/dark-pool/algebra-integration.md §3). Related: Erigon Archive, RPC-REQ-020 in pocs/services/REQUIREMENTS.md.
This layer lets cocoon index chain data into queryable, domain-scoped datasets and serve them over GraphQL (TheGraph-subgraph-compatible) and/or SQL (Dune-style) — without graph-node, postgres, or ipfs, because the node owns the datastore. It is not a new engine: it is a family of Consumers on the Map-Reduce Framework. The engine already provides the hard parts — parallel historical replay, per-extension domain namespaces, point-in-time persistence, live-stream attachment, and a WASM isolation tier — so this layer is mostly configuration + mapping logic + serving, not new infrastructure.
Where it sits on Map-Reduce
Map (parallel replay) Reduce (per-subgraph) Serve
───────────────────── ───────────────────── ─────
TracerFactory: "decode" ──▶ Consumer: run mappings ──▶ GraphQL (entity schema)
logs/traces → decoded (codegen-Go or WASM) SQL (decoded/curated tables)
events & calls (ABI) writes entities into
ext_<subgraph> domain
- Map = a decode tracer. ABI-driven decoding of
logs/tracesinto decoded events/calls is the shared substrate (this is exactly Dune'sevt_/call_decoded-table model). Both serving surfaces consume it. - Reduce = a mapping Consumer writing entities into the subgraph's own domain namespace (
ext_<subgraph>), reusing the framework'sDomainConsumer/AggregateConsumerpatterns. - Serve = GraphQL and/or SQL over those domains.
Mapping approaches — two orthogonal axes
Keep two axes separate: how mappings are written/executed (the language/runtime) and how data is served (the query surface). They mix freely — e.g. GhostGraph writes mappings in Solidity but serves GraphQL.
Mapping language / runtime — three options:
| TheGraph subgraph | GhostGraph | Dune | |
|---|---|---|---|
| Mapping written in | AssemblyScript → WASM | Solidity (handlers + entities as structs) | declarative ABI registration (no handler code) |
| Runtime | WASM VM (e.g. Wazero) | an EVM | SQL engine (DuneSQL/Trino) over decoded tables |
| Decode → map | imperative handlers you author | imperative handlers you author, in Solidity | mechanical ABI decode → per-event/per-function tables |
| Schema | GraphQL entities (schema.graphql) | GraphQL entities, defined as Solidity structs (schema.sol) + @many/@belongsTo | shared decoded-table namespace + curated SQL tables |
| Onboard a contract | write + deploy a subgraph (code) | write a Solidity indexer (code, same language as the contracts) | register an ABI (config) |
| Cross-contract | siloed per subgraph | per-indexer (can span contracts) | one warehouse, JOIN across all (natural) |
Query surface — two options (independent of the above): GraphQL (fixed entity schema, predictable cost — best for app reads) or SQL (ad-hoc joins/aggregates, variable cost — best for analytics/compliance).
GhostGraph (Solidity mappings) is the option to flag for cocoon. GhostGraph ("the first indexing solution that lets you write your index transformations in Solidity") authors entities as Solidity structs and handlers in Solidity, served over GraphQL. The architectural fit is unusually strong here: cocoon already runs an EVM (erigon), so Solidity mapping handlers can execute in the EVM we already have — no separate WASM/Wazero runtime to embed for AssemblyScript — and developers write mappings in the same language as the contracts being indexed. This makes Solidity-in-EVM a serious alternative to the WASM tier for the generalized layer.
The SQL-like API is the other first-class alternative we should note. Dune's model (raw → declarative ABI-decode → dbt/SQL transforms → SQL query API, engine = DuneSQL, a Trino fork) maps almost 1:1 onto erigon's substrate (erigon already produces blocks/txs/logs/traces, archive-grade). It is lower-onboarding-cost (ABI = config, not code), gives cross-contract compliance/analytics queries for free, and yields current+historical from one append-only store via SQL. The three mapping options + two surfaces are not mutually exclusive — the recommended internal shape is Dune-style decoded+curated tables with GraphQL exposed as a thin read API where the UI needs it, SQL for analytics/compliance/the test-agent harness, and Solidity-in-EVM as the ergonomic mapping path that reuses the node's own VM.
Current vs history — mostly free from the substrate
The Map-Reduce persistence layer writes into Erigon domain storage, which is already temporal: every value is keyed by txNum and read via GetAsOf. That means the subgraph layer inherits:
- Backfill (history): parallel replay from a subgraph's
startBlock→ head; erigon archive makes historicaleth_getLogs+eth_callcomplete and deterministic. - Current (live): live-stream attachment to the Execution component's
StateChangeEvent;PersistAndStreamConsumerunifies historic + live into one continuous stream. - Time-travel (
entity as of block N): native viaGetAsOf— no hand-rolled entity versioning (the thing graph-node+postgres must implement by hand). - Reorgs / finality: the current view follows the preconfirmed head (reorg-aware); the finalized view commits at finality. This maps directly onto the chain's preconfirmed-vs-finalized safety tags — the subgraph's current/history split is the chain's, not a bolt-on.
Domain per subgraph
Each subgraph is a self-contained, independently deployable domain:
| Layer | What it means | Provided by |
|---|---|---|
| Storage | own domain namespace ext_<subgraph>, independently backfilled/dropped/rebuilt | Map-Reduce persistence (RPC-REQ-004 pattern) |
| Schema/endpoint | own GraphQL endpoint /subgraphs/<id> and/or SQL schema | this layer |
| Lifecycle | deploy → sync → synced → undeploy, independent of other subgraphs | Map-Reduce service registry |
| On-chain identity (forward) | the subgraph as a chain-registered, DID-addressed, UCAN-gated, DBC-billed data domain in the services marketplace | chain-coordinated services |
The last row is the cocoon-specific reframing: "we run a subgraph" → "we vend an addressable data domain backed by a subgraph."
Requirements
SG-REQ-1 — Subgraph layer as a Map-Reduce Consumer family. The indexing layer must be built as TracerFactory (decode) + Consumer (map) on the existing engine, not a parallel indexer. Reuse parallel replay, ordered reduce, per-extension domains, and live+historic streaming.
SG-REQ-2 — Declarative ABI decode as shared substrate. A built-in "decode" tracer must turn logs/traces into decoded events/calls from registered ABIs (Dune-style), as the common input for both GraphQL-entity mapping and SQL tables. Support factory/dynamic decoding (one ABI covers many instances by contract_address).
SG-REQ-3 — Mapping tiers (multiple languages/runtimes). (a) Codegen/Go reduce consumers (near-term, compile-time, trusted Tier 0) — hand-written transforms over decoded data; the only tier needed for first-party subgraphs (e.g. Algebra). (b) Solidity-in-EVM mappings (GhostGraph-style) — execute Solidity handler logic in the node's own EVM; strong fit since cocoon already runs erigon's EVM and devs write the same language as the contracts; candidate alternative to (c). (c) WASM mappings — run upstream AssemblyScript subgraph handlers in the Wazero Tier-2 sandbox with graph-node-style host imports, to import third-party TheGraph subgraphs unmodified. (b)/(c) are the forward generalized paths; the store/serve/domain wrapper is shared across all tiers (SG-REQ open Q5).
SG-REQ-4 — Per-subgraph temporal domain. Each subgraph writes entities into its own ext_<subgraph> domain namespace, txNum-keyed for GetAsOf time-travel, with reorg rollback. No bespoke entity-versioning.
SG-REQ-5 — Current/finalized serving tags. Serving must expose both a current (preconfirmed-head, reorg-aware) and a finalized (historic, stable) view over one continuous stream, tagged consistently with the chain's safety levels.
SG-REQ-6 — Dual query surfaces (GraphQL + SQL). Provide (a) a GraphQL surface per subgraph (fixed entity schema, gqlgen codegen — see RPC-REQ-020) for app reads, and (b) a SQL-like query API over decoded/curated tables (Dune/Trino-style) for ad-hoc analytics + compliance. Neither is mandatory for a given dataset; both consume the same decoded substrate.
SG-REQ-7 — Curated/aggregate transform layer. A Spellbook/dbt analog (built on AggregateConsumer) must build normalized cross-contract tables (e.g. a unified settlements/trades view across OrderRegistry/DarkPool/SpentBook) incrementally, for compliance and analytics that siloed subgraphs make awkward.
SG-REQ-8 — Subgraph manifest import. Parse a TheGraph subgraph.yaml (datasources, ABIs, startBlock, handler bindings) to configure the decode tracer, mapping consumer, and backfill automatically — the generalized "import any subgraph component" path. Near-term we import only the schema; full manifest+WASM import is SG-REQ-3(b).
SG-REQ-9 — Domain-as-service (forward). A subgraph domain should be registrable as a chain-coordinated service: DID-addressed, UCAN-gated access, DBC-billed, discoverable via a registry — making indexed data a first-class addressable, access-controlled, billable domain.
SG-REQ-10 — Mapping-logic licensing. Ported third-party mapping math (e.g. Algebra's GPL-3.0 subgraph) must be re-implemented from spec in Go (clean) or, if run as WASM, treated under its copyleft. Keep ported logic separable and attributed.
SG-REQ-11 — Agent-defined datasets (forward). Via the Map-Reduce MCP interface, agents should be able to define subgraph/SQL datasets without Go — backfilled from genesis into a new domain and attached to the live stream, surviving restarts.
Monetization — an additional revenue stream for cocoon node operators
Running data domains should be a revenue source for cocoon node operators, not just a feature. Survey of how the incumbents monetize (see research):
Only two of ten providers reward an independent node operator directly — both decentralized stake/bond-to-serve marketplaces; everyone else is centralized SaaS where a company captures 100% and runs infra as a cost:
| Model | Operator reward | Precedent value for cocoon |
|---|---|---|
| The Graph — Indexer | stake ≥100k GRT → indexing rewards (3% inflation × stake × curation signal) + exponential query-fee rebates; delegators supply capital for a cut; curators signal demand | stake-to-serve + curation/delegation multi-sided market |
| Subsquid/SQD — Worker | bond 100k SQD (slashable) → token-APR plus Revenue Pools: up to 50% of real enterprise USDT fees to operators/stakers (capped pools), rest to workers + buyback | closest precedent — demand-backed (real fees, not inflation) |
| Dune / GhostGraph / Goldsky / Alchemy / QuickNode / Infura / Envio | nothing — centralized | only the metering menu below |
Pricing-primitive menu (what to bill with): per-query · per-entity/per-event (storage/output) · complexity-weighted compute units (the converged RPC/query unit — Alchemy/QuickNode/SQD all weight eth_getLogs/trace at multiples of a base call) · worker/compute-hours · processed-blocks+egress (streaming) · subscription tier + PAYG overage (near-universal base) · staking/delegation + rewards · revenue-pool share.
Going rate (what operators charge — the revenue line to size against, 2025–26):
| Charge unit | Market rate | Examples |
|---|---|---|
| Per-query / RPC (simple call) | ~$2–$20 / 1M requests | The Graph $20/1M subgraph queries; Goldsky Edge RPC $5/1M; SQD $2/1M (→$5); Alchemy/QuickNode eth_call ≈ $10–12/1M |
| Per-query (log/trace-heavy) | 3–10× the simple rate | Alchemy eth_getLogs ≈ $24–27/1M; Infura weights eth_call to ~$290/1M |
| Per-entity indexed | ~$10–$40 / 1M·month | Goldsky $40/1M; GhostGraph $25–50/1M overage |
| Per-event | ~$10 / 1M | Goldsky $1/100k |
| Always-on indexer (worker) | ~$17–37 / month | SQD $0.024/hr; Goldsky $0.05/hr |
| Storage | ~$0.60–1.50 / GB·month | SQD $0.60; Goldsky Postgres $1.50 |
| Streaming (processed blocks) | ~$3 / 1M blocks + ~$175 / TiB egress | Substreams |
| Subscription base (entry→mid) | ~$50–$1,000 / month | universal across indexing (GhostGraph $49–249), analytics (Dune $399–999), RPC (QuickNode $49–999, Infura $50–225) |
A cocoon operator's data-domain pricing slots into these same bands: single-digit-to-low-tens of dollars per million queries (more for heavy methods), tens of dollars per million indexed entities/events, ~$17–37/mo per always-on indexer, ~$1/GB·mo storage, under a ~$50–1,000/mo subscription base with PAYG overage.
Market data — the premium tier above commodity queries
Indexer query pricing is the floor (commodity bandwidth). Traditional market data is priced not per query but per license axis, and the operator-relevant axes run 100×–1,000× higher. A dark-pool/DEX venue is the sole source of its own trade/order data — the same structural monopoly that lets an exchange charge $22,400 for one non-display feed category. Global market-data spend is ~$49B/yr (Bloomberg + LSEG ≈ half), at 76–95% operating margins.
The fee-axis model to adopt (exchanges sell the sum of every axis that applies, not "the data"):
| Axis | Licenses | TradFi magnitude |
|---|---|---|
| Access / datafeed | right to receive the feed at all | $3k–8.4k/mo |
| Display, per-user | one human at a screen | $16–80/user/mo |
| Professional vs non-professional | modifier on display (non-pro 4–60× cheaper) | — |
| Non-display | automated/algo consumption (no human) — the largest line | $5k–75k per firm/category/mo |
| Redistribution / vendor | right to pass the feed onward | $250–29k/mo |
| Derived data | indices/VWAPs computed from the feed | $1.5k+/mo |
| Enterprise cap | flat cap replacing per-user counting | $25k–750k/mo |
Revenue stack (per-unit price climbs as you go up): raw indexer query $2–20/1M → cheap crypto market-data API $35–700/mo → institutional crypto data license (Kaiko) $10k–55k/yr → TradFi proprietary exchange feed (non-display $5k–75k/firm/mo + redistribution + derived) → terminal/vendor $12k–32k/seat/yr.
The premium over raw query comes from four ownable things: (1) non-display licensing — charge the algos, not screens (~90% of the money; nearly absent in crypto today); (2) redistribution + derived licensing — the same feed sold again to every downstream vendor and index (multiplicative); (3) latency + depth as the proprietary premium (the SIP-vs-direct-feed gap proves buyers pay 10–100× for better/faster, not just more); (4) compliance grade as a moat.
Regulatory positioning. MiFIR's "Reasonable Commercial Basis" (binding RTS in force Nov 2025, full compliance Aug 2026) forces cost-plus, unbundled, 15-min-delayed-free pricing on TradFi data; the EU/UK consolidated tapes are standing up now. MiCA imposes the transparency + order-book-record obligations but has NO data-pricing/RCB regime and NO crypto consolidated tape yet — a genuine gap. A cocoon venue building licensed, attributable, compliance-grade market data is positioning into a layer regulators are actively constructing in TradFi but that does not yet exist in crypto. Ties to market_data_compliance (MiCA/FCA/VARA, 8-yr retention) and best_execution_committee (the validators' consensus is best-execution data).
Cocoon model — bonded data domains in the services marketplace. This drops onto primitives cocoon already has (chain_coordinated_services, staking_set_and_spentbook, principal_agency_model "capital-light bonded service provider", lido_staking_design restaking, best_execution_committee):
- A data domain (SG-REQ-9) is a chain-registered, DBC-metered service; consumers pay per the metering menu in DBC.
- The operator earns a protocol-mediated share of real consumer fees, gated by a slashable bond (stake-to-serve) — wrong/withheld/stale data is slashable, mirroring the matcher/best-execution bonding.
- Demand-backed, not inflation-funded — prefer SQD's Revenue-Pool shape (real DBC fees → operator/staker split) over The Graph's inflation-reward model, matching cocoon's "no capital from the chain, fees amortized at the Stage-4 rollup boundary" stance (avoids per-query chain tax).
- Delegation + curation analogs: passive DBC capital can delegate to a data-domain operator for a cut (restaking); curation signal directs which domains are worth indexing.
Monetization requirements
SG-REQ-12 — Data domains are billable chain-registered services. A data domain is registered, DBC-metered, and access-gated (UCAN); consumption is metered and settled in DBC. Extends SG-REQ-9.
SG-REQ-13 — Metering primitives. Support complexity-weighted compute units for query/RPC, per-entity/per-event for indexing/storage, under a subscription-tier-plus-PAYG-overage billing shape. Metering is the operator's revenue base.
SG-REQ-14 — Stake-to-serve with slashing. Operator revenue rights are gated by a slashable bond; incorrect/stale/withheld data is slashable. Reuse the staking-set bonding + dispute machinery, not a bespoke mechanism. Support delegation (passive capital backs an operator for a share) and curation signal (directs demand).
SG-REQ-15 — Demand-backed payout, amortized at the rollup boundary. Operators earn a share of real consumer DBC fees (not chain inflation); fee settlement is netted at the Stage-4 rollup boundary to avoid a per-query chain tax. SQD Revenue Pools is the reference shape.
SG-REQ-16 — Market-data licensing by axis (the premium tier). Venue trade/order data must be priceable by license axis — display / non-display / professional-vs-non-professional / redistribution / derived-data / enterprise-cap — not just per-query. The non-display (automated-consumption) axis is the primary revenue line (where ~90% of TradFi data money sits). The venue is the sole source of its own data: this is a monopoly product (4–5-figure monthly per-firm licenses), not a commodity feed. Latency + depth tiers (real-time/full-depth vs delayed/top-of-book) are a separate premium axis.
SG-REQ-17 — Compliance-grade, attributable data as the moat. Real-time venue data must be licensed, attributable, and audit-ready (MiCA transparency + 8-yr retention + a MiFIR-RCB-style cost basis) so it is a regulated product, not a free scrape. Build the license taxonomy ahead of any MiCA data-pricing / consolidated-tape regime. Cross-ref dark-pool/market-data-compliance.md.
Open questions
- Store-all-versions vs recompute-from-archive. Domain storage versions everything (
GetAsOf); but because erigon is archive we could recompute a historical entity by replaying mappings to block N instead. Store-everything (fast, big) vs recompute (small, slow) — likely per-dataset policy. - Domain granularity. One domain per subgraph (graph-node model) or grouped multi-dataset domains?
- SQL engine choice. Embed a Trino-compatible engine over erigon-backed tables, or a lighter in-process SQL layer? DuneSQL types (
varbinary,uint256) matter for ergonomics. - Where the curated/transform layer runs. Incremental build-time (dbt-style materialization) vs query-time views — cost/freshness tradeoff.
- Same domain abstraction across both mapping tiers? Codegen-Go and WASM tiers should differ only in the map stage; store/serve/lifecycle/domain wrapper built once.
Relationship to other components
- Built on Map-Reduce Framework (engine) and Erigon Archive (storage sink /
BatchProcessorfor the decode + post-processing path). - GraphQL surface mechanics: RPC-REQ-020 (
pocs/services/REQUIREMENTS.md). - First instance + the partnership/no-3rd-party-infra rationale:
pocs/services/docs/dark-pool/algebra-integration.md§3.