infra: content gap analysis

[marc0olo]

Context

After the content migration sprints, a systematic comparison of all upstream sources against written docs is needed to find what's missing or outdated.

Steps

1. List all portal source pages in .sources/portal/docs/
2. Map each to its corresponding page in docs/ (use migration-plan.md as a starting point)
3. For pages that exist in portal but not in docs: evaluate if they belong (some portal content is not relevant for the rewrite)
4. For pages that exist in docs: spot-check for staleness against .sources/ upstream content
5. File individual issues for any meaningful gaps discovered
6. Update .docs-plan/migration-plan.md with gap findings

Output

A structured report in .docs-plan/ (or as a GitHub issue comment) listing gaps and their priority.

[marc0olo]

Content gap analysis: portal vs. developer-docs

Scope and premises

dfinity/developer-docs replaced dfinity/portal (archived, last commit ed74223 "docs: archive repo,
redirect to developer-docs"). Issue #118 asks for a systematic comparison of portal content against the
new docs to find what is missing or stale. The question this answers: is any important content that lived in the portal currently missing?

Two premises of the issue are out of date:

• Portal is no longer a .sources/ submodule, so step 1 (ls .sources/portal/docs/) cannot be run.
  This analysis used a local clone of dfinity/portal at its final archived commit instead.
• .docs-plan/ was deleted in 8ac59c8 ("post-migration cleanup"). The two artifacts this analysis
  needs were recovered from git history:
  • git show 8ac59c8^:.docs-plan/migration-plan.md (planned target page per portal source)
  • git show 8a5346c^:.docs-plan/portal-deep-dive.md (the original 335-file triage table)

The new site also restructured beyond the original plan (defi/ → digital-assets/, IC interface spec
split into references/ic-interface-spec/*, Learn Hub folded into concepts/, developer-tools/
promoted to top level), so a file-for-file diff is meaningless. Only topic-level coverage matters.

Method

1. Mapped 335 portal content pages (excluding _attachments, docs/tests/, 22 dfx release notes)
   against the 210 pages now on the new site.
2. Three parallel topic-scoped comparisons (chain fusion + network features; digital assets +
   governance; canisters + tooling + references), reading portal sources and grepping the new site per
   topic, classifying COVERED / THIN / MISSING.
3. Verified every high-priority finding by hand (greps against docs/, plus the ckETH minter Candid
   from dfinity/ic for the withdraw_erc20 question).
4. Cross-checked public/_redirects (433 rules) for portal URLs that land on a section index or a 404.

Interface spec split verified complete: portal's single 59,678-word ic-interface-spec.md is now
64,616 words across 7 files, with the certification section preserved and extended.

---

Summary

Coverage is broadly complete. All 335 portal pages have a redirect target, the security guide set,
canister management, testing, cycle costs, Candid reference, execution errors, subnet types, ICRC
standards, chain-key token flows, SNS launch/manage/test workflows, and the reference specs all port
at parity or better. The gaps cluster in five places:

1. vetKeys: 9 portal pages (~7,700 words) reduced to one 1,150-word concept page. The planned
   guides/security/encryption.md (GH docs: guides/security/encryption.md #113) was never written.
2. ICP ledger native interface: references/ledger.mdx (30 KB) has no successor.
3. Serving HTTP from a canister: the http_request + upgrade pattern exists only inside the
   HTTP gateway spec. There is no guide for building a canister HTTP or REST API.
4. Token operations: mint/burn semantics, ledger-suite upgrade procedure, index canister
   deployment, NFT implementation path.
5. Off-chain client concerns: Rust agent dynamic routing, JS identity from a key or seed,
   independent signature verification, identity key backup and recovery.

One factual error was found in current content (see P0).

P0: factual error in current docs

withdraw_erc20 is a real, separate minter method. Verified against
dfinity/ic:rs/ethereum/cketh/minter/cketh_minter.did:

withdraw_eth   : (WithdrawalArg)    -> (variant { Ok : RetrieveEthRequest;   Err : WithdrawalError });
withdraw_erc20 : (WithdrawErc20Arg) -> (variant { Ok : RetrieveErc20Request; Err : WithdrawErc20Error });

Two pages tell developers to use withdraw_eth for ckERC20 withdrawals, which will not work:

• docs/references/protocol-canisters.md:135: "used for both ETH and all ERC-20 withdrawals"
• docs/guides/digital-assets/chain-key-tokens.mdx:407: step 2 calls withdraw_eth for ckERC20

The ckERC20 path also needs two icrc2_approve calls (ckETH for gas, plus the ckERC20 token). Worth
adding while fixing: withdraw_eth uses a 21,000 gas limit, so withdrawals to contract addresses
typically fail (stated in the upstream Candid comment).

P1: high-priority gaps

Gap	Portal source	What is absent	Belongs
vetKeys implementation guide	network-features/vetkeys/* (9 files)	Motoko/Rust wrappers for vetkd_derive_key/vetkd_public_key, the context = domain-separator‖caller-principal construction, the frontend flow (TransportSecretKey.random() → publicKeyBytes() → EncryptedVetKey(...).decryptAndVerify(...)), KeyManager/EncryptedMaps backend + client usage, IBE (IbeCiphertext.encrypt/decrypt), timelock, offline MasterPublicKey derivation and key_1/test_key_1 values	new guides/security/encryption.md (GH #113) or guides/backends/vetkeys.mdx
eth_sendRawTransaction duplicate-submission semantics	chain-fusion/ethereum/evm-rpc/evm-rpc-canister.mdx	Because the outcall is replicated, providers legitimately return already known / NonceTooLow; that is the expected success path, and inclusion must be confirmed via eth_getTransactionCount at Latest. The current example (guides/chain-fusion/ethereum.mdx:443-500) traps on Consistent(Err) and Inconsistent, which is exactly the wrong behavior here	guides/chain-fusion/ethereum.mdx
ICP ledger native (non-ICRC) reference	references/ledger.mdx	AccountIdentifier derivation (CRC32(h) ‖ h, h = sha224("\x0Aaccount-id" ‖ principal ‖ subaccount)), legacy transfer/account_balance Candid (Tokens{e8s}, Memo : u64, TxTooOld/TxDuplicate/TxCreatedInFuture), query_blocks/archives semantics (certificate only on non-replicated queries, 2000-block cap, archived_blocks callbacks), encoded_block hashing, 24h dedup window and 60s clock drift. concepts/ledgers.md:29 describes AccountIdentifier in prose but gives no formula	new references/icp-ledger.md
Serving HTTP from a canister	network-features/using-http/gateways.mdx, http-certification/06-*	Every request arrives at http_request as a query regardless of method; upgrade = opt true re-runs it as http_request_update through consensus; http_request_update cannot be called directly with curl. Only documented inside references/http-gateway-protocol-spec.md; no guide covers building a canister HTTP or REST API	new guide, or a section in guides/frontends/certification.md
Identity key storage, backup, recovery	getting-started/identities.mdx	Where the private key lives, OS-keyring-first with password-encrypted-PEM fallback, seed-phrase recovery (identity import --seed-file), and that identity export yields the key but never the seed phrase. The new site has one sentence (guides/canister-management/cycles-management.mdx:31). This is an irrecoverable-loss class of mistake	short page, or an explicit handoff to cli.internetcomputer.org
Mint and burn semantics for token issuers	defi/token-ledgers/setup/*	Transfers from the minting account produce Mint blocks and to it produce Burn; only the minting account creates supply; burn needs fee = 0 and an amount above min_burn_amount; keep the minter identity separate from the ledger controller. guides/digital-assets/ledgers.mdx shows minting_account in init args without explaining the workflow	guides/digital-assets/ledgers.mdx
ICRC ledger-suite upgrade procedure	defi/token-ledgers/upgrading/icrc1_ledger_upgrade.mdx	Releases must be applied sequentially (version check fails on skips, heap→stable migrations are not backwards compatible), component order index → ledger → archive, snapshot first, post-upgrade verification via /metrics (ledger_transactions, ledger_balance_store_entries, ledger_num_approvals), identifying an unknown deployed version by module hash or git_commit_id	guides/digital-assets/ledgers.mdx
CMC MINT memo	references/system-canisters/index.mdx	references/system-canisters.md:96 documents notify_mint_cycles and the principal-encoded subaccount but omits that the ICP transfer must carry memo = blob "\4d\49\4e\54\00\00\00\00". Without it the notify step fails, so the documented flow does not work	references/system-canisters.md
Rust agent dynamic routing	interact-with-canisters/agents/rust-agent.mdx	Agent::builder().with_background_dynamic_routing(), the RouteProvider trait, DynamicRouteProvider with background API-BN discovery and health checks, round-robin vs latency-weighted strategies, and the static-vs-dynamic trade-off (a single gateway is a single point of failure). guides/canister-calls/calling-from-clients.md shows only with_url(...)	guides/canister-calls/calling-from-clients.md
Asset canister total-size ceiling	frontends/using-an-asset-canister.mdx, references/asset-canister.mdx	The ~1 GiB practical limit on total assets per frontend canister, the symptom (the canister fails to upgrade once exceeded), recovery (uninstall-code then redeploy, losing all assets), and sharding advice. No mention anywhere under guides/frontends/	guides/frontends/asset-canister.md

P2: medium-priority gaps

Grouped by area; each is a section-sized addition, not a new page unless noted.

Certification and HTTP

• ic-http-certification for dynamic responses: DefaultCelBuilder::full_certification / response_only_certification / skip_certification, IC-CertificateExpression, HttpCertificationTree with HttpCertificationPath::exact vs ::wildcard, re-certifying after each state change, JSON-API routing. Portal had ~1,200 lines (http-certification/03-*, 04-*); the new page defers to docs.rs.
• Asset canister default security policy: the actual header set behind security_policy: "standard", and the ICP-specific reasoning that nonces cannot be used (certified responses need static bodies) so script hashes plus strict-dynamic are required; the default connect-src allows calling any canister and should be narrowed.
• .ic-assets.json5 rule resolution: headers from multiple matching rules concatenate unless "headers": null resets; subdirectory rules override parent rules; last rule wins; ignored directories cannot have children un-ignored. references/application-canisters.md:80-108 shows a multi-rule example with none of the precedence rules.

Bitcoin and chain fusion

• Ordinals / Runes / BRC-20: commit-and-reveal inscription pattern, BRC-20 JSON in a Taproot witness script, Runes OP_RETURN Runestone and etching, the P2TR script-path requirement, Runes Indexer canister.
• Bitcoin address types and the two P2TR variants (key-path-only vs key-or-script, when to pass aux = bip341 { merkle_root_hash }).
• ckBTC withdrawal status variants from retrieve_btc_status_v2 (Pending, Signing, Sending, Submitted, AmountTooLow, Confirmed, Reimbursed, WillReimburse): a developer polling status cannot currently interpret the result.
• Threshold signature verification: BIP-340 requires dropping the first byte of the 33-byte SEC1 key from schnorr_public_key (currently only buried in references/ic-interface-spec/management-canister.md:493); Ed25519 verification; ECDSA r‖s layout. No verification example anywhere.
• EIP-1559 transaction construction and signing (build → Keccak-256 → sign_with_ecdsa → reassemble with V/R/S). mo:core has no Keccak, so Motoko developers have no path at all.
• eth_getLogs example plus RpcConfig.responseSizeEstimate: the canister doubles max_response_bytes and retries until the cycle budget is exhausted, so the estimate controls cost. The knob is never named.

HTTPS outcalls

• Error-string catalogue from references/https-outcalls-how-it-works.mdx, none of which appear on the new site. Includes one hard limit that is absent entirely: SysFatal - Transformed http response exceeds limit: 2045952, a separate cap on the transformed response (body plus headers) distinct from the 2 MB network limit.

Authentication and identity

• Independently verifying IC signatures off-chain: ic-validator-ingress-message (Rust) and @dfinity/standalone-sig-verifier-web (npm), with their caveats. The envelope fields are in the spec, but the "verify an II delegation chain in my backend" workflow has no home.
• Alternative auth providers: NFID, ic-auth, Sign in with Bitcoin / Ethereum / Solana (ic-siwb, ic-siwe, ic-siws), MSQ. The new site covers only Internet Identity and ICRC-25/27/49 signing, so a developer arriving from Ethereum or Solana gets no pointer.
• HSM-backed identities (references/using-hsm-with-identities.mdx): the OpenSC and PKCS#11 half is tool-agnostic and currently reduced to "use an HSM".
• AuthClient internals: the AuthClientStorage interface and the footgun that string-only custom storage must pass keyType: 'Ed25519'; the IdleManager 10-minute default logout with page reload.
• Creating a JS identity from a key or seed phrase for Node scripts, oracles, and CI. calling-from-clients.md covers Node.js only as anonymous.

Tokens and governance

• Index canister deployment and full API: artifacts, init args, status → num_blocks_synced (the standard way to confirm the index caught up), get_blocks, list_subaccounts.
• NFT implementation path: ICRC-7/37 are specified but there is no way to ship an NFT ledger (no reference implementation, no icrc7-mo/icrc37-mo/icrc3-mo, no mint or local-test steps).
• archive_options semantics: trigger_threshold vs num_blocks_to_archive (0 silently disables archiving), cycles_for_archive_creation must cover creation and install.
• Cycles ledger API and fees: deposit, the "leave a cycles buffer" gotcha, and the fee schedule. references/system-canisters.md#cycles-ledger lists no methods and no fees.
• ckERC20 creation and the ledger-suite orchestrator (vxkom-oyaaa-aaaar-qafda-cai, get_orchestrator_info, canister_ids(record { chain_id; address })). Absent site-wide.
• Rosetta neuron construction flows: neuron_index semantics and neuron-ID derivation, per-operation metadata, pre/postconditions for DISBURSE/SPAWN/STAKE_MATURITY. guides/digital-assets/rosetta.md:444 lists operation names only and defers to a source tree in dfinity/ic that has no equivalent prose docs (rs/rosetta-api/icrc1 has no README).
• Submitting an NNS proposal (as opposed to SNS): proposer eligibility, staking a proposal neuron, the manage_neuron → MakeProposal payload, wait-for-quiet.
• ManageDappCanisterSettings SNS proposal: absent site-wide, and it is the only route for a community-governed app to change canister settings after handover.

Canisters and tooling

• Running third-party canisters locally (ledger, Internet Identity, XRC): the whole workflow is missing and needs an icp-cli equivalent. This blocks local integration testing.
• Trap diagnosis: symptom → cause for "Heap out of bounds", "Stable memory out of bounds", "Unreachable" (a Rust panic, install the CDK panic hook), and when to trap deliberately.
• Exposing canister metrics: the ACL-guarded metrics() query and the /metrics text-exposition endpoint served from http_request. guides/security/observability-and-monitoring.md only links a blog post.
• Agent latency and UX guidance: batch with Promise.all, do not block the UI on 2 to 4 second update calls, optimistic rendering, pre-compute results.

P3: low-priority or niche

skip_certification / no_certification and when it is acceptable; ic-asset-certification advanced config (Redirect, fallback_for, encodings); local HTTP gateway proxy; Bitcoin canister get_config and the watchdog that disables the API when more than two blocks behind (an availability failure mode with zero coverage); regtest debugging commands; ckBTC deposit-address derivation detail (P2WPKH, deliberately not BIP-32); EVM RPC getProviders/updateApiKeys/providerId and ProviderError::TooFewCycles (note: Custom { url, headers } IS documented); supported-chains table rows dropped (Sui, TRON, Monero, Filecoin, Hedera, Kaspa, Stacks, Arweave); master public key hex constants for non-JS/Rust stacks; threshold signatures are non-deterministic; vetKD backup subnets; Candid gotchas (passing a func reference as [Principal, "method"], private candid:service metadata, 32-bit field-name hashing and didc hash, didc bind/test/random, import in .did files); ID-encoding spec test vectors and exact CRC-32 parameterization (the normative rule is already in references/ic-interface-spec/index.md:209); allowed_viewers 10-principal cap missing for log_visibility and snapshot_visibility; known-neuron registration; verifying NNS proposals with repro-check.sh; SNS asset canister set_permissions upgrade argument; direct-integration vs Rosetta decision framing and Rosetta node infrastructure requirements; token best-practices leftovers (embedding ICRC in an existing ledger, metadata quality for wallet UX); Rust ic-utils canister-oriented API; testnet FAQ and playground restrictions.

Intentional drops worth reconfirming

These were deliberate per portal-deep-dive.md, but they are the largest volume losses, so they should
be conscious decisions rather than defaults:

• All 80 tutorial pages (~93,000 words): Developer Liftoff (Motoko and Rust tracks) and the
  Hackathon Prep Course, every one redirecting to /getting-started/quickstart/. There is no
  long-form guided learning path on the new site; getting-started/ totals ~4,100 words.
• 12 NNS dapp end-user guides → internetcomputer.org/network/governance.
• 22 dfx release notes → dropped.
• Sample catalog (/references/samples/*, rendered from dfinity/examples) → redirects to the
  GitHub repo. There is no browsable examples index on the docs site.
• dfx.json reference, install, WSL, custom networks, chunk store, replica version →
  cli.internetcomputer.org. Fine, but note AGENTS.md/CLAUDE.md still reference
  docs/guides/tools/migrating-from-dfx.md, which no longer exists in docs/.

Separate finding: defects in public/_redirects

Not content gaps, but they break the portal-to-docs handoff. Found while mapping URLs (433 rules
checked against 335 portal URLs plus 210 existing pages):

1. 11 rules point at /getting-started/quick-start/, which does not exist (the page is
   /getting-started/quickstart/). Every one of them lands on the 404 page.
2. The portal quickstart URL itself 404s. The live portal path was
   /building-apps/getting-started/quickstart (confirmed in portal sidebars.js:12), but only
   /building-apps/getting-started/quick-start/ has a rule, and there is no
   /building-apps/getting-started/* wildcard, so it falls through to /* → /404.html. Same for
   /building-apps/getting-started/app-architecture, /identities, and /wsl-troubleshoot.
3. /building-apps/test/overview and /building-apps/test/pocket-ic have no rule at all and 404,
   even though /guides/testing/strategies/ and /guides/testing/pocket-ic/ exist.
4. Two trailing-dot typos: destinations /references/glossary. and /guides/backends/timers.
5. Destinations with no index page: /languages/motoko/fundamentals/, /icp-features/,
   /reference/ (3 rules each), and /getting-started/ (1 rule).
6. 9 rules pinned to cli.internetcomputer.org/0.2/ while the docs body uses 1.1.
7. Coarse wildcards bury precise targets. /building-apps/network-features/* →
   /concepts/#canisters--capabilities swallows randomness, timers, t-ECDSA, t-Schnorr, and
   verifiable credentials, all of which have specific pages. /references/* → /references/ does the
   same to execution-errors, http-gateway-protocol-spec, subnet-types,
   message-execution-properties, and dashboard-apis.

Suggested follow-ups

• Split the P0 fix and the P1 rows into individual documentation issues.
• Fix _redirects on an infra/ branch (items 1 to 5 are mechanical; item 7 needs per-page rules).
• Update AGENTS.md/CLAUDE.md: remove the docs/guides/tools/migrating-from-dfx.md reference.