progress.txt

# keen-pbr3 Progress

## Codebase Patterns
- Build system: Meson + Conan 2.x with PkgConfigDeps and MesonToolchain generators
- C++20 standard with size optimization (-Os, -ffunction-sections, -fdata-sections, --gc-sections, LTO)
- Optional features use Meson options (e.g., `with_api`) and C++ preprocessor guards (e.g., `#ifdef WITH_API`)
- Include path: `include/keen-pbr3/` for public headers
- Source path: `src/` for implementation files
- Cross-compilation: Conan profiles in `conan/profiles/`, Meson cross-files in `meson/cross/`
- Conan profiles use `include(base-embedded)` to share common embedded settings
- OpenWRT toolchain prefix pattern: `<arch>-openwrt-linux-musl-` (e.g., `mipsel-openwrt-linux-musl-gcc`)
- Keenetic toolchain prefix pattern: `<arch>-linux-musl-` (e.g., `mipsel-linux-musl-gcc`)
- Docker build: `docker/build.sh <arch>` runs full Conan + Meson build, outputs to `dist/<arch>/keen-pbr3`
- Source file layout: `src/<module>/<file>.hpp` and `src/<module>/<file>.cpp` (headers co-located with source, not in `include/`)
- Namespace: `keen_pbr3` for all project code
- Error handling: custom exception classes inheriting from `std::runtime_error`
- JSON config parsing: use nlohmann_json, parse duration strings ("30s", "5m", "24h") with `parse_duration()` helper; `ttl` in ListConfig accepts both duration string and integer (seconds)
- ListConfig TTL: applies to dnsmasq-resolved ipset entries only; user-supplied IPs/CIDRs are always permanent (timeout 0)
- Config errors: throw `ConfigError` (inherits `std::runtime_error`) with descriptive messages
- Outbound discrimination: JSON "type" field determines variant type ("interface", "table", "blackhole", "ignore", "urltest")
- Outbound variant: std::variant<InterfaceOutbound, TableOutbound, BlackholeOutbound, IgnoreOutbound, UrltestOutbound> — all std::visit lambdas must handle 5 types
- UrltestOutbound: has nested outbound_groups (weight + outbounds), retry (attempts, interval_ms), circuit_breaker (failure_threshold, success_threshold, timeout_ms, half_open_max_requests)
- Urltest validation: outbound_groups references validated in parse_config() after all outbounds parsed — must reference interface/table/blackhole only
- Local typecheck: `export PKG_CONFIG_PATH=/home/maksimkurb/Dev/keen-pbr3 && g++ -std=c++20 -fsyntax-only -I include $(pkg-config --cflags ...) <file>`
- IP/subnet matching: IpSet uses binary trie (IpTrie) with separate v4/v6 tries for efficient CIDR lookup
- Netlink: libnl3 requires `-I /usr/include/libnl3` for local typecheck; use RAII wrappers with custom deleters for nl_addr, rtnl_route, rtnl_nexthop, rtnl_rule
- Netlink: `rtnl_route_add_nexthop()` takes ownership of the nexthop pointer (use `.release()`, not `.get()`)
- Netlink: For ip rules with `family == 0`, add rules for both AF_INET and AF_INET6
- ICMP ping: Use SOCK_DGRAM (not SOCK_RAW) for ICMP sockets - avoids needing CAP_NET_RAW in many kernels
- ICMP ping: `IPPROTO_ICMPV6` and `IPPROTO_ICMP` are different unnamed enums; cast to `int` to avoid -Wenum-compare warning
- ICMP ping: For SOCK_DGRAM ICMP, kernel strips IP header from received packets; first byte is ICMP header directly
- Health checking: Outbounds without `ping_target` are always considered healthy (no check needed)
- Firewall: Abstract base class in `src/firewall/firewall.hpp`; backends use factory pattern via `create_firewall("auto"|"iptables"|"nftables")`
- Firewall: Backend implementations provide `create_iptables_firewall()` / `create_nftables_firewall()` forward-declared in firewall.cpp
- Firewall (iptables): Uses single `hash:net` ipset per logical set (supports both individual IPs and CIDRs)
- Firewall: `create_ipset()` accepts optional `timeout` parameter for TTL-based entry expiration (used by dnsmasq-resolved entries)
- Firewall: `add_to_ipset()` accepts optional `entry_timeout` parameter (-1=set default, 0=permanent) for per-entry timeout override
- Firewall (iptables): Mark rules in mangle table; cleanup removes rules before ipsets (dependency order)
- Firewall (nftables): Uses `inet` family table for dual-stack; single set with `flags interval` supports both IPs and CIDRs
- Firewall (nftables): Sets with timeout use `flags interval, timeout` and `timeout Ns` syntax
- Firewall (nftables): Cleanup simply deletes the entire table (cascades to all chains, rules, sets)
- Firewall (nftables): Rule deletion by handle: `nft -a list chain` to find handle, then `nft delete rule ... handle N`
- DNS: All DNS servers must be plain IPv4 or IPv6 addresses (no DoH, system, or blocked types)
- DNS: DnsServerConfig has tag, address, detour, resolved_ip (no type field, no doh_url)
- DNS: New module directory: `src/dns/` for DNS-related code
- DNS: DnsServerRegistry takes only `DnsConfig` (no ListManager dependency) for server tag lookup and fallback resolution
- DNS: DnsServerRegistry validates all server tags in constructor (fallback + rule references) - fail-fast on config errors
- DNS: Wildcard domain matching: `*.example.com` matches both subdomains and the base domain itself
- DNS: Dnsmasq ipset directives use plain set name (single `hash:net` set per list)
- DNS: Dnsmasq `server=` directives always generated for all DNS servers (all are plain IP)
- DNS: DnsmasqGenerator uses ListStreamer (not ListManager) and takes std::ostream& for streaming output; domains batched ~50 per ipset=/server= line
- Daemon: Epoll-based event loop with signalfd for signal handling; external fds registered via add_fd/remove_fd
- Daemon: signalfd requires signals blocked via sigprocmask first; use SFD_NONBLOCK | SFD_CLOEXEC flags
- Scheduler: timerfd_create with CLOCK_MONOTONIC + TFD_NONBLOCK | TFD_CLOEXEC; registered with Daemon's epoll via add_fd
- Scheduler: One-shot timers use it_interval={0,0}; repeating timers set both it_value and it_interval
- Scheduler: Must read(fd, &uint64_t, 8) to acknowledge timer expiration, otherwise timerfd stays readable
- API: cpp-httplib Server::listen() is blocking; run in std::thread, stop via Server::stop() (thread-safe)
- API: Pimpl pattern hides httplib.h from header; meson.build conditionally adds API sources inside `if get_option('with_api')` block
- API: Handlers use ApiContext struct with non-owning refs to subsystems; register_api_handlers() wires them to ApiServer routes before start()
- main.cpp: `<sys/socket.h>` needed for AF_INET; daemonize (fork/setsid) before creating epoll/signal fds but after loading config
- Shutdown order: API server → scheduler → route_table → policy_rules → firewall → PID file
- Visitor pattern: ListEntryVisitor (header-only abstract class) in `src/lists/list_entry_visitor.hpp`; EntryType enum (Ip, Cidr, Domain); FunctionalVisitor wraps std::function; EntryCounter counts without storing
- Streaming parser: ListParser::stream_parse(istream, visitor) reads line-by-line; classify_entry(string_view, visitor) dispatches a single entry
- HTTP conditional downloads: `download_conditional()` uses `If-None-Match`/`If-Modified-Since` request headers and captures `ETag`/`Last-Modified` from response via curl header callback
- CacheManager: `src/cache/cache_manager.hpp` manages download + metadata; cache files as `<cache_dir>/<name>.txt`, metadata as `<cache_dir>/<name>.meta.json`; uses nlohmann_json for metadata serialization
- ListStreamer: `src/lists/list_streamer.hpp` streams list entries from cache files and inline config through ListEntryVisitor without storing in memory; depends on CacheManager (const ref)
- Batch pipe visitors: IpsetRestoreVisitor (`ipset restore -exist`) and NftBatchVisitor (`nft -f -`) follow identical popen/fwrite/pclose pattern; both ignore Domain entries, only process Ip and Cidr
- Daemon mode: uses CacheManager + ListStreamer (no ListManager/ParsedList); firewall ipsets populated via create_batch_loader() + stream_list() + finish()
- Signal semantics: SIGUSR1 = re-evaluate failover outbound selection (no list reload); SIGHUP = full teardown + config re-read + rebuild
- Daemon mode: uses allocate_outbound_marks() for fwmark assignment; static routing tables/ip-rules set up once via setup_static_routing(); apply_firewall() lambda builds complete firewall transactionally (cleanup + create_firewall + rebuild + set_rules)
- Urltest change handling: UrltestManager change callback sets urltest selection in FirewallState then calls apply_firewall() for transactional firewall rebuild
- SIGUSR1 semantics: clear+re-add static routing tables/ip-rules (verify they exist) + trigger immediate URL tests for all urltest outbounds
- SIGHUP semantics: full teardown (urltest_manager.clear + routes + rules + firewall) → re-read config → re-allocate marks → re-create routing → re-register urltests → apply_firewall()
- API handlers: ApiContext holds CacheManager&, const map<string, ListConfig>&, const FirewallState&, and const UrltestManager&; GET /api/status reports fwmarks, rule mappings, and urltest selections; GET /api/health reports urltest per-child state
- libcurl SO_MARK: CURLOPT_MARK not available in libcurl 8.5.0; use CURLOPT_SOCKOPTFUNCTION + setsockopt(SOL_SOCKET, SO_MARK) instead for socket mark-based policy routing
- UrltestManager: per-urltest UrltestState holds per-child CircuitBreaker instances (not shared); select_outbound() uses state() const (not is_allowed()) to avoid side effects during selection
- Daemon lifecycle: run() does startup → event loop → shutdown; all business logic (routing, firewall, urltest, API) owned by Daemon as private methods
- Daemon::full_reload(): single implementation of teardown+rebuild used by SIGHUP and API reload — avoids duplication
- ApiContext: stored as unique_ptr<ApiContext> member under #ifdef WITH_API; holds references to Daemon-owned members (stable addresses across reload)
- Routing health: RoutingHealthChecker ties together FirewallVerifier + RoutingVerifier; must be constructed after firewall_, firewall_state_, route_table_, policy_rules_, netlink_; stored as unique_ptr in Daemon
- Testing: doctest at ; -- The CXX compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for C++ include format
-- Looking for C++ include format - not found
-- Configuring incomplete, errors occurred!
See also "/home/maksimkurb/Dev/keen-pbr3/CMakeFiles/CMakeOutput.log".
See also "/home/maksimkurb/Dev/keen-pbr3/CMakeFiles/CMakeError.log". builds keen-pbr3-tests; tests include only pure-function verifiers (NOT RoutingVerifier requiring real kernel)

## Iteration Log

## 2026-02-06 - US-001
- What was implemented: Meson build system, Conan dependency file, build options, version header
- Files changed:
  - `conanfile.py` - Conan 2.x recipe with dependencies (libcurl, nlohmann_json, libnl, mbedtls, optional cpp-httplib)
  - `meson.build` - Project definition with C++20, size optimization, dependency resolution, with_api feature flag
  - `meson_options.txt` - Build options: with_api (boolean), firewall_backend (combo)
  - `include/keen-pbr3/version.hpp` - Version macros (3.0.0)
- **Learnings for future iterations:**
  - The executable target in meson.build is commented out until source files are added (US-027 will finalize it)
  - libnl requires both `libnl-3.0` and `libnl-route-3.0` pkg-config packages
  - No local build verification possible without installing all dependencies; acceptance is based on file correctness
---

## 2026-02-06 - US-002
- What was implemented: Cross-compilation Conan profiles and Meson cross-files for all target architectures
- Files changed:
  - `conan/profiles/base-embedded` - Shared embedded settings (Linux, GCC, C++20, MinSizeRel)
  - `conan/profiles/mips-be-openwrt` - MIPS big-endian OpenWRT profile
  - `conan/profiles/mips-le-openwrt` - MIPS little-endian OpenWRT profile
  - `conan/profiles/arm-openwrt` - ARM (armv7hf) OpenWRT profile
  - `conan/profiles/aarch64-openwrt` - AArch64 (armv8) OpenWRT profile
  - `conan/profiles/x86_64-openwrt` - x86_64 OpenWRT profile
  - `conan/profiles/mips-le-keenetic` - MIPS little-endian Keenetic profile
  - `meson/cross/mips-be-openwrt.ini` - MIPS BE cross-file
  - `meson/cross/mips-le-openwrt.ini` - MIPS LE OpenWRT cross-file
  - `meson/cross/arm-openwrt.ini` - ARM cross-file
  - `meson/cross/aarch64-openwrt.ini` - AArch64 cross-file
  - `meson/cross/x86_64-openwrt.ini` - x86_64 cross-file
  - `meson/cross/mips-le-keenetic.ini` - MIPS LE Keenetic cross-file
- **Learnings for future iterations:**
  - OpenWRT toolchains use musl libc, toolchain prefix follows `<arch>-openwrt-linux-musl-` pattern
  - Keenetic toolchains differ: prefix is `<arch>-linux-musl-` (no "openwrt" in the name)
  - Conan profile `include()` directive uses relative paths from the profiles directory
  - Meson cross-files referenced from Conan profiles via `tools.meson:cross_file` conf
  - MIPS little-endian uses `mipsel` prefix, MIPS big-endian uses `mips` prefix
---

## 2026-02-06 - US-003
- What was implemented: Docker build infrastructure with Dockerfile and build script
- Files changed:
  - `docker/Dockerfile.openwrt` - Ubuntu 22.04-based build image with Conan 2.x, Meson, Ninja, and build tools
  - `docker/build.sh` - Executable build script accepting architecture argument, runs Conan install → Meson setup → Meson compile
  - `.gitignore` - Added `build/` and `dist/` directories
- **Learnings for future iterations:**
  - build.sh uses `--wipe` on meson setup to handle re-runs cleanly, falling back to fresh setup
  - Architecture names match Conan profile and Meson cross-file names (e.g., `mips-le-openwrt`)
  - Output goes to `dist/<arch>/keen-pbr3`
  - Conan's MesonToolchain generates `conan_meson_native.ini` which is passed as `--native-file` to meson setup
---

## 2026-02-06 - US-004
- What was implemented: HTTP client wrapper using libcurl for downloading IP/domain lists with HTTPS support
- Files changed:
  - `src/http/http_client.hpp` - HttpClient class declaration with HttpError exception, download(), set_timeout(), set_user_agent() methods
  - `src/http/http_client.cpp` - libcurl-based implementation with write callback, error handling, HTTPS/redirect support
  - `meson.build` - Added `src/http/http_client.cpp` to sources list
- **Learnings for future iterations:**
  - Source files go in `src/<module>/` directory, headers alongside the source (not in `include/`)
  - curl_global_init is done once via static local variable in constructor
  - HttpError carries status_code for HTTP-level errors (>= 400), 0 for transport-level errors
  - The version.hpp macro `KEEN_PBR3_VERSION_STRING` is used for default user-agent via string literal concatenation
  - meson.build executable target is still commented out (will be enabled in US-027)
---

## 2026-02-06 - US-005
- What was implemented: JSON configuration parser for daemon and outbound sections
- Files changed:
  - `src/config/config.hpp` - Config struct with DaemonConfig, InterfaceOutbound, TableOutbound, BlackholeOutbound, Outbound variant, parse_config() function
  - `src/config/config.cpp` - JSON deserialization using nlohmann_json, duration string parsing, outbound type discrimination
  - `meson.build` - Added `src/config/config.cpp` to sources list
  - `.gitignore` - Added Conan-generated files (*.pc, conan*.sh, etc.)
- **Learnings for future iterations:**
  - Conan generates .pc files and shell scripts in the project root; these must be in .gitignore
  - nlohmann_json `j.contains()` + `j.at().get<T>()` pattern for optional fields with defaults
  - Duration parsing supports "s" (seconds), "m" (minutes), "h" (hours) suffixes
  - Config struct is partial in US-005 (only daemon + outbounds); US-006 will extend it with lists, route, dns, api sections
  - Local conan install: `export PATH="$HOME/.asdf/installs/python/3.12.11/bin:$PATH" && conan install . --build=missing`
---

## 2026-02-06 - US-006
- What was implemented: JSON configuration parser extended with lists, route, DNS, and API sections
- Files changed:
  - `src/config/config.hpp` - Added ListConfig, SkipAction, RouteRule, RouteConfig, DnsServer, DnsRule, DnsConfig, ApiConfig structs; extended Config with all sections
  - `src/config/config.cpp` - Added parse functions for api, lists, route rules (single/failover/skip), route config, DNS servers, DNS rules, DNS config; extended parse_config() to parse all sections
- **Learnings for future iterations:**
  - Lists are stored as `std::map<std::string, ListConfig>` keyed by list name (JSON uses object with name keys, not array)
  - Route rules use three mutually exclusive JSON fields: `outbound` (single tag string), `outbounds` (failover chain array), or `action: "skip"`
  - DNS rules and route rules both use `"list"` as the JSON field name (array of list name strings)
  - Config sections are all optional - parse_config handles missing sections gracefully with defaults
  - SkipAction is an empty struct used as a variant alternative for route rule actions
---

## 2026-02-06 - US-007
- What was implemented: IP/domain list parser for mixed text files containing IPs, CIDRs, and domains
- Files changed:
  - `src/config/list_parser.hpp` - ParsedList struct and ListParser class with static parse method and validation helpers
  - `src/config/list_parser.cpp` - Implementation with IPv4/IPv6 address validation, CIDR notation parsing, domain name detection (including wildcards)
  - `meson.build` - Added `src/config/list_parser.cpp` to sources list
- **Learnings for future iterations:**
  - IPv4 detection uses `std::from_chars` for safe integer parsing of octets
  - IPv6 detection is basic but sufficient: checks for colons, hex digits, allows `::` shorthand and mapped v4
  - CIDR detection checks for `/` then validates IP part and prefix length (0-32 for v4, 0-128 for v6)
  - Domain detection requires at least one alpha character to distinguish from pure-numeric IPv4 addresses
  - Wildcard domains (`*.example.com`) are preserved as-is in the domains vector
  - Unrecognized lines are silently skipped (no error thrown)
  - Trim helper strips spaces, tabs, and carriage returns from line edges
---

## 2026-02-06 - US-008
- What was implemented: In-memory IP/subnet set using binary trie for efficient prefix matching
- Files changed:
  - `src/lists/ipset.hpp` - IpSet and IpTrie class declarations
  - `src/lists/ipset.cpp` - Implementation with IPv4/IPv6 parsing, trie insert/contains, CIDR support
  - `meson.build` - Added `src/lists/ipset.cpp` to sources list
- **Learnings for future iterations:**
  - Binary trie (radix tree on individual bits) gives O(W) lookup where W is address width (32 for IPv4, 128 for IPv6)
  - IpTrie::insert skips remaining bits if an existing shorter prefix already covers the new one (optimization)
  - IPv6 parsing handles `::` expansion, hex groups, and IPv4-mapped addresses (`::ffff:1.2.3.4`)
  - IpSet uses separate tries for v4/v6 - `contains()` tries v4 first, falls back to v6
  - `std::from_chars` with base 16 is used for hex group parsing; base 10 for IPv4 octets and CIDR prefix lengths
  - New module directory: `src/lists/` for list-related data structures
---

## 2026-02-06 - US-009
- What was implemented: List manager for downloading remote lists, reading local files, merging inline entries, caching to disk, and reloading
- Files changed:
  - `src/lists/list_manager.hpp` - ListManager class declaration with load/reload/get methods
  - `src/lists/list_manager.cpp` - Implementation with URL download+cache, local file reading, inline entry merging
  - `meson.build` - Added `src/lists/list_manager.cpp` to sources list
- **Learnings for future iterations:**
  - ListManager takes `std::map<std::string, ListConfig>` (from Config::lists) and a cache directory path
  - Downloaded lists are cached to `<cache_dir>/<list_name>.txt` for offline startup resilience
  - On download failure, falls back to cached copy if available; otherwise propagates the exception
  - Inline ip_cidrs entries are parsed through ListParser to classify them as IPs or CIDRs
  - Inline domain entries are added directly without parsing (they're already domain strings)
  - `reload()` clears all loaded data and calls `load()` again (full re-download)
  - `get()` returns `const ParsedList*` (nullptr if not found) to avoid exceptions for missing lists
  - `std::filesystem::create_directories` is used to ensure cache dir exists before downloading
---

## 2026-02-06 - US-010
- What was implemented: Routing target type definitions with RoutingDecision struct and failover chain resolution
- Files changed:
  - `src/routing/target.hpp` - RoutingDecision struct (skip/route_to/none factories), HealthCheckFn type alias, resolve_route_action() declaration
  - `src/routing/target.cpp` - Implementation of resolve_route_action() with single/failover/skip action handling
  - `meson.build` - Added `src/routing/target.cpp` to sources list
- **Learnings for future iterations:**
  - RoutingDecision uses `std::optional<const Outbound*>` to represent resolved outbound (nullopt = no match)
  - `is_skip` flag distinguishes "skip this rule" from "no outbound found"
  - HealthCheckFn is a `std::function<bool(const std::string&)>` - when null, all outbounds are considered healthy
  - `resolve_route_action()` takes the action variant from RouteRule and resolves it against the outbound list
  - For failover chains, iterates tags in order, selects first healthy outbound (or first found if no health check)
  - Helper `get_outbound_tag()` uses `std::visit` to extract tag from any outbound variant
  - New module directory: `src/routing/` for routing-related code
---

## 2026-02-06 - US-011
- What was implemented: Netlink route and policy rule management via libnl3
- Files changed:
  - `src/routing/netlink.hpp` - NetlinkManager class, RouteSpec and RuleSpec structs, NetlinkError exception
  - `src/routing/netlink.cpp` - Implementation using libnl3 (rtnl_route, rtnl_rule, rtnl_nexthop) with RAII wrappers
  - `meson.build` - Added `src/routing/netlink.cpp` to sources list
- **Learnings for future iterations:**
  - libnl3 headers are under `/usr/include/libnl3/` - local typecheck needs `-I /usr/include/libnl3`
  - RAII wrappers (unique_ptr with custom deleter) are essential for libnl objects that need explicit free/put
  - `rtnl_route_add_nexthop()` takes ownership of the nexthop, so use `.release()` not `.get()`
  - Route and rule operations use Impl (pimpl) pattern to hide libnl types from the header
  - RouteSpec supports: interface routing, gateway routing, table routing, and blackhole routes
  - RuleSpec supports fwmark-based ip rules with configurable mask and priority
  - When family == 0 for rules, add for both AF_INET and AF_INET6
  - `nl_addr_parse()` handles both plain IPs and CIDR notation automatically
  - Route add uses `NLM_F_CREATE | NLM_F_REPLACE` to be idempotent
  - Rule add uses `NLM_F_CREATE | NLM_F_EXCL` to fail if duplicate exists
---

## 2026-02-06 - US-012
- What was implemented: Abstract route table and policy rule management classes that track installed routes/rules to avoid duplicates and enable cleanup
- Files changed:
  - `src/routing/route_table.hpp` - RouteTable class declaration with add/remove/clear/size methods
  - `src/routing/route_table.cpp` - Implementation with duplicate tracking via vector comparison, reverse-order cleanup
  - `src/routing/policy_rule.hpp` - PolicyRuleManager class declaration with add/remove/clear/size methods
  - `src/routing/policy_rule.cpp` - Implementation with duplicate tracking via vector comparison, reverse-order cleanup
  - `meson.build` - Added both new .cpp files to sources list
- **Learnings for future iterations:**
  - RouteTable and PolicyRuleManager follow the same pattern: track installed specs in a vector, compare equality by value, skip duplicates, remove in reverse order during cleanup
  - Both classes take `NetlinkManager&` by reference (non-owning) - the caller must ensure NetlinkManager outlives these managers
  - Destructors do best-effort cleanup (catch all exceptions) to ensure no-throw guarantee
  - RouteSpec equality compares all fields: destination, table, interface, gateway, blackhole, family
  - RuleSpec equality compares all fields: fwmark, fwmask, table, priority, family
  - The `clear()` method removes routes/rules in reverse order (LIFO) to properly unwind dependencies
---

## 2026-02-06 - US-013
- What was implemented: Health checker with ICMP ping-based monitoring for interface outbounds
- Files changed:
  - `src/health/health_checker.hpp` - HealthChecker class declaration with HealthStatus enum, HealthResult struct, PingTarget internal struct
  - `src/health/health_checker.cpp` - Implementation using raw ICMP sockets (SOCK_DGRAM), SO_BINDTODEVICE for interface binding, poll() for timeout
  - `meson.build` - Added `src/health/health_checker.cpp` to sources list
- **Learnings for future iterations:**
  - SOCK_DGRAM ICMP sockets (instead of SOCK_RAW) work on Linux without CAP_NET_RAW on many kernels (ping_group_range sysctl)
  - ICMPv6 checksums are computed by the kernel; only IPv4 ICMP needs manual checksum computation
  - SO_BINDTODEVICE requires the interface name including null terminator (pass `.size() + 1`)
  - poll() is preferred over select() for single-fd timeout waiting
  - `IPPROTO_ICMPV6` and `IPPROTO_ICMP` need explicit `static_cast<int>()` in ternary to avoid `-Wenum-compare` warning
  - New module directory: `src/health/` for health check and circuit breaker code
---

## 2026-02-06 - US-014
- What was implemented: Circuit breaker for failover stability with three states (closed/open/half-open)
- Files changed:
  - `src/health/circuit_breaker.hpp` - CircuitBreaker class declaration with CircuitState enum, CircuitBreakerEntry struct, record_success/failure, is_allowed, state, failure_count, reset methods
  - `src/health/circuit_breaker.cpp` - Implementation with configurable failure threshold and cooldown period, state machine transitions
  - `meson.build` - Added `src/health/circuit_breaker.cpp` to sources list
  - `prd.json` - Marked US-014 as passes: true
- **Learnings for future iterations:**
  - Circuit breaker uses `std::chrono::steady_clock` (monotonic) for timing, not `system_clock` which can be adjusted
  - `entries_[tag]` default-constructs CircuitBreakerEntry with closed state and zero failures (via `get_or_create`)
  - State transitions: closed->open (after N failures), open->half_open (after cooldown), half_open->closed (on success), half_open->open (on failure)
  - `is_allowed()` has side effect: transitions open->half_open when cooldown expires (caller doesn't need separate transition call)
  - No dependencies on other modules (config, health_checker) - circuit breaker is a standalone state machine keyed by string tags
---

## 2026-02-06 - US-015
- What was implemented: Abstract firewall interface with pure virtual base class and factory function for backend detection
- Files changed:
  - `src/firewall/firewall.hpp` - Abstract Firewall base class with virtual methods (create_ipset, add_to_ipset, delete_ipset, create_mark_rule, delete_mark_rule, apply, cleanup), FirewallError exception, FirewallBackend enum, detect_firewall_backend() and create_firewall() factory functions
  - `src/firewall/firewall.cpp` - Implementation of detect_firewall_backend() (checks nft first, then iptables) and create_firewall() factory (auto/iptables/nftables backend preference)
  - `meson.build` - Added `src/firewall/firewall.cpp` to sources list
- **Learnings for future iterations:**
  - New module directory: `src/firewall/` for firewall-related code
  - Backend detection uses `command -v` via `std::system()` to check for nft/iptables availability
  - Factory function `create_firewall()` uses forward-declared `create_iptables_firewall()` and `create_nftables_firewall()` - these will be implemented in US-016 and US-017 respectively
  - Firewall interface is non-copyable (deleted copy ctor/assignment) with protected default constructor
  - `create_mark_rule` and `delete_mark_rule` have default chain parameter "PREROUTING" for mangle table marking
  - `create_ipset` takes `int family` (AF_INET or AF_INET6) to support both IPv4 and IPv6 sets
---

## 2026-02-06 - US-016
- What was implemented: iptables/ipset firewall backend implementing the abstract Firewall interface
- Files changed:
  - `src/firewall/iptables.hpp` - IptablesFirewall class declaration implementing Firewall, with MarkRule tracking struct
  - `src/firewall/iptables.cpp` - Full implementation using `ipset` and `iptables` CLI commands: create/add/delete ipsets, create/delete mangle mark rules, apply (no-op, commands are immediate), cleanup (reverse-order teardown)
  - `meson.build` - Added `src/firewall/iptables.cpp` to sources list
- **Learnings for future iterations:**
  - iptables/ipset uses single `hash:net` set per logical set (supports both individual IPs and CIDRs); optional `timeout` parameter enables TTL for dnsmasq-resolved entries
  - ipset `-exist` flag makes create/add idempotent (no error if already exists)
  - iptables mark rules go in the mangle table (`-t mangle`), using `-m set --match-set` to match against ipsets
  - Mark value formatted as hex (`0x...`) for consistency with routing fwmark values
  - `apply()` is a no-op for iptables backend since commands execute immediately; could be enhanced to use `ipset restore` for batched atomicity
  - Cleanup removes mark rules in reverse order (LIFO) before destroying ipsets, since iptables rules reference the ipset names
  - `exec_cmd` uses `std::system()` for shell command execution; `exec_cmd_checked` throws FirewallError on non-zero exit
  - IPv6 sets use `ip6tables` command instead of `iptables`; family is tracked per set name in `created_sets_`
---

## 2026-02-06 - US-017
- What was implemented: nftables firewall backend implementing the abstract Firewall interface using `nft` CLI commands
- Files changed:
  - `src/firewall/nftables.hpp` - NftablesFirewall class declaration implementing Firewall, with MarkRule tracking struct, table/chain creation tracking
  - `src/firewall/nftables.cpp` - Full implementation using `nft` CLI: create sets (flags interval), add elements, mark rules in prerouting chain, handle-based rule deletion, table-level cleanup
  - `meson.build` - Added `src/firewall/nftables.cpp` to sources list
- **Learnings for future iterations:**
  - nftables `inet` family is dual-stack (IPv4+IPv6 in one table) - simpler than iptables/ip6tables split
  - nft sets with `flags interval` support both individual IPs and CIDR prefixes in a single set; `flags interval, timeout` enables TTL for dnsmasq-resolved entries
  - nft rule deletion requires finding the rule handle first: `nft -a list chain` + grep + `nft delete rule ... handle N`
  - nft cleanup is simpler than iptables: `nft delete table inet <name>` removes everything (chains, rules, sets) in one command
  - nft chain creation for prerouting hook uses `type filter hook prerouting priority mangle` to match iptables mangle table behavior
  - `nft add` commands are idempotent for tables and chains (no error if already exists)
---

## 2026-02-06 - US-018
- What was implemented: DNS server type abstraction with address parsing and validation
- Files changed:
  - `src/dns/dns_server.hpp` - DnsServerType enum (PlainIP/DoH/System/Blocked), DnsServerConfig struct with parsed fields, DnsError exception, parse/validate functions
  - `src/dns/dns_server.cpp` - Implementation of address type detection (IPv4/IPv6 validation, DoH URL detection, system/rcode parsing), DnsServerConfig construction
  - `meson.build` - Added `src/dns/dns_server.cpp` to sources list
- **Learnings for future iterations:**
  - DnsServerType classifies addresses into 4 categories: PlainIP (raw IP), DoH (https:// URL), System ("system"), Blocked ("rcode://refused")
  - DnsServerConfig enriches the config-level DnsServer with a parsed type and extracted type-specific fields (resolved_ip for PlainIP, doh_url for DoH)
  - IPv4 validation uses `std::from_chars` for safe octet parsing (same pattern as list_parser)
  - IPv6 validation checks for colons, valid hex characters, and at most one `::` (basic but sufficient)
  - DoH detection is a simple `https://` prefix check
  - New module directory: `src/dns/` for all DNS-related code
---

## 2026-02-06 - US-019
- What was implemented: DNS rule matching and router that resolves domain names to DNS server configs based on configured rules and list membership
- Files changed:
  - `src/dns/dns_router.hpp` - DnsRouter class declaration with resolve(), get_server(), fallback() methods
  - `src/dns/dns_router.cpp` - Implementation with rule matching against ListManager domains, wildcard pattern support, config validation
  - `meson.build` - Added `src/dns/dns_router.cpp` to sources list
- **Learnings for future iterations:**
  - DnsRouter takes `const ListManager&` by reference - caller must ensure ListManager outlives DnsRouter (same pattern as RouteTable/PolicyRuleManager with NetlinkManager)
  - Constructor validates all server tag references (fallback + rules) and throws DnsError if any tag is missing - fail-fast on bad config
  - Wildcard pattern `*.example.com` matches both subdomains (like `sub.example.com`) and the base domain itself (`example.com`)
  - Domain matching only checks against `ParsedList::domains` vector (not IPs/CIDRs) since DNS routing is domain-based
  - Rule matching processes rules in config order with first-match-wins semantics, same as route rules
---

## 2026-02-06 - US-020
- What was implemented: Dnsmasq config file generator that produces ipset= and server= directives for domain-based routing
- Files changed:
  - `src/dns/dnsmasq_gen.hpp` - DnsmasqGenerator class declaration with generate() and write() methods
  - `src/dns/dnsmasq_gen.cpp` - Implementation generating dnsmasq directives from route rules, DNS rules, and loaded domain lists
  - `meson.build` - Added `src/dns/dnsmasq_gen.cpp` to sources list
- **Learnings for future iterations:**
  - DnsmasqGenerator takes const refs to DnsRouter, ListManager, RouteConfig, and DnsConfig - same non-owning reference pattern as other components
  - Ipset directives use the format `ipset=/domain/setname` (single `hash:net` set per list)
  - Only PlainIP DNS servers can be used in dnsmasq `server=` directives (DoH/system/blocked types are not supported by dnsmasq)
  - Wildcard domains (`*.example.com`) are stripped to base domain (`example.com`) for dnsmasq directives since dnsmasq inherently matches subdomains
  - Route rules with SkipAction are excluded from ipset generation (no routing needed)
  - Deduplication of domains per list prevents duplicate directives in generated config
  - `write()` creates parent directories if they don't exist (consistent with ListManager's cache directory pattern)
---

## 2026-02-06 - US-021
- What was implemented: Daemon event loop with epoll-based signal handling for SIGUSR1, SIGTERM, and SIGINT
- Files changed:
  - `src/daemon/daemon.hpp` - Daemon class declaration with run/stop, on_sigusr1, add_fd/remove_fd methods, FdCallback type alias
  - `src/daemon/daemon.cpp` - Epoll-based event loop with signalfd for signal handling, fd registration/dispatch
  - `meson.build` - Added `src/daemon/daemon.cpp` to sources list
- **Learnings for future iterations:**
  - New module directory: `src/daemon/` for daemon event loop and scheduler code
  - signalfd requires signals to be blocked via sigprocmask first; signals are then read from the fd
  - SFD_NONBLOCK and SFD_CLOEXEC flags used for signalfd; EPOLL_CLOEXEC for epoll_create1
  - Daemon destructor restores signal disposition by unblocking the masked signals
  - epoll_wait with timeout -1 blocks indefinitely; EINTR is handled by continuing the loop
  - FdCallback receives uint32_t epoll events so callers can distinguish EPOLLIN/EPOLLOUT/EPOLLERR
  - add_fd/remove_fd allow external components (scheduler timerfds, API server socket) to integrate with the event loop
---

## 2026-02-06 - US-022
- What was implemented: Periodic task scheduler using timerfd, integrated with Daemon epoll event loop
- Files changed:
  - `src/daemon/scheduler.hpp` - Scheduler class declaration with schedule_repeating(), schedule_oneshot(), cancel(), cancel_all() methods
  - `src/daemon/scheduler.cpp` - Implementation using timerfd_create/timerfd_settime with CLOCK_MONOTONIC, epoll integration via Daemon::add_fd/remove_fd
  - `meson.build` - Added `src/daemon/scheduler.cpp` to sources list
- **Learnings for future iterations:**
  - timerfd_create with CLOCK_MONOTONIC avoids issues with system clock adjustments (same reasoning as steady_clock for circuit breaker)
  - TFD_NONBLOCK | TFD_CLOEXEC flags match the pattern used by signalfd (SFD_NONBLOCK | SFD_CLOEXEC)
  - itimerspec.it_interval of {0,0} makes timerfd one-shot; non-zero makes it repeat automatically
  - Must read exactly sizeof(uint64_t) from timerfd to acknowledge expiration and re-arm for next interval
  - One-shot timers auto-remove from the scheduler after callback fires; the entry is copied before removal to avoid use-after-free
  - Scheduler takes Daemon& (non-owning ref) - same pattern as RouteTable/PolicyRuleManager with NetlinkManager
---

## 2026-02-06 - US-023
- What was implemented: REST API server using cpp-httplib, wrapped in #ifdef WITH_API guards, running in a background thread
- Files changed:
  - `src/api/server.hpp` - ApiServer class declaration with get/post/start/stop methods, pimpl pattern hiding httplib types
  - `src/api/server.cpp` - Implementation using httplib::Server in a background thread, address parsing from ApiConfig
  - `meson.build` - Conditionally adds `src/api/server.cpp` to sources when with_api option is true
- **Learnings for future iterations:**
  - cpp-httplib's Server::listen() is blocking; must run in a separate std::thread
  - Server::stop() is thread-safe and can be called from the main thread to terminate the listener
  - Server::is_running() provides a way to check if the server has started listening (spin-wait in start())
  - Pimpl pattern hides httplib.h from the header, keeping compile times down and hiding the library dependency
  - ApiConfig.listen is "host:port" format; parsed with rfind(':') to handle IPv6 hosts correctly
  - New module directory: `src/api/` for REST API server and handler code
  - meson.build conditionally adds API sources inside `if get_option('with_api')` block (source files also have #ifdef guards as defense-in-depth)
---

## 2026-02-06 - US-024
- What was implemented: REST API endpoint handlers for status, reload, and health check
- Files changed:
  - `src/api/handlers.hpp` - Handler declarations, ApiContext struct holding refs to subsystems (outbounds, ListManager, HealthChecker, reload callback)
  - `src/api/handlers.cpp` - Implementation of GET /api/status (version, outbounds, loaded list stats), POST /api/reload (triggers reload callback), GET /api/health (per-outbound health status)
  - `meson.build` - Added `src/api/handlers.cpp` to conditional API sources
- **Learnings for future iterations:**
  - ApiContext uses non-owning const refs for read-only data (outbounds, ListManager, HealthChecker) and std::function for the mutable reload action
  - register_api_handlers() takes ApiServer& and ApiContext& - called before ApiServer::start() to register routes
  - Outbound tag/type extraction uses std::visit with generic lambda and std::decay_t + std::is_same_v for variant dispatch
  - List stats in /api/status report counts (ips, cidrs, domains) per list name, not full list contents (avoids large responses)
  - Unmonitored outbounds (no ping_target) report as "healthy" in /api/health since they're always considered healthy
  - nlohmann_json included via `<nlohmann/json.hpp>` (not direct path) - resolved through pkg-config
---

## 2026-02-06 - US-025
- What was implemented: Main daemon entry point (main.cpp) that parses CLI flags, loads config, initializes all subsystems, and runs the daemon
- Files changed:
  - `src/main.cpp` - Complete daemon entry point: CLI parsing (--config, -d, --no-api, --version, --help), config loading, subsystem initialization (ListManager, HealthChecker, CircuitBreaker, Firewall, NetlinkManager, RouteTable, PolicyRuleManager, DnsRouter, DnsmasqGenerator, Scheduler, optional ApiServer), list download & apply, health check scheduling, periodic list update, SIGUSR1 reload handler, graceful shutdown with cleanup
  - `meson.build` - Added `src/main.cpp` to sources list
- **Learnings for future iterations:**
  - `<sys/socket.h>` must be included for AF_INET constant in main.cpp (not transitively included from other headers)
  - Daemonization (fork + setsid + redirect stdio) must happen before creating file descriptors (epoll, signalfd, timerfd) but after loading config
  - Firewall marks start at 0x10000 and increment per route rule; routing table IDs are derived as 100 + (fwmark & 0xFFFF)
  - Policy rule priority matches table ID for simplicity (100 + offset)
  - Health check scheduling: one repeating timer per interface outbound with ping_target, using the outbound's own ping_interval
  - SIGUSR1 callback and API reload both use the same reload_fn lambda for consistency
  - Shutdown order matters: stop API server first, cancel scheduled tasks, clear routes/rules, cleanup firewall, remove PID file
  - main.cpp compiles with and without WITH_API flag (verified both paths)
---

## 2026-02-06 - US-026
- What was implemented: GitHub Actions CI/CD workflow with matrix builds for all 6 target architectures
- Files changed:
  - `.github/workflows/build.yml` - CI workflow: matrix build (mips-be-openwrt, mips-le-openwrt, arm-openwrt, aarch64-openwrt, x86_64-openwrt, mips-le-keenetic), Docker-based builds via docker/build.sh, artifact upload per architecture
- **Learnings for future iterations:**
  - Workflow uses `fail-fast: false` in matrix strategy so one arch failure doesn't cancel others
  - Docker image is built per-job (each arch); could be optimized with a separate build-image job + caching
  - Artifact names include architecture suffix for unique identification: `keen-pbr3-<arch>`
  - The dist directory is bind-mounted (`-v`) so artifacts survive container removal
  - `if-no-files-found: error` in upload-artifact ensures build failures are caught
---

## 2026-02-06 - US-027
- What was implemented: Final meson.build integration - uncommented executable target, added static linking for cross-builds, verified all 22 source files are included
- Files changed:
  - `meson.build` - Uncommented executable target `keen-pbr3`, added `meson.is_cross_build()` guard for static linking (`-static`), updated comment from "will be added" to final form
- **Learnings for future iterations:**
  - `meson.is_cross_build()` is the proper way to conditionally add flags only for cross-compilation targets
  - Static linking via `-static` in link arguments ensures the binary is self-contained for embedded deployments
  - All 22 source files (20 core + 2 conditional API) compile cleanly with `-fsyntax-only` and all dependency flags
  - The project is now fully buildable: `conan install . --build=missing && meson setup build && meson compile -C build`
---

## 2026-02-09 - US-028
- What was implemented: ListEntryVisitor streaming interface with EntryType enum, abstract ListEntryVisitor class, FunctionalVisitor convenience wrapper, EntryCounter visitor, and streaming ListParser methods (stream_parse, classify_entry)
- Files changed:
  - `src/lists/list_entry_visitor.hpp` - New file: EntryType enum (Ip, Cidr, Domain), abstract ListEntryVisitor with on_entry/on_list_complete/finish virtual methods, FunctionalVisitor wrapping std::function, EntryCounter counting entries by type
  - `src/config/list_parser.hpp` - Added stream_parse(std::istream&, ListEntryVisitor&) and classify_entry(std::string_view, ListEntryVisitor&) static methods; added #include for istream and list_entry_visitor.hpp
  - `src/config/list_parser.cpp` - Implemented classify_entry() reusing existing is_ipv4/is_cidr_v4/is_ipv6/is_cidr_v6/is_domain methods, and stream_parse() reading line-by-line with std::getline
- **Learnings for future iterations:**
  - ListEntryVisitor is header-only (no .cpp needed) since it's an abstract interface + small inline implementations
  - classify_entry() returns bool indicating whether the entry was recognized (useful for callers that want to log unrecognized entries)
  - stream_parse() reuses the existing trim() helper from list_parser.cpp (file-local static function)
  - ParsedList struct and parse() method remain for backward compatibility until US-043 removes them
  - The relative include `../lists/list_entry_visitor.hpp` from config/ to lists/ works because meson.build doesn't add src/ to include path (files use relative includes between modules)
---

## 2026-02-09 - US-029
- What was implemented: Added configurable `cache_dir` to DaemonConfig (default `/var/cache/keen-pbr3`) and removed `list_update_interval` field since downloading is now a separate command
- Files changed:
  - `src/config/config.hpp` - Replaced `list_update_interval` with `cache_dir` in DaemonConfig
  - `src/config/config.cpp` - Updated `parse_daemon()` to parse `cache_dir` instead of `list_update_interval`
  - `src/main.cpp` - Replaced hardcoded `/var/cache/keen-pbr3` with `config.daemon.cache_dir` in both print-dnsmasq-config and daemon mode; removed periodic list update scheduler
  - `config.example.json` - Replaced `list_update_interval` with `cache_dir`
- **Learnings for future iterations:**
  - `parse_duration()` helper is still used by outbound ping_interval/ping_timeout and list TTL parsing, so it remains in config.cpp
  - The `<chrono>` include in config.hpp is still needed by InterfaceOutbound's ping_interval/ping_timeout fields
  - Typecheck uses build dir for pkg-config: `export PKG_CONFIG_PATH=.../build`
---

## 2026-02-09 - US-030
- What was implemented: Conditional HTTP downloads using ETag and If-Modified-Since headers to avoid re-downloading unchanged lists
- Files changed:
  - `src/http/http_client.hpp` - Added `ConditionalDownloadResult` struct (not_modified, body, etag, last_modified) and `download_conditional()` method declaration
  - `src/http/http_client.cpp` - Added `HeaderCapture` struct, `trim_header_value()` helper, `header_callback()` for capturing ETag/Last-Modified response headers, and `download_conditional()` implementation using `curl_slist_append` for request headers
- **Learnings for future iterations:**
  - curl header callback receives one header per call including CRLF terminator; must trim whitespace from values
  - `curl_slist_append` returns a new list head (or same if appending); must free with `curl_slist_free_all` after perform
  - HTTP 304 Not Modified still delivers response headers, so ETag/Last-Modified can be captured even on 304
  - Header names are case-insensitive per HTTP spec; use manual case-insensitive prefix matching
  - `<algorithm>` include added for `std::tolower` (used in case-insensitive header matching)
---

## 2026-02-09 - US-031
- What was implemented: CacheManager with metadata storage for downloading lists to cache with ETag/If-Modified-Since conditional request support
- Files changed:
  - `src/cache/cache_manager.hpp` - CacheManager class with CacheMetadata struct (etag, last_modified, url, download_time, optional counts), methods: ensure_dir(), download(), has_cache(), cache_path(), meta_path(), load_metadata(), save_metadata()
  - `src/cache/cache_manager.cpp` - Implementation using HttpClient::download_conditional() with stored ETag/Last-Modified from previous .meta.json, nlohmann_json for metadata serialization
  - `meson.build` - Added `src/cache/cache_manager.cpp` to sources list
- **Learnings for future iterations:**
  - New module directory: `src/cache/` for cache-related code
  - CacheManager::download() returns bool: true = content updated, false = 304 Not Modified
  - On download failure (exception), existing cache file is preserved (no overwrite)
  - CacheMetadata counts (ips, cidrs, domains) are std::optional<size_t> - left as nullopt after download, caller updates via save_metadata() after counting entries
  - Metadata JSON is written with 2-space indent for readability
  - `gmtime_r` used for thread-safe ISO 8601 timestamp generation
---

## 2026-02-09 - US-032
- What was implemented: ListStreamer for streaming list entries from cache files and inline config through a visitor without storing them in memory
- Files changed:
  - `src/lists/list_streamer.hpp` - ListStreamer class declaration with stream_list(), stream_cache(), and stream_file() methods; depends on CacheManager (const ref) and ListConfig
  - `src/lists/list_streamer.cpp` - Implementation: stream_list() sequentially streams cached URL file, local file, inline ip_cidrs via classify_entry(), inline domains as Domain entries; calls on_list_complete() after all sources
  - `meson.build` - Added `src/lists/list_streamer.cpp` to sources list
- **Learnings for future iterations:**
  - ListStreamer takes `const CacheManager&` (non-owning ref) - same pattern as other components
  - stream_list() checks `cache_.has_cache(name)` before attempting to stream cache file (graceful if not yet downloaded)
  - stream_file() is a private static helper that opens ifstream and calls ListParser::stream_parse()
  - Inline ip_cidrs use ListParser::classify_entry() to correctly dispatch as Ip or Cidr types
  - Inline domains are dispatched directly as EntryType::Domain without reclassification
  - No entries are stored in memory - everything flows through the visitor pattern
---

## 2026-02-09 - US-033
- What was implemented: IpsetRestoreVisitor that pipes IP/CIDR entries to 'ipset restore -exist' via popen for batch loading
- Files changed:
  - `src/firewall/ipset_restore_pipe.hpp` - IpsetRestoreVisitor class extending ListEntryVisitor with set_name, static_timeout constructor params, on_entry(), finish(), count() methods
  - `src/firewall/ipset_restore_pipe.cpp` - Implementation: popen("ipset restore -exist", "w"), writes 'add <setname> <entry> [timeout N]\n' per Ip/Cidr entry, pclose() with error checking in finish()
  - `meson.build` - Added `src/firewall/ipset_restore_pipe.cpp` to sources list
- **Learnings for future iterations:**
  - IpsetRestoreVisitor owns a FILE* from popen; finish() must be called to close the pipe and check exit status
  - Destructor calls finish() if not already called (best-effort, exceptions caught)
  - Domain entries are silently ignored (only Ip and Cidr types are written to ipset)
  - Per-entry timeout is appended only when static_timeout >= 0 (-1 means use set default)
  - fwrite used instead of fprintf for efficiency with pre-built string lines
---

## 2026-02-09 - US-034
- What was implemented: NftBatchVisitor that pipes nftables element additions to 'nft -f -' via popen for batch loading
- Files changed:
  - `src/firewall/nft_batch_pipe.hpp` - NftBatchVisitor class extending ListEntryVisitor with set_name, static_timeout constructor params, on_entry(), finish(), count() methods
  - `src/firewall/nft_batch_pipe.cpp` - Implementation: popen("nft -f -", "w"), writes 'add element inet keen_pbr3 <setname> { <entry> [timeout Ns] }\n' per Ip/Cidr entry, pclose() with error checking in finish()
  - `meson.build` - Added `src/firewall/nft_batch_pipe.cpp` to sources list
- **Learnings for future iterations:**
  - NftBatchVisitor follows identical pattern to IpsetRestoreVisitor (popen/fwrite/pclose lifecycle)
  - nft batch format: `add element inet keen_pbr3 <setname> { <entry> [timeout Ns] }` - note curly braces and `s` suffix on timeout
  - nft `-f -` flag reads commands from stdin, enabling pipe-based batch loading
  - Domain entries are silently ignored (only Ip and Cidr types written to nft sets)
---

## 2026-02-09 - US-035
- What was implemented: Added create_batch_loader() and flush_ipset() virtual methods to Firewall interface, implemented by both IptablesFirewall and NftablesFirewall backends
- Files changed:
  - `src/firewall/firewall.hpp` - Added forward declaration of ListEntryVisitor, added pure virtual create_batch_loader() and flush_ipset() methods
  - `src/firewall/iptables.hpp` - Added override declarations for create_batch_loader() and flush_ipset()
  - `src/firewall/iptables.cpp` - Implemented create_batch_loader() returning IpsetRestoreVisitor, flush_ipset() executing 'ipset flush <name>'
  - `src/firewall/nftables.hpp` - Added override declarations for create_batch_loader() and flush_ipset()
  - `src/firewall/nftables.cpp` - Implemented create_batch_loader() returning NftBatchVisitor, flush_ipset() executing 'nft flush set inet keen_pbr3 <name>'
- **Learnings for future iterations:**
  - Firewall::create_batch_loader() uses forward-declared ListEntryVisitor to avoid circular header dependency (firewall.hpp doesn't include list_entry_visitor.hpp)
  - IptablesFirewall::flush_ipset() uses checked exec ('ipset flush') since the set should exist; nftables uses same approach with 'nft flush set'
  - The batch loader pattern decouples streaming from the firewall backend: callers use the ListEntryVisitor interface and don't need to know which backend is in use
---

## 2026-02-10 - US-036
- What was implemented: Simplified DnsRouter to DnsServerRegistry by removing domain matching and ListManager dependency, keeping only server tag lookup and fallback resolution
- Files changed:
  - `src/dns/dns_router.hpp` - Renamed DnsRouter to DnsServerRegistry, removed resolve/domain_matches_list/domain_matches_pattern methods, removed list_manager_ and rules_ members, constructor takes only DnsConfig
  - `src/dns/dns_router.cpp` - Simplified implementation with only constructor, get_server(), and fallback() methods
  - `src/dns/dnsmasq_gen.hpp` - Updated to use DnsServerRegistry instead of DnsRouter
  - `src/dns/dnsmasq_gen.cpp` - Updated constructor and get_server() call to use dns_registry_
  - `src/main.cpp` - Updated print-dnsmasq-config path to construct DnsServerRegistry(config.dns) instead of DnsRouter(config.dns, list_manager)
- **Learnings for future iterations:**
  - DnsServerRegistry is a pure lookup class with no domain matching logic - domain matching will be handled differently in the streaming architecture
  - The ListManager #include was removed from dns_router.hpp, reducing header dependencies
  - DnsmasqGenerator still uses ListManager for now (will be switched to ListStreamer in US-037)
---

## 2026-02-10 - US-037
- What was implemented: Rewrote DnsmasqGenerator to use streaming architecture with ListStreamer and DnsServerRegistry, batched domain output (~50 per directive line)
- Files changed:
  - `src/dns/dnsmasq_gen.hpp` - Replaced ListManager dependency with ListStreamer; generate() now takes std::ostream& parameter; added lists_ map member for ListConfig lookup
  - `src/dns/dnsmasq_gen.cpp` - Rewrote generate() to use FunctionalVisitor for domain collection via ListStreamer::stream_list(); domains collected into std::set for dedup then output in batches of ~50; dedup set cleared between lists
  - `src/main.cpp` - Updated print-dnsmasq-config path to use CacheManager + ListStreamer + new DnsmasqGenerator API; added cache/cache_manager.hpp and lists/list_streamer.hpp includes
- **Learnings for future iterations:**
  - DnsmasqGenerator now takes ListStreamer& (non-const) because stream_list() is not const-qualified
  - Batching uses BATCH_SIZE constant (50) to split domains into groups per ipset=/server= line
  - The print-dnsmasq-config path in main.cpp now downloads lists only if not already cached (cache.has_cache check), deferring full streaming download support to US-040
  - FunctionalVisitor from list_entry_visitor.hpp is convenient for ad-hoc domain collection without needing a custom visitor class
---

## 2026-02-10 - US-038
- What was implemented: Added SIGHUP handler to daemon event loop for full config reload support
- Files changed:
  - `src/daemon/daemon.hpp` - Added on_sighup() method and sighup_cb_ member
  - `src/daemon/daemon.cpp` - Added SIGHUP to signalfd signal mask, handle_signal() dispatch, on_sighup() implementation, destructor signal unblock
- **Learnings for future iterations:**
  - SIGHUP follows the exact same pattern as SIGUSR1: blocked via sigprocmask, received via signalfd, dispatched to callback
  - Adding a new signal to the daemon requires 4 changes: sigprocmask block, signalfd mask, handle_signal() case, destructor unblock
  - The destructor must unblock ALL signals that were blocked in setup_signals(), including newly added ones
---

## 2026-02-10 - US-039
- What was implemented: 'keen-pbr3 download' CLI command that downloads all configured lists to cache with ETag/If-Modified-Since support, counts entries, and updates metadata
- Files changed:
  - `src/main.cpp` - Added `download_lists` field to CliOptions, `download` command parsing, download command handler: CacheManager creation, per-list download with EntryCounter for counting, metadata update, stderr status output
- **Learnings for future iterations:**
  - CLI commands are parsed as positional arguments (not --flags), matching the existing `print-dnsmasq-config` pattern
  - Lists without a URL are skipped with a status message (they only have inline entries or local files)
  - Download errors per-list are caught individually so one failed list doesn't abort others
  - EntryCounter + ListStreamer::stream_list() counts all entries (cache + inline + file), not just the downloaded URL content
---

## 2026-02-10 - US-040
- What was implemented: Verified that print-dnsmasq-config already uses streaming architecture (CacheManager, ListStreamer, DnsServerRegistry, streaming DnsmasqGenerator with std::ostream&) - this was implemented as part of US-037
- Files changed: (no code changes needed - only PRD/progress updates)
  - `prd.json` - Marked US-040 as passes: true
- **Learnings for future iterations:**
  - US-037 implemented both the DnsmasqGenerator rewrite AND the print-dnsmasq-config main.cpp path update in one story, effectively completing US-040's acceptance criteria early
  - The print-dnsmasq-config path in main.cpp (lines 194-210) does not reference ParsedList or ListManager at all - those are only used in the daemon mode path
  - When stories overlap, verify acceptance criteria independently rather than assuming incomplete
---

## 2026-02-10 - US-041 + US-042
- What was implemented: Rewrote daemon mode to use streaming firewall population via batch loaders, new SIGUSR1/SIGHUP signal semantics, removed periodic list update scheduler and all ListManager/ParsedList usage. Also updated API handlers to use CacheManager metadata instead of ListManager.
- Files changed:
  - `src/main.cpp` - Replaced ListManager with CacheManager + ListStreamer in daemon mode; firewall ipsets populated via create_batch_loader() + stream_list() + finish(); SIGUSR1 re-evaluates failover outbound selection via resolve_route_action(); SIGHUP does full teardown + config re-read + rebuild; periodic list update scheduler removed; health check scheduler retained; API context uses CacheManager
  - `src/api/handlers.hpp` - ApiContext now holds CacheManager& and const map<string, ListConfig>& instead of ListManager&; removed list_manager.hpp include
  - `src/api/handlers.cpp` - GET /api/status reads entry counts from CacheManager::load_metadata() per list plus inline config sizes; POST /api/reload triggers SIGHUP-like full reload
- **Learnings for future iterations:**
  - When US stories are tightly coupled (US-041 daemon rewrite + US-042 API handler update), they may need to be implemented together to maintain compilation
  - SIGUSR1 handler uses `rule_outbound_tags` vector to track per-rule resolved outbound tag for detecting changes during failover re-evaluation
  - SIGHUP handler re-creates the firewall backend via create_firewall("auto") after cleanup() to get a fresh state
  - The apply_all() lambda captures all subsystem references and encapsulates the full firewall/routing setup, making it reusable for initial setup, SIGHUP reload, and API reload
  - List TTL determination in streaming mode uses config-level `has_domains` heuristic (checks inline domains, URL presence, file presence) rather than parsed content
---

## 2026-02-11 - US-043
- What was implemented: Removed now-unused ListManager class and ParsedList struct, updated meson.build and includes
- Files changed:
  - `src/lists/list_manager.hpp` - Deleted
  - `src/lists/list_manager.cpp` - Deleted
  - `src/config/list_parser.hpp` - Removed ParsedList struct and parse() method declaration; removed unused `<string>` and `<vector>` includes
  - `src/config/list_parser.cpp` - Removed parse() implementation and `<sstream>` include
  - `src/main.cpp` - Removed unused `#include "config/list_parser.hpp"`
  - `meson.build` - Removed `src/lists/list_manager.cpp` from sources list
- **Learnings for future iterations:**
  - After streaming architecture migration, the only ListParser methods still in use are `stream_parse()` and `classify_entry()` - both are used by ListStreamer
  - main.cpp included list_parser.hpp but didn't use it directly (EntryCounter comes from list_entry_visitor.hpp, streaming from ListStreamer)
  - Grep for all references before deletion ensures nothing is missed; all ParsedList/ListManager usage was confined to the deleted files themselves
---

## 2026-02-11 - US-044
- What was implemented: Added FwmarkConfig and IprouteConfig structs, JSON parsing for fwmark/iproute config sections, fwmark mask validation (contiguous nibble pairs), and allocate_outbound_marks() helper to assign sequential fwmarks to interface/table outbounds
- Files changed:
  - `src/config/config.hpp` - Added FwmarkConfig (start, mask), IprouteConfig (table_start), OutboundMarkMap type alias, allocate_outbound_marks() declaration; extended Config struct with fwmark and iproute fields
  - `src/config/config.cpp` - Added parse_fwmark(), parse_iproute(), validate_fwmark_mask() (checks exactly two adjacent hex nibbles set to F with nibble alignment), allocate_outbound_marks() implementation; added `<iomanip>` and `<sstream>` includes
  - `config.example.json` - Added fwmark section (start: 65536, mask: 16711680) and iproute section (table_start: 150)
  - `prd.json` - Marked US-044 as passes: true
- **Learnings for future iterations:**
  - Fwmark mask validation: isolate lowest set bit via `mask & (~mask + 1)`, then `mask / lowest` must equal 0xFF for exactly 8 contiguous set bits (two hex nibbles)
  - Nibble alignment check: bit position of lowest set bit must be a multiple of 4
  - allocate_outbound_marks() step calculation: step = lowest set bit of mask (e.g., 0x10000 for mask 0x00FF0000), giving 256 available marks
  - Only InterfaceOutbound and TableOutbound get fwmarks; BlackholeOutbound (and future IgnoreOutbound/UrltestOutbound) do NOT
  - JSON config uses integer values for start/mask (not hex strings) since nlohmann_json parses integers directly
---

## 2026-02-11 - US-045
- What was implemented: Simplified DNS server types — removed DoH, System, Blocked types; all DNS servers must now be plain IPv4 or IPv6 addresses
- Files changed:
  - `src/dns/dns_server.hpp` - Removed DnsServerType enum, removed `type` and `doh_url` fields from DnsServerConfig, removed `parse_dns_address_type()` declaration
  - `src/dns/dns_server.cpp` - Removed `is_doh_url()`, `is_system()`, `is_rcode_refused()`, `parse_dns_address_type()` functions; simplified `parse_dns_server()` to only validate IPv4/IPv6 and set resolved_ip; simplified `validate_dns_address()` to only accept valid IPs
  - `src/dns/dnsmasq_gen.cpp` - Removed PlainIP-only check when generating server= directives (all servers are now plain IP)
  - `src/config/config.hpp` - Updated DnsServer.address comment to 'IPv4 or IPv6 address'
  - `config.example.json` - Removed DoH and rcode://refused server examples
  - `prd.json` - Marked US-045 as passes: true
- **Learnings for future iterations:**
  - DNS simplification was clean since dnsmasq only supports plain UDP DNS servers in server= directives
  - The is_valid_ipv4/is_valid_ipv6 helpers are reused from the original implementation (same validation logic)
  - config.example.json had DoH and blocked entries that needed removal alongside the code changes
---

## 2026-02-11 - US-046
- What was implemented: Added IgnoreOutbound and UrltestOutbound types, removed ping settings from InterfaceOutbound, updated all dependent code
- Files changed:
  - `src/config/config.hpp` - Removed `<chrono>` include; removed `ping_target`, `ping_interval`, `ping_timeout` from InterfaceOutbound; added IgnoreOutbound, OutboundGroup, RetryConfig, CircuitBreakerConfig, UrltestOutbound structs; updated Outbound variant to 5-type variant
  - `src/config/config.cpp` - Removed ping field parsing from InterfaceOutbound; added JSON parsing for `type: "ignore"` and `type: "urltest"` (with outbound_groups, retry, circuit_breaker sub-objects); added urltest outbound_groups reference validation (must point to interface/table/blackhole, not ignore/urltest)
  - `src/main.cpp` - Removed `#include "health/health_checker.hpp"`; removed HealthChecker initialization, register_outbound calls, and periodic ping health check scheduling; simplified `make_health_fn()` to only use circuit_breaker; removed `health_checker` from ApiContext; added comment for IgnoreOutbound/UrltestOutbound no-op in apply_routing
  - `src/api/handlers.hpp` - Removed `#include "../health/health_checker.hpp"` and `HealthChecker&` from ApiContext
  - `src/api/handlers.cpp` - Updated `outbound_type()` to handle 5 outbound types; removed `ping_target` from outbound_to_json; added UrltestOutbound fields to JSON; removed `health_status_string()` and HealthChecker references from /api/health endpoint
  - `src/health/health_checker.hpp` - Removed `#include "../config/config.hpp"`; replaced `register_outbound(InterfaceOutbound&)` with `register_target(tag, interface, target, timeout)` to decouple from config types
  - `src/health/health_checker.cpp` - Updated `register_outbound` → `register_target` with explicit parameters
  - `config.example.json` - Removed ping settings from interface outbound; added ignore and urltest outbound examples
  - `prd.json` - Marked US-046 as passes: true
- **Learnings for future iterations:**
  - Outbound variant now has 5 types: InterfaceOutbound, TableOutbound, BlackholeOutbound, IgnoreOutbound, UrltestOutbound — all std::visit lambdas that match on variant types need updating
  - UrltestOutbound has nested config objects (outbound_groups, retry, circuit_breaker) — each parsed with optional fields and defaults
  - urltest outbound_groups validation happens after all outbounds are parsed in parse_config(), since forward references are allowed
  - HealthChecker is decoupled from InterfaceOutbound — uses register_target() with explicit params now; will be fully removed in US-056
  - allocate_outbound_marks() already correctly handles new types via is_routable check (only InterfaceOutbound/TableOutbound get marks)
---

## 2026-02-12 - US-047
- What was implemented: Verified all acceptance criteria already satisfied — RouteRule uses plain std::string outbound (not variant), SkipAction struct removed, JSON parsing rejects 'outbounds' array and 'action' field with descriptive errors suggesting ignore/urltest outbound alternatives
- Files changed: (no code changes needed — already implemented across US-046 and prior stories)
  - `prd.json` - Marked US-047 as passes: true
- **Learnings for future iterations:**
  - Route rule simplification was progressively implemented: US-046 removed SkipAction and failover chain types, config.cpp already throws ConfigError for deprecated fields
  - When verifying already-complete stories, grep for removed types (SkipAction, HealthCheckFn) to confirm no remaining references
---

## 2026-02-12 - US-048
- What was implemented: Verified all acceptance criteria already satisfied — resolve_route_action() takes single string tag (not variant), HealthCheckFn removed, IgnoreOutbound returns skip(), UrltestOutbound returns route_to(), no failover chain handling
- Files changed: (no code changes needed — already implemented across US-046 and prior stories)
  - `prd.json` - Marked US-048 as passes: true
- **Learnings for future iterations:**
  - resolve_route_action() was simplified in the same iteration that added IgnoreOutbound/UrltestOutbound types
  - The target.cpp implementation is minimal: lookup by tag, check IgnoreOutbound → skip, everything else → route_to
---

## 2026-02-12 - US-049
- What was implemented: Removed static linking for cross-compilation targets, switching to dynamic linking to reduce binary size and leverage system libraries on routers
- Files changed:
  - `meson.build` - Removed the `if meson.is_cross_build()` block that added `-static` link argument (lines 78-81)
- **Learnings for future iterations:**
  - Minimal single-file change; the static linking block was 3 lines plus a comment
  - Dynamic linking is preferred for OpenWRT/Keenetic targets since system libraries (musl libc, libcurl, etc.) are already present on the router
---

## 2026-02-12 - US-050
- What was implemented: Updated CircuitBreaker to accept CircuitBreakerConfig struct, added success_threshold-based half_open→closed transitions, half_open_max_requests concurrency limiting, and begin_request/end_request methods
- Files changed:
  - `src/health/circuit_breaker.hpp` - Constructor now takes CircuitBreakerConfig instead of individual params; CircuitBreakerEntry extended with success_count_in_half_open and half_open_active_requests; added begin_request() and end_request() methods; includes config.hpp for CircuitBreakerConfig
  - `src/health/circuit_breaker.cpp` - Constructor stores CircuitBreakerConfig; record_success() in half_open increments success count and transitions to closed only when >= success_threshold; is_allowed() in half_open checks half_open_active_requests < half_open_max_requests; cooldown uses config.timeout_ms (milliseconds); begin_request/end_request manage active request count; failure/state-reset clears new counters
- **Learnings for future iterations:**
  - CircuitBreaker now depends on config.hpp for CircuitBreakerConfig struct — this creates a header dependency from health/ to config/
  - Cooldown changed from std::chrono::seconds to std::chrono::milliseconds (config_.timeout_ms) for finer granularity
  - success_count_in_half_open and half_open_active_requests are reset on any state transition (to closed, to open, and when entering half_open from open)
  - begin_request/end_request are designed to be called by the URL tester (US-051) around probe requests
---

## 2026-02-12 - US-051
- What was implemented: HTTP URL tester that measures response time through specific outbounds using SO_MARK to route test traffic via the outbound's assigned fwmark
- Files changed:
  - `src/health/url_tester.hpp` - URLTester class declaration with URLTestResult struct (success, latency_ms, error), test() and test_once() methods
  - `src/health/url_tester.cpp` - Implementation using libcurl with CURLOPT_SOCKOPTFUNCTION + setsockopt(SO_MARK) for fwmark-based routing, retry logic with configurable attempts and interval, latency measurement via steady_clock
  - `meson.build` - Added `src/health/url_tester.cpp` to sources list
- **Learnings for future iterations:**
  - CURLOPT_MARK is not available in libcurl 8.5.0; use CURLOPT_SOCKOPTFUNCTION + setsockopt(SOL_SOCKET, SO_MARK) instead to set socket mark for policy routing
  - The fwmark is passed to the sockopt callback via CURLOPT_SOCKOPTDATA using reinterpret_cast<void*>(static_cast<uintptr_t>(fwmark)) to safely pass a uint32_t through a void pointer
  - test() returns on first successful attempt (no need to try all retries if one succeeds)
  - Response body is discarded (discard_callback) since we only care about latency and success status
  - SO_MARK requires CAP_NET_ADMIN or root on Linux; this is expected for a routing daemon
---

## 2026-02-12 - US-052
- What was implemented: UrltestManager that runs periodic URL tests for all urltest outbounds, tracks per-child-outbound latencies and circuit breaker states, and selects the best outbound using the weighted group algorithm
- Files changed:
  - `src/routing/urltest_manager.hpp` - UrltestManager class declaration, UrltestState struct (config, last_results, circuit_breakers, selected_outbound, scheduler_task_id), UrltestChangeCallback type alias
  - `src/routing/urltest_manager.cpp` - Implementation: register_urltest() creates per-child CircuitBreakers and schedules repeating tests; run_tests() tests each child via URLTester with circuit breaker gating (begin_request/end_request); select_outbound() sorts groups by weight ascending, filters by circuit breaker state and successful results, selects fastest within tolerance; triggers on_change callback when selection changes
  - `meson.build` - Added `src/routing/urltest_manager.cpp` to sources list
- **Learnings for future iterations:**
  - Scheduler uses std::chrono::seconds; urltest interval_ms is converted by dividing by 1000 (minimum 1 second)
  - Circuit breaker state check in select_outbound() uses state() const method (not is_allowed() which has side effects) to avoid mutating state during selection
  - Outbounds without fwmarks (e.g., blackhole) are skipped during URL testing since they can't be routed to
  - run_tests() immediately runs an initial test on register_urltest() so there's no delay before first selection
  - select_outbound() returns empty string when all outbounds are exhausted, signaling blackhole fallback to the caller
  - UrltestState holds per-child CircuitBreaker instances (not shared) since each urltest outbound has its own circuit_breaker config
---

## 2026-02-12 - US-053
- What was implemented: In-memory FirewallState class for tracking current firewall rule-to-outbound mappings, fwmark assignments, and urltest selections
- Files changed:
  - `src/routing/firewall_state.hpp` - FirewallState class declaration with RuleActionType enum (Mark/Drop/Skip), RuleState struct (rule_index, list_names, set_names, outbound_tag, action_type, fwmark), and FirewallState class with set/get methods for rules, outbound marks, urltest selections, plus resolve_effective_outbound()
  - `src/routing/firewall_state.cpp` - Implementation of all FirewallState methods
  - `meson.build` - Added `src/routing/firewall_state.cpp` to sources list
- **Learnings for future iterations:**
  - FirewallState is a simple state container with no dependencies on firewall backends or routing subsystems — it only depends on config.hpp for OutboundMarkMap
  - resolve_effective_outbound() checks urltest_selections_ map to dereference urltest outbound tags to their currently selected child tag
  - RuleState tracks both list_names (from config) and set_names (firewall set names) since they may differ in future implementations
  - RuleActionType::Skip is for ignore outbounds that don't produce any firewall rule
---

## 2026-02-12 - US-054
- What was implemented: Added create_drop_rule() virtual method to Firewall abstract interface, implemented by both iptables and nftables backends, with proper cleanup tracking
- Files changed:
  - `src/firewall/firewall.hpp` - Added pure virtual `create_drop_rule(set_name, chain)` method to Firewall abstract class
  - `src/firewall/iptables.hpp` - Added `create_drop_rule()` override and `DropRule` tracking struct with `drop_rules_` vector
  - `src/firewall/iptables.cpp` - Implemented `create_drop_rule()` using `iptables -t mangle -A <chain> -m set --match-set <set_name> dst -j DROP`; updated `cleanup()` to remove drop rules in reverse order before mark rules
  - `src/firewall/nftables.hpp` - Added `create_drop_rule()` override and `DropRule` tracking struct with `drop_rules_` vector
  - `src/firewall/nftables.cpp` - Implemented `create_drop_rule()` using `nft add rule inet keen_pbr3 <chain> ip daddr @<set_name> drop`; updated `cleanup()` to clear `drop_rules_`
- **Learnings for future iterations:**
  - DROP rules follow the same tracking pattern as mark rules: stored in a vector, cleaned up in reverse order during cleanup()
  - iptables DROP rules use the mangle table (same as mark rules) for consistency: `-t mangle -A PREROUTING -m set --match-set <set> dst -j DROP`
  - nftables cleanup via `nft delete table` cascades to all rules including drop rules, so only the vector needs clearing
  - iptables cleanup must remove drop rules AND mark rules before destroying ipsets (both reference ipset names)
---

## 2026-02-12 - US-055
- What was implemented: Rewrote daemon mode in main.cpp to use mark-based routing architecture: static routing tables/ip-rules set up once at startup via allocate_outbound_marks(), transactional firewall rule rebuilds via apply_firewall() lambda, blackhole handled by firewall DROP rules (no routing tables), in-memory FirewallState tracking, UrltestManager with change callback that triggers apply_firewall(), and SIGUSR1 to verify routing tables and trigger immediate URL tests
- Files changed:
  - `src/main.cpp` - Complete rewrite of daemon mode: replaced sequential fwmark allocation with allocate_outbound_marks(); added setup_static_routing() lambda for one-time routing table/ip-rule creation per InterfaceOutbound/TableOutbound; replaced apply_all() with apply_firewall() that builds complete firewall transactionally and updates FirewallState; added URLTester and UrltestManager initialization with change callback; SIGUSR1 now clears+rebuilds static routing and triggers immediate urltest; SIGHUP clears UrltestManager, re-allocates marks, re-creates routing, re-registers urltests; removed HealthChecker, make_health_fn(), rule_outbound_tags tracking, and periodic ping scheduler
- **Learnings for future iterations:**
  - apply_firewall() does cleanup() + create_firewall("auto") at start to get fresh firewall state before rebuilding — this is the transactional rebuild pattern
  - For urltest outbound resolution, apply_firewall() reads firewall_state.get_urltest_selections() to find the currently selected child, then uses the child's fwmark for the mark rule
  - setup_static_routing() uses config.iproute.table_start + offset for table IDs and config.fwmark.mask for ip rule fwmask — these are configurable, not hardcoded
  - UrltestManager's change callback sets the urltest selection in FirewallState BEFORE calling apply_firewall(), so the new selection is visible during rebuild
  - SIGUSR1 does clear()+setup_static_routing() to re-verify routing tables exist, not just check — simpler and more robust than checking individual routes
  - API reload lambda duplicates SIGHUP logic since lambdas capture by reference and need the same steps
  - Shutdown order: urltest_manager.clear() → scheduler.cancel_all() → route_table.clear() → policy_rules.clear() → firewall->cleanup() → PID file
---

## 2026-02-12 - US-056
- What was implemented: Removed ICMP-based HealthChecker (deleted files + removed from meson.build), updated API handlers to use FirewallState and UrltestManager for status and health reporting
- Files changed:
  - `src/health/health_checker.hpp` - Deleted
  - `src/health/health_checker.cpp` - Deleted
  - `meson.build` - Removed `src/health/health_checker.cpp` from sources list
  - `src/api/handlers.hpp` - Added FirewallState& and UrltestManager& to ApiContext struct; added includes for firewall_state.hpp and urltest_manager.hpp
  - `src/api/handlers.cpp` - GET /api/status now includes fwmark assignments per outbound, current rule-to-outbound mappings (with action type and effective outbound), and urltest selections; GET /api/health reports urltest state per urltest outbound (per-child latencies, circuit breaker states, selected outbound), interface/table outbounds report 'always healthy'
  - `src/main.cpp` - Added firewall_state and urltest_manager to ApiContext initialization
- **Learnings for future iterations:**
  - ApiContext now holds `const FirewallState&` and `const UrltestManager&` for read-only access to firewall and urltest state
  - CircuitBreaker::state() is a const method taking tag parameter — safe to call from const UrltestManager reference
  - format_hex() helper uses `<iomanip>` and `<sstream>` for hex formatting with zero-padding
  - GET /api/health uses try/catch for UrltestManager::get_state() which throws std::out_of_range for unknown tags
---

## 2026-02-15 - US-057
- What was implemented: Moved all runtime subsystem ownership into Daemon class with parameterized constructor, moved helper functions to keen_pbr3 namespace, simplified main.cpp daemon path
- Files changed:
  - `src/daemon/daemon.hpp` - Already had new structure (Config, config_path, DaemonOptions constructor, all subsystem members, private business logic method declarations)
  - `src/daemon/daemon.cpp` - Replaced no-arg constructor with `Daemon(Config, string, DaemonOptions)` that initializes all subsystems in member initializer list; removed `on_sigusr1()`/`on_sighup()` callback methods and `sigusr1_cb_`/`sighup_cb_` members; added `get_outbound_tag()` and `find_outbound()` as keen_pbr3 namespace free functions; signal dispatch calls `handle_sigusr1()`/`handle_sighup()` directly; added stub implementations for business logic methods (US-058 will implement them); included api/server.hpp under `#ifdef WITH_API` for unique_ptr<ApiServer> destructor
  - `src/main.cpp` - Daemon mode reduced to: parse args → load config → daemonize if needed → construct `Daemon(config, config_path, opts)` → `daemon.run()`; removed all subsystem construction, lambdas, signal handler registration, API setup, and shutdown cleanup from main.cpp; removed local `get_outbound_tag()`/`find_outbound()`/`write_pid_file()`/`remove_pid_file()` from anonymous namespace; non-daemon commands (download, print-dnsmasq-config) unchanged
- **Learnings for future iterations:**
  - unique_ptr<T> destructor needs T to be a complete type — forward declaration is not enough in the .cpp where the destructor runs; must include the full header
  - Member initializer list order must match declaration order in the class: config_ before cache_ (since cache_ depends on config_.daemon.cache_dir)
  - Scheduler is constructed as unique_ptr in constructor body (not initializer list) because it takes Daemon& which isn't ready until after construction begins
  - UrltestManager created later during startup (register_urltest_outbounds) since it needs the change callback wired up
---

## 2026-02-15 - US-058
- What was implemented: Moved all routing/firewall logic and run lifecycle into Daemon class private methods, replacing stub implementations with full business logic
- Files changed:
  - `src/daemon/daemon.hpp` - Added ApiContext forward declaration and api_ctx_ unique_ptr member under #ifdef WITH_API
  - `src/daemon/daemon.cpp` - Implemented all stub methods: setup_static_routing() (creates routing tables/ip-rules per outbound), apply_firewall() (transactional firewall rebuild with RuleState tracking), download_uncached_lists() (downloads lists not yet cached), register_urltest_outbounds() (creates UrltestManager with change callback), full_reload() (single implementation of teardown+rebuild for SIGHUP and API), write_pid_file()/remove_pid_file() (PID file management), setup_api() (API server lifecycle). Updated run() with full startup sequence (PID → download → routing → urltest → firewall → API → event loop → shutdown). Implemented handle_sigusr1() and handle_sighup() to call business logic directly.
  - `src/main.cpp` - Removed unnecessary includes (filesystem, memory, sys/socket.h, firewall, netlink, route_table, policy_rule, target, urltest_manager, scheduler, health/url_tester, firewall_state, and WITH_API includes) since all business logic is now in Daemon
- **Learnings for future iterations:**
  - ApiContext holds references to Daemon members (config_.outbounds, cache_, config_.lists, firewall_state_, *urltest_manager_). After config_ = parse_config() in full_reload(), the member addresses are stable since Config is a member, so references remain valid.
  - ApiContext stored as unique_ptr<ApiContext> to outlive setup_api() — cannot be a local variable since handlers capture references to it
  - full_reload() is the single implementation for both SIGHUP and API reload — eliminates the 3x duplication from old main.cpp
  - Daemon::run() is now the complete lifecycle: startup → event loop → shutdown, not just the event loop
  - handle_sigusr1() and handle_sighup() call business logic directly (no callback indirection) — callbacks removed
---

## 2026-02-15 - US-059
- What was implemented: Verified all acceptance criteria already satisfied — signal handlers are internal to Daemon (handle_sigusr1/handle_sighup private methods), no external callback API (on_sigusr1/on_sighup removed), API server lifecycle managed by Daemon via setup_api() private method
- Files changed: (no code changes needed — already implemented across US-057 and US-058)
  - `prd.json` - Marked US-059 as passes: true
- **Learnings for future iterations:**
  - US-057 through US-059 were tightly coupled — moving subsystem ownership (US-057), business logic (US-058), and removing external callback API (US-059) all happened together naturally
  - When verifying already-complete stories, grep for removed interfaces (on_sigusr1, on_sighup, sigusr1_cb_, sighup_cb_) to confirm complete removal
---

## 2026-02-15 - US-060
- What was implemented: Verified all acceptance criteria already satisfied — main.cpp is a thin CLI entry point with only arg parsing, config loading, non-daemon commands, and Daemon construction+run; all business logic (subsystem construction, routing, firewall, signal handlers, API setup, shutdown) is in Daemon class
- Files changed: (no code changes needed — already implemented across US-057 and US-058)
  - `prd.json` - Marked US-060 as passes: true
- **Learnings for future iterations:**
  - US-057 through US-060 formed a natural refactoring arc where each story's implementation implicitly satisfied the next story's criteria
  - main.cpp includes beyond config.hpp and daemon.hpp are justified by non-daemon commands (download needs CacheManager/ListStreamer/EntryCounter, print-dnsmasq-config needs DnsServerRegistry/DnsmasqGenerator)
---

## 2026-02-27 - US-061
- What was implemented: Routing health data structures, Firewall::backend(), and accessor methods
- Files changed:
  - `src/health/routing_health.hpp` - New file with CheckStatus enum and five structs (FirewallChainCheck, FirewallRuleCheck, RouteTableCheck, PolicyRuleCheck, RoutingHealthReport)
  - `src/firewall/firewall.hpp` - Moved FirewallBackend enum before Firewall class; added pure virtual backend() const = 0
  - `src/firewall/iptables.hpp` - Added backend() const override declaration
  - `src/firewall/nftables.hpp` - Added backend() const override declaration
  - `src/firewall/iptables.cpp` - Implemented backend() returning FirewallBackend::iptables
  - `src/firewall/nftables.cpp` - Implemented backend() returning FirewallBackend::nftables
  - `src/routing/route_table.hpp` - Added get_routes() const returning const std::vector<RouteSpec>&
  - `src/routing/policy_rule.hpp` - Added get_rules() const returning const std::vector<RuleSpec>&
  - `prd.json` - Marked US-061 as passes: true
- **Learnings for future iterations:**
  - FirewallBackend enum was already declared in firewall.hpp but needed to be moved before the Firewall class so backend() could use it in the class definition
  - routing_health.hpp is a pure header (no .cpp needed) — all types are data structs with no non-trivial logic
  - CheckStatus has only 3 values: ok, missing, mismatch (no "error" variant despite US-065 mentioning error field in RoutingHealthReport — errors go in the report's string error field)
---

## 2026-02-27 - US-062
- What was implemented: FirewallVerifier abstract interface + IptablesFirewallVerifier with parse_iptables_save()
- Files changed:
  - src/firewall/firewall_verifier.hpp (new): CommandRunner type alias, run_command_capture() declaration, FirewallVerifier abstract class, create_firewall_verifier() factory
  - src/firewall/firewall_verifier.cpp (new): run_command_capture() using popen, factory returning IptablesFirewallVerifier (nftables throws "not yet implemented" for US-063)
  - src/firewall/iptables_verifier.hpp (new): ParsedIptablesRule, ParsedIptablesState structs, parse_iptables_save() declaration, IptablesFirewallVerifier class
  - src/firewall/iptables_verifier.cpp (new): parse_iptables_save() parsing chain declaration/prerouting jump/mark+drop rules; verify_chain() aggregates v4+v6 results; verify_rules() returns FirewallRuleCheck per (RuleState, set_name) pair
  - CMakeLists.txt: added firewall_verifier.cpp and iptables_verifier.cpp to SOURCES
- **Learnings for future iterations:**
  - FirewallVerifier factory in firewall_verifier.cpp uses iptables_verifier.hpp; nftables case throws until US-063 implements it
  - parse_iptables_save() detects chain via `:KeenPbrTable` prefix; prerouting jump via exact string match `-A PREROUTING -j KeenPbrTable`; mark/drop rules by scanning `-A KeenPbrTable -m set --match-set <set> dst -j ...`
  - Mark values written as hex (`0x100`) by iptables.cpp, parsed with std::stoul(..., nullptr, 0) to handle both hex and decimal
  - verify_rules() builds combined set_name→rule map from v4 then v6 (v6 only inserted if not already present), then matches against RuleState.set_names
  - CommandRunner injection pattern allows testing without system() calls
---
## 2026-02-27 - US-063
- What was implemented: NftablesFirewallVerifier with parse_nft_json() for verifying live nftables state against expected configuration
- Files changed:
  - `src/firewall/nftables_verifier.hpp` - ParsedNftRule, ParsedNftablesState structs; parse_nft_json() declaration; NftablesFirewallVerifier class; create_nftables_verifier() factory
  - `src/firewall/nftables_verifier.cpp` - Implementation of parse_nft_json() using nlohmann_json; NftablesFirewallVerifier::verify_chain() and verify_rules() methods
  - `src/firewall/firewall_verifier.cpp` - Updated factory to use create_nftables_verifier() instead of throwing; added nftables_verifier.hpp include
  - `CMakeLists.txt` - Added src/firewall/nftables_verifier.cpp to SOURCES list
- **Learnings for future iterations:**
  - nft -j list ruleset JSON structure: top-level {"nftables": [...]} array with objects keyed by "table", "chain", "rule", "metainfo"
  - Table lookup: elem["table"] where family=="inet" and name=="KeenPbrTable"
  - Chain lookup: elem["chain"] where table=="KeenPbrTable" and name=="prerouting"; hook=="prerouting" indicates active hook
  - Rule lookup: elem["rule"] where table=="KeenPbrTable" and chain=="prerouting"; expr array contains match/mangle/drop objects
  - Named set reference in match.right: string "@setname" (strip '@' to get set_name); some versions may use {"set": "@setname"}
  - Mark value from mangle expr: mangle.key.meta.key == "mark" and mangle.value is the uint32_t fwmark
  - Drop verdict: expr object has key "drop" (value is null/void)
  - IPv6 detection: match.left.payload.protocol == "ip6"
  - nlohmann_json parse failures caught with catch(...) -> return empty state (defensive parsing)
---
## 2026-02-27 - US-064
- What was implemented: NetlinkManager dump methods + RoutingVerifier
- Files changed:
  - `src/routing/netlink.hpp` - added DumpedRoute and DumpedRule structs; added dump_routes_in_table() and dump_policy_rules() method declarations
  - `src/routing/netlink.cpp` - implemented both dump methods using rtnl_route_alloc_cache/rtnl_rule_alloc_cache + nl_cache_foreach; added nl_addr_to_ip_str() helper and CachePtr RAII wrapper
  - `src/routing/routing_verifier.hpp` - new RoutingVerifier class with verify_route_table() and verify_policy_rule() methods
  - `src/routing/routing_verifier.cpp` - implementations using netlink dump methods
  - `CMakeLists.txt` - added src/routing/routing_verifier.cpp to SOURCES
- **Learnings for future iterations:**
  - libnl3 dump pattern: alloc cache via rtnl_route_alloc_cache(sock, family, 0, &cache) then nl_cache_foreach() with void* context struct (no captures needed)
  - rtnl_route_get_nnexthops() + rtnl_route_nexthop_n(route, 0) to get first nexthop; rtnl_route_nh_get_ifindex() + if_indextoname() for interface name
  - nl_addr2str() includes prefix length ("/N"), strip with find('/') for plain IP representation
  - rtnl_route_get_type() == RTN_BLACKHOLE to detect blackhole routes; RTN_BLACKHOLE from <netlink/route/route.h>
  - nl_addr_get_prefixlen(dst) == 0 identifies default route (0.0.0.0/0 or ::/0)
  - CachePtr RAII: unique_ptr<nl_cache, CacheDeleter> where CacheDeleter calls nl_cache_free()
  - Local structs used as C callback context (void*) work fine in C++20 captureless lambdas
  - include <netlink/cache.h> for nl_cache_foreach (even if often included transitively)
  - include <arpa/inet.h> for inet functions alongside <net/if.h>
---

## 2026-02-27 - US-065
- What was implemented: RoutingHealthChecker orchestrator that ties together FirewallVerifier and RoutingVerifier, plus routing_health_report_to_json() for JSON serialization
- Files changed:
  - `src/health/routing_health_checker.hpp` - new header declaring RoutingHealthChecker class and routing_health_report_to_json() free function
  - `src/health/routing_health_checker.cpp` - implementation; check() builds table_id->outbound_tag map from policy_rules + outbound_marks, then calls verifiers
  - `CMakeLists.txt` - added src/health/routing_health_checker.cpp to SOURCES list
- **Learnings for future iterations:**
  - routing_health_checker.cpp outbound_tag lookup for route tables: use policy_rules (fwmark->table) + outbound_marks (tag->fwmark) to build table_id->outbound_tag map
  - hex_str() uses std::format("0x{:08x}", v) — requires GCC 13+ (CMakeLists.txt already checks for <format>)
  - overall_ok only when ALL sub-checks have CheckStatus::ok; only when no exception; error field populated on catch
  - JSON output: "overall" is "ok"/"degraded"/"error"; fwmark/fwmask as hex strings; firewall chain in "firewall" object
---

## 2026-02-27 - US-066
- What was implemented: Wired GET /api/health/routing into API handlers and Daemon; RoutingHealthChecker owned by Daemon and passed to ApiContext
- Files changed:
  - `src/api/handlers.hpp` - added RoutingHealthChecker& field to ApiContext struct
  - `src/api/handlers.cpp` - added GET /api/health/routing handler calling routing_health_checker.check() + routing_health_report_to_json()
  - `src/daemon/daemon.hpp` - added routing_health_checker_ member (unique_ptr<RoutingHealthChecker>) and #include routing_health_checker.hpp
  - `src/daemon/daemon.cpp` - constructed routing_health_checker_ at end of Daemon constructor; passed to ApiContext in setup_api()
- **Learnings for future iterations:**
  - RoutingHealthChecker must be constructed AFTER firewall_, firewall_state_, route_table_, policy_rules_, and netlink_ are initialized (all are in member initializer list, so end of constructor body is safe)
  - ApiContext struct initialization order must match field declaration order in handlers.hpp
  - server.get() RouteHandler only returns std::string (HTTP 200); exceptions propagate to server wrapper (HTTP 500 with {"error":"..."})
  - For /api/health/routing: check() catches all exceptions internally; handler's try/catch re-throws with proper JSON format for 500 responses
---

## 2026-02-27 - US-067
- What was implemented: doctest-based unit tests for parse_iptables_save(), parse_nft_json(), IptablesFirewallVerifier, and NftablesFirewallVerifier with injectable CommandRunner; 26 tests, 90 assertions
- Files changed:
  - `third_party/doctest` - git submodule added (doctest/doctest.h single header)
  - `.gitmodules` - new entry for doctest submodule
  - `tests/test_firewall_verifier.cpp` - 26 test cases covering all required scenarios
  - `tests/CMakeLists.txt` - builds keen-pbr3-tests from iptables_verifier, nftables_verifier, firewall_verifier object files + doctest header
  - `CMakeLists.txt` - added `option(BUILD_TESTS ...)` and `add_subdirectory(tests)` when ON
- **Learnings for future iterations:**
  - doctest is single-header at `third_party/doctest/doctest/doctest.h`; include path is `third_party/doctest` → `#include <doctest/doctest.h>`
  - Tests built with `cmake -DBUILD_TESTS=ON && cmake --build build --target keen-pbr3-tests`
  - Tests include only iptables_verifier, nftables_verifier, firewall_verifier.cpp (NOT routing_verifier — requires real kernel)
  - RoutingVerifier tests are NOT included (requires real kernel via libnl3; covered by manual testing)
  - CommandRunner injection: `IptablesFirewallVerifier verifier([&canned](const std::string&) { return canned; });`
  - DROP rule in nft JSON: `{"drop": null}` (null value for drop key)
---