Skip to content

Commit d668146

Browse files
shaypal5claude
andauthored
feat: Milestone 2 — narrative layer (NarrativeSpec, WorldSpec, dataset card) (#5)
* feat: Milestone 2 — narrative layer (NarrativeSpec, WorldSpec, dataset card) - narrative/spec.py: frozen dataclasses for NarrativeSpec hierarchy (CompanySpec, ProductSpec, MarketSpec, GtmMotionSpec, PersonaSpec, FunnelStageSpec) with validated from_dict() classmethods - narrative/dataset_card.py: render_dataset_card() produces Markdown dataset card from WorldSpec (header, narrative summary, task, stubs for table inventory and feature categories, use cases, caveats) - core/models.py: WorldSpec.narrative field (NarrativeSpec | None) - api/generator.py: world_spec property; from_recipe() resolves the recipe's narrative.yaml into a NarrativeSpec and populates WorldSpec - 51 new tests covering spec validation, card rendering, and Generator integration (110 total); ruff + mypy clean Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address Copilot review comments on Milestone 2 PR - spec.py: _require_keys now guards against non-dict input (COPILOT-3) - spec.py: NarrativeSpec.from_dict validates each personas/funnel_stages element is a dict before passing to sub-from_dict (COPILOT-3) - spec.py: GtmMotionSpec.from_dict validates channels is a list of strings, rejects bools for share floats, and enforces [0, 1] range (COPILOT-1) - spec.py: PersonaSpec.from_dict validates title_variants is a list of strings instead of silently splitting a bare string (COPILOT-2) - spec.py: ProductSpec.from_dict requires free_trial_available / demo_available to be actual bools; rejects int/str coercion (COPILOT-6) - spec.py: MarketSpec.from_dict validates icp_industries and geographies are lists of strings (COPILOT-7) - generator.py: Generator.__init__ takes only world_spec; config property derives from world_spec.config (single source of truth) (COPILOT-4) - dataset_card.py: stub text changed to "Narrative unavailable for this dataset." (COPILOT-5); test updated to match Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent f7dff3f commit d668146

8 files changed

Lines changed: 863 additions & 32 deletions

File tree

.agent-plan.md

Lines changed: 26 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,44 +6,50 @@
66

77
## Current System State
88

9-
**v0.2.0 in progress.** Typed `Recipe` model, `GenerationConfig` with full validation, config
10-
precedence system, `RNGRoot` with deterministic substreams, `Generator.from_recipe()` fully
11-
implemented, `core/hashing.py`, `core/serialization.py`, and recipe narrative/difficulty-profile
12-
assets for `b2b_saas_procurement_v1`. 59 tests passing.
9+
**v0.2.0 in progress — Milestone 2 complete (PR open).** Typed `NarrativeSpec` hierarchy, `WorldSpec`
10+
with narrative field, `Generator.from_recipe()` populates `world_spec`, dataset card renderer, and
11+
full test coverage. 110 tests passing.
1312

1413
---
1514

16-
## Active Task Breakdown — Milestone 2: Narrative Layer (v0.2.0 cont.)
15+
## Active Task Breakdown — Milestone 3: Schema Layer (v0.2.0 cont.)
1716

18-
Goal: Build the concrete company/product/market story objects that anchor all later simulation.
17+
Goal: Define the relational entity schema (accounts, contacts, leads, etc.) and feature dictionary.
1918

20-
- [ ] **1. Narrative models**
21-
- Implement typed dataclasses in `narrative/`: `CompanySpec`, `ProductSpec`, `MarketSpec`,
22-
`PersonaSpec`, `FunnelSpec`
23-
- Loader: parse `narrative.yaml` into these models with validation
19+
- [ ] **1. Entity schema**
20+
- Implement `schema/entities.py`: typed dataclasses for `Account`, `Contact`, `Lead`
21+
- Implement `schema/events.py`: `Touch`, `SalesActivity`, `Opportunity` etc.
2422

25-
- [ ] **2. WorldSpec population**
26-
- Flesh out `WorldSpec` to hold a resolved `NarrativeSpec`
27-
- Wire into `Generator.from_recipe()` so `gen.world_spec` is populated after construction
23+
- [ ] **2. Feature dictionary**
24+
- Implement `schema/features.py` + `schema/dictionaries.py`
25+
- Generate `feature_dictionary.csv` stub
2826

29-
- [ ] **3. Dataset card generation**
30-
- Implement `narrative/dataset_card.py`: render a Markdown dataset card from `WorldSpec`
31-
- Tests: round-trip model → YAML → model, dataset-card text contains expected fields
27+
- [ ] **3. Task schema**
28+
- Implement `schema/tasks.py`: `converted_within_90_days` task manifest structure
3229

3330
---
3431

3532
## Context Pointers
3633

37-
- Milestone 2 scope: `docs/leadforge_implementation_plan.md` §5 "Milestone 2"
34+
- Milestone 3 scope: `docs/leadforge_implementation_plan.md` §6 "Milestone 3"
3835
- Full milestone dependency graph: `docs/leadforge_implementation_plan.md` §6
39-
- Narrative spec: `docs/leadforge_architecture_spec.md` §7
40-
- Recipe assets: `leadforge/recipes/b2b_saas_procurement_v1/narrative.yaml`
36+
- Schema spec: `docs/leadforge_architecture_spec.md` §8
37+
- Recipe assets: `leadforge/recipes/b2b_saas_procurement_v1/`
4138

4239
---
4340

4441
## Completed Phases
4542

46-
### Milestone 1 — Canonical Config, Recipe & Model Objects ✓ (v0.2.0 in PR)
43+
### Milestone 2 — Narrative Layer ✓ (v0.2.0 in PR)
44+
- `leadforge/narrative/spec.py`: frozen dataclasses `NarrativeSpec`, `CompanySpec`, `ProductSpec`,
45+
`MarketSpec`, `GtmMotionSpec`, `PersonaSpec`, `FunnelStageSpec` — all with validated `from_dict()`
46+
- `leadforge/narrative/dataset_card.py`: `render_dataset_card(world_spec)` — Markdown card
47+
- `leadforge/core/models.py`: `WorldSpec` gets `narrative: NarrativeSpec | None` field
48+
- `leadforge/api/generator.py`: `world_spec` property; `from_recipe()` resolves narrative into
49+
`WorldSpec`
50+
- 51 new tests (spec validation, dataset card, Generator integration); total 110 passing
51+
52+
### Milestone 1 — Canonical Config, Recipe & Model Objects ✓ (v0.2.0 merged)
4753
- `leadforge/core/rng.py`: `RNGRoot` with SHA-256-derived named substreams
4854
- `leadforge/core/hashing.py`: `hash_config()` — stable SHA-256 digest of `GenerationConfig`
4955
- `leadforge/core/serialization.py`: `load_yaml`, `load_json`, `dump_json`

leadforge/api/generator.py

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
from typing import Any
66

77
from leadforge.core.enums import DifficultyProfile, ExposureMode
8-
from leadforge.core.models import GenerationConfig, WorldBundle
8+
from leadforge.core.models import GenerationConfig, WorldBundle, WorldSpec
99
from leadforge.core.rng import RNGRoot
1010
from leadforge.core.sentinels import _MISSING
1111

@@ -23,17 +23,22 @@ class Generator:
2323
bundle = gen.generate(n_leads=5000, difficulty="intermediate")
2424
bundle.save("./out/demo_bundle")
2525
26-
``from_recipe`` is implemented in Milestone 1. Full generation
27-
(``generate``) is implemented across Milestones 2–9.
26+
``from_recipe`` is implemented in Milestone 1–2. Full generation
27+
(``generate``) is implemented across Milestones 3–9.
2828
"""
2929

30-
def __init__(self, config: GenerationConfig) -> None:
31-
self._config = config
32-
self._rng = RNGRoot(config.seed)
30+
def __init__(self, world_spec: WorldSpec) -> None:
31+
self._world_spec = world_spec
32+
self._rng = RNGRoot(world_spec.config.seed)
3333

3434
@property
3535
def config(self) -> GenerationConfig:
36-
return self._config
36+
return self._world_spec.config
37+
38+
@property
39+
def world_spec(self) -> WorldSpec:
40+
"""The resolved world specification, including narrative."""
41+
return self._world_spec
3742

3843
@classmethod
3944
def from_recipe(
@@ -69,15 +74,16 @@ def from_recipe(
6974
Applied after recipe defaults but before explicit kwargs.
7075
7176
Returns:
72-
A configured :class:`Generator` instance ready to call
73-
:meth:`generate` on.
77+
A configured :class:`Generator` with a populated
78+
:attr:`world_spec` (narrative resolved from the recipe).
7479
7580
Raises:
7681
:class:`~leadforge.core.exceptions.InvalidRecipeError`: if the
7782
recipe does not exist, is malformed, or the requested
7883
exposure mode / difficulty is not supported.
7984
"""
8085
from leadforge.api.recipes import Recipe
86+
from leadforge.narrative.spec import NarrativeSpec
8187
from leadforge.recipes.registry import load_recipe
8288

8389
raw = load_recipe(recipe_id)
@@ -93,7 +99,12 @@ def from_recipe(
9399
output_path=output_path,
94100
override=override,
95101
)
96-
return cls(config)
102+
103+
narrative_data = recipe.load_narrative()
104+
narrative = NarrativeSpec.from_dict(narrative_data) if narrative_data else None
105+
world_spec = WorldSpec(config=config, narrative=narrative)
106+
107+
return cls(world_spec)
97108

98109
def generate(
99110
self,

leadforge/core/models.py

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,15 @@
33
from __future__ import annotations
44

55
from dataclasses import dataclass, field
6-
from typing import Any
6+
from typing import TYPE_CHECKING, Any
77

88
from leadforge.core.enums import DifficultyProfile, ExposureMode
99
from leadforge.core.exceptions import InvalidConfigError
1010
from leadforge.version import __version__
1111

12+
if TYPE_CHECKING:
13+
from leadforge.narrative.spec import NarrativeSpec
14+
1215

1316
def _require_positive_int(value: Any, name: str) -> None:
1417
"""Raise ``InvalidConfigError`` unless *value* is a positive plain ``int``.
@@ -74,10 +77,13 @@ def __post_init__(self) -> None:
7477
class WorldSpec:
7578
"""Fully instantiated hidden world specification (post-sampling, pre-simulation).
7679
77-
Populated in Milestone 2 (narrative/schema) through Milestone 6 (mechanisms).
80+
Populated incrementally across milestones:
81+
- M2: config + narrative
82+
- M3–M6: schema, structure, mechanisms
7883
"""
7984

8085
config: GenerationConfig = field(default_factory=GenerationConfig)
86+
narrative: NarrativeSpec | None = None
8187

8288

8389
@dataclass
Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
"""Dataset card renderer.
2+
3+
Produces the ``dataset_card.md`` artifact from a :class:`WorldSpec`.
4+
The card follows the structure required by the architecture spec (§14.3).
5+
"""
6+
7+
from __future__ import annotations
8+
9+
from typing import TYPE_CHECKING
10+
11+
if TYPE_CHECKING:
12+
from leadforge.core.models import WorldSpec
13+
14+
15+
def render_dataset_card(world_spec: WorldSpec) -> str:
16+
"""Return a Markdown dataset card string for *world_spec*.
17+
18+
Sections present at all milestones:
19+
- Header (recipe id, version, seed, exposure mode)
20+
- Narrative summary (company, product, market, GTM)
21+
- Primary task and label definition
22+
- Suggested use cases
23+
- Caveats
24+
25+
Sections populated in later milestones (rendered as stubs here):
26+
- Table inventory
27+
- Feature categories
28+
"""
29+
cfg = world_spec.config
30+
narrative = world_spec.narrative
31+
32+
lines: list[str] = []
33+
34+
# ------------------------------------------------------------------
35+
# Header
36+
# ------------------------------------------------------------------
37+
lines += [
38+
"# leadforge dataset card",
39+
"",
40+
"| Field | Value |",
41+
"|---|---|",
42+
f"| Recipe | `{cfg.recipe_id}` |",
43+
f"| Package version | `{cfg.package_version}` |",
44+
f"| Seed | `{cfg.seed}` |",
45+
f"| Exposure mode | `{cfg.exposure_mode}` |",
46+
f"| Difficulty | `{cfg.difficulty}` |",
47+
f"| Horizon | {cfg.horizon_days} days |",
48+
"",
49+
]
50+
51+
# ------------------------------------------------------------------
52+
# Narrative summary
53+
# ------------------------------------------------------------------
54+
lines.append("## Narrative summary")
55+
lines.append("")
56+
if narrative is not None:
57+
c = narrative.company
58+
p = narrative.product
59+
m = narrative.market
60+
gtm = narrative.gtm_motion
61+
lines += [
62+
f"**Vendor:** {c.name} ({c.stage}, founded {c.founded_year},"
63+
f" {c.hq_city}, {c.hq_country})",
64+
"",
65+
f"**Product:** {p.name}{p.category}. "
66+
f"Deployment: {p.deployment}. "
67+
f"Pricing: {p.pricing_model}. "
68+
f"ACV range: ${p.acv_range_usd[0]:,}–${p.acv_range_usd[1]:,}.",
69+
"",
70+
f"**Target market:** {m.icp_employee_range[0]}{m.icp_employee_range[1]}-employee"
71+
f" firms in {', '.join(m.geographies)}. "
72+
f"Key industries: {', '.join(m.icp_industries)}. "
73+
f"Average deal size: ${m.avg_deal_size_usd:,}. "
74+
f"Average sales cycle: {m.avg_sales_cycle_days} days.",
75+
"",
76+
f"**GTM motion:** {', '.join(gtm.channels)} "
77+
f"({gtm.inbound_share:.0%} inbound / "
78+
f"{gtm.outbound_share:.0%} outbound / "
79+
f"{gtm.partner_share:.0%} partner).",
80+
"",
81+
"**Buyer personas:**",
82+
"",
83+
]
84+
for persona in narrative.personas:
85+
ellipsis = "…" if len(persona.title_variants) > 2 else ""
86+
lines.append(
87+
f"- **{persona.role}** ({persona.decision_authority}) — "
88+
f"{', '.join(persona.title_variants[:2])}{ellipsis}"
89+
)
90+
lines.append("")
91+
else:
92+
lines += ["*Narrative unavailable for this dataset.*", ""]
93+
94+
# ------------------------------------------------------------------
95+
# Primary task
96+
# ------------------------------------------------------------------
97+
lines += [
98+
"## Primary task",
99+
"",
100+
"**Task:** `converted_within_90_days`",
101+
"",
102+
"**Label definition:** A lead is considered converted if a `closed_won` event "
103+
"is recorded within 90 days of the lead's snapshot anchor date. "
104+
"The label is derived from simulated events — it is never sampled directly.",
105+
"",
106+
]
107+
108+
# ------------------------------------------------------------------
109+
# Table inventory (stub — populated in later milestones)
110+
# ------------------------------------------------------------------
111+
lines += [
112+
"## Table inventory",
113+
"",
114+
"*Table counts will appear here once the simulation layer is implemented (v0.3.0+).*",
115+
"",
116+
]
117+
118+
# ------------------------------------------------------------------
119+
# Feature categories (stub)
120+
# ------------------------------------------------------------------
121+
lines += [
122+
"## Feature categories",
123+
"",
124+
"*Feature dictionary will appear here once the schema layer is implemented (v0.3.0+).*",
125+
"",
126+
]
127+
128+
# ------------------------------------------------------------------
129+
# Suggested use cases
130+
# ------------------------------------------------------------------
131+
lines += [
132+
"## Suggested use cases",
133+
"",
134+
"- Teaching binary classification on realistic CRM data",
135+
"- Portfolio projects demonstrating end-to-end ML pipelines",
136+
"- Benchmarking lead-scoring models under controlled signal/noise conditions",
137+
"- Research on causal structure in funnel conversion data",
138+
"",
139+
]
140+
141+
# ------------------------------------------------------------------
142+
# Caveats
143+
# ------------------------------------------------------------------
144+
lines += [
145+
"## Caveats",
146+
"",
147+
"- This is **synthetic** data. It does not represent any real company, product, or market.",
148+
"- The hidden world structure varies by motif family and stochastic rewiring; "
149+
"no two seeds produce the same DGP.",
150+
"- Features are anchored at the snapshot date. No post-anchor data is "
151+
"included (leakage-free by construction).",
152+
"- In `student_public` mode, the latent world graph, mechanism summary, "
153+
"and full world spec are withheld.",
154+
"",
155+
]
156+
157+
return "\n".join(lines)

0 commit comments

Comments
 (0)