Commit 5164ddc
docs: fix modifying-prompts-metrics guide with correct API (vibrantlabsai#2486)
## Issue Link / Problem Description
The modifying-prompts-metrics guide had significant factual errors that
would cause code examples to fail:
- Used non-existent class names (`FaithfulnessInput`,
`FaithfulnessPrompt`, etc.)
- Referenced wrong attribute names (`scorer.prompt` instead of
`scorer.statement_generator_prompt`)
- Documented save/load functionality that doesn't exist on the
collections `BasePrompt` class
## Changes Made
- **Fixed incorrect class names**: `FaithfulnessInput` →
`StatementGeneratorInput`, `FaithfulnessOutput` →
`StatementGeneratorOutput`, etc.
- **Fixed wrong attribute names**: `scorer.prompt` →
`scorer.statement_generator_prompt` / `scorer.nli_statement_prompt`
- **Removed non-existent save/load section**: ~175 lines documenting
functionality that doesn't exist on
`ragas.prompt.metrics.base_prompt.BasePrompt`
- **Added cross-references**: Quick start pointing to
`DiscreteMetric`/`NumericMetric` for simpler custom metrics
- **Converted to pure markdown**: Deleted `.ipynb` source, created clean
`.md` file
- **Fixed mkdocstrings link**: Changed
`[BasePrompt][ragas.prompt.metrics.base_prompt.BasePrompt]` to
`[BasePrompt][ragas.prompt.BasePrompt]` (public API)
- **Added docs rule**: Added API references guidance to
`.cursor/rules/docs-structure.mdc`
## Testing
### How to Test
- [x] Automated tests added/updated
- [x] Manual testing steps:
1. Run `uv run python tests/docs/test_modifying_prompts_metrics.py` -
all 7 tests pass
2. Run `make serve-docs` and verify the guide renders correctly
3. Verify all code examples in the guide work
### Test Results
```
✓ Faithfulness has statement_generator_prompt and nli_statement_prompt
✓ Generated prompt string successfully
✓ Custom FactualCorrectness prompt evaluation completed. Score: 1.0
✓ Custom Faithfulness NLI prompt evaluation completed. Score: 1.0
✓ NLI prompt string generated successfully
✓ Faithfulness evaluation completed. Score: 1.0
✓ Language adaptation completed. Score: 1.0
✓ All tests passed!
```
## References
- Related to PR vibrantlabsai#2394 (documentation structure update)
- Related to PR vibrantlabsai#2407 (metrics collections API documentation)
- Fixes errors introduced in commits `691ad9f` and `e62ff19`1 parent c4b44e1 commit 5164ddc
7 files changed
Lines changed: 191 additions & 629 deletions
File tree
- .cursor/rules
- docs
- concepts/metrics/overview
- howtos/customizations/metrics
- references
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
35 | 40 | | |
36 | 41 | | |
37 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
| 64 | + | |
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| |||
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
98 | | - | |
| 98 | + | |
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
| 109 | + | |
| 110 | + | |
109 | 111 | | |
110 | 112 | | |
111 | 113 | | |
| |||
Lines changed: 0 additions & 350 deletions
This file was deleted.
0 commit comments