Skip to content

Commit 52fe43a

Browse files
rvernicaanistark
authored andcommitted
Fix 2440 Adjust LLM parameters in evals.py (vibrantlabsai#2441)
## Issue Link / Problem Description - Fixes vibrantlabsai#2440 ## Changes Made Adjust llm_factory arguments as the default values do not work for GPT-5 mini - `temperature=1` - `top_p=None` ## Testing ### How to Test - [ ] Automated tests added/updated - [x] Manual testing steps: 1. `python -m ragas_examples.improve_rag.evals` ## References - Documentation: https://docs.ragas.io/en/stable/howtos/applications/evaluate-and-improve-rag/#run-initial-rag-experiment ## Screenshots/Examples (if applicable) N/A
1 parent a09396d commit 52fe43a

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • examples/ragas_examples/improve_rag

examples/ragas_examples/improve_rag/evals.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@ async def run_experiment(mode: str = "naive", model: str = "gpt-5-mini", name: O
206206
dataset,
207207
name=name or f"{datetime.now().strftime('%Y%m%d-%H%M%S')}_{'agenticrag' if mode == 'agentic' else 'naiverag'}",
208208
rag=rag,
209-
llm=llm_factory("gpt-5-mini", client=openai_client)
209+
llm=llm_factory("gpt-5-mini", client=openai_client, temperature=1, top_p=None)
210210
)
211211

212212
# Print basic results

0 commit comments

Comments
 (0)