Skip to content

RAG_ASK_ENDPOINT.md

Latest commit

 

History

History
210 lines (157 loc) · 7.03 KB

RAG_ASK_ENDPOINT.md

File metadata and controls

210 lines (157 loc) · 7.03 KB
Raw

Provider-backed Grounded Ask Endpoint

POST /api/documents/ask retrieves authorized tenant/owner-scoped chunks first, then delegates answer construction to a configured IAnswerGenerator.

The default provider remains deterministic, local, credential-free, and extractive. An optional OpenAI-compatible Chat Completions provider can be enabled explicitly without changing the retrieval or source-response contract.

Request

{
  "question": "Who approves vendor contracts?",
  "topK": 3
}

topK is optional and defaults to 3.

Successful grounded response

{
  "question": "Who approves vendor contracts?",
  "answer": "The Finance Director approves vendor contracts [S1].",
  "sourceCount": 3,
  "sources": [
    {
      "documentId": "00000000-0000-0000-0000-000000000000",
      "fileName": "contract-policy.txt",
      "chunkIndex": 0,
      "score": 0.91,
      "text": "The Finance Director approves vendor contracts."
    }
  ],
  "answerStatus": "answered",
  "answerProvider": "openai-compatible",
  "answerModel": "configured-model",
  "isGrounded": true,
  "reasonCode": null
}

The original question, answer, sourceCount, and sources fields remain in the same order. Provider metadata is additive. Retrieved sources are produced before answer generation and remain independent from provider-generated text.

Insufficient evidence

The endpoint returns HTTP 200 with an explicit non-answer when:

  • no visible source was retrieved;
  • the highest source score is below AnswerGeneration:MinimumSourceScore;
  • the two highest sources from different documents are within the configured conflict delta;
  • the provider returns the exact INSUFFICIENT_EVIDENCE sentinel.

Example:

{
  "question": "What is the data retention period?",
  "answer": "I could not find enough indexed document evidence to answer this question safely.",
  "sourceCount": 0,
  "sources": [],
  "answerStatus": "insufficient_evidence",
  "answerProvider": "deterministic",
  "answerModel": "local-extractive-v1",
  "isGrounded": false,
  "reasonCode": "no_evidence"
}

Reason codes:

  • no_evidence;
  • low_confidence;
  • conflicting_evidence;
  • provider_declined.

The provider is not called for missing, low-confidence, or conflicting evidence.

Provider failure responses

Provider failures preserve the retrieved sources but do not return a model answer:

{
  "question": "Who approves vendor contracts?",
  "message": "The configured answer provider is temporarily unavailable.",
  "code": "answer_provider_unavailable",
  "retryable": true,
  "sourceCount": 3,
  "sources": []
}

Controlled mappings:

Condition HTTP Code Retryable
timeout 504 answer_provider_timeout yes
network/5xx 503 answer_provider_unavailable yes
rate limit 503 answer_provider_rate_limited yes
provider credential rejection 502 answer_provider_authentication_failed no
invalid JSON 502 answer_provider_invalid_response no
empty answer 502 answer_provider_empty_response no
missing/out-of-range citation 502 answer_provider_ungrounded_response no
other rejected request 502 answer_provider_rejected_request no

Provider response bodies and credentials are not returned to clients or stored in audit metadata.

Provider selection

Default configuration:

AnswerGeneration__Provider=Deterministic

Optional OpenAI-compatible provider:

AnswerGeneration__Provider=OpenAiCompatible
AnswerGeneration__OpenAiCompatible__Endpoint=https://provider.example/v1/chat/completions
AnswerGeneration__OpenAiCompatible__ApiKey=<secret>
AnswerGeneration__OpenAiCompatible__Model=<model-name>
AnswerGeneration__OpenAiCompatible__TimeoutSeconds=20
AnswerGeneration__OpenAiCompatible__MaxOutputTokens=500

The external provider is Fail-Closed:

  • endpoint, API key, and model are mandatory when selected;
  • the endpoint must use HTTPS, except a loopback HTTP endpoint for local testing;
  • timeout must be between 1 and 120 seconds;
  • maximum output tokens must be between 1 and 8192;
  • invalid configuration prevents application startup.

Do not commit provider credentials. Supply them through an approved secret-management mechanism.

Grounding boundary

Before any provider call:

  1. JWT authentication and tenant/owner authorization are applied.
  2. Retrieval runs through the existing tenant-aware semantic-index path.
  3. source count is limited by AnswerGeneration:MaxSources;
  4. combined source text is limited by AnswerGeneration:MaxContextCharacters;
  5. the question is limited to 4,000 characters in the provider prompt;
  6. each source receives a stable request-local marker such as [S1].

The system prompt states that source content is untrusted data. Instructions, role changes, credential requests, or prompt-injection text inside a document must not be followed.

A provider answer is accepted only when it contains at least one valid citation marker referring to a supplied source. Source metadata is not parsed from provider output.

Deterministic provider

DeterministicAnswerGenerator remains the default and requires no external service. It returns an extractive answer based on the highest-ranked acceptable source and includes [S1].

This mode is intended for local development, deterministic tests, and environments that have not approved external data transfer.

Telemetry and audit

Operational signals include:

  • provider name and answer status;
  • generation duration;
  • controlled failure code and retryability;
  • provider-reported input/output token counts when available;
  • source count and topK.

The following are excluded from audit details and metric tags:

  • question text;
  • source/chunk text;
  • generated answer text;
  • bearer tokens;
  • provider API keys;
  • provider response bodies.

Answer-quality evaluation

Run the credential-free evaluation from the repository root:

dotnet run --project tools/answer-evaluation/EnterpriseDocumentAssistant.AnswerEvaluation.csproj

The versioned dataset under evaluation/answers/ covers:

  • grounded deterministic and scripted-provider answers;
  • missing evidence;
  • low-confidence evidence;
  • conflicting near-tie evidence;
  • provider-declined answers;
  • uncited answers;
  • out-of-range citations.

CI requires 100% accuracy for the initial eight-case grounding-gate baseline and uploads a machine-readable JSON report for fourteen days.

Privacy, cost, and deployment review

Enabling an external provider may transfer authorized document excerpts and the user question outside the deployment boundary. Before activation, review:

  • provider data retention and training terms;
  • geographic residency and subprocessors;
  • contractual and regulatory restrictions;
  • tenant confidentiality requirements;
  • token limits and truncation behavior;
  • request and token costs;
  • rate limits and availability targets;
  • incident response and key rotation.

This repository does not bundle a production provider account, secret manager, data-processing agreement, or factual-accuracy guarantee.