Skip to content

API-CLIENT-TESTING-PLAN.adoc

Latest commit

 

History

History
200 lines (142 loc) · 6.24 KB

API-CLIENT-TESTING-PLAN.adoc

File metadata and controls

200 lines (142 loc) · 6.24 KB

ReleaseHx Demo API Client Testing

Overview

This document focuses on testing the ReleaseHx API clients using the demo configurations and mock data available in this repository. It complements the comprehensive authentication setup and testing guidance in the main ReleaseHx documentation.

Demo Data Testing

JIRA Demo Testing

Test with existing mock JSON data:

# Test basic JIRA configuration with mock data
bundle exec rhx 1.1.0 --api-data _payloads/jira-customfield-note-1.1.0.json --config _configs/jira-customfield.yml --yaml

# Test description-based release notes
bundle exec rhx 1.1.0 --api-data _payloads/jira-description-note-1.1.0.json --config _configs/jira-description.yml --md

# Test changelog-only output
bundle exec rhx 1.1.0 --api-data _payloads/jira-customfield-note-1.1.0.json --config _configs/jira-customfield-changelog.yml --html

# Test heavy configuration with all features
bundle exec rhx 1.1.0 --api-data _payloads/jira-customfield-note-1.1.0.json --config _configs/jira-customfield-heavy.yml --pdf

GitHub Demo Testing

Test with GitHub mock data and different tag strategies:

# Test checkbox-based tagging
bundle exec rhx 1.1.0 --api-data _payloads/github-checkbox-tags-1.1.0.json --config _configs/github-basic.yml --yaml

# Test label-based tagging
bundle exec rhx 1.1.0 --api-data _payloads/github-label-tags-1.1.0.json --config _configs/github-label-tags.yml --adoc

# Test with actual GitHub data (if available)
# bundle exec rhx 0.2.0 --api-data _payloads/github-issuer-actual-0.2.0.json --config _configs/github-basic.yml --html

GitLab Demo Testing

Test with GitLab mock data configurations:

# Test checkbox-based tagging
bundle exec rhx 1.1.0 --api-data _payloads/gitlab-checkbox-tags-1.1.0.json --config _configs/gitlab-basic.yml --yaml

# Test label-based tagging
bundle exec rhx 1.1.0 --api-data _payloads/gitlab-label-tags-1.1.0.json --config _configs/gitlab-label-tags.yml --md

Generic Demo Testing

Test with version progression using generic demo data:

# Test version 1.1.1
bundle exec rhx 1.1.1 --api-data _payloads/generic-1.1.1.json --config _configs/github-basic.yml --yaml

# Test version 1.1.2
bundle exec rhx 1.1.2 --api-data _payloads/generic-1.1.2.json --config _configs/gitlab-basic.yml --adoc

# Test version 1.2.0
bundle exec rhx 1.2.0 --api-data _payloads/generic-1.2.0.json --config _configs/jira-customfield.yml --html

Testing Phases

Configuration Validation

Syntax testing

Ensure YAML files parse correctly

Template rendering

Verify Liquid templates render with test data

Authentication

Test auth header generation with mock credentials

Static JSON Testing

JIRA

Test with existing mock JSON files

GitHub/GitLab

Test with downloaded sample responses

Mapping validation

Ensure data maps correctly to RHYML format

Live API Testing

GitHub

Test with real GitHub repository

GitLab

Test with real GitLab project

Error handling

Test with invalid tokens, missing repos, etc.

Integration Testing

End-to-end

Full workflow from API → RHYML → AsciiDoc/HTML/PDF

Authentication methods

Test different auth modes

Pagination

Test with large result sets

Edge cases

Empty responses, API errors, network timeouts

Test Data Requirements

GitHub Repository Setup

Create a test repository with:

  • Multiple issues with different labels

  • At least one milestone named "1.1.0"

  • Issues assigned to the milestone

  • Mix of open/closed issues

  • Issues with and without release note content

GitLab Project Setup

Create a test project with:

  • Multiple issues with different labels

  • At least one milestone named "1.1.0"

  • Issues assigned to the milestone

  • Mix of open/closed issues

  • Issues with merge request links

Expected Blockers & Solutions

Authentication Issues

Problem

Invalid tokens, wrong scopes

Solution

Clear error messages, token validation

API Rate Limiting

Problem

GitHub/GitLab rate limits

Solution

Implement retry logic, respect rate limit headers

Data Format Variations

Problem

Different response formats between API versions

Solution

Flexible mapping configurations, validation

Missing Data

Problem

Required fields not present in API responses

Solution

Default values, conditional mapping

Success Criteria

Client Criteria

Configuration file loads without errors

Test YAML parsing and validation

Authentication headers generate correctly

Verify proper auth token/header formatting

API requests execute successfully

When credentials available, test actual API calls

JSON responses map to valid RHYML format

Ensure data transformation works correctly

Generated AsciiDoc/HTML renders correctly

Validate final output quality

Error conditions handled gracefully

Test failure scenarios and error messages

Integration Criteria

All three APIs work with demo configurations

JIRA, GitHub, and GitLab clients functional

API → AsciiDoc → HTML/PDF pipeline works end-to-end

Complete workflow validation

Documentation accurately reflects capabilities

Ensure docs match implementation

Error messages are helpful for troubleshooting

User-friendly error reporting

Next Steps

Immediate

Test JIRA with static JSON files

Short-term

Set up GitHub/GitLab test repositories

Medium-term

Implement comprehensive error handling

Long-term

Add support for additional API endpoints (pull requests, etc.)