ChestXpert

A comprehensive AI-powered medical imaging analysis platform for chest X-ray diagnosis and triage prioritization.

Project Overview

This system provides automated pneumonia detection from chest X-ray images using deep learning, with features including AI-powered attention heatmaps, triage prioritization, physician validation workflows, and automated medical report generation.

Core Features

1. AI-Powered Image Analysis

• Binary Pneumonia Detection: Classifies chest X-rays as Normal or Pneumonia
• Vision Transformer Model: Uses state-of-the-art ViT architecture fine-tuned for chest X-ray analysis
• Confidence Scoring: Provides prediction confidence levels (0-100%)
• Batch Processing: Supports multiple image uploads and analysis

2. Attention Heatmap Visualization

• Grad-CAM Integration: Generates attention heatmaps showing AI focus areas
• Visual Interpretation: Highlights regions of interest in X-ray images
• Side-by-Side Comparison: Original X-ray and heatmap displayed together
• Automatic Fallback: Creates alternative visualizations if attention fails

3. Intelligent Triage System

• Priority Classification: Automatically assigns Critical, Urgent, or Normal priority
• Confidence-Based Logic: Higher confidence pneumonia cases flagged as Critical (>70%)
• Dashboard Organization: Cases sorted by priority level for efficient workflow
• Color-Coded Badges: Visual priority indicators (red/yellow/green)

4. Validation Workflow

• Three-State System: Pending, Validated, Rejected status tracking
• Radiologist Review: Medical professionals can validate or reject AI findings
• Validation Notes: Optional comments for decision documentation
• Audit Trail: Timestamp and validator tracking for all validations

5. AI-Generated Medical Reports

• Local LLM Integration: Uses BioGPT-Large for medical text generation
• Professional PDF Format: Hospital-grade report layout with branding
• Comprehensive Sections: Clinical impression, detailed findings, recommendations
• Image Inclusion: X-ray and heatmap embedded in report
• One-Click Generation: Automatic report creation from analysis data

6. Patient Management

• Patient Records: Name, birth date, and notes tracking
• Analysis History: Complete timeline of all patient X-rays and results
• Privacy-First: Local database storage, no cloud data transmission
• Bulk Operations: Multi-select delete functionality

Technology Stack

Backend

Framework & API

• FastAPI: Modern Python web framework for REST API
• Uvicorn: ASGI server with auto-reload support
• Pydantic: Data validation and settings management
• Python-Jose: JWT token authentication

Database

• SQLAlchemy: ORM for database operations
• SQLite: Lightweight relational database
• Alembic: Database migration management

AI/ML Stack

• PyTorch 2.6+: Deep learning framework
• Transformers 4.46+: Hugging Face model library
• Timm: Vision model utilities
• Model: nickmuchi/vit-finetuned-chest-xray-pneumonia
  • Architecture: Vision Transformer (ViT)
  • Training: Fine-tuned on chest X-ray pneumonia dataset
  • Classes: Normal, Pneumonia
  • Input Size: 224x224 RGB images

Report Generation

• BioGPT-Large: Medical language model for report text
  • Model: microsoft/BioGPT
  • Size: ~1.5GB
  • Device: Auto-detect GPU/CPU
  • Purpose: Generate clinical impressions and recommendations
• ReportLab: Professional PDF generation
• Pillow: Image processing for PDF embedding
• Sacremoses: BioGPT tokenizer dependency

Image Processing

• OpenCV: Computer vision operations
• Matplotlib: Heatmap visualization and plotting
• Scikit-learn: Additional ML utilities

Frontend

Core Framework

• React 18: Component-based UI library
• Vite: Fast build tool and dev server
• React Router: Client-side routing

Styling

• Tailwind CSS: Utility-first CSS framework
• Custom Components: Reusable UI elements
• Dark Mode Support: Theme switching capability

State Management

• React Hooks: useState, useEffect for local state
• Context API: Global authentication state
• Local Storage: JWT token persistence

HTTP Client

• Axios: Promise-based HTTP client
• Interceptors: Automatic token attachment and error handling

System Architecture

Request Flow

1. User uploads X-ray image via React frontend
2. FastAPI receives image, creates patient record
3. Vision Transformer analyzes image
4. GradCAM generates attention heatmap
5. Priority level calculated based on prediction + confidence
6. Results stored in SQLite database
7. Response sent to frontend with image URLs
8. Dashboard displays results with validation options
9. Optional: Generate AI-powered PDF report

Security

• JWT Authentication: Token-based user authentication
• Password Hashing: Bcrypt for secure password storage
• CORS Protection: Configured for localhost development
• Input Validation: Pydantic schemas for all API requests
• File Type Validation: Image format checking on upload

Data Storage

data/
├── uploads/          # Original X-ray images
├── heatmaps/         # Generated attention heatmaps
└── reports/          # AI-generated PDF reports

backend/
└── medical_triage.db # SQLite database

Database Schema

Users Table

• id, name, email, password_hash, role, created_at
• Relationships: validated_analyses

Patients Table

• id, name, birth_date, notes, created_at
• Relationships: images

Images Table

• id, patient_id, filepath, original_filename, upload_date, image_type
• Relationships: patient, analysis

Analysis Results Table

• id, image_id, prediction, confidence, all_probabilities
• priority, priority_score, heatmap_path
• validated, validation_status, validated_by, validation_notes, validation_date
• created_at, updated_at
• Relationships: image, validator

Model Performance

Vision Transformer

• Accuracy: High accuracy on pneumonia detection
• Speed: ~2-5 seconds per image on GPU, ~10-15 seconds on CPU
• Input: 224x224 RGB chest X-ray images
• Output: Binary classification with confidence scores

BioGPT Report Generation

• Quality: Medical-grade text with clinical terminology
• Speed: 30-60 seconds first run (model download), ~10-20 seconds subsequent
• Fallback: Template-based reports if LLM fails
• Device Support: CUDA GPU or CPU inference

API Endpoints

Authentication

• POST /auth/register - Create new user account
• POST /auth/login - Login and receive JWT token
• GET /auth/me - Get current user info

Analysis

• POST /analysis/upload - Upload X-ray image
• POST /analysis/run - Run AI analysis
• GET /analysis/triage - Get triage dashboard data
• GET /analysis/{id} - Get analysis details
• PUT /analysis/{id}/validate - Validate/reject analysis
• DELETE /analysis/{id} - Delete analysis and files
• POST /analysis/{id}/generate-report - Generate PDF report

Patients

• POST /patients - Create patient record
• GET /patients - List all patients
• GET /patients/{id} - Get patient details

Installation & Setup

Prerequisites

• Python 3.10+
• Node.js 16+
• GPU with CUDA (optional, for faster inference)

Backend Setup

cd backend
python -m venv venv
venv\Scripts\activate  # Windows
pip install -r requirements.txt
python setup.py  # Initialize database
uvicorn main:app --reload

Frontend Setup

cd frontend
npm install
npm run dev

First Run Notes

• BioGPT model downloads automatically (~1.5GB) on first report generation
• ViT model downloads from Hugging Face on first analysis
• Create initial user via /auth/register endpoint

Development Environment

Backend (Port 8000)

• API Documentation: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc
• Auto-reload enabled for code changes

Frontend (Port 5173)

• Development Server: http://localhost:5173
• Hot Module Replacement (HMR) enabled

Key Design Decisions

Local-First Architecture

• All data stored locally, no cloud dependencies
• Privacy-compliant for medical data
• Offline-capable after initial model downloads

GPU/CPU Flexibility

• Automatic device detection (CUDA/CPU)
• Graceful degradation on CPU-only systems
• Performance warnings for slower hardware

Binary Classification Choice

• Focused on pneumonia detection accuracy
• Simpler than multi-disease models
• Better calibration than 14-disease alternatives
• Easier validation workflow for radiologists

Validation Workflow

• Three-state system balances simplicity and functionality
• Pending default prevents false confidence
• Notes field for documentation without complexity

Report Generation Strategy

• Local LLM avoids API costs and privacy concerns
• Template fallback ensures reliability
• Professional PDF format for clinical use

Future Enhancements

Potential Features

• Multi-disease classification support
• DICOM format support
• Integration with PACS systems
• Mobile app for field use
• Batch report generation
• Advanced analytics dashboard
• Model performance tracking
• User role management (admin, radiologist, technician)

Technical Improvements

• Model fine-tuning on custom datasets
• Real-time analysis streaming
• Automated model updates
• Performance monitoring
• Load balancing for production
• Docker containerization
• CI/CD pipeline setup

License & Usage

This project is for educational and research purposes. Medical use requires appropriate validation, regulatory approval, and clinical oversight.

Credits

• Vision Transformer Model: nickmuchi/vit-finetuned-chest-xray-pneumonia
• Medical Language Model: Microsoft BioGPT
• Framework: FastAPI, React
• UI Components: Tailwind CSS

---

Version: 1.0.0
Last Updated: December 2025
Status: Production-Ready