User Guide - Imagines Nummorum VLM Data Extraction

Quick Start

Basic Usage

Prepare your images: Place images in an input directory
Configure the script: Set input/output paths
Run processing: Execute the main script
Review results: Check JSON outputs and extracted images

Simple Example

# Activate virtual environment
source .venv/bin/activate  # Linux/macOS
# .venv\Scripts\activate   # Windows

# Run processing
python src/coin_card_information_extraction.py

Step-by-Step Tutorial

Step 1: Organize Your Images

Create a well-organized input directory structure:

input_images/
├── collection_A/
│   ├── coin_card_001.jpg
│   ├── coin_card_002.jpg
│   └── text_page_001.jpg
├── collection_B/
│   ├── form_001.png
│   └── form_002.png
└── single_items/
    ├── item_001.tiff
    └── item_002.jpg

Supported formats: JPG, JPEG, PNG, TIFF, TIF, BMP

Step 2: Configure Processing Parameters

Edit the main script (src/coin_card_information_extraction.py) to set your paths:

# Essential configuration
INPUT_IMAGE_DIRECTORY = "C:/path/to/your/input_images"
OUTPUT_JSON_DIRECTORY = "C:/path/to/your/output_results"

# Model configuration
MODEL_ID = "Qwen/Qwen2.5-VL-32B-Instruct"  # or 7B for smaller model
MODEL_CACHE_DIR = "C:/path/to/model_cache"

# OCR strategy for text pages
OCR_STRATEGY_FOR_TEXT_PAGES = "both"  # tesseract_hocr_only, qwen_text_only, both

Step 3: Run the Processing Pipeline

Execute the main script:

Command Line:

python src/coin_card_information_extraction.py

Jupyter Notebook: If working in a Jupyter environment, you can run the script in a notebook cell:

%run src/coin_card_information_extraction.py

Alternative execution:

# From the project root directory
python -m src.coin_card_information_extraction

You'll see progress output like:

Loading Qwen-VL model...
Set Hugging Face cache directory to: C:\model_cache
Using device: cuda
Processing images: 100%|████████████| 50/50 [02:15<00:00,  2.70s/it]
Batch processing completed.
Total images processed: 50

Step 4: Review Results

The output directory will mirror your input structure:

output_results/
├── collection_A/
│   ├── coin_card_001.json
│   ├── coin_card_001_extracted_images/
│   │   ├── coin_card_001_1.png
│   │   └── coin_card_001_2.png
│   ├── coin_card_002.json
│   └── text_page_001.json
├── collection_B/
│   ├── form_001.json
│   └── form_002.json
└── single_items/
    ├── item_001.json
    └── item_002.json

Understanding Output Files

JSON Structure

Each processed image generates a JSON file with the following structure:

{
  "image_path_original": "/path/to/original/image.jpg",
  "image_type": "form",
  "handwritten_content": true,
  "data": {
    "form_data": {
      "coins": [
        {
          "id": "coin_1",
          "description": "Ancient Greek silver tetradrachm",
          "bounding_box": {
            "x": 150,
            "y": 100,
            "width": 200,
            "height": 200
          }
        }
      ],
      "card_fields": {
        "Atelier": "Athens",
        "Date": "440-430 BC",
        "Métal": "Silver",
        "Poids": "17.2g",
        "Diamètre": "24mm"
      }
    }
  },
  "images_extracted": ["coin_card_001_extracted_images/coin_card_001_1.png"],
  "status": "success",
  "error_message": null
}

Complete JSON Output Structure

The JSON structure varies by image type. Here's the complete format:

{
  "image_path_original": "/path/to/original/image.jpg",
  "image_type": "form|text_page|empty_page",
  "handwritten_content": true|false,
  "data": "varies by image_type - see detailed sections below",
  "images_extracted": ["relative/path/to/extracted_image.png"],
  "status": "success|classification_failed|form_extraction_failed|text_extraction_failed",
  "error_message": "detailed error message if failed"
}

Status Values:

"success": Processing completed successfully
"classification_failed": Stage 1 (image type detection) failed
"form_extraction_failed": Form data extraction failed
"text_extraction_failed": OCR processing failed

Data Structure by Image Type:

Empty Pages:

{
  "data": null
}

Text Pages:

{
  "data": {
    "ocr_results": [
      {
        "source": "tesseract|qwen",
        "type": "hocr_xml|plain_text",
        "content": "extracted text or hOCR XML",
        "status": "success|failed",
        "error_message": null
      }
    ]
  }
}

Forms:

{
  "data": {
    "form_data": {
      "coins": [...],
      "card_fields": {...}
    }
  }
}

Image Types and Their Outputs

1. Form Images

Contains: Coin images and metadata cards
Output: Structured data + cropped coin images
Use case: Museum catalog cards, documentation forms

Example output:

{
  "image_type": "form",
  "data": {
    "form_data": {
      "coins": [...],
      "card_fields": {...}
    }
  },
  "images_extracted": ["image_1.png", "image_2.png"]
}

Cropped Image Details:

Directory: Saved in {original_name}_extracted_images/ subdirectory
Naming: {original_name}_{coin_number}.png (e.g., coin_card_001_1.png)
Format: PNG with optimized margins
Margins: Automatically detected based on background uniformity

2. Text Pages

Contains: Textual content, manuscripts, printed pages
Output: Extracted text using OCR
Use case: Historical documents, catalog pages

Example output:

{
  "image_type": "text_page",
  "data": {
    "ocr_results": [
      {
        "source": "tesseract",
        "type": "hocr",
        "content": "<xml>hOCR formatted text</xml>",
        "status": "success"
      },
      {
        "source": "qwen_vl",
        "type": "plain_text",
        "content": "Plain text extraction",
        "status": "success"
      }
    ]
  }
}

3. Empty Pages

Contains: Blank or nearly empty pages
Output: Minimal processing
Use case: Separator pages, blank forms

Advanced Usage

Batch Processing Large Collections

For processing thousands of images:

Monitor system resources:

# Linux/macOS - monitor memory usage
watch -n 5 free -h
htop

# Windows - Task Manager or PowerShell
Get-Process | Sort-Object WorkingSet -Descending

Process in chunks if needed:

# Modify the script to process subdirectories separately
import os
subdirs = [d for d in os.listdir(INPUT_IMAGE_DIRECTORY)
          if os.path.isdir(os.path.join(INPUT_IMAGE_DIRECTORY, d))]

for subdir in subdirs:
    input_path = os.path.join(INPUT_IMAGE_DIRECTORY, subdir)
    output_path = os.path.join(OUTPUT_JSON_DIRECTORY, subdir)
    # Process this subdirectory

Custom OCR Strategies

Choose the appropriate OCR strategy based on your content:

# Available OCR strategies for text pages:

# Option 1: Tesseract only (outputs hOCR XML format)
OCR_STRATEGY_FOR_TEXT_PAGES = "tesseract_hocr_only"

# Option 2: Qwen-VL only (outputs plain text)
OCR_STRATEGY_FOR_TEXT_PAGES = "qwen_text_only"

# Option 3: Try Tesseract first, fallback to Qwen-VL if it fails
OCR_STRATEGY_FOR_TEXT_PAGES = "tesseract_then_qwen_fallback"

# Option 4: Try Qwen-VL first, fallback to Tesseract if it fails
OCR_STRATEGY_FOR_TEXT_PAGES = "qwen_then_tesseract_fallback"

# Option 5: Run both methods independently (maximum accuracy)
OCR_STRATEGY_FOR_TEXT_PAGES = "both"

Strategy Selection Guidelines:

For multilingual text: Use "tesseract_hocr_only" - Tesseract has better language support
For handwritten content: Use "qwen_text_only" - AI models handle handwriting better
For maximum accuracy: Use "both" - Compare results from both methods
For speed: Use "qwen_text_only" or "tesseract_hocr_only"
For reliability: Use fallback strategies with primary + backup method

Optimizing Coin Extraction

Adjust cropping parameters for different image qualities:

# For high-resolution images
CROP_MARGIN_PIXELS = 60
MAX_CROP_MARGIN = 150

# For low-resolution or noisy images
INITIAL_CROP_MARGIN = 30
COLOR_SIMILARITY_THRESHOLD = 40
EDGE_UNIFORMITY_THRESHOLD = 0.75

Data Analysis Workflow

Convert to CSV for Analysis

After processing, convert JSON results to CSV:

python src/json_to_csv.py

This creates coin_data.csv with flattened data suitable for:

Excel analysis
Database import
Statistical analysis
Machine learning training

Validate Processing Results

Check processing completeness:

python src/validate_data.py

Sample validation output:

=== Processing Verification Results ===
Total images found: 1247
Successfully processed: 1189
Success rate: 95.3%

❌ Missing JSON files (12):
  - /path/to/image1.jpg → /path/to/image1.json
  - /path/to/image2.jpg → /path/to/image2.json

❌ Failed processing (46):
  - /path/to/image3.jpg
    Status: classification_failed
    Error: Unable to classify image content

Working with Specific Content Types

Museum Catalog Cards

Typical workflow for museum documentation:

Scan catalog cards at 300+ DPI
Organize by collection or period
Use "both" OCR strategy for maximum text extraction
Review card_fields extraction for metadata accuracy
Export to CSV for database integration
Batch process by date/collection
Monitor for handwritten annotations

Quality Assurance

Reviewing Results

Check success rates:

python src/validate_data.py

Sample random outputs:

# Review a random sample of JSON files
ls output_results/**/*.json | shuf -n 10 | xargs -I {} jq '.status' {}

Validate extracted images:

import os
from PIL import Image

def check_extracted_images(output_dir):
    for root, dirs, files in os.walk(output_dir):
        for file in files:
            if file.endswith('.png'):
                try:
                    img = Image.open(os.path.join(root, file))
                    if img.size[0] < 50 or img.size[1] < 50:
                        print(f"Small image: {file} - {img.size}")
                except Exception as e:
                    print(f"Corrupted image: {file} - {e}")

Common Quality Issues

Poor OCR results: Check image quality, adjust OCR strategy
Missing coin extractions: Verify bounding box detection
Incorrect classifications: Review problematic images manually
Incomplete metadata: Check card field extraction accuracy

Performance Optimization

Hardware Optimization

Use GPU acceleration:

# Verify GPU usage
import torch
print(f"Using GPU: {torch.cuda.get_device_name(0)}")

Optimize memory usage:

# Clear cache between batches
torch.cuda.empty_cache()

Adjust batch sizes based on available memory

Processing Optimization

Pre-filter images to remove obvious non-content
Use smaller model for initial classification, larger for extraction
Parallel processing for independent operations
Resume interrupted processing using existing JSON checks

Integration with External Tools

Database Integration

Export to various database formats:

import pandas as pd
import sqlite3

# Load CSV data
df = pd.read_csv('output/coin_data.csv')

# Export to SQLite
conn = sqlite3.connect('numismatic_data.db')
df.to_sql('coin_records', conn, if_exists='replace', index=False)

Web Interface Integration

Create a simple web viewer:

from flask import Flask, render_template, jsonify
import json
import os

app = Flask(__name__)

@app.route('/api/results')
def get_results():
    results = []
    for root, dirs, files in os.walk('output_results'):
        for file in files:
            if file.endswith('.json'):
                with open(os.path.join(root, file)) as f:
                    results.append(json.load(f))
    return jsonify(results)

Export Formats

Convert to various formats for different applications:

# Export to Excel with multiple sheets
with pd.ExcelWriter('numismatic_analysis.xlsx') as writer:
    coins_df.to_excel(writer, sheet_name='Coins', index=False)
    metadata_df.to_excel(writer, sheet_name='Metadata', index=False)

# Export to TEI XML for digital humanities
def export_to_tei(data):
    # Convert structured data to TEI format
    pass

Troubleshooting Common Issues

Processing Failures

Check input image format and quality
Verify sufficient disk space and memory
Review error messages in JSON outputs
Test with single images first

Performance Issues

Monitor system resources during processing
Adjust model size based on available hardware
Process smaller batches if memory is limited
Use CPU-only mode if GPU issues occur

For detailed troubleshooting, see TROUBLESHOOTING.md.

Best Practices

File Organization

Use descriptive directory names
Maintain consistent naming conventions
Keep original images in separate backup location
Document your organizational scheme

Processing Workflow

Start with small test batches
Validate results before large-scale processing
Monitor system resources during long runs
Keep processing logs for troubleshooting

Data Management

Regular backups of processing results
Version control for configuration changes
Document any manual corrections or annotations
Maintain chain of custody for source materials

Next Steps

After mastering basic usage:

Review Technical Documentation for advanced features
Explore API Reference for customization options
Check Examples directory for specific use cases
Consider contributing improvements via GitHub

Uh oh!

FilesExpand file tree

USER_GUIDE.md

Latest commit

History