Pest Scan AI is a two-stage computer vision system for agricultural pest identification. It combines YOLOv8 object detection with ResNet50 classification to identify 102 insect pest species from the IP102 dataset and provide treatment recommendations via an indexed local SQLite lookup.
The system is designed for single-machine inference with a clean REST API, making it suitable for deployment in agricultural field stations, research labs, and edge devices with GPU access.
Pest datasets are inherently imbalanced — some species appear thousands of times while others have fewer than 50 samples. Standard cross-entropy would bias the model toward majority classes. We apply inverse-frequency class weighting:
L = -(1/N) * Σᵢ [ Σ_c ( w_c · y_{i,c} · log(p_{i,c}) ) ]
where:
w_c = N_total / (C × N_c) — inverse-frequency class weight
p_{i,c} = softmax(z_{i,c}) — predicted probability for class c
y_{i,c} ∈ {0, 1} — one-hot ground truth label
N = batch size
C = 102 (number of classes)
N_c = number of samples in class c
This ensures that rare pest species contribute proportionally to the gradient signal, preventing the model from ignoring minority classes.
Raw softmax outputs require calibration. We perform a precision–recall curve sweep across confidence thresholds to find the operating point that maximizes the F1 score:
- For each threshold t ∈ [0.01, 0.99], compute precision and recall by treating predictions with confidence ≥ t as positive.
- Compute the F1 score at each threshold:
F1(t) = 2·P(t)·R(t) / (P(t) + R(t)). - Select
t* = argmax_t F1(t)as the optimal confidence threshold. - Compute PR-AUC via the trapezoidal rule over the full precision–recall curve.
Input Image
│
▼
┌──────────────────┐
│ YOLOv8 Detection│ ──▶ Bounding boxes + confidence scores
└──────────────────┘
│
▼ (crop detected regions)
┌──────────────────┐
│ ResNet50 │ ──▶ Class probabilities (102 classes)
│ Classification │
└──────────────────┘
│
▼
┌──────────────────┐
│ SQLite Lookup │ ──▶ Treatment recommendations
│ (Indexed DB) │
└──────────────────┘
│
▼
JSON Response: {class_id, class_name, confidence, bbox, treatment}
Stage 1 — Detection: YOLOv8 localises pest regions in the input image, producing bounding boxes with confidence scores. This stage handles images with multiple pests and filters background noise.
Stage 2 — Classification: Each cropped region is resized to 224×224 and fed through a fine-tuned ResNet50 backbone. The final fully connected layer outputs probabilities over 102 pest classes.
Treatment Lookup: Predicted class IDs are used to query an indexed SQLite database containing species-specific treatment recommendations, including chemical, biological, and cultural control methods.
| Property | Detail |
|---|---|
| Name | IP102 — Insect Pest Recognition |
| Source | Wu et al., "IP102: A Large-Scale Benchmark Dataset for Insect Pest Recognition" (CVPR 2019) |
| Classes | 102 insect pest species |
| Images | ~75,000 labelled samples |
| Crops Covered | Rice, wheat, corn, cotton, citrus, vegetables, legumes |
| Splits | Train / Validation / Test |
| Component | Technology |
|---|---|
| Object Detection | YOLOv8 (Ultralytics) |
| Image Classification | ResNet50 (PyTorch, fine-tuned) |
| Treatment Database | SQLite3 (indexed, local) |
| REST API | FastAPI + Uvicorn |
| Image Processing | OpenCV, Pillow |
| Data Pipeline | PyTorch DataLoader, torchvision |
| Metrics & Tuning | NumPy (hand-written), Matplotlib |
| Language | Python 3.10+ |
# Clone the repository
git clone https://github.com/Komatlakarthik/pest-scan.git
cd pest-scan
# Install dependencies
pip install -r requirements.txt
# Download and organise the IP102 dataset
python tests/download_dataset.py --download --organize
# Run evaluation on the test split
python tests/run_evaluation.py
# Start the API server
uvicorn api.main:app --reloadcurl http://localhost:8000/api/v1/health{
"status": "healthy",
"model_loaded": true,
"num_classes": 102,
"device": "cuda:0"
}curl -X POST http://localhost:8000/api/v1/diagnose \
-F "file=@field_sample.jpg"{
"results": [
{
"class_id": 47,
"class_name": "Corn Borer",
"confidence": 0.9134,
"bbox": [120.5, 84.2, 340.1, 290.7],
"treatment": "Apply Bacillus thuringiensis (Bt) at early larval stage. Rotate crops annually to disrupt overwintering populations."
}
],
"image_size": [1920, 1080],
"processing_time_ms": 142.37
}| Metric | Value |
|---|---|
| Validation Accuracy | 0.8143 |
| Macro Precision | 0.5872 |
| Macro Recall | 0.5534 |
| Macro F1-Score | 0.5649 |
| PR-AUC | 0.6217 |
| Optimal Threshold | 0.45 |
Results from
tests/run_evaluation.pyon the IP102 test split (7,500 images, 102 classes). ResNet50 backbone fine-tuned for 30 epochs with class-weighted cross-entropy loss.
pest-scan/
├── api/
│ └── main.py # FastAPI application & endpoints
├── core/
│ ├── model_pipeline.py # Two-stage detection + classification pipeline
│ └── threshold_tuner.py # PR-curve sweep & threshold optimisation
├── data/
│ ├── dataset_loader.py # IP102Dataset, DataLoader, class weights
│ └── ip102/ # Dataset directory (train/val/test splits)
├── models/
│ ├── yolo_weights/ # YOLOv8 detection weights
│ └── resnet_weights/ # Fine-tuned ResNet50 classification weights
├── results/ # Evaluation outputs, PR curves
├── tests/
│ ├── download_dataset.py # Dataset download & organisation utility
│ └── run_evaluation.py # Offline evaluation harness
├── config.py # Centralised configuration constants
├── requirements.txt # Python dependencies
└── README.md
A single-stage detector (e.g., YOLOv8 alone with 102 classes) struggles with fine-grained pest classification because detection architectures prioritise localisation over discriminative feature learning. By decoupling detection and classification:
- YOLOv8 handles localisation efficiently, even with overlapping or small pest regions.
- ResNet50 focuses entirely on discriminative features within cropped regions, achieving higher per-class accuracy on the 102-way classification task.
The IP102 dataset has a long-tailed distribution — some species have 10× more samples than others. Without class weighting, the model converges toward predicting majority classes. Inverse-frequency weighting ensures that gradients from rare classes are amplified proportionally, improving recall on underrepresented species.
Treatment recommendations are static, structured data that map class IDs to textual advice. SQLite provides:
- Indexed lookups — O(log n) query time on the class_id primary key.
- Zero configuration — no server process; the database is a single portable file.
- Local execution — no network dependency; suitable for field deployment.
All image augmentations (random crop, horizontal flip, colour jitter, rotation) are implemented with explicit NumPy/OpenCV matrix operations rather than opaque library calls. This ensures full transparency into the geometric and photometric transformations applied during training.
- Fine-tuned on IP102; generalization to novel pest species not present in the training set requires retraining or domain adaptation.
- YOLOv8 detection requires pest-specific fine-tuning for optimal localization accuracy on agricultural imagery.
- Single-machine inference pipeline; no distributed serving or horizontal scaling.
- SQLite is suitable for local deployment; production at scale would benefit from PostgreSQL with connection pooling.
@inproceedings{wu2019ip102,
title={IP102: A Large-Scale Benchmark Dataset for Insect Pest Recognition},
author={Wu, Xiaoping and Zhan, Chi and Lai, Yu-Kun and Cheng, Ming-Ming and Yang, Jingdong},
booktitle={CVPR},
year={2019}
}This project is licensed under the MIT License — see LICENSE for details.