trendXtract: AI Research Intelligence system

Search. Synthesize. Discover. Build. The AI-native research platform for researchers, engineers, and founders who need to move faster than Google Scholar.

--

Problem

Researchers waste hours searching, reading, and synthesizing papers—only to find their questions unanswered and connections between papers invisible. Existing tools (Google Scholar, Semantic Scholar) are passive: they retrieve papers, but don't reason about them.

Solution

trendXtract is an AI Research Intelligence Engine that:

Searches millions of arXiv papers in milliseconds
Synthesizes insights across multiple papers using Claude AI
Extracts structured knowledge (methodology, contributions, limitations)
Compares papers with normalized differences
Generates implementation guides from papers
Detects research gaps to spark new ideas

Result: What took 2 hours now takes 2 minutes.

Key Features

1. Multi-Paper Intelligence 🤖

Ask a question and get answers synthesized across 1-5 papers with inline citations. Claude reasons across papers to reveal patterns humans miss.

User: "How do these papers approach context window limitations?"
→ trendXtract reads all papers and generates a synthesized answer with citations

2. Structured Paper Extraction 📋

One-click breakdown of any paper into:

Problem Statement — What's being solved?
Methodology — How does it work?
Key Contributions — What's novel?
Limitations — What's missing?
Applications — Real-world impact?

3. Paper Comparison Engine 📊

Select 2-5 papers. Get a normalized comparison table:

Methods & algorithms
Datasets & benchmarks
Performance metrics
Computational complexity
Year & novelty

4. Build from Paper 🛠️

For any paper, auto-generate:

Project Ideas — 3 implementable ideas based on the paper
Starter Code — Python skeleton to get you moving
Tech Stack — Recommended libraries & tools
Implementation Steps — Key milestones

5. Research Gap Detector 🔍

Analyze 1-5 papers and instantly discover:

Research Gaps — What's NOT being addressed?
Weaknesses — Limitations in current approaches
Future Directions — High-impact research paths
Startup Ideas — Commercializable opportunities
Hackathon Ideas — Quick-win projects

📊 Architecture

┌─────────────────────────────────────────────────────┐
│                      Frontend (React)                │
│  - Search Bar  - Paper Cards  - Feature Tabs        │
│  - Multi-Intelligence  - Structured Panel            │
│  - Comparison  - Build Guide  - Gap Detector         │
└──────────────────┬──────────────────────────────────┘
                   │ (REST API + SSE Streaming)
┌──────────────────▼──────────────────────────────────┐
│                   Backend (Flask)                    │
├─────────────────────────────────────────────────────┤
│  ┌─────────────────────────────────────────────┐   │
│  │ API Endpoints                               │   │
│  │ - GET  /api/search       (arXiv Search)     │   │
│  │ - POST /api/multi-intelligence (Synthesize) │   │
│  │ - POST /api/structured   (Extract Info)     │   │
│  │ - POST /api/compare      (Compare Papers)   │   │
│  │ - POST /api/build        (Implementation)   │   │
│  │ - POST /api/gaps         (Gap Detection)    │   │
│  └─────────────────────────────────────────────┘   │
│                                                      │
│  ┌─────────────────────────────────────────────┐   │
│  │ Services Layer                              │   │
│  │ - arxiv_service     (Paper retrieval)       │   │
│  │ - ai_service        (Claude integration)    │   │
│  │ - cache_service     (LRU + TTL cache)       │   │
│  └─────────────────────────────────────────────┘   │
└──────────────────┬──────────────────────────────────┘
                   │
        ┌──────────┴──────────┐
        │                     │
    ┌───▼─────┐         ┌──────▼──┐
    │ arXiv   │         │ Claude  │
    │ API     │         │ API     │
    └─────────┘         └─────────┘

Key Design Decisions

No heavy ML dependencies — Uses Claude API for reasoning instead of local embeddings
Stream-based responses — SSE for real-time feedback
Smart caching — 1-hour TTL on synthesized results
Multi-paper context — Papers passed in structured XML prompts to Claude
Production-ready — gunicorn, error handling, logging, Render-optimized

🛠️ Tech Stack

Backend:

Flask 3.0 — Lightweight web framework
Anthropic SDK — Claude AI integration
requests — HTTP library for arXiv
cachetools — LRU + TTL caching
gunicorn — Production WSGI server

Frontend:

React 19 — Modern UI library
Tailwind CSS 4 — Utility-first styling
react-markdown — Markdown rendering
react-syntax-highlighter — Code highlighting

APIs:

arXiv API — Paper metadata & search
Claude API (Opus 4.1) — Reasoning & synthesis

Deployment:

Render — Unified platform for backend + frontend
Python + Node runtime

📥 Installation

Prerequisites

Python 3.10+
Node 18+
Anthropic API key (get one at console.anthropic.com)

Local Setup

Clone & install backend

cd backend
pip install -r requirements.txt
cp .env.example .env
# Edit .env and add your ANTHROPIC_API_KEY
python app.py

Backend runs on http://localhost:10000

Install & run frontend (in a new terminal)
```
cd trendxtract-frontend
npm install
npm start
```
Frontend runs on http://localhost:3000
Test
- Open http://localhost:3000
- Search for "transformers"
- Select 2 papers
- Try "Compare" or "Multi-Intelligence"

🚀 Deployment on Render

One-Click Deploy

Fork this repo
Go to render.com
Create a new "Blueprint" service
Connect your GitHub fork
Add environment variable: ANTHROPIC_API_KEY=sk-ant-...
Deploy!

Manual Setup

# Backend Service
Service: Web
Environment: Python 3.10
Build: pip install -r backend/requirements.txt
Start: cd backend && gunicorn app:app --bind 0.0.0.0:$PORT
Env: ANTHROPIC_API_KEY, FLASK_ENV=production

# Frontend Service
Service: Web
Environment: Node 18
Build: cd trendxtract-frontend && npm install && npm run build
Start: cd trendxtract-frontend && npm run build && npx serve -s build -l 3000
Env: REACT_APP_BACKEND_URL=https://your-backend-url.onrender.com

📖 API Reference

Search Papers

GET /api/search?q=transformers&max=10

Response:
{
  "papers": [
    {
      "id": "2312.10997",
      "title": "Attention Is All You Need",
      "abstract": "...",
      "authors": ["Ashish Vaswani", ...],
      "published_year": "2017",
      "arxiv_url": "https://arxiv.org/abs/2312.10997"
    }
  ]
}

Multi-Paper Intelligence (Streaming)

POST /api/multi-intelligence

Request:
{
  "query": "How do these papers approach context windows?",
  "paper_ids": ["2312.10997", "2401.02461"]
}

Response: Server-Sent Events stream with chunks of text

Structured Extraction (Streaming)

POST /api/structured

Request:
{
  "paper_id": "2312.10997"
}

Response: SSE stream containing JSON with problem, methodology, etc.

Compare Papers (Streaming)

POST /api/compare

Request:
{
  "paper_ids": ["2312.10997", "2401.02461", "2312.14993"]
}

Response: SSE stream with comparison analysis

Build from Paper (Streaming)

POST /api/build

Request:
{
  "paper_id": "2312.10997"
}

Response: SSE stream with project ideas, starter code, tech stack

Detect Research Gaps (Streaming)

POST /api/gaps

Request:
{
  "paper_ids": ["2312.10997", "2401.02461"]
}

Response: SSE stream with gaps, weaknesses, opportunities

🎯 Use Cases

For Researchers

Literature reviews — Synthesize 5 papers instead of reading 50
Gap analysis — Find unexplored areas in your field
Trend spotting — Discover what's hot in your domain

For Engineers

Quick learning — Understand a new technique in 2 minutes
Implementation — Get starter code from papers
Comparison — Choose between competing approaches

For Founders

Idea validation — Spot gaps = startup opportunities
Competitive analysis — How do competing papers approach the problem?
Investment thesis — What's emerging in AI?

🏗️ Project Structure

trendXtract/
├── backend/
│   ├── app.py                    # Main Flask app (9 routes)
│   ├── core/                     # Config, logging, feature flags
│   ├── services/                 # Enhanced services layer
│   │   ├── arxiv_service.py      # Paper search & retrieval
│   │   ├── ai_service.py         # Claude AI integration
│   │   └── cache_service.py      # Caching layer
│   ├── agents/                   # Multi-agent system (6 agents)
│   ├── pipelines/                # Composite workflows
│   ├── plugins/                  # Extensible plugin system
│   ├── experiments/              # Novel research features
│   ├── api/                      # REST API blueprints
│   ├── utils/                    # Utilities & helpers
│   ├── requirements.txt
│   └── .env.example
│
├── trendxtract-frontend/
│   ├── src/
│   │   ├── components/           # React components
│   │   │   ├── SearchBar.js
│   │   │   ├── PaperCard.js
│   │   │   ├── MultiIntelligence.js
│   │   │   ├── StructuredPanel.js
│   │   │   ├── ComparisonTable.js
│   │   │   ├── ImplementationPanel.js
│   │   │   ├── GapDetector.js
│   │   │   ├── TrendingTopics.js
│   │   │   └── Skeleton.js
│   │   ├── hooks/
│   │   │   └── useStream.js      # SSE streaming hook
│   │   ├── services/
│   │   │   └── api.js            # API client
│   │   ├── App.js                # Main app
│   │   └── index.css             # Tailwind styles
│   ├── package.json
│   └── tailwind.config.js
│
├── ARCHITECTURE.md               # Complete architecture guide
├── EXPANSION_SUMMARY.md          # Implementation summary
├── render.yaml                   # Render deployment config
└── README.md                     # This file

🔧 Configuration

Environment Variables

Backend (.env):

ANTHROPIC_API_KEY=sk-ant-...
PORT=10000
FLASK_ENV=development
DEFAULT_MODEL=claude-3-5-sonnet-20241022
CACHE_TTL_SECONDS=3600
ENABLE_AGENTS=true
ENABLE_EXPERIMENTS=true

Frontend (.env in trendxtract-frontend):

REACT_APP_BACKEND_URL=http://localhost:10000

📊 Performance

Search latency — <500ms (arXiv API)
Synthesis latency — 3-8s (Claude streaming)
Cold start — <5s (Render)
Cache hit rate — ~30% (1-hour TTL, 200 slots)
Concurrent users — 50+ (Render Pro)

🛡️ Limitations & Future

Current Limitations

arXiv only (no PubMed, IEEE Xplore yet)
Max 5 papers per analysis
1-hour cache TTL
No authentication/user accounts

Roadmap

PubMed + CrossRef integration
Vector similarity search (for better paper discovery)
User accounts + saved analyses
Citation network visualization
Research notifications (alerts for gap-addressing papers)
Fine-tuned embedding model for papers
Batch analysis (100+ papers)

🤝 Contributing

Contributions welcome! Areas:

New data sources (PubMed, IEEE, bioRxiv)
UI/UX improvements
Better LLM prompts
Performance optimizations
Additional features (visualization, export, etc)

📜 License

MIT

🙋 Support

Bug reports — GitHub Issues
Feedback — Contact
Documentation — See ARCHITECTURE.md and EXPANSION_SUMMARY.md

🎓 Built With

Claude API — AI reasoning backbone
arXiv — Paper metadata
React — Beautiful, interactive UX
Flask — Production-grade backend
Render — Seamless deployment

Start searching smarter. Try it now →

Made for researchers, engineers, and founders who want to move faster than manual literature reviews.

Name		Name	Last commit message	Last commit date
Latest commit History 17 Commits
backend		backend
trendxtract-frontend		trendxtract-frontend
ARCHITECTURE.md		ARCHITECTURE.md
EXPANSION_SUMMARY.md		EXPANSION_SUMMARY.md
README.md		README.md
render.yaml		render.yaml
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

trendXtract: AI Research Intelligence system

Problem

Solution

Key Features

1. Multi-Paper Intelligence 🤖

2. Structured Paper Extraction 📋

3. Paper Comparison Engine 📊

4. Build from Paper 🛠️

5. Research Gap Detector 🔍

📊 Architecture

Key Design Decisions

🛠️ Tech Stack

📥 Installation

Prerequisites

Local Setup

🚀 Deployment on Render

One-Click Deploy

Manual Setup

📖 API Reference

Search Papers

Multi-Paper Intelligence (Streaming)

Structured Extraction (Streaming)

Compare Papers (Streaming)

Build from Paper (Streaming)

Detect Research Gaps (Streaming)

🎯 Use Cases

For Researchers

For Engineers

For Founders

🏗️ Project Structure

🔧 Configuration

Environment Variables

📊 Performance

🛡️ Limitations & Future

Current Limitations

Roadmap

🤝 Contributing

📜 License

🙋 Support

🎓 Built With

About

Topics

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages