Skip to content

FEATURES.md

Latest commit

 

History

History
489 lines (484 loc) · 22.8 KB

FEATURES.md

File metadata and controls

489 lines (484 loc) · 22.8 KB

Features

The features that have been marked as check have been implemented while those that are planned to be implemented are marked as unchecked.

Completion/In-Progress

  • is a completed task

  • [-] is an in-progress task

  • is a pending task

  • User Authentication

  • User Registration

  • User Management System

    • List Users (Admin Access)
    • Add New User (Admin Access)
    • Edit User (Admin Access)
    • Delete User (Admin Access)

  • Beta Users Management System

    • List Beta Users (Admin Access)
    • Convert Beta User To Actual User (Admin Access)

  • Blog Article Publishing System: Articles for the sole reason to help market the platform will be implemented and admin users will be allowed to write and manage articles that will be published.

    • Add Article (Admin Access)
      • Save Article As Draft (Admin Access)
      • Publish Article (Admin Access)
    • List Articles (Admin Access)
    • Edit Article (Admin Access)
      • Article Versioning (Admin Access)
    • Delete Article (Admin Access)
    • Add Category (Admin Access)
    • List Categories (Admin Access)
      • Delete Category (Admin Access)
      • Edit Category (Admin Access)
    • Display Published Article (Public Access)
    • Display List of Published Articles (Public Access)

  • Database Backup & Restore

  • Projects: A project is a concept that is being used to group together data and related information of an analysis so that a user can easily manage the multiple analysis that the user will perform on different data sets that may not be related to one another. Having projects allows us to organize the data in neatly packed containers so that the user can easily focus on his/her data analysis activities.

  • [-] Data Sources: A data source can be database that contains data that a user wants to bring into the platform to analyse. The aim is to provide support for multiple datasets so that the users can perform all of their analysis in this single platform without having to go back and forth from one software to another.

    • Traditional Databases
      • PostgreSQL
      • MySQL
      • MariaDB
      • MongoDB
    • File-Based Sources
      • Excel Files (Includes multiple sheets support)
      • CSV Files (Comma-separated values with automatic type inference)
      • PDF Files (Includes multiple pages support; data in the form of tables and paragraph text are supported)
    • Marketing & Analytics Platforms
      • Google Analytics (OAuth 2.0, multiple report types, custom date ranges)
      • Google Ads (OAuth 2.0, campaign performance, custom date ranges)
      • Google Ad Manager (OAuth 2.0, revenue & geography reports)
      • [-] Meta Ads
        • OAuth 2.0 Authentication
        • Campaigns, Ad Sets, Ads
        • Performance Insights (impressions, clicks, spend, conversions)
        • Ad Creatives (images, videos, copy, CTAs) - Planned (see META_ADS_ATTRIBUTION_ENHANCEMENT.md)
        • Custom Conversions (pixel events, conversion tracking) - Planned (see META_ADS_ATTRIBUTION_ENHANCEMENT.md)
      • LinkedIn Ads (OAuth 2.0, campaign/creative sync, daily & monthly analytics)
      • HubSpot (OAuth 2.0, CRM data sync)
      • Klaviyo (API key authentication, email marketing data sync)

  • Data Models: A data model is the representation of the data that the user will build which will then be used for analysis. Essentially raw data will not be saved in the platform's database, but data models will be built which will form the foundation for the analysis that the user will perform.

  • Custom Calculated Fields: The user will be able to create new custom fields which will further aid the user in his/her analysis.

  • Data Model Builder: The data model builder is simple easy to use drag and drop UI based control mechanism that allows the user to build data models. The user can select the columns from multiple tables and even performs auto joins between tables that have a relation between them.

    • AI Data Modeler: Instantly generate production-ready data models with a single click or converse with the agent to analyze your data sources and build you production ready data models. AI analyzes your database schema and provides intelligent preset templates (e.g., Sales Analysis, User Engagement, Performance Metrics) that automatically configure tables, JOINs, columns, and aggregations. No SQL knowledge required—just click and apply.
    • Intelligent Join Suggestions: AI-powered join recommendations analyze table relationships and suggest optimal join conditions with confidence scoring
    • Join Catalog: Save and reuse common join patterns across data models with a centralized join catalog management system

  • AI Insights: AI-powered data analysis that samples your connected data sources and generates actionable insights using Google Gemini. Supports follow-up conversation to drill deeper into findings.

    • Initialize Insight Session (select project + one or more data sources)
    • AI-Generated Insight Reports (automatic analysis of sampled data)
    • Conversational Follow-up Chat (ask follow-up questions about the insights)
    • Save Insight Reports
    • View Saved Reports (per project)
    • Delete Insight Reports
    • Cancel Active Session
    • Redis-based Session Persistence
    • Rate-limited AI Operations (dedicated insightsLimiter)

  • Subscription Tier Management System: Complete subscription tier management with dynamic limits and real-time enforcement

    • Admin Subscription Tiers Panel (CRUD Operations)
    • Create Custom Subscription Tiers
    • Edit Tier Limits (Projects, Data Sources, Data Models, Dashboards, AI Generations)
    • Custom Tier Naming Support
    • Tier Activation/Deactivation
    • Price Configuration (Monthly USD)
    • 5 Pre-configured Tiers (FREE, PRO, TEAM, BUSINESS, ENTERPRISE)

  • Subscription Tier Enforcement: Real-time limit enforcement across all platform resources

    • Row Limit Per Data Model (Automatic SQL LIMIT Injection)
    • Maximum Projects Per User
    • Maximum Data Sources Per Project
    • Maximum Data Models Per Data Source
    • Maximum Dashboards Per User
    • AI Generations Per Month Limit
    • Unlimited Tier Support (Value: -1)
    • Middleware-based Resource Validation
    • Tier Upgrade/Downgrade Support

  • User Subscription Management: User-facing subscription features and controls

    • View Current Subscription Tier
    • View Tier Limits and Features
    • Real-time Usage Statistics Dashboard
    • Usage Indicators on All Resource Pages
    • Resource Capacity Warnings
    • Subscription History Tracking
    • Auto-assignment of FREE Tier for New Users

  • Advanced UI/UX Features

    • Dynamic Usage Indicators (Projects, Data Sources, Data Models, Dashboards)
    • "Unlimited" Display for Unlimited Tiers
    • Per-Resource Capacity Calculations
    • Cross-Data Source Model Capacity (Data Sources × Limit)
    • Single Data Source Model Capacity (Per-Source Limit)
    • Informational Tooltips for Limit Explanations
    • Tier Badge Display on User Interface
    • Resource Creation Prevention at Limit
    • Tier Limit Modal Warnings

  • Scheduled Database Backups: Automated backup system with configurable schedules

    • Recurring Backup Schedules (Daily, Weekly, Monthly, Custom Cron)
    • Background Job Queue Processing
    • Real-time Progress Updates (Socket.IO)
    • Backup Retention Management
    • Automatic Cleanup of Old Backups
    • Email Notifications on Success/Failure
    • Admin Control Panel for Backup Schedules
    • Manual Trigger Support

  • Privacy & Compliance Features

    • GDPR-Compliant Cookie Consent Banner
    • Cookie Consent Management (Essential vs Analytics)
    • Google Analytics GA4 Integration with Consent Mode
    • IP Anonymization for Analytics
    • Cookie Preferences Storage (localStorage)
    • 180-Day Re-consent Mechanism
    • Privacy Policy with Cookie Disclosure
    • Cookie Deletion on Opt-out
    • Customizable Cookie Settings Modal

  • OAuth Integration & Security

    • Google OAuth 2.0 Authentication
    • OAuth Session Management (Redis-based)
    • State Token Validation (CSRF Protection)
    • OAuth Token Encryption
    • Automatic Token Refresh
    • OAuth Callback Security
    • Session Expiry Management

  • Performance & Monitoring

    • Performance Metrics Tracking
    • Operation Timing Statistics
    • Bottleneck Detection
    • Slowest Operations Identification
    • Performance API Endpoints
    • Metrics Cleanup Operations

  • Rate Limiting: Multi-level rate limiting for API protection

    • General API Rate Limiter (1000 req/15min)
    • Authentication Rate Limiter (10 req/15min)
    • Expensive Operations Limiter (30 req/15min)
    • OAuth Callback Limiter (10 req/15min)
    • AI Operations Limiter (20 req/hour)
    • Google Ad Manager Rate Limiting
    • Rate Limit Status Endpoints

  • Data Security & Encryption

    • Automatic Field-Level Encryption (AES-256-CBC)
    • Connection Details Encryption
    • OAuth Credentials Encryption
    • TypeORM ValueTransformer Integration
    • Encrypted Backup Support
    • Secure Key Management

  • [-] Dashboard: A dashboard is essentially a collection of visualizations/charts which will help the user tell a story and show the bigger picture.

    • Droppable Text
    • Table
    • Pie Chart
    • Donut Chart
    • Vertical Bar Chart
    • Horizontal Bar Chart
    • Vertical Bar Chart With Line
    • Stacked Bar Chart
    • Multi-line Chart
    • Treemap Chart
    • Bubble Chart
    • Stacked Area Chart
    • Map Chart
    • Export Capabilities
      • Export to PNG/Image
      • PDF Export
      • Dashboard Sharing

  • Multi-Tenant Architecture: Complete organization and workspace isolation for team collaboration

    • Organizations (top-level tenant isolation)
      • Create Organization
      • Update Organization (name, domain, logo)
      • Delete Organization
      • List User Organizations
      • Switch Between Organizations
      • Organization-Level Subscription Management
      • Organization Branding (logo, domain)
    • Workspaces (project grouping within organizations)
      • Create Workspace
      • Update Workspace (name, slug, description)
      • Delete Workspace
      • List Organization Workspaces
      • Switch Between Workspaces
      • Workspace-Level Access Control
    • Organization Invitations
      • Invite Users to Organization
      • Accept/Reject Invitations
      • Resend Invitations
      • Cancel Pending Invitations
      • Email Notifications for Invitations
      • Role-Based Organization Access (Owner, Admin, Member)
    • Workspace Invitations
      • Invite Users to Workspace
      • Role-Based Workspace Access (Admin, Editor, Viewer)
      • Invitation Management

  • Project Team Collaboration

    • Project Member Management
      • Add Team Members to Projects
      • Remove Team Members
      • Update Member Roles
      • View Project Members List
    • Project Invitations
      • Send Project Invitations
      • Accept/Decline Invitations
      • Email Notifications
      • Invitation Expiry Management
    • Role-Based Access Control (RBAC)
      • Project Owner Role
      • Analyst Role
      • Viewer Role
      • Permission-Based Feature Access

  • Marketing Attribution Engine: Multi-touch attribution system with 5 attribution models

    • Event Tracking
      • Page View Tracking
      • Interaction Event Tracking
      • Conversion Event Tracking
      • UTM Parameter Parsing
      • Referrer Domain Extraction
      • Custom Event Metadata
    • Attribution Models
      • First-Touch Attribution (100% to first interaction)
      • Last-Touch Attribution (100% to last interaction)
      • Linear Attribution (equal distribution across all touchpoints)
      • Time-Decay Attribution (exponential decay, 7-day half-life)
      • U-Shaped Attribution (40-20-40 distribution)
    • Channel Management
      • Automatic Channel Creation from UTM Parameters
      • Channel Performance Metrics
      • ROI Calculation per Channel
      • Traffic Source Categorization (organic, paid, social, email, direct, referral)
    • Conversion Path Analysis
      • Top Conversion Paths
      • Path Length Analysis
      • Time-to-Conversion Tracking
      • Multi-Channel Journey Mapping
    • Funnel Analysis
      • Multi-Step Funnel Tracking
      • Drop-off Rate Analysis
      • Step Completion Rates
      • Average Time to Complete
    • Attribution Reports
      • Generate Attribution Reports
      • Model Comparison
      • Channel Performance Reports
      • User Journey Reports
      • Save and Export Reports

  • Data Quality & Health Monitoring: Automated data quality analysis and health tracking

    • Data Quality Analysis
      • Null Value Detection
      • Duplicate Detection
      • Data Type Consistency Checks
      • Outlier Detection
      • Quality Score Calculation
      • Quality Remediation Suggestions
    • Data Model Health Tracking
      • Health Status Classification (Healthy, Warning, Blocked)
      • Automatic Health Status Updates
      • Health Issue Detection
      • Row Count Monitoring
      • Source Row Count Comparison
      • Health History Tracking
    • Admin Health Monitoring
      • Platform-Wide Health Summary
      • Blocked Models Dashboard
      • Warning Models Dashboard
      • Health Metrics by Project
      • Health Issue Analytics

  • Medallion Architecture (Data Layer Classification): Bronze/Silver/Gold data layer management

    • Data Layer Types
      • Bronze Layer (Raw Data) - Direct source imports
      • Silver Layer (Refined Data) - Cleaned and validated data
      • Gold Layer (Business Ready) - Aggregated and business-optimized data
    • Layer Classification
      • Automatic Layer Detection
      • Manual Layer Assignment
      • Layer Validation Rules
      • Layer Migration Support
    • Layer-Based Features
      • Layer Filtering in UI
      • Layer-Specific Query Optimization
      • Layer Promotion Workflows
      • Layer Dependency Tracking
    • Admin Layer Management
      • Migration Candidate Detection
      • Bulk Layer Classification
      • Layer Analytics Dashboard
      • Unclassified Model Reports

  • Data Synchronization & Refresh System

    • Sync History Tracking
      • Sync Status Monitoring (pending, running, completed, failed)
      • Sync Duration Tracking
      • Records Synced/Failed Counters
      • Error Message Logging
      • Sync Timestamp Tracking
    • Data Model Refresh
      • Manual Data Model Refresh
      • Refresh History Tracking
      • Cascade Refresh (source → dependent models)
      • Background Refresh Processing
      • Refresh Status Monitoring
    • Scheduled Data Source Sync
      • Cron-Based Scheduling
      • Daily/Weekly/Monthly Schedules
      • Custom Cron Expressions
      • Next Sync Time Calculation
      • Enable/Disable Scheduling
    • Real-Time Sync Updates
      • Socket.IO Progress Events
      • Live Sync Status Updates
      • Progress Percentage Tracking
      • Completion Notifications

  • Notifications System: In-app notification center with real-time updates

    • Notification Types
      • System Notifications
      • Subscription Updates
      • Invitation Notifications
      • Data Sync Notifications
      • Admin Notifications
    • Notification Management
      • Unread Count Display
      • Mark as Read/Unread
      • Mark All as Read
      • Delete Notifications
      • Notification History
    • Notification Delivery
      • Real-time In-App Notifications
      • Email Notifications (configurable)
      • Notification Expiry (auto-delete after 90 days)
      • Batch Cleanup of Expired Notifications

  • Email Preferences & Communication

    • Email Preference Management
      • Subscription Update Emails (on/off)
      • Expiration Warning Emails (on/off)
      • Renewal Reminder Emails (on/off)
      • Promotional Emails (on/off)
    • Email Templates
      • Welcome Emails
      • Invitation Emails
      • Subscription Confirmation
      • Cancellation Confirmation
      • Data Export Ready
    • Email Delivery
      • SMTP Integration
      • Email Queue Management
      • Delivery Status Tracking

  • Lead Generation & Content Distribution

    • Lead Generators (Downloadable Resources)
      • Upload PDF Resources
      • Custom Landing Pages (slug-based)
      • Email Capture Forms
      • One-Time Download Tokens (1-hour expiry)
      • Download Count Tracking
      • Lead Email Collection
      • Email Confirmation Workflow
    • Admin Lead Generator Management
      • Create Lead Generator
      • Update Lead Generator Metadata
      • Delete Lead Generator
      • View Download Statistics
      • Export Lead Emails

  • Promotional & Discount System

    • Promo Code Management
      • Create Promo Codes
      • Code Validation
      • Usage Limits (per-code and per-user)
      • Expiration Date Support
      • Tier-Specific Codes
    • Discount Types
      • Percentage Discounts
      • Fixed Amount Discounts
      • Tier Upgrade Codes (free tier upgrade)
      • Billing Cycle Discounts (monthly/annual)
    • Promo Code Redemption
      • Real-time Code Validation
      • Discount Calculation
      • Redemption Tracking
      • Redemption History
    • Admin Promo Management
      • Create/Edit/Delete Codes
      • View Redemption Statistics
      • Bulk Code Generation
      • Usage Analytics

  • Account Lifecycle Management

    • Account Cancellation System
      • Self-Service Cancellation Request
      • Cancellation Reason Collection (category + feedback)
      • Immediate Cancellation (30-day grace period)
      • End-of-Billing-Period Cancellation
      • Data Export Before Deletion
        • Export Projects as JSON
        • Export Data Models (CSV, Excel, JSON)
        • Export Dashboards
      • 30-Day Data Retention
      • Automatic Data Deletion After 30 Days
      • Email Notifications (confirmation, data export ready, deletion complete)
    • Admin Cancellation Management
      • View All Cancellation Requests
      • Filter by Status (pending, confirmed, completed, data_deleted)
      • Cancel Cancellation (user retention)
      • Manual Data Deletion Trigger
      • Cancellation Analytics
    • Downgrade Request System
      • User Downgrade Requests
      • Admin Approval Workflow
      • Pending Request Queue
      • Approve/Reject Downgrade
      • Automatic Tier Change on Approval

  • Enterprise Features

    • Enterprise Contact Requests
      • Enterprise Inquiry Form
      • Contact Request Tracking
      • Admin Review Dashboard
      • Status Management (pending, contacted, converted, rejected)
      • Notes and Follow-up Tracking
    • Enterprise Subscription Management
      • Custom Tier Creation for Enterprise Clients
      • Unlimited Resource Limits
      • Custom Pricing Configuration

  • Admin Analytics & Monitoring

    • Subscription Analytics
      • Active Subscriptions by Tier
      • Revenue Metrics
      • Churn Rate Analysis
      • Subscription Growth Trends
      • Tier Distribution Charts
    • Platform Statistics
      • Total Users Count
      • Active Projects Count
      • Data Sources by Type
      • Data Models Statistics
      • Dashboards Count
      • Storage Usage Metrics
    • Admin Dashboard
      • Real-Time Platform Metrics
      • User Activity Monitoring
      • System Health Indicators
      • Resource Usage Alerts

  • Payment Integration

    • Paddle Webhook Integration
      • Subscription Created
      • Subscription Updated
      • Subscription Cancelled
      • Payment Succeeded
      • Payment Failed
      • Refund Processed
    • Paddle Sync Service
      • Manual Subscription Sync
      • Automatic Webhook Processing
      • Subscription Status Updates
      • Payment History Tracking

  • Platform Administration

    • Platform Settings Management
      • Global Platform Configuration
      • Feature Flags
      • System Maintenance Mode
      • Email Service Configuration
    • Admin Job Management
      • Background Job Monitoring
      • Job Queue Statistics
      • Failed Job Retry
      • Job Cleanup Operations
    • Sitemap Generation
      • Automatic Sitemap.xml Generation
      • Article Sitemap Integration
      • Dynamic URL Discovery
      • SEO Optimization

  • Geolocation & Tracking

    • IP Geolocation Service
      • Country Detection
      • Region/State Detection
      • City Detection
      • Timezone Detection
      • Coordinate Detection (latitude/longitude)
    • Offline Event Tracking
      • Queue Offline Events
      • Sync When Online
      • Event Replay Support
      • Data Loss Prevention