research-metrics.md

Research Metric Catalog

This is the English reference for every metric rendered by Forecast Evaluation Reports, Backtest Runs, and Validation Reports. The desktop UI uses the same stable IDs and version 1.0.0 definitions.

Forecast metrics describe Model prediction quality. Strategy metrics describe simulated profitability, risk, and execution. Validation metrics aggregate retained Strategy evidence. None is a universal investment-quality threshold or trading recommendation.

Forecast Evaluation

Stable ID and version	Definition
forecast.aligned-count@1.0.0	Aligned predictions. Count of Forecast rows with an available prediction and verifiable realized label. Formula: count(aligned prediction-label rows). More evidence can improve precision, but sample size does not prove out-of-sample or representative evidence. Range: integers from zero through the evaluation row count. Always defined; zero means no row could be evaluated.
forecast.coverage@1.0.0	Coverage. Share of evaluation rows with aligned evidence. Formula: aligned rows / evaluation rows. Higher means fewer unavailable rows. Range: [0, 1]. It measures availability, not prediction quality or Strategy profitability. Defined as zero when there are no aligned rows.
forecast.missingness@1.0.0	Missingness. Share of evaluation rows without aligned evidence. Formula: 1 - coverage. Lower means fewer unavailable rows. Range: [0, 1]. Inspect retained unavailable-row reasons because missingness may be systematic. Defined from coverage for every report.
forecast.mae@1.0.0	MAE. Mean absolute Forecast error in Target-native units. Formula: mean(abs(prediction - realized)). Lower means smaller average absolute error. Range: [0, +infinity). Scale depends on the Target; this is Model prediction quality, not Strategy profitability. Unavailable without aligned, verifiable continuous labels.
forecast.rmse@1.0.0	RMSE. Root mean squared Forecast error in Target-native units. Formula: sqrt(mean((prediction - realized)^2)). Lower means smaller error and large errors receive more weight. Range: [0, +infinity). Scale depends on the Target; this is not Strategy profitability. Unavailable without aligned, verifiable continuous labels.
forecast.mean-bias@1.0.0	Mean bias. Average signed Forecast error. Formula: mean(prediction - realized). Closer to zero means less average signed bias. Range: (-infinity, +infinity). Positive and negative errors can cancel, so no universal quality threshold applies. Unavailable without aligned, verifiable continuous labels.
forecast.pearson-correlation@1.0.0	Pearson correlation. Linear association between aligned predictions and realized labels. Formula: covariance(prediction, realized) / (prediction standard deviation x realized standard deviation). Range: [-1, 1]; interpret sign and magnitude in context. It is not profitability, causality, or a universal score. Unavailable with fewer than two rows or a constant series.
forecast.brier-score@1.0.0	Brier Score. Mean squared error between probability and binary label. Formula: mean((probability - label)^2). Lower means smaller probability error. Range: [0, 1]. Interpret with class balance and calibration; no universal threshold applies. Unavailable without aligned binary labels.
forecast.log-loss@1.0.0	Log Loss. Mean binary cross-entropy. Formula: -mean(label x ln(p) + (1-label) x ln(1-p)). Lower is better and confident errors cost more. Approximate range: [0, 34.539] after clipping p to [1e-15, 1-1e-15]. Interpret with class balance. Unavailable without aligned binary labels.
forecast.roc-auc@1.0.0	ROC AUC. Probability that a positive label ranks above a negative label, with ties worth one half. Formula: (concordant pairs + 0.5 x ties) / positive-negative pairs. Higher means stronger ranking separation. Range: [0, 1]. No universal investment-quality threshold applies. Unavailable unless both classes occur.
forecast.calibration@1.0.0	Calibration. Agreement between mean Forecast probability and observed positive frequency in ten fixed buckets. Formula: compare mean(probability) with mean(label) in each bucket. Both means range from zero to one; closer is better. Empty buckets remain explicit and small buckets are weak evidence. A bucket mean is unavailable when it has no aligned rows.
forecast.pearson-ic@1.0.0	Time-series Pearson IC. Linear association between Score and realized Target in one single-Instrument time-series. Formula: covariance(score, target) / (score standard deviation x target standard deviation). Range: [-1, 1]; interpret in context. This is not cross-sectional IC, Strategy profitability, or a universal score. Unavailable with fewer than two rows or a constant series.
forecast.spearman-rank-ic@1.0.0	Time-series Spearman Rank IC. Rank association between Score and realized Target for one Instrument, preserving ties. Formula: Pearson correlation of deterministic average ranks. Range: [-1, 1]; interpret in context. This is not future cross-sectional IC or profitability. Unavailable with fewer than two rows or a constant ranked series.
forecast.window-icir@1.0.0	Window ICIR. Mean ordered-window Pearson IC divided by its population standard deviation. Formula: mean(valid window IC) / population standard deviation(valid window IC). Range: (-infinity, +infinity); interpret with every window and sample count. This is single-Instrument stability evidence, not profitability or turnover. Unavailable unless two valid window IC values exist and vary.
forecast.five-quantiles@1.0.0	Five quantiles. Realized Targets grouped by ascending Score while tied Scores stay together. Formula: five deterministic rank buckets. Inspect monotonicity and every bucket count; neither direction is universally favorable. Five buckets are explicit and some may be empty. This is descriptive single-Instrument evidence, not portfolio return.

Backtest Strategy

Stable ID and version	Definition
strategy.total-return@1.0.0	Total return. Strategy equity change over the Backtest period. Formula: final equity / initial equity - 1. Higher means greater simulated return. Range: [-1, +infinity). Results depend on period, costs, execution assumptions, and risk. Unavailable without valid initial and final equity.
strategy.cagr@1.0.0	CAGR. Annualized compounded Strategy equity growth. Formula: (final equity / initial equity)^(1 / years) - 1. Higher means greater annualized simulated growth. Range: [-1, +infinity). Short-period annualization can mislead. AdaQ reports zero when duration or positive final equity cannot support compounding.
strategy.max-drawdown@1.0.0	Max drawdown. Largest peak-to-trough Strategy equity decline. Formula: minimum over time of equity / prior peak equity - 1. Closer to zero means a smaller observed decline. Range: [-1, 0]. It is path- and period-dependent and does not bound future loss. Unavailable without an equity curve.
strategy.sharpe@1.0.0	Sharpe. Annualized excess mean Strategy return per unit of annualized return volatility. Formula: (annualized mean return - risk-free rate) / annualized volatility. Higher means more measured excess return per volatility unit. Range: (-infinity, +infinity). Period, sampling, distribution, and risk-free assumptions matter; no universal threshold applies. AdaQ reports zero when volatility is zero.
strategy.sortino@1.0.0	Sortino. Annualized excess mean Strategy return per unit of annualized downside deviation. Formula: (annualized mean return - risk-free rate) / annualized downside deviation. Higher means more measured excess return per downside unit. Range: (-infinity, +infinity). Period, sample size, and downside definition matter. AdaQ reports zero when downside deviation is zero.
strategy.excess-return@1.0.0	Excess return. Strategy total return minus frozen benchmark return. Formula: Strategy total return - benchmark total return. Positive means simulated outperformance for this period. Range: (-infinity, +infinity). Benchmark, period, fees, and execution assumptions matter. Unavailable without both equity series.
strategy.final-equity@1.0.0	Final equity. Ending simulated Strategy equity in quote-asset units. Formula: final cash + final base quantity x final price. Interpret relative to initial equity and risk. Range: [0, +infinity) quote units. Nominal amounts are not comparable across allocations, quote assets, or periods. Unavailable when the portfolio cannot be valued.
strategy.realized-pnl@1.0.0	Realized P&L. Sum of simulated P&L realized by fills. Formula: sum(fill realized P&L). Positive is profit and negative is loss. Range: (-infinity, +infinity) quote units. It excludes value change in the remaining position. Defined as zero when no fill realizes P&L.
strategy.unrealized-pnl@1.0.0	Unrealized P&L. Value change in the remaining simulated base position. Formula: final base quantity x final price - remaining cost basis. Positive is gain and negative is loss. Range: (-infinity, +infinity) quote units. It depends on the final mark and is not locked in. Defined as zero without an open position.
strategy.total-fees@1.0.0	Total fees. Sum of simulated execution fees. Formula: sum(fill fee). Lower means less fee drag for comparable evidence. Range: [0, +infinity) quote units. Fee schedule, fill policy, turnover, and market path matter. Defined as zero without a fee-bearing fill.
strategy.win-rate@1.0.0	Win rate. Share of positive realized outcomes among non-zero realized outcomes. Formula: winning outcomes / non-zero outcomes. Higher means a greater winning share. Range: [0, 1]. It ignores win and loss size, so high win rate can coexist with negative P&L. AdaQ reports zero without non-zero realized outcomes.
strategy.fill-count@1.0.0	Fills. Number of simulated order fills. Formula: count(fills). Descriptive only; neither direction is inherently favorable. Range: non-negative integers. Fill count is not trade count and depends on partial-fill behavior. Defined as zero when no order fills.
strategy.realized-trade-count@1.0.0	Trades. Count of non-zero simulated realized P&L outcomes. Formula: count(fill realized P&L not equal to zero). Descriptive only. Range: non-negative integers. This engine-level count may differ from round-trip conventions. Defined as zero when no fill realizes P&L.

Execution evidence

Stable ID and version	Definition
execution.order-count@1.0.0	Orders. Number of simulated Orders. Formula: count(orders). Descriptive only. Range: non-negative integers. It depends on Strategy decisions and execution policy. Defined as zero when none are recorded.
execution.order-quantity@1.0.0	Order quantity. Requested base-asset quantity. Formula: requested base quantity. Interpret with price, equity, and Instrument units. Range: non-negative base units. It may differ from filled quantity. Unavailable without a valid quantity.
execution.limit-price@1.0.0	Limit price. Quote-asset price limit attached to an Order. Formula: Order limit price. Interpret with side and market evidence. Range: non-negative quote units per base unit. It does not prove a Fill price. Unavailable without a valid price.
execution.fill-quantity@1.0.0	Filled quantity. Base-asset quantity executed by one Fill. Formula: executed base quantity. Compare with requested quantity. Range: non-negative base units. One Order may have partial Fills. Unavailable without a valid quantity.
execution.requested-quantity@1.0.0	Requested quantity. Original quantity requested by the Fill's Order. Formula: originating Order quantity. Compare with filled quantity. Range: non-negative base units. It can exceed a partial Fill. Unavailable without the originating quantity.
execution.fill-price@1.0.0	Fill price. Quote-asset price per base unit for one Fill. Formula: Fill quote value / filled quantity. Interpret with side and market evidence. Range: non-negative quote units per base unit. Execution assumptions matter. Unavailable without a valid price.
execution.fill-fee@1.0.0	Fill fee. Simulated fee for one Fill. Formula: Fill quote value x frozen fee rate. Lower means less drag for otherwise identical execution. Range: non-negative quote units. Fee schedule, role, quantity, and price matter. Defined as zero without a fee.
execution.fill-realized-pnl@1.0.0	Fill realized P&L. Profit or loss realized by one Fill. Formula: closed quantity x (Fill price - cost basis), signed by side. Positive is profit and negative is loss. Range: any quote amount. It excludes remaining unrealized P&L. Defined as zero when no quantity closes.

Validation

Stable ID and version	Definition
validation.completed@1.0.0	Completed. Validation windows or markets that completed with metric evidence. Formula: count(items without failure). More completed items broaden retained evidence but do not imply favorable or independent results. Range: zero through configured item count. Always defined; zero means none completed.
validation.failed@1.0.0	Failed. Validation windows or markets retained with failure evidence. Formula: configured items - completed items. Lower means fewer failures. Range: zero through configured item count. Inspect every failure rather than excluding it from conclusions. Defined as zero when all items complete.
validation.total-fees@1.0.0	Total fees. Sum of fees across completed sample-in and sample-out Runs or markets. Formula: sum(completed validation Run fees). Descriptive only. Range: non-negative quote units. Overlapping windows can count a period more than once; fee schedules and failures affect the total. Defined as zero when no completed Run records a fee.
validation.realized-trade-count@1.0.0	Trades. Sum of non-zero realized P&L outcomes across completed sample-in and sample-out Runs or markets. Formula: sum(completed validation Run trade count). Descriptive only. Range: non-negative integers. Overlapping windows can count a period more than once and this is not a round-trip convention. Defined as zero when none realize P&L.
validation.average-sample-out-return@1.0.0	Average sample-out return. Arithmetic mean Strategy total return across completed held-out windows or markets. Formula: sum(completed sample-out returns) / completed count. Higher means greater average simulated return. Range: [-1, +infinity). This is not compounded performance and failures remain separate evidence. AdaQ reports zero when none complete.
validation.average-sample-in-return@1.0.0	Average sample-in return. Arithmetic mean Strategy total return across completed sample-in windows. Formula: sum(completed sample-in returns) / completed count. Use it as comparison context. Range: [-1, +infinity). It is not held-out evidence; cross-market validation has no sample-in leg. AdaQ reports zero without sample-in results.
validation.worst-sample-out-drawdown@1.0.0	Worst drawdown. Most negative max drawdown among completed held-out windows or markets. Formula: minimum(completed sample-out max drawdown). Closer to zero means a smaller worst observed decline. Range: [-1, 0]. Selection determines observed risk and it does not bound future loss. AdaQ reports zero when none complete.
validation.average-sample-out-sharpe@1.0.0	Average Sharpe. Arithmetic mean Backtest Sharpe across completed held-out windows or markets. Formula: sum(completed sample-out Sharpe) / completed count. Higher means greater average measured risk-adjusted simulated return. Range: (-infinity, +infinity). Averaging ratios hides dispersion; inspect each item and use no universal threshold. AdaQ reports zero when none complete.
validation.cross-market-return-spread@1.0.0	Total return spread. Difference between highest and lowest completed Strategy return across validation markets. Formula: max(completed market return) - min(completed market return). Lower means less observed dispersion, not necessarily better profitability. Range: [0, +infinity). Market selection and sample size matter. Unavailable unless two markets complete.

Validation evidence also renders strategy.total-return, strategy.max-drawdown, and strategy.sharpe using the same definitions above.