Silent data loss in the Prometheus pipeline: stale TRUTH.PROMETHEUS constant and exact-name table dispatch in PrometheusReader

[sevmag]

Two related problems in the Prometheus conversion/training path. Both fail silently — you end up with truth-less or photon-less datasets and no error. Verified against graphnet main @ ddc54b1 and Prometheus (Harvard-Neutrino) main @ 39f26dd.

1. TRUTH.PROMETHEUS does not match what Prometheus writes (silent loss of all truth labels)

PrometheusTruthExtractor extracts the schema that current Prometheus actually produces (prometheus_extractor.py:62-71):

interaction, initial_state_energy, initial_state_type, initial_state_zenith,
initial_state_azimuth, initial_state_x, initial_state_y, initial_state_z

matching Prometheus' Injection.to_dict() (injection.py).

But TRUTH.PROMETHEUS (constants.py:90-117) still lists a much older schema — injection_energy, injection_type, primary_lepton_1_*, primary_hadron_1_*, total_energy, … — none of which exist in an mc_truth table produced by the current extractor.

The failure is silent: Dataset._remove_missing_columns (dataset.py:555-596) strips missing columns with only a warning, so a user who converts current Prometheus files and trains with truth = TRUTH.PROMETHEUS gets a dataset with an (almost) empty truth dict and no error.

Affected example scripts (all use truth = TRUTH.PROMETHEUS):
examples/04_training/01_train_dynedge.py:27, 02_train_tito_model.py:29, 05_train_RNN_TITO.py:33, 06_train_icemix_model.py:35, 07_train_normalizing_flow.py:33, 08_train_grit_model.py:27.

These currently "work" only because the shipped example database (data/examples/sqlite/prometheus/prometheus-events.db) was produced by a very old Prometheus version — its mc_truth genuinely has injection_energy/primary_lepton_1_* columns, and its pulsemap table is named total rather than photons. Any freshly converted file exposes the mismatch.

Suggested fix: update TRUTH.PROMETHEUS to the interaction + initial_state_* list (i.e. keep it in lockstep with PrometheusTruthExtractor, which graphnet.datasets.prometheus_datasets.PublicPrometheusDataset already uses), update the example scripts, and consider raising instead of warning in _remove_missing_columns when all requested truth columns are missing.