CUBSO

Catálogo CUBSO · read-only

NATIVE · CANONICAL_COMPLETE

CUBSO_A_SCALE_QUERY_PROJECTION_LC_V1

Track CUBSO_A_ONLY_SUPERVISED_RETRIEVAL

Experimentos

Pregunta científica

Does a supervised query-only orthogonal alignment improve frozen A/C0 exact retrieval on held-out hard-group DEV while A, C0 and BGE-M3 remain unchanged?

An orthogonal Procrustes transform learned only from TRAIN improves DEV Recall@100 by at least 2 percentage points without reducing Recall@20 or Recall@50.

Cohort

CUBSO_A_SCALE_DEV

DEVELOPMENT

Exposure: INFORMED_HYPOTHESIS · Sealed: SEALED

212409444c17fb058878658b886f2557a46e91d5ea93396437604d25c7f645be

Baseline

Frozen normalized A/BGE-M3 query embeddings against unchanged frozen C0.

Treatment

Frozen A/BGE-M3 query embeddings transformed by one TRAIN-fitted 1024x1024 orthogonal Procrustes matrix; candidate vectors remain unchanged.

Variable causal

supervised query-side orthogonal Procrustes transform only

Modelo y ejecución

BAAI/bge-m3 + orthogonal Procrustes query transform

5617a9f
COMPLETE
PASS_REMOTE_HASHES_CANONICAL_DEV_JOIN_9730

Runtime: Kaggle Python 3.12 / torch 2.10.0+cu128 / CUDA 12.8 / 2x Tesla T4 · Provider: kaggle:fredericklied/sicoti-cubso-a-scale-procrustes-lc-v1-320c8b7ddb0d

e2595e128a11973bb1c61f59868020a8c9863c18

Decisión científica

KEEP

KEEP on preregistered DEV gate; strong seen-target/service lift, but target-unseen remains the limiting slice and prevents any claim of product readiness.

Regla: KEEP_QUERY_PROJECTION

Resultado: matched preregistered full-data DEV decision gate

External validation: no · Replication required: sí

Métricas

No hay dos métricas primarias con semántica de visualización compatible; los valores exactos permanecen disponibles en la tabla.

MétricaValor exacto
DeltaMRR0.0779442561684
DeltaRecall@1000.0726618705036
DeltaRecall@200.0921891058582
DeltaRecall@500.0821171634121
DeltaTop10.0695786228160
MRR0.392461447202
Recall@1000.748920863309
Recall@200.630421377184
Recall@500.704213771840
Top10.304008221994

Tests estadísticos

cluster_bootstrap_repetitions
2000
cluster_bootstrap_seed
20260823
cluster_count
5038
recall_at_100_losses
117
recall_at_100_mcnemar_exact_p
1.378829575001173e-131
recall_at_100_wins
824
recall_at_20_losses
141
recall_at_20_mcnemar_exact_p
2.921761163857805e-169
recall_at_20_wins
1038
recall_at_50_losses
128
recall_at_50_mcnemar_exact_p
4.779730442353226e-150
recall_at_50_wins
927

Incertidumbre

DeltaMRR
[0.07112454739012941,0.08515411716572528]
DeltaRecall@100
[0.06497392186724642,0.08094869572564607]
DeltaRecall@20
[0.08339363462917225,0.10190670160237579]
DeltaRecall@50
[0.07366017294265712,0.09164499162235011]
DeltaTop1
[0.06178641720999159,0.07749303151237484]

Lineage

Informed by

CUBSO_A_SCALE_FROZEN_DENSE_REPLICATION_V1

Same population as

Code bindings

workspaces/conosce/conosce/cubso/experiments/retrieval/a_scale/projection_evaluation.py

PRESERVED_ARTIFACT

c5bfac0ecbadd5f8cd0144f53cc10ac78cb4a325d4a906b6ed37eb5de884c54b

workspaces/conosce/conosce/cubso/experiments/retrieval/a_scale/projection_worker.py

GIT_BLOB

e2595e128a11973bb1c61f59868020a8c9863c18
7f286db62de0b70fef6483b2ba48b3a7590d47029979bd0408ce4a13a9e69747

Artefactos y evidencia

  • HISTORICAL_CODE_SNAPSHOT

    artifacts/derived/procurement_need/registry/control-plane/code-snapshots/c5bfac0ecbadd5f8cd0144f53cc10ac78cb4a325d4a906b6ed37eb5de884c54b.py

    c5bfac0ecbadd5f8cd0144f53cc10ac78cb4a325d4a906b6ed37eb5de884c54b

  • LEARNING_CURVE

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/providers/kaggle-train/320c8b7ddb0d/results/learning_curve.json

    45f246345ba02790c998c4816c684755a6b4fd2817a305d7296ac5e5e5f0c275

  • ORTHOGONAL_PROJECTION

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/providers/kaggle-train/320c8b7ddb0d/results/full_projection.parquet

    18d8b990a6b072c6b48ed0b34fec183ab527d7435950938f0d52103ad93b6495

  • PAIRED_DEV_EVALUATION

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/evaluation.json

    e62e8311a0accac219b3665988ac1a8285b2a27c3cc99ac7ce9547eed3b15821

  • PREREGISTRATION

    docs/research/cubso/preregistrations/CUBSO_A_SCALE_QUERY_PROJECTION_LC_V1.json

    59e03667b9382f4cac2dd3716ae246871fb073e7388195c91fa75dc5cb5ca40c

  • PROVIDER_JOB_SPEC

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/providers/kaggle-train/320c8b7ddb0d/dataset/job-spec.json

    daacf7b88d4844bed416c38e067078d1a6ef20d631a9b0d7401a5ab3403b478a

  • PROVIDER_JOB_SUMMARY

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/providers/kaggle-train/320c8b7ddb0d/results/job_summary.json

    43ae78105cc1d5ba71489f8a1fe269726e32f9f52670112f933eaecfa0076177

  • REMOTE_DEV_RANKINGS

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/providers/kaggle-train/320c8b7ddb0d/results/full_dev_rankings.parquet

    8ba37c7e922649f0ddbce8c8f5176bbfcf2c72aa5a828671f45a3e3e58c2f187

  • TRAINING_INPUT

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/candidate_embeddings.parquet

    6e663819fb44c92759603a557d8900da04bfae9b75b656fc3bfab048609801c5

  • TRAINING_INPUT

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/dev_mapping.parquet

    30c25f47791c1758fd738470d19715df27cb7eda931eb17a19d4c3587a6e4356

  • TRAINING_INPUT

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/dev_query_embeddings.parquet

    0b94e69f3dcb99e9417b477f485d69f0cb4443c3b89049f62a8ad90427f5d398

  • TRAINING_INPUT

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/train_mapping.parquet

    758f96536f3685fdf12a866cfd81819ec7e47484d52ca15b05b46a45573d1735

  • TRAINING_INPUT

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/train_query_embeddings.parquet

    acfe4c8e2f50da34e73626c463668d43f5d4bff39d98928fa574d8f42e2b4f83

  • VALIDATED_DEV_RANKINGS

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_query_projection_lc_v1_20260823/full_dev_rankings_validated.parquet

    5c2e94ee5cf629488c48160d59f78e4ee047ca044d5837e5edcea730110f8ced

Provenance relacionado

Entities

a-scale:projection:preregistration
PREREGISTRATION
a-scale:projection:train-membership
TRAIN_MEMBERSHIP

Activities

a-scale:projection:preregistration-activity
PREREGISTRATION_GIT_BOUND · 6e3fdcbf614d7f3085af677139dec954b60f8d09 · 2026-08-23T22:54:52-05:00 · 2026-08-23T22:54:52-05:00

Edges

a-scale:projection:edge:prereg-generated
WAS_GENERATED_BY: a-scale:projection:preregistration → a-scale:projection:preregistration-activity
a-scale:projection:edge:prereg-used-train
USED: a-scale:projection:preregistration-activity → a-scale:projection:train-membership

Source authority SHA-256

ab2d1066ba1a483a08a5bff24bee85f68197ebef177d93d999a26c059b93f088