CUBSO

Catálogo CUBSO · read-only

NATIVE · CANONICAL_COMPLETE

CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1

Track CUBSO_A_ONLY_RETRIEVAL_FUSION

Experimentos

Pregunta científica

Does the unchanged RRF(k=60) treatment that passed CALIBRATION show the same recall direction and materiality on the already-inspected DEV cohort?

Without retuning k, source weights or retrievers, RRF(k=60) satisfies the same material-gain gate on DEV; this is a same-cohort follow-up, not fresh validation.

Cohort

CUBSO_A_SCALE_DEV

DEVELOPMENT

Exposure: INFORMED_HYPOTHESIS · Sealed: SEALED

212409444c17fb058878658b886f2557a46e91d5ea93396437604d25c7f645be

Baseline

Retained full-data Procrustes ranking on the already-inspected A-scale DEV cohort.

Treatment

The exact CALIBRATION-kept treatment: full 85,711-candidate RRF(k=60), equal contribution from retained Procrustes and frozen BGE-M3-5k.

Variable causal

rank aggregation only: full-catalog Procrustes + BGE5K -> RRF(k=60)

Modelo y ejecución

fixed reciprocal-rank fusion over retained Procrustes and BGE-M3-5k

Procrustes:CUBSO_A_SCALE_QUERY_PROJECTION_LC_V1;BGE5K:ca27492deb375a01a369c549cac486450770752cc3d06d8f6b26c6ce597579b4
COMPLETE
PASS_17_OF_17_CANONICAL_CPU_RRF_PROBES

Runtime: CUBSO_A_SCALE_FULL_RRF_GPU_EVALUATION_V1 · Provider: kaggle

3333d8f3dd41e900465cd772ff6544246218f98c

Decisión científica

KEEP

CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1 was evaluated only on the preregistered DEV development cohort; TEST and TEMPORAL_TEST remain sealed.

Regla: KEEP_PROCRUSTES_BGE5K_RRF60

Resultado: matched preregistered DEV screening decision gate

External validation: no · Replication required: sí

Métricas

Sólo se visualizan métricas primarias que comparten semántica de display; el gráfico no autoriza comparaciones entre familias incompatibles.

MétricaValor exacto
DeltaMRR-0.00261586510531
DeltaRecall@1000.0454265159301
DeltaRecall@200.0361767728674
DeltaRecall@500.0504624871531
DeltaTargetUnseenMRR0.00535290450491
DeltaTargetUnseenRecall@1000.0528879610299
DeltaTargetUnseenRecall@200.0487125956855
DeltaTargetUnseenRecall@500.0521920668058
DeltaTargetUnseenTop1-0.00487125956855
DeltaTop1-0.0131551901336
McNemar_R100_p3.39070376126e-75
MRR0.389845551432
Recall@1000.794347379239
Recall@200.666598150051
Recall@500.754676258993
TargetUnseenMRR0.256740857970
TargetUnseenRecall@1000.688239387613
TargetUnseenRecall@200.514265831594
TargetUnseenRecall@500.628392484342
TargetUnseenTop10.176061238692
Top10.290853031860

Tests estadísticos

overall_recall_at_100_losses
97
overall_recall_at_100_mcnemar_exact_p
3.3907037612643287e-75
overall_recall_at_100_ties
9094
overall_recall_at_100_wins
539
overall_recall_at_20_losses
340
overall_recall_at_20_mcnemar_exact_p
2.584268375121685e-28
overall_recall_at_20_ties
8698
overall_recall_at_20_wins
692
overall_recall_at_50_losses
144
overall_recall_at_50_mcnemar_exact_p
2.6173112242766577e-74
overall_recall_at_50_ties
8951
overall_recall_at_50_wins
635
target_unseen_recall_at_100_losses
17
target_unseen_recall_at_100_mcnemar_exact_p
7.237227129476197e-14
target_unseen_recall_at_100_ties
1327
target_unseen_recall_at_100_wins
93
target_unseen_recall_at_20_losses
50
target_unseen_recall_at_20_mcnemar_exact_p
8.020158388607808e-8
target_unseen_recall_at_20_ties
1267
target_unseen_recall_at_20_wins
120
target_unseen_recall_at_50_losses
27
target_unseen_recall_at_50_mcnemar_exact_p
1.881974752113246e-11
target_unseen_recall_at_50_ties
1308
target_unseen_recall_at_50_wins
102

Incertidumbre

DeltaMRR
[-0.009302745613121265,0.004155429404529384]
DeltaRecall@100
[0.03977070859619628,0.051590549563123894]
DeltaRecall@20
[0.028520702716996898,0.044082555840814226]
DeltaRecall@50
[0.04394549093659868,0.05764889325659599]
DeltaTop1
[-0.02216078896078318,-0.0042478728193568365]
TargetUnseenDeltaMRR
[-0.006459634144098458,0.01781887091321563]
TargetUnseenDeltaRecall@100
[0.03538096943608557,0.0718466457368838]
TargetUnseenDeltaRecall@20
[0.030797273426362903,0.06762321468787004]
TargetUnseenDeltaRecall@50
[0.03397511127311716,0.07137517938513227]
TargetUnseenDeltaTop1
[-0.02083596570397112,0.011602000762800744]

Lineage

Informed by

CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_CALIBRATION_V1
CUBSO_A_SCALE_QUERY_PROJECTION_LC_V1
CUBSO_A_SCALE_BGE_M3_DENSE_FINETUNE_5K_V1

Same population as

Code bindings

workspaces/conosce/conosce/cubso/experiments/retrieval/a_scale/rrf_evaluation.py

GIT_BLOB

3333d8f3dd41e900465cd772ff6544246218f98c
0d78e0d5731df75149904c343a09b86dbff96b04e54b3fad584e7eec23712816

workspaces/conosce/conosce/cubso/experiments/retrieval/a_scale/rrf_gpu_worker.py

GIT_BLOB

3333d8f3dd41e900465cd772ff6544246218f98c
a84b31bf5459373afc28c12bb074c190a47f2d1741bd7d6d29cb8d0bed4c021f

Artefactos y evidencia

  • BASE_C0_EMBEDDINGS

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/dataset/base_candidate_embeddings.parquet

    6e663819fb44c92759603a557d8900da04bfae9b75b656fc3bfab048609801c5

  • BGE5K_C0_EMBEDDINGS

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/dataset/bge5k_candidate_embeddings.parquet

    72a3af050a43762b4d2cc418180894dc2b31239c19d4ff65c547a1725c0972e6

  • BGE5K_DEV_EMBEDDINGS

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/dataset/bge5k_query_embeddings.parquet

    983509072a4581c8914c7a62a838838cf2c6a3edf98cb993359a754610ca8663

  • DEV_MAPPING

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/dataset/query_mapping.parquet

    245754f04ddfa1228ff46d86b93b82718f09d01edb2f5b9aed0c898da67e3fc2

  • EXACT_DEV_RANKING

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/results/rrf_ranking.parquet

    11bab59c84bf18cccc6b971a4c52c0c652de3da18cb7233fbbd24bb1024638ae

  • GPU_SUMMARY

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/results/job_summary.json

    cf3e0f617b610aca88df384f5110107c3bc13f8dfbb41da35c66dfe758733bde

  • PAIRED_DEV_EVALUATION

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/evaluation.json

    6becf6717a3a2629f12003286336e812a95d8502b8625502a9e87c61700e2b52

  • PREREGISTRATION

    docs/research/cubso/preregistrations/CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1.json

    9aa0ec1ac460574d9af8249d14761b3dbdd711856049d08452c6bc1ce6060e32

  • PROCRUSTES_DEV_EMBEDDINGS

    artifacts/derived/procurement_need/tasks/cubso/experiments/a_scale_procrustes_bge_rrf_dev_followup_v1_20260824/providers/kaggle-rrf/c71c902e9cfd/dataset/procrustes_query_embeddings.parquet

    e52c9b8d81e69427d2e27877db3f565341536359a2edd9d31f86ee3146ecb77d

Provenance relacionado

Entities

a-scale:rrf-dev:preregistration
PREREGISTRATION
a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:decision
SCIENTIFIC_DECISION_EVIDENCE
a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:evaluation
EVALUATION

Activities

a-scale:rrf-dev:preregistration-activity
PREREGISTRATION_GIT_BOUND · 8bdf7cc7fc2981e7acb2cf336ad27bfdb34ec585 · 2026-08-24T01:58:44-05:00 · 2026-08-24T01:58:44-05:00
a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:binding-activity
RESULT_BINDING · 3333d8f3dd41e900465cd772ff6544246218f98c · 2026-08-24T12:09:13.897000Z · 2026-08-24T12:09:13.897000Z

Edges

a-scale:rrf-dev:edge:prereg-generated
WAS_GENERATED_BY: a-scale:rrf-dev:preregistration → a-scale:rrf-dev:preregistration-activity
a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:edge:1
WAS_GENERATED_BY: a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:evaluation → a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:binding-activity
a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:edge:2
WAS_GENERATED_BY: a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:decision → a-scale:screening-result:CUBSO_A_SCALE_PROCRUSTES_BGE5K_RRF_DEV_FOLLOWUP_V1:binding-activity

Source authority SHA-256

ab2d1066ba1a483a08a5bff24bee85f68197ebef177d93d999a26c059b93f088