Reproducibility package

Jev router study: scripts and data

Everything behind Testing Jev in High Volume Retrieval and Routing: claim-level data, results, scripts and pre-registrations. The package does not include the dossiers, claim text or evidence text.

Download everything (.zip, 4.9 MB) Back to the report

Read me first

What each file holds and how the package was built.

README.md8 KB

Claim-level data

One CSV per test set. Vendor names are replaced by codes (V001 to V153).

data/devset_calls.csv99 KB
data/devset_claims.csv87 KB
data/founder_f3.csv4 KB
data/pilot.csv42 KB
data/scaleup.csv174 KB
data/ws4_planted.csv385 KB
data/ws4_plants.csv3 KB

Aggregate results

Results files and rendered charts for each workstream.

results/charts/adj-1.png227 KB
results/charts/adj-1.svg138 KB
results/charts/adj-2.png218 KB
results/charts/adj-2.svg131 KB
results/charts/adj-3.png153 KB
results/charts/adj-3.svg100 KB
results/charts/f4-1.png184 KB
results/charts/f4-1.svg112 KB
results/charts/f4-2.png198 KB
results/charts/f4-2.svg129 KB
results/charts/f4-3.png240 KB
results/charts/f4-3.svg137 KB
results/charts/specs_adj.json6 KB
results/charts/specs_f4.json6 KB
results/charts/specs_overrides.json41 KB
results/charts/ws1-1.png182 KB
results/charts/ws1-1.svg101 KB
results/charts/ws1-2.png125 KB
results/charts/ws1-2.svg90 KB
results/charts/ws1-3.png336 KB
results/charts/ws1-3.svg198 KB
results/charts/ws1_charts.json3 KB
results/charts/ws2-1.png293 KB
results/charts/ws2-1.svg218 KB
results/charts/ws2-2.png289 KB
results/charts/ws2-2.svg162 KB
results/charts/ws2-3.png318 KB
results/charts/ws2-3.svg189 KB
results/charts/ws2-4.png296 KB
results/charts/ws2-4.svg182 KB
results/charts/ws2_charts.json27 KB
results/charts/ws3-1.png276 KB
results/charts/ws3-1.svg179 KB
results/charts/ws3-2.png201 KB
results/charts/ws3-2.svg130 KB
results/charts/ws3-3.png258 KB
results/charts/ws3-3.svg239 KB
results/charts/ws3-4.png259 KB
results/charts/ws3-4.svg169 KB
results/charts/ws3_charts.json5 KB
results/charts/ws4-1.png226 KB
results/charts/ws4-1.svg150 KB
results/charts/ws4-2.png213 KB
results/charts/ws4-2.svg143 KB
results/charts/ws4-3.png121 KB
results/charts/ws4-3.svg84 KB
results/charts/ws4-4.png106 KB
results/charts/ws4-4.svg76 KB
results/charts/ws4_charts.json5 KB
results/devset_cause_codes.json25 KB
results/devset_explor_or_robustness.json1 KB
results/devset_label_provenance.json50 KB
results/devset_tables.json166 KB
results/devset_ws2_results.json155 KB
results/evidence_channel_measure_armb.out2 KB
results/founder_f3_SCORE.md1 KB
results/pilot_adjudication.json8 KB
results/pilot_analysis.json59 KB
results/scaleup_adjudication.json3 KB
results/scaleup_build_summary.json1 KB
results/ws4_results.json36 KB

Scripts

Analysis, scoring and chart scripts. Run python scripts/recount.py from the package folder to recompute the headline counts.

scripts/adjudication/append_label.py4 KB
scripts/adjudication/build_inputs_pilot.py4 KB
scripts/adjudication/build_inputs_recheck_window.py4 KB
scripts/adjudication/build_inputs_scaleup.py4 KB
scripts/adjudication/score_f3.py5 KB
scripts/adjudication/score_pilot.py7 KB
scripts/adjudication/score_recheck_window.py5 KB
scripts/adjudication/score_scaleup.py10 KB
scripts/charts/build_specs.py17 KB
scripts/charts/gen_index.py32 KB
scripts/charts/render_all.py33 KB
scripts/devset/ws2_analysis.py18 KB
scripts/devset/ws2_make_charts.py7 KB
scripts/devset/ws3_analyze.py10 KB
scripts/devset/ws3_explor.py2 KB
scripts/devset/ws3_tally_causes.py4 KB
scripts/evidence_channel/measure_armb.py5 KB
scripts/pilot/analyze.py12 KB
scripts/pilot/jev_shadow.py14 KB
scripts/pilot/make_packet.py2 KB
scripts/recount.py2 KB
scripts/scaleup/make_packet.py4 KB
scripts/scaleup/run_f4.py7 KB
scripts/scaleup/spend.py1 KB
scripts/scaleup/summarize.py3 KB
scripts/t2_raw_excerpt.py5 KB
scripts/ws4/analyze.py14 KB
scripts/ws4/make_charts.py5 KB
scripts/ws4/plant.py6 KB
scripts/ws4/run_arms.py6 KB
scripts/ws4/run_glm.py1 KB
scripts/ws4/ws4lib.py14 KB

Pre-registrations

Each workstream's pre-registration or protocol, with HASHES.md to check they were not changed after the runs.

prereg/adj_PROTOCOL.md7 KB
prereg/adj_recheck-window-only_PROTOCOL.md4 KB
prereg/f4_adj_PROTOCOL.md7 KB
prereg/f4_PREREG.md8 KB
prereg/HASHES.md2 KB
prereg/ws1_PREREG.md8 KB
prereg/ws2_PREREG.md8 KB
prereg/ws3_PREREG.md12 KB
prereg/ws4_PREREG-ADDENDUM-1.md5 KB
prereg/ws4_PREREG.md9 KB