Download the Tutorial Data#
Every later chapter assumes the tutorial files are already on disk under a
data/ directory and refers to them by path alone. If a command in a later
chapter cannot find its input, the fix is here, not there.
The Atacama study behind tier 1 and the ASV-1 … ASV-13 labelling are covered in
Atacama Soil Microbiome.
Where the files go#
Pick one project directory and stay in it. Every command in the tutorial is
written relative to that directory, with a data/ prefix:
mkdir -p ~/q2-hdstats-tutorial/data
cd ~/q2-hdstats-tutorial
Tip
A few of the high-dimensional chapters show bare filenames
(atacama-top-300-correlation.qza rather than data/atacama-top-300-correlation.qza).
Either cd data before running those, or add the prefix. Nothing else changes.
The Zenodo record#
The Atacama artifacts (tiers 1 and 2) are published as a single Zenodo record,
q2-hdstats-tutorial-data v1.0.0. Publishing them rather than pointing at
the plugin repositories keeps the exact bytes citable and fixed.
Note
The DOI has not been minted yet. Wherever a base URL is needed below, the
literal placeholder ZENODO_DOI_PENDING stands in for it. The tier 1 and tier 2
commands therefore cannot be run as written yet — they carry one outstanding
substitution. Search the sources for ZENODO_DOI_PENDING to find every place
that has to change when the record goes live.
Set the base URL once and reuse it. Every tier 1 and tier 2 block below assumes this variable is exported in your shell:
# Substitute the real record URL here once the DOI is minted.
export ZENODO_BASE=ZENODO_DOI_PENDING
The manifest#
docs/_data/manifest.tsv in this repository is the machine-checkable index of
everything the tutorial downloads. It has five tab-separated columns:
Column |
Meaning |
|---|---|
|
Basename as it lands on disk |
|
|
|
Expected size |
|
Expected checksum |
|
Where it came from (Zenodo for both tiers) |
Prose can drift out of date. The manifest cannot, because it is what the verification steps below read. Treat it as authoritative when the two disagree, and open an issue.
Define this helper once — every tier below uses it to turn the manifest into a
checklist that sha256sum -c understands. It needs a checkout of this
repository; point MANIFEST at wherever yours is:
export MANIFEST=~/q2-hdstats-docs/docs/_data/manifest.tsv
tier_checklist () {
awk -F'\t' -v tier="$1" 'NR > 1 && $2 == tier { print $4 " " $1 }' "$MANIFEST"
}
Note
Every bytes and sha256 value in the manifest is real — all 16 rows — and
each was verified against the file it describes. tier_checklist N | sha256sum -c
reports OK for every tier, so the verification steps below work today.
The url column for tiers 1 and 2 is the one entry still outstanding: it reads
ZENODO_DOI_PENDING until the record is minted. You can verify files you
already have, but the curl commands cannot fetch them yet.
Tier 1 — Atacama, 13 ASVs#
Every action in both plugins gets its one canonical demonstration on this table, which is small enough to print in full.
cd ~/q2-hdstats-tutorial/data
curl -L -O "${ZENODO_BASE}/atacama-counts.qza"
curl -L -O "${ZENODO_BASE}/classification.qza"
curl -L -O "${ZENODO_BASE}/selected-atacama-sample-metadata.tsv"
curl -L -O "${ZENODO_BASE}/atacama-selected-covariates-veg.tsv"
File |
What it is |
|---|---|
|
|
|
|
|
Sample metadata passed to |
|
The five covariates used by q2-classo: |
Verify:
tier_checklist 1 > ~/q2-hdstats-tutorial/data/SHA256SUMS.tier1
cd ~/q2-hdstats-tutorial/data && sha256sum --ignore-missing -c SHA256SUMS.tier1
--ignore-missing matters: it lets you verify a partial download against a
checklist that also names files you deliberately skipped, instead of failing on
their absence.
You need classification.qza even for the pure network chapters:
qiime gglasso transform-features declares --i-taxonomy as a required input
although its function body never reads it. See
Troubleshooting for that and the other
registration warts.
Note
On the sample count. The tier 1 table is 13 features × 50 samples, read
directly from the artifact. Earlier drafts said 49 in prose while every command
passed --p-n-samples 50; the commands were right. The metadata TSV has 75 rows
because it covers the full Atacama tutorial, of which 50 samples appear in this
table.
Tier 2 — Atacama, 300 ASVs#
The same study, scaled up: the 300 most abundant ASVs, which demonstrate model selection and latent-rank choice in a regime where you cannot eyeball the matrix.
cd ~/q2-hdstats-tutorial/data
curl -L -O "${ZENODO_BASE}/atacama-top-300-table.qza"
curl -L -O "${ZENODO_BASE}/atacama-top-300-clr.qza"
curl -L -O "${ZENODO_BASE}/atacama-top-300-correlation.qza"
curl -L -O "${ZENODO_BASE}/atacama-taxonomy-silva138.qza"
curl -L -O "${ZENODO_BASE}/sample-metadata.tsv"
curl -L -O "${ZENODO_BASE}/top-300-asvs.tsv"
curl -L -O "${ZENODO_BASE}/atacama-classo-outcomes-mean-imputed.tsv"
File |
What it is |
|---|---|
|
|
|
The transformed table, shipped so you can skip the transform step |
|
|
|
|
|
Full sample metadata, including |
|
Total abundance per feature, keyed on |
|
The regression outcomes with missing values mean-imputed, as used by the q2-classo cross-validation chapter |
Verify:
tier_checklist 2 > ~/q2-hdstats-tutorial/data/SHA256SUMS.tier2
cd ~/q2-hdstats-tutorial/data && sha256sum --ignore-missing -c SHA256SUMS.tier2
Recomputing the transformed table and the correlation matrix from
atacama-top-300-table.qza is the first exercise of the tier 2 chapters, and
holding the reference versions on disk makes a mismatch visible immediately
rather than letting it propagate into the network.
The cocoa appendix has no download#
No feature table is published for the MOSHPIT cocoa fermentation example (14 shotgun metagenomes, BioProject PRJNA552479), so there is nothing to fetch. Reproducing one means running assembly and binning and holding local Kraken2/Kaiju databases. See Appendix: Shotgun Metagenomics (MOSHPIT cocoa) for what that involves before you commit compute to it.
Verifying the download#
A truncated .qza is still a valid ZIP prefix. QIIME 2 will often load it
without complaint, and the failure surfaces much later, inside a solver, as
something that looks like a numerical problem rather than a broken file. Ten
seconds of sha256sum saves an afternoon of debugging the wrong layer.
Once the checksums pass, confirm that QIIME 2 reads the artifacts as the types the tutorial expects:
cd ~/q2-hdstats-tutorial
qiime tools peek data/atacama-counts.qza
qiime tools peek data/classification.qza
Note
The expected qiime tools peek output — UUID, semantic type and format — is
pending verification against QIIME 2 2026.7. Most of this book has not yet been
re-run against that release, and the UUID differs with every download, so no
captured output accompanies the commands above.
The resulting tree#
After both tiers:
~/q2-hdstats-tutorial/
└── data/
├── atacama-counts.qza
├── classification.qza
├── selected-atacama-sample-metadata.tsv
├── atacama-selected-covariates-veg.tsv
├── atacama-top-300-table.qza
├── atacama-top-300-clr.qza
├── atacama-top-300-correlation.qza
├── atacama-taxonomy-silva138.qza
└── sample-metadata.tsv
Everything else the tutorial uses is derived: each chapter writes its
transformed tables, covariance matrices, solutions and visualizations back into
data/, so the directory grows as you work through the book. Only the files
above have to be fetched.
Note
The tier 2 chapters also build a design-matrix table and a mean-imputed outcomes TSV for the q2-classo section. Whether those ship on the Zenodo record or are produced by the chapter commands is being settled as the record is assembled. The manifest will say which.
Next#
With the data in place, continue to Prerequisites & Installation, then Verifying Your Installation — the last step of which reads one of the artifacts you just downloaded. To see which chapter demonstrates which command before you start, the Command Coverage Matrix is the map.