Overview#
The Atacama chapters introduce no new QIIME 2 action. Every command on the following pages has already been demonstrated in Tier 1 on a 13-ASV table you could read off the screen. What changes is the size of the problem — 300 ASVs across 54 samples of the Atacama soil microbiome — and with it the need to argue for the parameter values. At \(p = 13\) you can pick \(\lambda_1\) by eye. At \(p = 300\), with \(p \gg n\), the penalty is the model: it decides how many of the 44,850 possible edges you are willing to believe.
Three problems come with the larger table:
Choosing \(\lambda_1\) with a criterion rather than a guess (Selecting lambda).
Sizing the latent block when you cannot set its rank directly (Choosing the Latent Rank), and reading the resulting components against measured variables (Latent Components & Covariates).
Asking whether the two plugins agree — whether the structure the network attributes to unobserved drivers is the same structure the log-contrast models are exploiting (Interpretation).
The canonical parameterization#
One parameterization covers the Atacama analysis. Every page uses it, and any figure or artifact carrying other values comes from an older bundle.
Setting |
Value |
Where it is argued |
|---|---|---|
|
|
fixed by the table |
|
|
|
|
|
|
|
|
|
|
on, for the SLR models |
|
|
|
Change these three last — they are the ones people change first and regret later.
\(\lambda_1 = 0.8\) minimises the extended BIC along a linear path. Move it down and the graph densifies fast: the edge count is a steep function of \(\lambda_1\) in this regime, so a value that looks “only slightly less conservative” can multiply the edge set several-fold.
\(\gamma = 0.3\) is the eBIC’s extra edge penalty. \(\gamma\) and \(\lambda_1\)
are selected jointly, in the sense that changing \(\gamma\) changes which
\(\lambda_1\) wins. Foygel and Drton’s conventional \(\gamma = 0.5\)
[M2] and the tier-1 default of 0.01 both select a
different network on this table. If you report a network, report the \(\gamma\)
that produced it.
\(\mu_1 = 15\) is the nuclear-norm penalty on the low-rank block, and the only
way to influence the rank — --p-rank raises NotImplementedError on every
released GGLasso. Larger \(\mu_1\) gives a smaller rank. The value 15 was found by
scouting at \(\lambda_1 = 0.8\), and does not transfer to another \(\lambda_1\) or
another table.
Important
Read the --p-mu1-* value as a penalty, never as a rank. The rank is an output.
Choosing the Latent Rank gives the procedure for turning a
target rank into a \(\mu_1\) and for reading back the rank you got.
Two further SLR fits, at \(\mu_1 = 10\) and \(\mu_1 = 7.5\), appear on
03 and 04. They give you something to
argue the rank-2 choice against, and — because qiime gglasso pca caps
--p-n-components at the achieved rank — they supply a solution with room for
more than the single pair-plot panel a rank-2 fit allows. Neither is an
alternative canonical model.
The superseded \(\lambda = 0.95\) bundle#
An earlier exploratory bundle of this analysis circulated with \(\lambda = 0.95\) and \(\mu = 10.5\), and drafts, notebooks and slide figures carrying those values still exist. Do not mix them with anything on these pages.
Whatever the merits of 0.95 in isolation, the two bundles answer different questions:
The 0.95 bundle predates the linear-path eBIC sweep and the choice of \(\gamma = 0.3\). Its \(\lambda\) was hand-picked at the conservative end of the path rather than selected by a criterion.
Its \(\mu = 10.5\) was scouted against \(\lambda = 0.95\). The sparse and low-rank blocks compete for the same covariance structure, so the \(\mu \rightarrow \text{rank}\) map is \(\lambda\)-specific: a rank obtained at \(\lambda = 0.95\) is not the rank the same \(\mu\) gives at \(\lambda = 0.8\), and the two are not comparable.
An artifact, figure or table mentioning \(\lambda = 0.95\) or \(\mu = 10.5\) comes from the exploratory bundle, and its edge counts, ranks and eBIC values cannot be quoted alongside the ones here. The canonical bundle is \(\lambda_1 = 0.8\), \(\gamma = 0.3\), \(\mu_1 = 15\).
Recompute status#
Note
Recomputed and confirmed. The single-graphical-lasso \(\lambda\) path on
Selecting lambda has been re-run under QIIME 2 2026.7:
the selected \(\lambda = 0.8\), its 216 edges and the \(\gamma\) sensitivity all
hold, and the recompute script generates the eBIC table on that page into
docs/_data/atacama-lambda-path.tsv instead of anyone typing it in.
The achieved ranks and the \(\mu_1 \rightarrow \text{rank}\) map
(Choosing the Latent Rank) have likewise been re-run at the
selected \(\lambda = 0.8\): \(\mu_1 = 15/10/7.5\) give rank \(2/5/10\), and that table
is generated into docs/_data/atacama-mu-rank-map.tsv.
Still pending. Nothing else has been re-run. The SLR rank-comparison edge counts and the \(m_t\)-versus-\(q_t\) Spearman correlation on Selecting lambda, the component-to-covariate correlations (Latent Components & Covariates), the cross-validated \(R^2\) values and selected features (Log-Contrast Models at Scale), and the SGL-versus-SLR edge-set comparison and named taxa (Interpretation) are all pending verification against QIIME 2 2026.7.
Treat the pending values as the expected result of the recompute rather than a confirmed one.
Once the recompute has run, the numbers will not be typed into these pages by hand. Each figure comes from a pinned artifact bundle plus a recompute script: the bundle fixes the inputs (the count table, the taxonomy, the metadata, and their checksums), the script fixes the commands and the parameter values in the table above, and the page cites both. Only then can a reader who disagrees with a number tell whether it came from a different input or a different command.
Note
The artifact bundle will be published as the Zenodo record
q2-hdstats-tutorial-data v1.0.0. The DOI has not been minted yet. Wherever a
download URL is required, these pages carry the literal placeholder
ZENODO_DOI_PENDING; grep for it to find every occurrence and replace them in
one pass when the record goes live.
The pages#
Page |
What it adds |
Actions used |
|---|---|---|
provenance of the table, taxonomy and transformed artifacts |
|
|
eBIC on a linear \(\lambda\) path; the \(\gamma\) sensitivity |
|
|
the \(\mu_1 \rightarrow\) rank scouting procedure |
|
|
projecting samples on the latent axes and matching them to measured variables |
|
|
cross-validated log-contrast regression for each covariate |
|
|
reading the sparse and low-rank parts together, and against the regression coefficients |
— |
Conventions#
Commands assume you are working in the directory where the Tier 2 bundle was
unpacked, so artifacts appear as bare filenames
(atacama-top-300-correlation.qza, not data/atacama-top-300-correlation.qza).
Adjust the paths if you keep them elsewhere;
Download the Tutorial Data gives
the layout the rest of the book assumes.
These pages assume both plugins are installed and registered — see Verifying Your Installation — and that you have worked through at least Single Graphical Lasso, Sparse + Low-Rank and Latent-Component PCA, none of which is explained again here.
Troubleshooting collects the failures
you are most likely to hit at this scale — --p-rank raising, pca refusing a
non-latent solution, a model-selection run that silently collapsed to a single
fit. The full parameter lists are in the
q2-gglasso and
q2-classo references.