10Selecting the active space

Reduce the orbital problem without losing sight of the chemistry

Lesson 9 gave us methods for optimizing determinant amplitudes. The remaining difficulty is that allowing every occupation of every orbital is usually too expensive. Active-space selection makes a different reduction: choose which orbitals can participate in explicit electron correlation at all.

Our question remains the electronic energy of neutral singlet N₂ at 1.85 Å. The starting cc-pVDZ representation has 28 spatial orbitals and fourteen electrons. Selecting a smaller active space does not shrink the molecule or change its nuclear charge. It changes the many-electron model built from that orbital representation.

We will follow all four registered native selectors and the optional AVAS adapter, then trace the actual nitrogen tutorial. Code is illustrative and unexecuted. Numerical reference values below are attributed to repository tests, not calculations performed during this preparation.

Active, inactive, and excluded are physical modeling choices

An inactive orbital is held occupied in the reference partition—doubly occupied in the restricted paired-electron case used here. An active orbital may change occupation across determinants. An excluded virtual orbital remains empty in the correlated model. Beware the word virtual: an orbital unoccupied in HF can still be active in CASCI, where excitations populate it.

We will write CAS($N_e$e, $N_o$o) for $N_e$ active electrons in $N_o$ active spatial orbitals. Separately record $N_\alpha$ and $N_\beta$ because the solver takes those counts. For a singlet-like paired sector, CAS(6e,6o) uses three alpha and three beta electrons, giving 400 determinants before further symmetry reductions. Equal counts alone are still not a total-spin certificate.

For our fourteen-electron reference, a CAS(6e,6o) model with four inactive orbitals accounts for eight inactive plus six active electrons. If eighteen of the 28 spatial orbitals are excluded, the bookkeeping is four inactive, six active, eighteen excluded. The original AO/MO data can remain available even though the active Hamiltonian acts on only six spatial orbitals.

Selecting six orbitals and solving every determinant within them is not the same as selecting a few determinants across all 28 orbitals. Both reductions can be used together: selected CI inside a candidate active orbital space is a useful preliminary calculation. Their errors must not be merged into one unexplained “active-space approximation.”

Frozen electrons still change the active Hamiltonian

We do not merely delete the inactive-electron rows and forget their interactions. For real restricted orbitals, let $i,j$ label inactive doubly occupied orbitals and $p,q$ active orbitals. The effective one-body matrix of the frozen-core active model is

The inactive electrons contribute a Coulomb field and exchange, just as occupied electrons did in HF. The constant energy is

For an ECP model, the nuclear/core constant and one-body operator must use the effective-Hamiltonian convention already established. The remaining active–active two-electron integrals are retained. Correlation involving changes of inactive occupancy or occupation of excluded orbitals is omitted by this bare active-space model.

The native Hamiltonian constructor implements this through an inactive density, inactive Coulomb/exchange matrices, the inactive Fock matrix, and an inactive energy constant. Its output therefore carries more than a smaller subarray. Later correlated solvers add the stored core constant to the active eigenvalue. This is frozen-core embedding, not the more general correlation-dependent downfolding discussed in Lesson 11.

What a selector actually returns

The interface is selector.run(wavefunction), returning a new Wavefunction with orbital-partition metadata. Native registration provides exactly qdk_valence, qdk_occupation, qdk_autocas, and qdk_autocas_eos. The factory default is qdk_autocas_eos, not the valence selector labeled “default” in some prose. An entropy-based default still needs suitable entropy information; creating it does not make that information exist.

The source file named entropy_active_space.cpp implements qdk_autocas_eos; it does not register an additional qdk_entropy method. The registration code is the catalog to follow. The optional PySCF implementation is pyscf_avas.

Native selection helpers retain the coefficient representation for metadata-only selections and derive inactive indices from Aufbau occupied positions outside the selected active list. They downselect from the input’s candidate active indices; they do not retrieve an orbital that was already removed from that candidate space. The returned wavefunction is not generally the correlated input restricted with all amplitudes preserved. A one-determinant input has its occupation record projected to the new active ordering. For a multi-determinant input, the helper constructs an Aufbau occupation record and then projects it. Correlated amplitudes and RDM payloads are not propagated as a new correlated solution.

Thus “selected wavefunction” usually means a model/reference carrier ready for Hamiltonian construction and a fresh solve. That is consistent with the localizer warning in Lesson 8. Keep the original correlated evidence used for selection and do not present the selector’s single determinant as that evidence itself.

Manual indices are supported as data, not a separate native selector

If chemical analysis already supplies the exact indices, a new Orbitals object can carry explicit active_indices and inactive_indices. The public spin_index_set helper constructs symmetry-aware index sets from sorted alpha/beta lists.

For illustration only, this specifies a six-orbital partition in the original restricted HF representation:

from qdk_chemistry.data import Orbitals
from qdk_chemistry.data.symmetry import spin_index_set

original = reference.get_orbitals()
n_mo = original.get_num_molecular_orbitals()
active = list(range(4, 10))
inactive = list(range(4))
manual_orbitals = Orbitals(
    coefficients=original.coefficients(), energies=original.energies(),
    ao_overlap=original.get_overlap_matrix(), basis_set=original.get_basis_set(),
    active_indices=spin_index_set(n_mo, active, active),
    inactive_indices=spin_index_set(n_mo, inactive, inactive),
)

A Hamiltonian constructor can consume manual_orbitals; a CASCI solver would separately need the consistent active alpha/beta counts. The same integers applied to natural or localized orbitals can describe a different physical subspace. This manual HF partition is therefore not automatically the tutorial’s final gauge-fixed natural-orbital model, even though its index lists look identical.

Ensure active/inactive indices are disjoint and valid, electron counts fit the selected orbitals, and the chosen spin sector is possible. Creating metadata does not project a correlated state with a guaranteed retained norm or construct all omitted interactions. No index list by itself contains that scientific argument.

qdk_valence: an explicit frontier-window model

The valence selector uses num_active_electrons and num_active_orbitals, both defaulting to invalid sentinel -1 until supplied. It requires restricted orbitals carrying orbital energies, positive requested sizes, sufficient candidates, no more active electrons than total electrons, and an even number of electrons left inactive.

Its actual rule computes the number of inactive pairs as

then selects a contiguous block in the candidate-index ordering beginning after that offset. It does not scan the numerical orbital energies to discover the best chemical valence subspace. The expected canonical ordering makes this a frontier-window heuristic.

The HOMO is the highest occupied molecular orbital and the LUMO the lowest unoccupied molecular orbital of a specified reference. The Fermi level is a filling boundary or chemical-potential notion; for a small finite molecule, an occupied/unoccupied frontier is the more direct picture. Selecting occupied orbitals near that boundary together with nearby unoccupied partners can capture important bonding/antibonding flexibility, but an energy-neighbor need not be the chemically relevant orbital in a metal complex, diffuse basis, or unusual excited state.

Stored energies are only a structural prerequisite in this implementation, not an independent proof that columns are actually canonical and energy-ordered. Repeated downselection of an already reduced candidate list also follows index/offset rules rather than a universal chemical tracking algorithm. Inspect the returned lists and active counts instead of relying solely on requested sizes. The code’s checks are not a substitute for independently ensuring $N_\alpha,N_\beta\le N_o$ and appropriate inactive pairing.

The helper compute_valence_space_parameters(reference, charge) provides a starting size, not a selector by itself. It estimates valence electrons from element identities relative to smaller noble-gas cores and supplies a period-based orbital count, capped by available orbital space. The implementation counts nitrogen as five valence electrons and four valence orbitals, so N₂ suggests CAS(10e,8o). Its optional include_double_d_shell=True adds five correlating d-like directions per supported d-block atom to the suggested orbital count; it does not create those functions or prove that their correlation is adequately represented. Elements beyond the implemented period-six range receive a warning. Charges and ECP-derived explicit counts must remain consistent with the chosen reference.

Starting from the original fourteen-electron, 28-orbital RHF reference, the nitrogen tutorial’s valence window is indices 2 through 9, leaving orbitals 0 and 1 inactive. That is a manageable candidate for subsequent analysis, not a claim that all other correlation has disappeared.

qdk_occupation: threshold fractional occupations with the right provenance

The occupation selector uses occupation_threshold=0.1. For an occupation $n_p$, its actual test is

where $t$ is the threshold. For physical occupations in $[0,2]$, this selects $t\le n_p\le2-t$, including equality. An orbital almost full or almost empty is omitted; a singly occupied or substantially fractional orbital is retained. The implementation requires restricted orbitals, a candidate active space, and an even total electron count. It rejects an empty selection. It does not explicitly impose equality of alpha/beta counts despite some older descriptions calling it strictly closed-shell.

This method needs a meaningful occupation source. A paired HF determinant has only zero/two total occupations, so with the default positive threshold it supplies no evidence for a correlated active space and can select nothing. Fractional occupations from a correlated density may be informative, but fractional spatial occupations from spin-traced broken-symmetry references or uncontrolled approximations need separate interpretation.

There is an important data-convention issue from Lesson 9. The wavefunction occupation accessor used here can return descending eigenvalues of each spin 1-RDM without rotating the attached MO coefficients. The selector then associates those entries with candidate indices. Therefore apply it only when occupation ordering is consistent with the orbital representation—particularly a validated restricted closed-shell natural-orbital representation—or inspect the mapping explicitly. A spectrum alone does not label arbitrary original canonical orbitals. A spin-traced natural basis of a more general open-shell density need not diagonalize both spin blocks separately.

The threshold is a heuristic about one-electron occupation, not a bound on energy lost by omitting a weakly occupied orbital. Many individually small contributions can sum to important dynamical correlation. Nor does a scalar occupation tell us which other orbitals form a coupled bonding or magnetic partner. That motivates entropy-based and chemically targeted approaches.

Entropy measures uncertainty of an orbital’s occupation

A spatial orbital has four local possibilities: empty, alpha-only, beta-only, or doubly occupied. Write its spin occupations as $n_\alpha,n_\beta$ and double-occupation expectation as $d$. Its probabilities are

The double-occupation quantity comes from the opposite-spin two-particle RDM. The single-orbital von Neumann entropy is

The code uses natural logarithms, so the maximum for four equally likely local states is $\ln4$, not two bits. Zero entropy means the local occupation is definite. For a pure correlated global state, local uncertainty can reflect entanglement with other orbitals. The value depends on the chosen orbital subsystems: rotating orbitals can change these local entropies without changing the exact full many-electron state.

A deterministic HF occupation gives zero local entropy in its own occupied/virtual orbital description, even if the true molecule would be strongly correlated. A selector cannot infer missing correlation from an approximation that never represented it. The wavefunction implementation can compute single-orbital entropy from available real spin-resolved active 1/2-RDMs or use an explicitly supplied entropy payload. Generic complex entropy evaluation is not implemented. Ask the preliminary solver for the necessary data; amplitude-container storage alone does not provide it.

The shared entropy-selection helper requires matching alpha/beta active index lists, one entropy per candidate orbital, and available entropy data. It sorts entropy values together with their actual orbital indices, keeping their identity during ranking. With normalize_entropies=True, it divides positive values by their maximum, not by $\ln4$ and not by the entropy sum. Thus normalized values express relative prominence, while the raw maximum still contains information about the overall scale.

qdk_autocas: a plateau in selected count, not a cluster of ten orbitals

The autoCAS family automates complete-active-space selection using correlated orbital diagnostics. The original native variant qdk_autocas uses defaults num_bins=100, min_plateau_size=10, entropy_threshold=0.14, and normalize_entropies=True.

The implementation first checks the raw maximum entropy. If it is below 0.14, it returns an empty active selection with a warning. This is an algorithmic threshold decision, not proof that HF is quantitatively accurate.

Otherwise it evaluates thresholds $b_k=k/\texttt{num\_bins}$ and counts how many sorted entropies are strictly above each threshold. A plateau is a run of adjacent thresholds for which that selected-orbital count stays unchanged. min_plateau_size=10 means ten threshold bins, not ten selected orbitals. With 100 bins, it asks for a relatively wide interval over which the same candidate count is suggested.

The code identifies sufficiently long plateaus, takes their positive selected counts, and chooses the largest such count. It retains that many highest-entropy orbitals, then sorts the selected physical indices for downstream use. If no qualifying positive count exists, it returns an empty space rather than silently inventing one.

Its entropy_threshold is a maximum-entropy gate in this implementation; the later plateau loop does not independently require each chosen orbital’s raw entropy to exceed that setting. Turning off normalization also does not rebuild histogram bins over the raw entropy range: thresholds remain from zero to just below one. Those precise semantics are more useful than interpreting every “entropy threshold” as one universal per-orbital cutoff.

The bin count must be at least two and plateau size at least one. Their resolution affects which gaps count as robust. A broad plateau often indicates a natural-looking separation in this diagnostic, but it is not an error-controlled proof that everything below it is chemically negligible.

qdk_autocas_eos: direct gaps in ranked entropy

EOS is enhanced orbital selection, as named in the repository documentation. The default native selector qdk_autocas_eos uses consecutive entropy differences instead of histogram bins. Its settings are normalize_entropies=True, diff_threshold=0.1, and entropy_threshold=0.14.

Let $\widetilde s_i$ denote the descending entropies after optional normalization. For positive raw maximum $s_{\max}$, the normalized absolute threshold is $0.14/s_{\max}$. The code considers a cut after position $i$ when

It chooses the largest qualifying prefix, not necessarily the first or largest gap. After normalization the first test still expresses an absolute raw-entropy condition, because the threshold is divided by the same maximum. The difference threshold, however, then measures a relative gap; with normalization disabled it measures a raw-entropy gap.

For a purely illustrative ranked list $[0.80,0.78,0.60,0.58,0.03,0.02]$, normalization gives approximately $[1,0.975,0.75,0.725,0.0375,0.025]$. With the defaults, cuts after the second and fourth orbitals pass both tests. EOS selects four because that is the largest qualifying count. This is not a nitrogen entropy calculation, and a count of four says nothing by itself about how many electrons remain active; original orbital identities and the inactive partition still determine that.

Unlike the histogram variant, EOS only considers adjacent differences. Equal high entropies with no qualifying gap need not produce an all-orbital selection; no boundary can mean an empty result. A one-candidate list likewise has no adjacent cut. The current EOS code lacks a separate explicit zero-maximum guard before threshold normalization. Treat nonempty meaningful entropy data as a prerequisite and check the result; do not rely on a generic promised exception or assume that absence of a selected space proves absence of correlation.

The selector tests use controlled synthetic entropy arrays to test normalization, threshold changes, non-contiguous indices, and empty selections. These synthetic records are algorithm tests, not molecular evidence.

What autoCAS does not do automatically

Neither native autoCAS selector runs an SCF, CASCI, ASCI, or orbital-localization calculation internally. Their settings contain no nested energy solver that creates missing entropies. The workflow supplies a preliminary correlated state. This is important both for cost accounting and for understanding why simply calling the default selector on a paired HF state is not a production active-space strategy.

A conservatively chosen valence candidate can be analyzed by CASCI if small enough. For larger candidates, a bounded ASCI calculation or another justified approximate correlated density can provide preliminary evidence. Those entropies need not be quantitatively exact to suggest relevant orbitals, but their stability under enlarging the determinant support or candidate space should be checked. An omitted orbital outside the preliminary candidate space is invisible to the entropy ranking; the algorithm cannot discover its importance from no data.

The selector’s explicit symmetry guard is matching alpha/beta candidate indices. It does not perform a general total-spin adaptation, enforce every point-group multiplet, or guarantee that a chosen cut preserves complete natural-occupation-degenerate blocks. The inactive helper relies on an Aufbau-style partition. Verify active electron counts, open-shell partners, and required symmetry-related orbitals yourself. The gauge fixer in Lesson 8 can reject a split occupation-degenerate block; that is a separate safeguard, not a property automatically guaranteed by entropy selection.

For practical bookkeeping, choose a meaningful candidate reference once and downselect from it deliberately. Repeatedly applying a size-based selector to an already reduced candidate list is not a universal way to grow or track a chemical space. Preserve a broader reference if you later need to reconsider excluded orbitals.

AVAS asks a chemically different question

AVAS, automated valence active space, selects orbitals by their overlap with target atomic functions. It can start from an appropriate single-determinant reference without first computing correlated entropies. This makes it useful when chemical reasoning identifies a metal d shell, ligand p manifold, or another local target.

Conceptually, let $\{\chi_\mu^T\}$ span a target atomic-function space with overlap matrix $S_T$. Its projector is

AVAS examines matrices such as $A_{ij}=\langle\phi_i|P_T|\phi_j\rangle$ within occupied and virtual orbital groups. Diagonalizing these projected-overlap matrices creates orbital combinations ranked by target character. Eigenvalues above a threshold identify strongly represented target directions. This is a projection/rotation method, not an energy-ranking rule or an entropy calculation.

The QDK PySCF adapter delegates that method to pyscf.mcscf.avas.AVAS. It exposes ao_labels=[], threshold=0.2 with a zero-to-one bound, canonicalize=False, and openshell_option=2. The target labels must be supplied; an empty list raises an error. The external implementation supplies its atomic-reference details; the QDK schema does not expose every upstream AVAS option or an arbitrary new target-basis keyword.

A source-backed input pattern for nitrogen would be:

avas_selector = create(
    "active_space_selector", "pyscf_avas",
    ao_labels=["N 2s", "N 2p"], threshold=0.2, canonicalize=False,
)
avas_reference = avas_selector.run(reference)

This requires the optional plugin and a compatible restricted single-determinant input; it was not run and no selected count is promised. Labels have an atom and orbital part, such as "N 2p". The adapter preserves explicit indexed labels and wildcards, and can expand an unindexed element label to match multiple indexed atoms in the converted molecule. A misspelled element/target is not silently replaced by a chemically guessed one.

Unrestricted input is rejected. Restricted open-shell references have explicit options: option two includes singly occupied orbitals in the alpha projection procedure, while option three keeps singly occupied orbitals in the active space. The adapter passes this option upstream rather than enforcing a complete alternative spin theory itself. It has tests on restricted water and ROHF oxygen cases. Its open-shell output is constructed with separate alpha/beta blocks, even when the returned coefficient arrays are equal; check downstream restricted-Hamiltonian requirements rather than assume every solver will accept it unchanged.

AVAS can rotate or canonicalize orbitals and reorder them into inactive, active, and excluded groups. It returns a new single-determinant Wavefunction carrier with metadata and invalidated orbital energies, not just a list of the original column indices. Multi-determinant input is not supported by this adapter. For nonstandard or already reduced seeds, verify the returned occupations and partition instead of assuming arbitrary correlated amplitudes were transported through its rotations.

Follow the actual nitrogen tutorial without losing the correlated state

The current tutorial workflow starts from the same 1.85-Å neutral singlet and native cc-pVDZ RHF reference used in earlier lessons. It then performs these distinct scientific stages.

First, the valence-size helper suggests ten electrons in eight orbitals. qdk_valence selects indices 2–9 and a CASCI solver receives active populations five and five. This is $\binom85^2=3136$ determinants, vastly smaller than full fourteen-electron CI in 28 orbitals.

Second, CASCI is run with ci_residual_tolerance=1e-10, calculate_one_rdm=True, and calculate_two_rdm=True. The density defines a natural-orbital rotation. The tutorial does not take the localizer’s returned Aufbau carrier as the correlated answer: it rebuilds the Hamiltonian in those natural orbitals and runs CASCI again. Only then does it read the new representation’s single-orbital entropies.

The essential handoff can be written using the existing restricted nitrogen reference:

from qdk_chemistry.data.symmetry import SymmetryLabel, axes
from qdk_chemistry.utils import compute_valence_space_parameters

ne, no = compute_valence_space_parameters(reference, 0)
valence = create(
    "active_space_selector", "qdk_valence",
    num_active_electrons=ne, num_active_orbitals=no,
).run(reference)
alpha = SymmetryLabel([axes.alpha()])
indices = list(valence.get_orbitals().active_indices().indices(alpha))
na, nb = valence.get_active_num_electrons()
build_h = create("hamiltonian_constructor", "qdk")
cas = create("multi_configuration_calculator", "macis_cas",
             ci_residual_tolerance=1e-10,
             calculate_one_rdm=True, calculate_two_rdm=True)
valence_energy, valence_ci = cas.run(build_h.run(valence.get_orbitals()), na, nb)
natural_carrier = create("orbital_localizer", "qdk_natural_orbitals").run(
    valence_ci, indices, indices,
)
natural_energy, natural_ci = cas.run(build_h.run(natural_carrier.get_orbitals()), na, nb)
selected = create("active_space_selector", "qdk_autocas_eos").run(natural_ci)

These calls are unexecuted. The existing solver objects may be reused with their unchanged locked settings, as explained in Lesson 3. A changed tolerance would require a new configured instance. Neither selector contains the CAS solver; the workflow explicitly owns the preliminary computational cost.

Third, the tutorial inspects the EOS-selected partition and explicitly errors if no active orbitals were selected. Its recorded reference retains indices 4–9, makes 0–3 inactive, and leaves eighteen excluded virtuals. The resulting model is CAS(6e,6o) with active populations three and three.

Fourth, it gauge-fixes the selected complete degenerate blocks while still supplying the broader correlated natural-orbital state to the gauge fixer. The returned orbitals still carry the broad input active space, so the workflow explicitly reapplies the EOS active/inactive index sets to a new Orbitals object. It then constructs the final reduced Hamiltonian and runs a fresh CASCI solve with the reduced counts. This is a careful separation of evidence, representation choice, model reduction, and coefficient optimization—not a chain of objects all representing one unchanged correlated state.

The tutorial regression test records:

These are source reference values, not newly computed outputs. The test’s optional entropy snapshot has two low-entropy valence orbitals and six above 0.5; the six include two moderately high and four very high entropies. EOS selects the largest qualifying group, not only the four largest entries. Exact per-orbital snapshot numbers depend on the reference configuration and should not be generalized to another geometry or basis.

The energy increase from the natural-orbital valence model to the reduced model is approximately 7.318 milliHartree by subtraction of those recorded references. That quantifies an omission effect relative to this larger candidate model. It is not the full error against exact chemistry, and it is not zero merely because an automated selector found a clean entropy cut.

The separate examples/qpe_stretched_n2.ipynb uses 1.270025 Å. None of its energies, active-space diagnostics, or resource numbers should be merged into this 1.85-Å sequence.

Assess the reduction as a scientific decision

An active-space proposal should be tested against the intended observable: a relative energy, bond-stretching profile, spin splitting, density feature, or another stated target. Enlarge the candidate orbital set, increase preliminary selected-CI support, vary thresholds, inspect orbital character, and compare to an appropriate larger reference where affordable. A stable entropy cut can be useful evidence, but it is not a certified chemical-accuracy theorem.

Occupation thresholds miss some weak but collectively important dynamical correlation. Entropy methods only see correlation represented in their supplied state and orbital basis. AVAS can miss chemistry outside the specified atomic target. A contiguous valence window can miss energetically separated but chemically essential partners. These are different failure modes; using several diagnostics can expose disagreement without treating their intersection as automatically correct.

Along a geometry path, keeping the same integer indices is not the same as keeping the same physical subspace. Orbitals can exchange order, change character, or rotate within degenerate blocks. A threshold crossing can abruptly change active electron/orbital count and produce a discontinuous model. For a reaction or bond scan, compare orbital overlaps and chemical character and consider a consistent union of relevant subspaces. The current per-geometry selectors do not supply a universal cross-geometry orbital-tracking guarantee. The derivative workflow’s reuse of index metadata likewise does not solve this problem automatically.

Reducing eight active spatial orbitals to six reduces an untapered occupation encoding from sixteen to twelve electron spin modes. But total quantum cost also depends on the Hamiltonian’s coefficients and terms, preparation overlap, precision, repetitions, and encoding. A smaller model may be harder to prepare accurately or may introduce unacceptable chemical error. Resource estimation should compare complete model/workload choices at a common accuracy target, not just qubit counts.

We now know exactly which electrons remain variable, which are held filled, and what source evidence supports the chosen reduction. The next question is how to repair limitations of fixed-orbital CASCI and how excluded-space correlation can influence a smaller model. That leads to MCSCF, dynamical correlation, and the precise effective-Hamiltonian capabilities that this repository actually offers.