11Refining and reducing the chemical model

Optimizing orbitals, adding correlation, and downfolding are different operations.

~27 min reading18 sections3 knowledge checks

Progress is saved only in this browser. No account or tracking.

Three distinct refinement routes

Concept plate 11
ReferenceMCSCF: adapt orbitalsCorrelation correctionEffective HamiltonianDashed: interface
MCSCF changes the orbitals for a multiconfigurational state; dynamical correlation treats additional effects; an effective-Hamiltonian route changes the reduced operator. The dashed branch denotes an interface rather than a supplied solver.
  • 1MCSCF: orbitals and coefficients
  • 2Correlation: effects beyond a reference
  • 3Downfolding: an effective operator
Original technical lesson · 1 contextual figure addedDownload original MarkdownJump to the interactive model

Three different ways to improve a reduced calculation

The previous lesson produced a clear active-space model for our fixed 1.85-Å nitrogen problem. A CASCI calculation can optimize every determinant coefficient inside that space and still have two important limitations: its orbitals were chosen elsewhere, and correlation involving excluded orbital occupations is absent.

There are several responses, and they are not interchangeable. MCSCF adapts orbitals and determinant coefficients together. Reference-derived correlation methods, such as MP2 and coupled cluster, approximate additional electronic correlation around a reference with their own assumptions. Downfolding aims to incorporate effects of a larger space into an effective smaller-space Hamiltonian. A smaller active space plus a scalar energy correction is not automatically the same thing as such an effective Hamiltonian.

The repository’s current implementation boundary is important enough to state immediately:

Interface and implementation What is actually available here
multi_configuration_scf / pyscf Optional PySCF CASSCF engine with a nested QDK MC solver
dynamical_correlation_calculator / qdk_mp2_calculator Native restricted/unrestricted MP2 energy and amplitude construction within supported input cases
dynamical_correlation_calculator / pyscf_coupled_cluster Optional PySCF CCSD adapter, including tested unrestricted-reference use
effective_hamiltonian_constructor Interface and validation support, but no bundled concrete downfolding implementation or default choice

This is a lesson about those actual methods and contracts, not a promise that every related method in the chemistry literature can be invoked by changing a string. No calculations or tests were executed in preparing it. Any new small numerical example is explicitly illustrative; recorded tests using other geometries remain separate from our 1.85-Å case.

MCSCF changes the orbitals for the correlated state

HF optimizes orbitals for one determinant. CASCI optimizes coefficients in fixed orbitals. Multi-configurational self-consistent field (MCSCF) optimizes both. If the configuration expansion is complete within a specified active space, the usual name is complete active space SCF (CASSCF).

A useful parameterization is

∣Ψ(c,κ)⟩=∑I∈DactivecI∣DI(Ceκ)⟩.|\Psi(c,\kappa)\rangle= \sum_{I\in\mathcal D_{\mathrm{active}}}c_I |D_I(Ce^{\kappa})\rangle.

cIc_I are configuration amplitudes, CC the starting MO coefficients, and κ\kappa orbital-rotation parameters, represented by an anti-Hermitian matrix so orthonormality is preserved. The energy depends on both sets of variables. Optimizing only cc gives CASCI; allowing appropriate rotations of CC gives MCSCF.

The useful rotations are not all independent physical changes. For a complete active space, rotations wholly inside that active space can be compensated by re-expression of its full CI coefficients. Rotations inside an inactive occupied space or an excluded empty space are likewise redundant for the corresponding state. Rotations mixing inactive, active, and excluded spaces can change the correlated wavefunction available to the approximation. In effect, a fixed number of active orbitals can adapt its shape inside the larger AO/MO span.

This is different from the localization of Lesson 8. A localizer chooses a representation objective inside a specified subspace and usually returns a reference carrier. MCSCF optimizes the electronic energy and performs correlated coefficient solves as the physically relevant orbital subspace changes.

For example, a six-orbital active model in 28 MOs can optimize which six orbital directions best describe its correlated electrons while keeping the active-space size fixed. It has not expanded to full CI over all 28 orbitals. At an ideal variational minimum it can improve on the corresponding fixed-orbital CASCI starting point, but a practical nonlinear iteration can reach different local solutions or fail. A lower final energy is not by itself a global-minimum or correct-state certificate.

Macro and micro iterations have different jobs

At fixed orbitals, the CI equation gives optimized amplitudes and reduced density matrices. Those densities describe the correlated response needed to form orbital gradients. A proposed orbital rotation changes integrals, which changes the coefficient problem; orbital and CI changes are coupled.

A macroiteration updates the larger, self-consistent orbital/coefficient problem with refreshed integral and CI information. Microiterations use local response or curvature approximations to make cheaper orbital progress inside that larger cycle. A second-order or augmented-Hessian strategy uses information about orbital energy curvature rather than only the first derivative. These concepts explain why one MCSCF step is much more than a new SCF diagonalization, and why the cost includes repeated integral transformations and correlated subproblems.

Two coupled optimization problems, not two independent switches#

Supplementary · Conceptual diagram

Macroiteration: refresh the coupled problem

  1. Orbitals → integralsA new orbital basis changes matrix elements.
  2. CI solve → densitiesOptimize coefficients for those integrals.
Inner orbital microiterations

Use local gradients and curvature to propose cheaper rotations.

Model and source. The diagram explains the macro/micro distinction in this section. Actual orbital-update machinery is delegated to the supported external solver; this is not a recorded iteration trace.

How to read this. Changing orbitals changes integrals and therefore the CI problem. Cheap local orbital steps still need a refreshed correlated solution.

Return to this explanation

The current QDK adapter delegates the actual orbital-update and microiteration machinery to PySCF’s mcscf.CASSCF(...).kernel(...). It does not reimplement that optimizer in the Python shim or expose a menu choosing arbitrary first-order, two-step, and augmented-Hessian variants. We should not infer a more detailed upstream algorithm configuration than the adapter actually supplies.

The MultiConfigurationScf interface takes Orbitals and two active electron counts, returning (energy, wavefunction). Older prose mentioning a Hamiltonian and several algorithm objects as run arguments is not the current signature. The C++ factory has no registered native implementation; its default implementation name is pyscf, available through the optional plugin. Bundled MACIS contains MCSCF-related library routines, but their existence is not a separately registered native QDK MCSCF solver.

Trace the PySCF–QDK CASSCF bridge

The adapter requires restricted orbitals and identical alpha/beta active and inactive index sets. Unequal active alpha/beta counts are allowed for an appropriate restricted open-shell problem; the tests include both a singlet nitrogen case and an oxygen triplet case with four active alpha and two active beta electrons.

It builds a PySCF mean-field object from the supplied orbitals, occupations, molecular basis, charge, and spin information. It constructs PySCF CASSCF with the number of active orbitals and active electrons. The QDK selection indices are converted to PySCF’s one-based sort_mo request, placing the active orbitals into the layout expected by the external optimizer.

The nested setting multi_configuration_calculator defaults to AlgorithmRef("multi_configuration_calculator", "macis_cas"). The adapter wraps that QDK calculator to look like PySCF’s FCI solver. When PySCF needs a CI solution, it passes active one-electron integrals, two-electron integrals, and a core constant. The wrapper restores full four-index integrals, packages a QDK Hamiltonian over temporary ModelOrbitals, and calls the nested MC calculator. Thus PySCF supplies the orbital optimization and current integrals; MACIS can supply the correlated CI solve.

The wrapper supplies make_rdm1 and make_rdm12 callbacks using the most recently returned QDK wavefunction. It forces calculate_one_rdm=True and calculate_two_rdm=True on the child before use, regardless of false values in an initial child reference. Its ci0 parameter is not used as a new CI warm start by this shim. A source comment mentions a two-RDM scaling, but the current callback only reshapes the already supplied QDK tensor; do not apply an extra factor from that comment.

After PySCF reports convergence, the adapter attaches optimized molecular coefficients, energies, basis/overlap information, and the last CI coefficients/determinants to the output. It carries spin-resolved densities when available and falls back to stored spin-traced densities otherwise. Unlike a localizer’s Aufbau carrier, this output is intended to contain the actual correlated solver result in the optimized orbital representation.

There is a layout limitation worth handling deliberately. The adapter sorts/reorders MOs for PySCF, then copies the input active/inactive index lists into its output metadata rather than returning a general inverse permutation. A conventional contiguous inactive-then-active layout, as used in the tutorial, avoids treating arbitrary noncontiguous input indices as automatically preserved identities. Check returned orbital ordering and model consistency when using custom partitions.

Configure the supported MCSCF workflow, not a hypothetical one

The PySCF MCSCF settings are max_cycle_macro=50, verbose=0, and the nested MC reference. The outer schema does not expose arbitrary conv_tol, max_cycle_micro, root-selection, or state-average weights. Child CI controls such as ci_residual_tolerance are configured inside the reference and have the semantics from Lesson 9; in particular max_solver_iterations controls MACIS’s Davidson subspace.

For the valence reference from Lesson 10, a valid unexecuted pattern is:

from qdk_chemistry.algorithms import create
from qdk_chemistry.data import AlgorithmRef

active_orbitals = valence.get_orbitals()
na, nb = valence.get_active_num_electrons()
mcscf_solver = create(
    "multi_configuration_scf", "pyscf",
    max_cycle_macro=50,
    multi_configuration_calculator=AlgorithmRef(
        "multi_configuration_calculator", "macis_cas",
        ci_residual_tolerance=1e-10,
    ),
)
casscf_energy, optimized_ci = mcscf_solver.run(active_orbitals, na, nb)

This chooses orbital adaptation of the valence candidate. It does not secretly retain the fixed-orbital final energy from Lesson 10. The same design can start from a different supported contiguous active partition, but the comparison must state which one was optimized.

A selected-CI child such as macis_asci can be substituted at the interface, provided its returned state and RDM capabilities satisfy the callbacks. With incomplete selected CI, however, the effective orbital/coefficient model and convergence depend on determinant selection. Smooth, fully variational CASSCF properties should not be transferred uncritically to every selected child, particularly when selected support changes with orbitals.

The adapter explicitly raises if PySCF CASSCF does not converge. It does not promise global optimization or automatic tracking of one electronic root along a reaction coordinate. State averaging, which optimizes orbitals for a weighted set of states, is an important general MCSCF concept, but no corresponding state-average/root-control settings are exposed here. Likewise, an ordinary lowest-state MC child is not an excited-state targeting API.

The standalone documentation example and nitrogen MCSCF test use a 2.0-Å molecule. They are evidence for the method/interface, not a CASSCF energy baseline for our 1.85-Å case. No such new nitrogen number is reported here.

Orbital adaptation does not recover all outside-space correlation

“Static” or strong near-degeneracy correlation describes situations where several configurations compete prominently, as in bond breaking. “Dynamical” correlation often describes many smaller amplitude changes by which electrons avoid one another. The distinction is useful but not a sharp separation into two independent physical forces. Both depend on the reference and orbital space.

CASSCF can improve a chosen correlated model by adapting orbitals, but it still omits determinant changes involving excluded orbitals and fixed inactive occupancies. Conversely, a single-reference perturbative or coupled-cluster correction can capture extensive dynamical correlation while failing badly when its reference is qualitatively inappropriate. Neither label is an automatic accuracy ranking.

Do not add a generic MP2 number to a CASCI energy and call it a validated multireference perturbation method. A consistent post-CAS treatment must define its reference, excitation spaces, denominators, and double-counting rules. The actual reference-derived methods below are not general CASPT2, NEVPT2, or multireference coupled-cluster implementations exposed by this repository.

The dynamical-correlation contract packages a reference and Hamiltonian together

An Ansatz pairs a Hamiltonian with a reference wavefunction. The DynamicalCorrelationCalculator consumes that pair and returns

energy, ket_wavefunction, bra_wavefunction = calculator.run(ansatz)

The energy is a total energy, not only the correlation increment. A ket is the right state representation. The optional bra is a left-state representation used by methods whose practical equations are not Hermitian. It is not necessarily just a copy whose coefficients can be complex-conjugated mechanically. Why coupled cluster needs a left representation becomes clear below.

The native factory defaults to qdk_mp2_calculator; the optional PySCF registration adds pyscf_coupled_cluster. There is not a separate required coupled_cluster_calculator registry type for the current plugin. The shared interface is extensible, but each concrete method still has narrower scientific input assumptions.

MP2 estimates the leading correlation around canonical HF

Møller–Plesset second-order perturbation theory (MP2) separates the Hamiltonian into a solvable mean-field part and a residual interaction. It estimates leading corrections assuming the reference is a sensible starting point and the residual mixing is sufficiently small.

Let i,ji,j label occupied orbitals, a,ba,b virtual orbitals, and ε\varepsilon their canonical orbital energies. For the real restricted spatial-orbital formulation used here,

tijab=(ia∣jb)εi+εj−εa−εb,t_{ij}^{ab}=\frac{(ia|jb)} {\varepsilon_i+\varepsilon_j-\varepsilon_a-\varepsilon_b},
EMP2corr=∑ijabtijab[2(ia∣jb)−(ib∣ja)].E_{\mathrm{MP2}}^{\mathrm{corr}} =\sum_{ijab}t_{ij}^{ab}\left[2(ia|jb)-(ib|ja)\right].

The numerator is an electron-repulsion coupling and the denominator is a reference excitation-energy difference. For ordinary occupied-to-virtual gaps the denominator is negative, giving the familiar negative leading correction for a stable conventional reference. Same-spin unrestricted contributions use antisymmetrized couplings; opposite-spin contributions do not contain the same exchange subtraction.

As a purely illustrative contribution, a coupling of 0.1 Hartree and denominator −0.5 Hartree gives amplitude −0.2 and a simple numerator-squared/denominator contribution −0.02 Hartree. If that denominator becomes −0.01 while the coupling remains fixed, the amplitude magnitude becomes ten: the small-correction assumption is no longer credible. These are invented perturbation numbers, not an N₂ calculation.

MP2 uses orbital-energy information in an essential way. A localized or natural-orbital object with no energies cannot simply be inserted into this formula. Nor is a stale copied energy array evidence that the current orbitals diagonalize the needed mean-field operator. DFT or correlated-reference orbitals may be useful elsewhere, but passing them to a routine that assumes canonical HF does not automatically define a justified generalized perturbation theory.

The native MP2 implementation is an actual solver, but not an RDM producer

The native implementation reads active alpha/beta counts, extracts orbital energies corresponding to active indices, and uses the Hamiltonian’s molecular two-electron tensors to form the correction and amplitudes. Its restricted case requires equal alpha/beta counts. If the Hamiltonian is unrestricted, it evaluates same- and opposite-spin terms separately.

An unequal-spin reference with a restricted Hamiltonian is not a supported RO-MP2 shortcut: the dispatch selects the unrestricted branch, but that branch explicitly requires an unrestricted Hamiltonian. Both spin active spaces must have compatible sizes, populations must fit, and the ordering is assumed to place occupied active orbitals before active virtuals. The public method checks energy availability but does not independently prove all canonical-reference assumptions.

These kernels divide by the excitation denominators directly; they do not apply a general small-gap regularization or guarantee that a pathological denominator produces a clean diagnostic instead of non-finite amplitudes. Strongly stretched nitrogen is exactly a case where perturbative reliability must not be assumed from a returned number.

The calculator has no method-specific convergence/settings schema: MP2 here is direct algebra on supplied integrals, not an iterative CC residual solve. It computes T1/T2 amplitudes eagerly; T1 is zero in its canonical-HF MP2 model. It returns reference_energy + correlation_energy, where reference_energy comes from ansatz.calculate_energy(). Thus the stored core constant is included once in the reference energy, and should not be added again.

Its result is an AmplitudeContainer tagged MollerPlesset, not a determinant expansion. Both ket and bra return slots refer to that same generated amplitude-wavefunction object in this implementation. The generic container nevertheless carries the NotSelfDual metadata convention; a duplicate result handle is not a newly computed normalized bra or an overlap calculation.

Most importantly, this native MP2 calculator does not produce a one- or two-particle RDM. It therefore cannot directly supply the RDM-bearing state required by qdk_natural_orbitals. The deprecated MP2-natural-orbital localizer has its own MACIS density-related computation, but that does not give the general native MP2 amplitude result new capabilities. Suitable MP2 RDM data must be produced/imported through an explicitly supported route. This clarifies the earlier natural-orbital handoff rather than promising an unavailable automatic conversion.

There are producer-specific amplitude details. Restricted MP2 stores a spatial T2 block; it must not be read as three independently antisymmetrized spin tensors merely because getters expose three block positions. The unrestricted same-spin kernel writes independent i<j,a<bi<j,a<b entries into full-sized vectors without filling every antisymmetry-related partner. In the current active-space UMP2 amplitude construction, the beta-beta call also receives the full beta energy vector rather than the extracted active beta vector, so nontrivial active-space amplitude interpretation requires particular care even when the energy path used the extracted energies. The course’s simple MP2 handoff uses a full restricted reference and does not assume interchangeable amplitude layouts across producers.

The MP2 tests cover restricted and unrestricted energy references, active restricted energy cases, and amplitude serialization. They are not a guarantee of reliability for a strongly correlated N₂ reference or every custom active ordering.

Coupled cluster uses an exponential, not a truncated CI list

In coupled cluster (CC), the right-hand state is parameterized as

∣ΨCC⟩=eT∣Φ0⟩,T=T1+T2+⋯ .|\Psi_{\mathrm{CC}}\rangle=e^T|\Phi_0\rangle, \qquad T=T_1+T_2+\cdots.

Φ0\Phi_0 is a reference determinant. T1T_1 generates single excitations and T2T_2 double excitations. In spin-orbital notation,

T1=∑iatiaaa†ai,T2=14∑ijabtijabaa†ab†ajai.T_1=\sum_{ia}t_i^a a_a^\dagger a_i, \qquad T_2=\frac14\sum_{ijab}t_{ij}^{ab} a_a^\dagger a_b^\dagger a_j a_i.

The amplitudes now solve coupled nonlinear equations rather than just the second-order denominator formula. CCSD keeps connected single and double excitation operators, T=T1+T2T=T_1+T_2.

The exponential still generates terms such as T22/2T_2^2/2, corresponding to simultaneous double excitations on different parts of the system. These are disconnected contributions: products of lower-rank cluster effects, not new independent connected quadruple amplitudes. This is why CCSD is not the same as CI restricted to singles and doubles, even though the names mention the same ranks.

For noninteracting fragments with a compatible factorized reference, TAT_A and TBT_B commute, and eTA+TB=eTAeTBe^{T_A+T_B}=e^{T_A}e^{T_B}. This structure underlies standard CC’s size consistency: the energy of separated fragments agrees with the sum of separately calculated energies under the corresponding assumptions. It does not rescue a qualitatively wrong reference at bond dissociation.

Standard projective CC introduces the similarity-transformed Hamiltonian

H‾=e−THeT.\overline H=e^{-T}He^T.

The reference projection gives the energy, and projections onto single/double excited determinants are set to zero:

E=⟨Φ0∣H‾∣Φ0⟩,0=⟨Φia∣H‾∣Φ0⟩,0=⟨Φijab∣H‾∣Φ0⟩.E=\langle\Phi_0|\overline H|\Phi_0\rangle, \qquad 0=\langle\Phi_i^a|\overline H|\Phi_0\rangle, \qquad 0=\langle\Phi_{ij}^{ab}|\overline H|\Phi_0\rangle.

This transformed problem is generally non-Hermitian because an excitation-only TT is not anti-Hermitian. Standard CCSD energy is not a variational upper bound obtained by minimizing a normalized expectation value. Converged amplitudes can still be physically poor when the single-reference picture fails.

The “UCCSD” wording in these PySCF tests means unrestricted-reference CCSD, not the unitary coupled-cluster quantum ansatz eT−T†e^{T-T^\dagger}. That latter construction is a different method. Also, the current adapter calls CCSD only; it does not expose a perturbative triples CCSD(T) calculation merely because the external PySCF project can perform one in another interface.

A left-state solution is additional information

For properties and response in projective CC, the useful left representation is commonly written

⟨Ψ~∣=⟨Φ0∣(1+Λ)e−T,\langle\widetilde\Psi|= \langle\Phi_0|(1+\Lambda)e^{-T},

where Λ\Lambda contains de-excitation amplitudes obtained from additional left/lambda equations. It is not generally the Hermitian conjugate of the stored right-amplitude list. This is why the reference-derived result type has a separate optional bra slot.

The PySCF CC adapter can request that computation with compute_bra=True, then checks converged_lambda. To retain the returned lambda amplitudes in QDK storage, store_amplitudes=True is also needed. Otherwise the external equations may be solved while the returned bra container does not retain their amplitude arrays. A result object’s existence and a stored useful amplitude payload are different facts.

Trace the current CCSD adapter and its input boundary

The implementation name is dynamical_correlation_calculator/pyscf_coupled_cluster. The code constructs pyscf.cc.CCSD, applies exposed controls, runs kernel, and explicitly checks pyscf_cc.converged. It then adds pyscf_cc.e_corr to the QDK Ansatz reference expectation. Optional lambda equations are solved and checked separately.

For ordinary molecular orbitals, the conversion helper reconstructs a PySCF SCF-like object from the molecule, basis, orbital coefficients, energies and occupations. It does not perform a fresh SCF optimization in that conversion. The external CC engine then uses its own molecular integral machinery.

This is an important limitation of the apparently broad Hamiltonian input: when molecular coefficient data are available, hamiltonian_to_scf reroutes to the orbital conversion and does not directly install arbitrary modified one-/two-body tensors from the supplied QDK Hamiltonian. A reduced or downfolded molecular Hamiltonian must therefore not be assumed faithfully consumed by this route. With a reduced active-space reference, PySCF’s full molecular correlation problem and the QDK amplitude container’s active dimensions can also disagree. The supported examples use a consistent full molecular HF reference/Hamiltonian pair.

For coefficient-free model orbitals, the conversion instead constructs a synthetic restricted SCF object containing the supplied one-/two-body integrals, identity overlap, core-energy function, identity MO coefficients, and occupations. That explicit-integral path rejects unrestricted model Hamiltonians, open-shell occupations, and reduced active spaces. It is an interoperability mechanism, not proof that an arbitrary model determinant is a converged canonical HF reference.

The adapter does not implement multireference CC by accepting a correlated Wavefunction. Its occupation-based conversion and reference assumptions remain those of the supported single-reference paths. Passing a CASCI state must not be advertised as a valid post-CAS coupled-cluster correction without a separate method and matching theory.

The PySCF integration tests demonstrate molecular closed-shell CCSD, genuine unrestricted-reference CCSD, and stored amplitude round trips. The synthetic C++ test named for coupled-cluster calculators checks the extensible interface with a mock implementation; it is not a native CCSD solver or a molecular benchmark.

CCSD settings have concrete names and narrower scope than old prose

The actual exposed controls are conv_tol=1e-7, conv_tol_normt=1e-5, max_cycle=50, diis_space=6, diis_start_cycle=0, direct=False, async_io=True, incore_complete=True, store_amplitudes=False, and compute_bra=False.

conv_tol and conv_tol_normt control PySCF’s energy/amplitude-update convergence criteria; the latter name includes the final t. They are not the native SCF orbital residual threshold. max_cycle caps the CC iteration, while the DIIS controls determine history size and when extrapolation begins. direct and the I/O controls change the external engine’s integral/memory strategy, not the CCSD excitation rank. incore_complete=True is an external CC control, not a promise that the entire surrounding workflow performs no file operations.

Older docstrings also mention frozen, max_memory, and verbose, and the method body contains optional lookups for them, but the shipped PyscfCoupledClusterSettings constructor does not register these keys. They are not ordinary settable factory keyword controls in this current schema. Inspect settings rather than promise a frozen-orbital or memory-budget feature from documentation alone.

A conventional dense molecular implementation stores T2 data of size roughly o2v2o^2v^2, with oo occupied and vv virtual orbitals. Leading contractions commonly scale as o2v4o^2v^4, giving the familiar sixth-power cost when both grow with system size. Full integral storage and scratch arrays add substantial memory beyond the amplitudes. These are algorithmic scaling estimates, not timings or measured memory for this environment. Tightening tolerances does not fix reference breakdown; large amplitudes, difficult convergence, and strong near-degeneracy require scientific reassessment.

Amplitude storage is not a quantum-ready state vector

Both MP2 and CC use AmplitudeContainer, with an AmplitudeType specifying whether the amplitudes belong to Møller–Plesset or coupled-cluster theory. The same arrays do not mean the same ansatz: perturbative doubles are not automatically an exponential CC state. The container stores the reference wavefunction, orbitals, and optional T1/T2 blocks, and validates provided sizes/spin-component completeness against the reference active dimensions.

Getters return T1 alpha/beta blocks and T2 in the order opposite-spin, alpha-alpha, beta-beta. The PySCF adapter flattens its arrays for storage. Canonical tensor axis meaning and antisymmetry conventions must be retained alongside these flattened values. A shape check cannot determine whether a supplied spatial or spin-adapted tensor has been interpreted correctly.

The container is pure storage and marks amplitude wavefunctions NotSelfDual. It does not build their determinant expansion, normalize them, compute overlaps, or generate reduced density matrices. norm() and overlap() throw. Availability of a stored reference determinant does not make that determinant the correlated state. The right and lambda results have the same amplitude-type tag, so their role also comes from the return position and calculation provenance.

Consequently, a coefficient-based state-preparation algorithm cannot directly consume a CC result as if get_coefficients() returned normalized determinant amplitudes. Nor can qdk_natural_orbitals, population analysis, or entropy selection simply generate missing correlated RDMs from it. Those next steps require an explicit implemented conversion/property calculation, not a property-table assumption.

Using the separate small H₂ reference/Hamiltonian pair from Lesson 9, the actual invocation pattern is:

from qdk_chemistry.data import Ansatz

reference_ansatz = Ansatz(h2_hamiltonian, h2_reference)
mp2_energy, mp2_ket, mp2_bra = create(
    "dynamical_correlation_calculator", "qdk_mp2_calculator",
).run(reference_ansatz)

ccsd = create(
    "dynamical_correlation_calculator", "pyscf_coupled_cluster",
    store_amplitudes=True, compute_bra=True,
)
ccsd_energy, cc_ket, cc_bra = ccsd.run(reference_ansatz)

These examples are unexecuted and use the full small-model HF reference, not the truncated/gauge-fixed nitrogen CASCI state. The dynamical-correlation documentation example uses a similar H₂ setup with its explicitly specified basis. No native calculate_one_rdm flag or CCSD(T) option should be invented for these calls.

What downfolding would add beyond discarding orbitals

To understand the effective-Hamiltonian interface, partition a larger orbital window WW into retained target orbitals PP and external orbitals QQ. In a formal many-electron derivation, use P^\hat P and Q^\hat Q for projectors onto the corresponding retained/excluded many-electron spaces. The orbital-index lists in the API are not themselves these many-electron projection matrices.

Write the Schrödinger equation in blocks:

HPPΨP+HPQΨQ=EΨP,HQPΨP+HQQΨQ=EΨQ.H_{PP}\Psi_P+H_{PQ}\Psi_Q=E\Psi_P, \qquad H_{QP}\Psi_P+H_{QQ}\Psi_Q=E\Psi_Q.

When the relevant inverse exists, eliminate the excluded component:

ΨQ=(E−HQQ)−1HQPΨP,\Psi_Q=(E-H_{QQ})^{-1}H_{QP}\Psi_P,
Heff(E)=HPP+HPQ(E−HQQ)−1HQP.H_{\mathrm{eff}}(E)=H_{PP} +H_{PQ}(E-H_{QQ})^{-1}H_{QP}.

The second term remembers virtual excursions into the excluded space. Simply keeping HPPH_{PP} discards them. Exact elimination is not free: the result depends on the unknown energy and the inverse of a potentially enormous excluded-space operator. Practical downfolding methods introduce approximations, references, truncation orders, or transformations to make this tractable.

For an illustrative two-state matrix with retained diagonal −1.0 Hartree, excluded diagonal −0.5 Hartree, and coupling 0.1 Hartree, elimination gives the scalar equation

E=−1.0+0.01E+0.5.E=-1.0+\frac{0.01}{E+0.5}.

Ignoring the external state gives −1.0; the exact lower eigenvalue is approximately −1.019258 Hartree. Replacing EE in the denominator by −1.0 gives an approximate −0.02-Hartree correction. This is a toy explanation of exclusion-space influence, not an implemented QDK downfolding calculation or a nitrogen correction.

Exact elimination can induce effective three-body and higher-body interactions even when the starting Hamiltonian contains only one- and two-body terms. A method returning the usual molecular constant/one-/two-body representation must explain how such terms are represented or neglected. A smaller orbital count can therefore shift work and error into classical preprocessing rather than simply remove cost.

The current effective-Hamiltonian capability is an interface, not a solver

At this revision, EffectiveHamiltonianConstructor defines

effective_hamiltonian = concrete_constructor.run(reference, hamiltonian, p_indices)

This is an interface signature, not an executable example selecting a shipped implementation. The factory implementation registers no concrete instances and its default name is empty. The documentation explicitly states the same limitation. No built-in perturbative downfolding order, CC-based decoupling algorithm, or numerical downfolding result can honestly be taught as currently runnable here.

An external/custom implementation could register into this interface, but its presence and scientific assumptions would require separate evidence. The current tests use a mock that returns an input Hamiltonian to exercise registration and validation. They are not a physical downfolding demonstration.

The interface still defines a useful scientific handoff. Inputs are a reference Wavefunction, a Hamiltonian over the larger window, and a SymmetryBlockedIndexSet of absolute MO indices for the target. If a Hamiltonian’s active indices are [0, 2, 4, 6], target [0, 4] identifies those actual MOs; [1, 3] does not mean “second and fourth active positions.”

The output contract requires the target indices as its active space, preserved original inactive orbitals, method-consistent inactive/empty classification of removed orbitals, a consistent inactive Fock matrix when present, and the folded scalar contribution in the constant energy term. Other excluded-space contributions belong in the returned effective integrals. The reference and selected approximation must decide how to handle occupations and information in the removed space; the interface alone cannot decide those physics.

Common input validation is opt-in: base run locks settings but does not automatically call _validate_inputs. A concrete implementation may invoke that helper, which checks null inputs, compatible AO/MO dimensions and basis hashes, spin restrictions/symmetries, coefficient and energy consistency, and target/reference active windows as subsets of the Hamiltonian window. The helper permits a target not contained in the reference’s own active subset as long as both are within the larger Hamiltonian window; method-specific information requirements remain the implementer’s responsibility. It is not a generic assurance that two rotated but differently represented states can be combined without transforming their tensors.

The base has no common downfolding settings. Therefore there is no legitimate current order, energy_shift, or method="..." keyword to guess. The correct answer to “which downfolding method did this repository use?” is “none through a bundled concrete implementation at this revision,” unless a separately identified implementation has been supplied.

Keep ECPs, frozen core, orbital optimization, and downfolding distinct

An ECP changes the underlying electronic model by replacing selected atomic core electrons with an effective operator. Frozen-core active-space construction retains a reference inactive population and its mean-field effect/constant while omitting its excitations. CASSCF adapts orbital shapes and coefficients within its chosen many-electron model. Downfolding aims to retain effects of a larger excluded space through a modified smaller-space Hamiltonian.

An MP2 or CCSD scalar correlation energy is not, by itself, a new effective Hamiltonian with corrected eigenstates and observables. Adding a full-space correction to a reduced correlated result without a derivation can double count some correlation and miss other terms. Likewise, if an effective Hamiltonian had already folded in QQ, subsequently re-correlating that same QQ contribution would need explicit double-counting control.

A fair comparison would keep geometry, basis, particle/spin target, energy reference, and accuracy goal aligned, then compare a bare active model, an orbital-optimized model, and a specifically defined effective model against suitable larger-space evidence. It would include the cost of obtaining the reference and effective integrals, not just the smaller quantum register. Our recorded 1.85-Å CASCI reference cannot be combined with the 2.0-Å MCSCF example or the 1.270025-Å resource notebook as if they were successive stages of one calculation.

The handoff to integral representation

At this point the model choices are explicit. We can obtain a correlated fixed-orbital state, optimize a supported CASSCF problem through PySCF with native CI, or run supported single-reference MP2/CCSD calculations while respecting their amplitude and conversion boundaries. We also know where the repository provides an extension contract rather than a completed scientific method.

The next lesson changes a different layer: how the Hamiltonian’s one- and two-electron information is stored, transformed, and factorized. Factorization can reduce storage or circuit-construction cost for a chosen operator, but it does not by itself decide which electrons were modeled, recover missing outside-space correlation, or convert an amplitude-only result into a normalized quantum state. Keeping these layers separate is what makes later resource estimates scientifically interpretable.

Make the idea your own

Supplementary learning

The original lesson above is preserved in full, with labeled supplemental figures placed beside the relevant explanations. The model and questions below are also learning aids, not newly executed chemistry results.

Mix two configurations

Interactive model

The toy mixes only two configurations. It cannot reproduce MP2, CCSD, MCSCF, or a downfolding method.

Illustrative model, not a scientific calculation. An exactly diagonalized real symmetric 2 by 2 matrix with diagonal entries 0 and the chosen gap, and equal off-diagonal coupling. Energies use arbitrary units. This is not HF, a molecular CI calculation, or an orbital optimizer.

Enable JavaScript to explore this model. The assumptions and original lesson remain readable without it.

Check your understanding

Supplementary learning

Reason through these 3 questions. Your answers do not mark the lesson as read, and nothing is locked.

1. What does MCSCF optimize beyond fixed-orbital CI?
2. Why are CC amplitudes not a ready-made pure-state vector?
3. What should an effective-Hamiltonian interface without a provided solver be called?

Enable JavaScript to check and save answers locally.

Source notes & references (24)

Read-only source/test tracing at97a3ff4f; no imports, solver execution or tests run. MCSCF is optional PySCF CASSCF with nested QDK CI, not a shipped native orbital optimizer; outer settings limited and arbitrary input MO-index permutation is not restored generically. MP2 directly computes amplitudes without RDMs or small-denominator regularization; restricted open-shell dispatch is rejected by unrestricted-Hamiltonian requirement, and active UMP2 beta-beta amplitude energy indexing needs care. AmplitudeContainer is pure storage/NotSelfDual; native MP2 returns same object in ket/bra slots, CC optional lambda result requires store_amplitudes for retained data. PySCF molecular CC conversion rebuilds from orbitals rather than honoring arbitrary reduced/effective tensors; full molecular/reference examples only. Only CCSD is invoked, not triples; frozen/max_memory/verbose not registered despite optional code lookups. Effective-Hamiltonian factory has no concrete implementation; formal P/Q derivation is illustrative, validation opt-in. One directly related Lesson8 sentence corrected to remove implication that native MP2 produces the RDM required by natural-orbital localizer.

Public source: microsoft/qdk-chemistry, pinned to 97a3ff4fb86e. This course describes that revision, not every later release.

Look up a term

On this page

  1. Three different ways to improve a reduced calculation
  2. MCSCF changes the orbitals for the correlated state
  3. Macro and micro iterations have different jobs
  4. Trace the PySCF–QDK CASSCF bridge
  5. Configure the supported MCSCF workflow, not a hypothetical one
  6. Orbital adaptation does not recover all outside-space correlation
  7. The dynamical-correlation contract packages a reference and Hamiltonian together
  8. MP2 estimates the leading correlation around canonical HF
  9. The native MP2 implementation is an actual solver, but not an RDM producer
  10. Coupled cluster uses an exponential, not a truncated CI list
  11. A left-state solution is additional information
  12. Trace the current CCSD adapter and its input boundary
  13. CCSD settings have concrete names and narrower scope than old prose
  14. Amplitude storage is not a quantum-ready state vector
  15. What downfolding would add beyond discarding orbitals
  16. The current effective-Hamiltonian capability is an interface, not a solver
  17. Keep ECPs, frozen core, orbital optimization, and downfolding distinct
  18. The handoff to integral representation
  19. Interactive model
  20. Knowledge check
  21. Source notes

The complete journey

Course homeGlossarySavedAbout

Search the course

Search titles and the full lesson text.

    Tab to a result, Enter to open. Escape closes search.