22Overlap tests and success amplification

An energy number does not describe the whole state

Two trial states can have similar energy expectations but different overlaps with the desired eigenstate. Two time-evolution circuits can agree on occupation probabilities yet differ in relative phase. After learning expectation estimation and QPE, we need tools that reveal coherent overlap information and, under specific assumptions, increase the probability of a desired outcome.

The continuing chemistry problem is neutral singlet N₂ at 1.85 Å in the selected active model. These methods do not change that model or supply a missing exact ground-state oracle. Their benefit depends on the trial state, the definition of success, the cost of testing success, and the validity of reversible circuit composition.

This lesson first derives the Hadamard test, then amplitude amplification, and finally the repository’s QPE-based subspace-marking oracle. The last has an important exact-phase-grid restriction: its current temporary-ancilla design is not a general robust off-grid spectral filter. All examples and calculations here are source-backed descriptions or hand-worked toys, not executions.

The complex quantity measured by a Hadamard test

Let a compatible state-preparation operation produce $|\psi\rangle$, and let $U$ be a unitary. Define

This number can be complex. Its magnitude measures how much the evolved state overlaps the original; its phase carries information not visible in a simple probability comparison. For time evolution, $z(t)=\sum_k|c_k|^2e^{-iE_kt}$ when $|\psi\rangle=\sum_k c_k|E_k\rangle$. Thus it depends on the energy distribution, not only on its mean or its minimum.

Prepare a control in $|+\rangle$ and apply controlled U. The joint state becomes

The control’s X expectation is $\operatorname{Re}z$, and its Y expectation is $\operatorname{Im}z$. Accordingly,

where zero means the +1 eigenvalue of the selected control observable. Two separate measurement ensembles are needed to estimate both components. One shot cannot reveal the complete complex overlap.

The sign of the imaginary part depends on the precise controlled-U and measurement conventions. The repository’s HadamardTest.qs uses H, then the supplied controlled operation, then MResetX or MResetY. Its tests compare the Y result with the positive imaginary part of $\langle\psi|U|\psi\rangle$, not its negative.

A local decomposition of Y measurement would apply $S^\dagger$ followed by H before Z measurement. A different S/S-dagger convention changes the sign. For an illustrative eigenstate with $U|\psi\rangle=e^{-i\pi/4}|\psi\rangle$, the X expectation is $1/\sqrt2$ and Y expectation is $-1/\sqrt2$. This is analytic phase arithmetic, not a measured molecular result.

The Q# routine resets the system before measuring the control. An unconditional local reset of the system preserves the reduced state of the control, so this does not change the overlap statistic. It does mean the system is not returned for reuse. Each shot starts with state preparation again.

Counts, errors, and state-to-state overlaps

For either basis, the estimated component is $(n_0-n_1)/(n_0+n_1)$. Under independent repeated shots, a component with true mean m has sample-mean variance $(1-m^2)/N$, at most $1/N$. A plug-in estimate and finite-sample confidence qualifications follow the discussion in Lesson 20. Zero observed variance does not prove perfect accuracy, and coherent circuit approximation or noise can bias the mean independently of shot count.

The two parts may be combined into $\widehat z=\widehat x+i\widehat y$. Its magnitude squared is not automatically an unbiased fidelity estimate, and $|z|^2$ here is the overlap between $|\psi\rangle$ and $U|\psi\rangle$, not necessarily the ground-state fidelity. Always state which states are being compared.

For two pure states prepared by unitary extensions $A_\psi$ and $A_\phi$, their overlap can in principle be written

That suggests a Hadamard test of a composed unitary, but it requires a realizable adjoint, consistent ancillas, and correct relative phases. A nonunitary density object, CC amplitude container, or arbitrary pair of wavefunction files cannot simply be passed to this interface as an overlap oracle. The shipped high-level Hadamard method accepts a UnitaryRepresentation, not two Wavefunctions or an arbitrary second Circuit in place of U. A supported representation/mapper or an explicitly engineered compatible unitary is still needed.

A global phase of a standalone U becomes a relative control-branch phase and changes z. The up-to-global-phase circuit rewrites discussed in Lesson 17 are therefore not automatically safe before adding control. An overall global phase of the complete controlled experiment is different and does not affect its probabilities.

The actual Hadamard-test workflow

The registered algorithm is hadamard_test/qdk; the separately registered builder is hadamard_test_circuit_builder/qdk. The orchestrator uses that builder by fixed name rather than exposing a general nested builder-selection setting.

Its settings are test_basis="X", controlled_circuit_mapper=AlgorithmRef("controlled_circuit_mapper","pauli_sequence"), and circuit_executor=AlgorithmRef("circuit_executor","qdk_full_state_simulator"). X and Y are the supported basis choices; Z is rejected. The run signature is run(state_preparation_circuit, unitary, shots), requiring a Circuit, a UnitaryRepresentation, and a positive integer shot count.

The builder base sets control index zero on the nested controlled mapper. The native builder requires Q# callables for preparation and controlled evolution and creates a Q# factory on [control | target register]. It does not automatically convert a QASM-only preparation into a composable Q# operation.

Its target width is unitary.get_num_qubits(). For a Pauli product formula this is the system width. For a block encoding it includes structural block ancillas, and the state preparation is handed that enlarged register. A chemical state loader expecting exactly the system-only width is not universally compatible. A loader adding workspace beyond the unitary container’s structural width is not automatically accommodated either. There is no explicit shared-gradient ownership logic or complete encoding/orbital compatibility proof in this builder.

The default Pauli mapper therefore has a clear supported pattern: first build a time-evolution UnitaryRepresentation, then test its overlap on a compatible state. Other configured mappers retain their own limits, including CSWAP vacuum preservation and PSP layout/workspace assumptions. Merely changing the AlgorithmRef does not prove an arbitrary block/SOSSA unitary is supported.

The high-level run returns CircuitExecutorData directly: count dictionaries, shot information, and executor data, not a precomputed complex number or an EnergyExpectationResult. It has no noise argument in its own run signature and does not forward a noise profile. To study an explicitly noisy test through available executor support, construct the circuit with the builder and invoke a compatible executor separately; that is a different workflow, not an undocumented high-level parameter.

A source-backed, unexecuted usage pattern is:

from qdk_chemistry.algorithms import create
from qdk_chemistry.data import AlgorithmRef

# trial_circuit and unitary_plan already satisfy the same register/encoding contract.
real_test = create(
    "hadamard_test", "qdk", test_basis="X",
    circuit_executor=AlgorithmRef("circuit_executor", "qdk_full_state_simulator", seed=42),
)
# In an authorized execution: counts = real_test.run(trial_circuit, unitary_plan, shots=1000).bitstring_counts
# Repeat with a separately configured test_basis="Y" to estimate the imaginary part.

The repository example builds the target evolution before calling this method. Its hydrogen example and the water test fixture are not the stretched-nitrogen baseline. Their references compare with the same constructed finite product-formula unitary, not a claim of exact arbitrary molecular propagation.

Amplification needs a precisely defined good subspace

An amplitude-amplification problem starts with a preparation $A|0\rangle=|\psi\rangle$ and a good subspace, represented by a projector $\Pi_G$. The success probability is

The good subspace may mean a flagged computational pattern or a specified spectral region. It is not necessarily one ground eigenvector. A predicate must distinguish good from bad coherently, without revealing which branch occurred.

Write the normalized state as $|\psi\rangle=\sin\theta|G\rangle+\cos\theta|B\rangle$, with $p=\sin^2\theta$. The normalized good and bad components span a two-dimensional plane. Define reflections

Applying the good reflection and then the prepared-state reflection rotates the state in this plane. With these conventions, after k rounds the good probability is

Different texts attach an overall minus sign to a differently defined reflection; the physical probability formula does not justify dropping relative phases inside the implementation. The repository’s actual order follows the two reflections defined above.

For a hand-worked example, $p=1/4$ means $\theta=\pi/6$. One round gives success one; two rounds return it to one-quarter. This is overshooting: more rounds are not always better. If p is unknown, choosing rounds requires an estimate or an additional robust strategy. This implementation supplies fixed integer rounds, not automatic amplitude estimation, fixed-point amplification, or a general unknown-success scheduler.

For small known p, the first maximum occurs at a round count of order $1/\sqrt p$. Rejection sampling would need order $1/p$ independent preparations to obtain one success under its assumptions. The quadratic statement counts ideal oracle uses; an expensive coherent predicate, adjoint, or state preparation can erase its practical advantage.

How the generic implementation realizes the reflections

The algorithm amplitude_amplification/qdk_base takes state_prep_oracle: Circuit and good_state_oracle: Circuit. The only numerical setting is rounds=1; zero is allowed and negative rounds raise. It returns a Circuit, not an executed probability estimate.

The preparation callable must have the action $A$ on its declared argument register and a valid adjoint. The good-state callable has the form (register, flag), flipping the flag if the register is good. A simple example in the tests uses a multi-controlled X to mark the all-ones basis state. That custom predicate is explicit Q# logic, not a newly registered generic “find my target state” method.

AmplitudeAmplification.qs reflects about the prepared state by applying $A^\dagger$, reflecting about all-zero, and applying A. It reflects about good states by computing the flag, applying Z to the flag, and uncomputing the flag. The initial A is executed once, and each round adds one $A^\dagger$ and one A. Thus the literal preparation/adjoint count is $2k+1$, with two flag-oracle uses per round before any compiler simplification.

The Python wrapper checks that both Circuits contain Q# callables, but does not mathematically prove their unitarity, clean-workspace action, adjoint correctness, or encoding agreement. The Q# operation types require Adj. A measured circuit, unsupported callable capability, or dirty released ancilla cannot be made reversible merely by naming it an oracle.

The amplified Circuit’s factory measures the whole amplified register. Its qsharp_op exposes the same preparation/amplification without measurement for further composition. The returned Circuit currently omits explicit width, encoding, and shared-gradient metadata; preserve those contracts externally rather than treating the output as self-describing for every downstream consumer.

The input preparation’s declared width is used when available. Otherwise the wrapper attempts a resource estimate and reads its logical qubit count. That fallback can conflate peak scratch-inclusive width with argument width if a custom preparation does not declare its interface precisely. It can also fail if the input Circuit lacks an estimatable representation. A declaration from the actual producer is preferable to relying on that fallback.

A garbage-entangled preparation is not universally invalid for amplification: if all garbage is retained inside the amplified register and A has the correct inverse, the reflected state is the complete joint state. But the desired good projector must refer to that joint contract. An alias-sampling index marginal is not a clean chemical trial state, and a phase-gradient catalyst sitting outside the reflected all-zero convention needs special handling. This generic wrapper does not implement the shared-gradient exclusions that the unary QPE builder does.

The shipped QPE-subspace oracle marks energies above a bound

The separate registered family is amplitude_amplification_oracle, whose implementation/default is qdk_qpe_subspace. Its source has two settings: required finite energy_lower_bound (default NaN) and qpe_circuit_builder=AlgorithmRef("qpe_circuit_builder","qdk_standard"). Phase bits, unitary settings, and controlled mapper belong inside that nested builder, not at the oracle’s top level.

Despite the course’s ground-state motivation, this predicate marks decoded energies greater than or equal to the lower bound. There is no accompanying upper-bound setting, no default “lowest-energy” predicate, and no automatic ground-state identification. For an ordinary Pauli Hamiltonian, deliberately negating H can turn a lower-energy target of H into an upper-energy target of −H, with consistent thresholds and phase decoding. That is a model/operator transformation you must perform and track; it is not an automatic option and cannot be transferred blindly to structured SOS data.

The oracle’s run takes the Hamiltonian alone. It builds standard coherent QPE with a no-op state preparation, because the supplied register already contains the state under test. It uses the builder’s measurement-free Q# callable, not its measured standalone factory. Iterative builders are rejected: a classical sequence of majority votes and reprepared systems is not a coherent circuit that can be uncomputed. A Qiskit standard builder can pass the Python class check but does not supply the required native Q# callable in this path. Use the actual supported native composition rather than infer support from inheritance.

Supported useful combinations include a Pauli product formula with controlled Pauli mapping and an LCU walk with controlled PSP, subject to their width/phase contracts. A bare LCU block lacks the needed energy-phase law. SOSSA is not made supported by possessing a decoder: no standard controlled mapper accepts its Adj-only block. Unary QPE is likewise not a substitute for the required standard builder.

Enumerating phase bins and respecting aliases

For m phase qubits the implementation examines all $2^m$ bins. It calls the selected container’s eigenvalue_from_phase for each bin, keeps bins whose decoded energy reaches the lower bound, and coalesces consecutive accepted indices into sorted, disjoint half-open intervals [lower, upper). For quantum-walk containers it canonicalizes mirrored bins so that an inclusive energy threshold treats both walk branches consistently.

This is exhaustive classical bin enumeration, not a constant-cost analytic threshold calculation. Its work grows exponentially with the phase-register width even if the final accepted range list is short. It does not use the actual eigenvalues of the Hamiltonian to know which accepted bins have support.

No accepted bins or every bin accepted are rejected because neither identifies a proper phase-register subspace to amplify. The bounds follow the container’s actual decoder, including time-evolution wrapping or the cosine walk law. A time-evolution energy alias chosen by the principal branch is not corrected by this enumeration. The energy bound must use the same energy convention as the decoder: ordinary molecular Pauli/LCU input omits the core scalar, whereas a decoder that includes a shift would require its total-energy convention. Do not use a total molecular bound against active-only decoded energies.

The Q# MarkPhaseRange routine marks integer bins through equality or lower/upper comparisons. Interior intervals use temporary comparison flags that are uncomputed. MarkAcceptedPhase combines interval predicates; disjointness matters because a bin flipped twice would become unmarked. The generated ranges have that property; the low-level operation is not a general automatic repair of overlapping user intervals.

The tests verify exact inclusive energy comparisons, wraparound ranges, mirrored walk bins, missing/invalid bounds, and exact-grid interior eigenstates. For example, the shipped upper-band demonstration marks bins [0,3) and [14,16) of a 16-bin walk register. That is a specific reference fixture, not a computed nitrogen filter.

Finite-resolution tagging is not an exact spectral oracle

The intended coherent marking pattern is: allocate phase and signal ancillas, run QPE, flip a flag for accepted bins, then run QPE-adjoint. If every relevant eigenphase lies exactly on a resolved bin and both walk branches are classified consistently, the phase/workspace can return clean while the flag records good/bad membership.

Off-grid phases produce a distribution $a_y(E)$ over bins. A measured phase filter would accept an eigencomponent with probability $\sum_{y\in T}|a_y(E)|^2$, where T is the accepted set. That is a soft finite-resolution spectral response, not a perfect projector onto a sharply bounded energy eigenspace. Nearby energies, degeneracies, approximation errors, and wrap boundaries influence the response.

There is a stronger current implementation limit. Copying the accepted/not-accepted bit to a flag can entangle it with the finite-resolution phase distribution. QPE-adjoint then does not generally return the temporary phase register to zero. The shipped MarkQPEPhase allocates and releases those ancillas internally, and its own Python documentation warns that off-bin phases leave residual amplitude there. This is not a robust arbitrary off-grid oracle with an automatically quantified approximation; it can violate the clean-ancilla promise required for the construction.

Increasing m can reduce leakage but does not by itself prove exact clean release. A different robust filtering design might retain phase/workspace as part of the amplified Hilbert space or explicitly account for approximation and cleanup; that is not what this wrapper presently implements. Consequently the ideal two-dimensional amplification formula must not be promised for arbitrary nitrogen phases using this oracle merely because the circuit object can be constructed.

The successful supplied amplification fixtures deliberately use exact phase-grid energies, including nontrivial interior walk eigenvalues with both branches. They establish meaningful implemented behavior under that promise, not a general theorem for all molecular spectra. No such tests or circuits are executed here.

A source-backed exact-grid demonstration

The following is an unexecuted version of the repository demonstration. It is a two-qubit toy, deliberately targeting the upper state rather than claiming a nitrogen ground-state filter:

import math
import numpy as np
from qdk_chemistry.algorithms import create
from qdk_chemistry.data import AlgorithmRef, Configuration, ModelOrbitals, QubitOperator, StateVectorContainer, Wavefunction

h = QubitOperator(["ZI", "IZ"], np.array([-math.pi / 4, -math.pi / 4]))
guide = Wavefunction(StateVectorContainer(
    np.array([math.sqrt(0.91), 0.3]),
    [Configuration.from_bitstring("00"), Configuration.from_bitstring("11")],
    ModelOrbitals(2),
))
prepare = create("state_prep", "dense_pure_state").run(guide)
mark = create(
    "amplitude_amplification_oracle", "qdk_qpe_subspace",
    energy_lower_bound=h.schatten_norm / 2,
    qpe_circuit_builder=AlgorithmRef(
        "qpe_circuit_builder", "qdk_standard", num_bits=4,
        unitary_builder=AlgorithmRef("hamiltonian_unitary_builder", "lcu", quantum_walk=True),
        controlled_circuit_mapper=AlgorithmRef("controlled_circuit_mapper", "prepare_select_prepare"),
    ),
).run(h)
amplified = create("amplitude_amplification", "qdk_base", rounds=2).run(prepare, mark)

The Hamiltonian has $|11\rangle$ at $+\pi/2$, $|00\rangle$ at $-\pi/2$, and the other basis energies at zero. Its LCU walk phases are exact grid points for this register. The guiding probability on the marked state is 0.09. The ideal two-round prediction is $\sin^2(5\arcsin0.3)$; the example’s separately executed comparison is designed to test that relation. We have only written the instructional pattern into this lesson, not created callables, compiled it, or observed a success rate.

The returned amplified object is a circuit description. Executing it and estimating success requires a separate executor and a precise rule for interpreting output bitstrings. A high observed probability on this toy upper state is not a result about chemical ground-state preparation.

Count the expensive oracle, not only Grover rounds

In the literal generic circuit, k rounds require $2k+1$ preparation/adjoint uses and two good-flag oracle uses per round. A QPE-subspace flag oracle itself contains forward and inverse coherent QPE, so before compiler cancellation that can mean four coherent-QPE-or-inverse operations per round. Each QPE contains controlled powers, phase-register transforms, and possibly LCU PREPARE/SELECT/reflections and extra signal ancillas. The register and rotation precision are part of the cost.

An oracle promising a narrow energy window can therefore be much more expensive than the state whose success it amplifies. Unknown success probability introduces additional work. Noise, approximate reflections, and invalid cleanup can spoil the ideal rotation picture. The quadratic query advantage is conditional on efficient coherent oracles and controlled error, not a universal reduction in total runtime.

Postselection instead measures and discards unsuccessful runs. It destroys the coherent branch information but can be simpler; its repetitions cost approximately one over the acceptance probability for independent identical trials. Coherent amplification changes probabilities before final measurement and requires reusable adjoints and reflections. Neither option avoids paying for what defines success.

For the nitrogen workflow, Hadamard tests can characterize a supported evolution’s coherent overlap, while amplification offers a possible preparation improvement only after the target predicate and implementation promises are established. These tools complement, rather than replace, model validation, state alignment, and correct QPE statistics. We next move from fixed spectral questions to systems whose Hamiltonian explicitly changes with time.