33Engineering and maintaining the whole package

An algorithm is useful only when the surrounding software works

We have followed the scientific calculation from a molecular question to quantum circuits and conditional resource estimates. We have also learned how artifacts, remote jobs, and user interfaces connect the stages. There is one large workstream left: making the package build, install, behave predictably, and remain understandable as the code evolves.

This work is sometimes treated as peripheral to the science. In reality, a wrong tensor convention at a language boundary can invalidate an energy, an unavailable binary dependency can prevent a method from being used, and an example written for the wrong release can make a correct implementation appear broken. Numerical research and software delivery are coupled.

This lesson is a tour of the engineering contracts in the repository, not a request to compile or install it. The build configurations and test sources described here were inspected read-only. No chemistry package builds, installations, or scientific tests were executed for the course.

1. Read the repository as a system of responsibilities

The top-level directories divide the work into recognizable responsibilities:

This map is a guide to responsibility, not an assertion that each folder can be released independently or that a class appearing in a public header has a concrete algorithm behind it. Earlier lessons distinguished abstract interfaces from implemented capabilities, notably for effective-Hamiltonian downfolding.

The public C++ headers form the supported interface. Internal implementation headers beneath the source tree are a different boundary. The C++ CMake configuration explicitly makes those internal headers available to the library itself without installing them as the public API.

2. Why C++, Python, and Q# all appear

C++ is used for numerical data structures and demanding classical kernels. Python makes it convenient to compose workflows, integrate community packages, analyze results, and implement orchestration. Q# expresses quantum operations and interacts with the broader QDK compiler, simulator, and resource tooling.

These are engineering roles, not a division into classical physics and quantum physics. The C++ and Python chemistry calculations solve quantum-mechanical models on ordinary computers. Q# circuit descriptions are also often compiled or simulated on ordinary computers during development.

The Python build creates a compiled _core extension using pybind11. It includes bindings for structures, bases, orbitals, configurations, wavefunctions, Hamiltonians, symmetry data, algorithm interfaces, derivatives, constants, and numerical utilities.

pybind11 is the bridge that makes C++ classes and functions callable from Python. It manages conversions and object lifetimes; it is not itself an electronic-structure method. Errors can occur at this boundary if shapes, numerical types, ownership, or overload selection are misunderstood.

A type stub describes an interface to Python tools without performing the operation. The project supplies/generated stubs for compiled classes and registry-driven creation so an editor can suggest valid operations and settings. Stubs improve discoverability and catch some mistakes early, but they cannot prove that a given active space is scientifically appropriate or that a numerical algorithm will converge.

3. The numerical dependency stack has distinct scientific jobs

The package does not reimplement every mathematical primitive. It integrates specialized libraries:

BLAS, Basic Linear Algebra Subprograms, supplies building-block vector and matrix operations. LAPACK, Linear Algebra Package, supplies routines such as solving systems and eigenproblems. A faster implementation can change performance without changing the mathematical task, although floating-point ordering and backend details can affect numerical reproducibility.

The build has to make these components agree on conventions such as integer widths, calling conventions, precision, and linked runtime libraries. A calling convention is the low-level agreement for how a compiled function receives arguments and returns results. This is why a dependency mismatch can be more serious than a missing Python import.

For native exchange-correlation evaluation, our traced source uses ExchCXX built-in functional kernels through GauXC. Broad documentation credits Libxc in the ecosystem, but that does not mean every native DFT call invokes Libxc. Likewise, the presence of GPU-capable GauXC code does not mean an arbitrary installed wheel is GPU-enabled.

The MACIS source package includes its own broader numerical infrastructure, including sparse and iterative-solver support. Its standalone capabilities and defaults should not be confused with the narrower way QDK Chemistry configures and calls it. In the inspected C++ build, the bundled MACIS configuration disables its own Python interface, standalone examples, and MPI integration when that dependency is built through this path. Its source can contain capabilities that are not exposed by the installed QDK interface.

4. A build turns source and dependencies into compatible artifacts

A compiler translates source code into machine code. A linker combines compiled objects with the libraries they call. A build system decides what to compile, in which order, with which options, and when a changed input requires rebuilding.

QDK Chemistry uses CMake to describe the native build. CMake can generate work for a build executor such as Ninja. The C++ project requires C++20 and creates the chemistry library and its exported qdk::chemistry target.

Python packaging uses scikit-build-core, which connects Python package building to CMake. The package manifest declares the build dependencies, runtime dependencies, optional extras, command entry points, and packaging configuration.

The Python CMake flow first tries to find a compatible installed native chemistry library. If it does not find one, it can build the native source as a subproject. If it does use a preinstalled library, it still must include the chemistry resource files in the Python distribution. The build explicitly checks for those resources instead of shipping a binary that later cannot find its basis data.

This illustrates the difference between a compiled program and a usable package. Basis archives, configuration data, Python modules, quantum source files, and compiled components all have to arrive in compatible locations.

FetchContent can obtain missing build dependencies. Some dependencies are identified through pinned manifests or versions; not every configured fetch is an immutable commit. For example, the inspected linear-algebra CMake-module fetch follows its main branch. A reproducible engineering record should therefore include the actual resolved dependency revisions, not assume the top-level source revision alone fixes every external input.

The vcpkg manifest declares native packages such as Eigen, HDF5 C++ support, Boost components, test libraries, JSON, logging/formatting libraries, and OpenBLAS. The vcpkg configuration pins a registry baseline and includes overlay ports. An overlay port overrides or adapts how a dependency is built; it is not a new chemistry algorithm. The included libaec and OpenBLAS overlays address dependency delivery rather than molecular modeling.

5. Build options are capabilities, not universal runtime promises

The native configuration exposes choices for coverage instrumentation, GPU acceleration, MPI integration, logging, resource locations, and OpenMP. Several are off by default.

OpenMP coordinates shared-memory work across threads within a machine. MPI, Message Passing Interface, coordinates work among processes, potentially on different machines. These mechanisms do not automatically parallelize every algorithm or make every combination supported.

The inspected configuration disables OpenMP on Windows and under particular Apple compiler conditions. It also checks microarchitecture consistency when using an externally installed MACIS library. Microarchitecture refers to CPU instruction capabilities: a binary optimized for instructions unavailable on another machine can fail there.

GPU acceleration requires the relevant build and dependencies, and it applies only to paths implemented to use it. It does not turn a classical simulator into a quantum processor. Conversely, a future quantum resource estimate does not describe the memory needed to compile C++ or run classical preprocessing.

An application binary interface, or ABI, is the low-level compatibility agreement between compiled components. Compiler/runtime differences, incompatible libraries, or incorrect architecture targeting can break that agreement even if all source code is individually valid.

These platform details are part of why the project publishes prebuilt distributions: most users should not have to solve the entire native dependency problem merely to try the chemistry interface.

6. Installation and development are different use cases

A wheel is a built Python distribution designed for a particular supported environment. A compatible wheel avoids compiling the package from source on the user machine. The installation guide describes published platform families, Python requirements, and native runtime prerequisites.

Optional extras select additional dependency groups. The minimal chemistry package is different from installing plugin, Qiskit, OpenFermion, network-graph, notebook, MCP, Discovery, documentation, testing, or resource-estimation support. [all] is a convenience union subject to platform/version markers, not a promise that unsupported dependencies magically become available.

Native Windows lacks the packaged PySCF dependency in this setup. Windows ARM64 has additional exclusions for dependencies without suitable wheels. The core native methods remain a different capability from those optional integrations. WSL, the Windows Subsystem for Linux, uses the Linux environment and is often the route when a needed plugin is not available natively.

The installed package version and example source must match. A notebook from a moving development branch can refer to APIs not present in the latest published wheel. Checking out source corresponding to the actual installed release is part of reproducibility, not unnecessary ceremony.

For development, the .devcontainer recipe describes an environment with native/Python tooling and editor configuration. A development container packages an environment recipe so contributors can start from similar dependencies. It reduces setup differences; it does not make dependency downloads instantaneous or replace version records and tests.

An editable Python installation can reflect changes to Python files promptly, while changes to C++ or pybind11 code still require the relevant rebuild. Recompiling every dependency after every Python edit would waste time; skipping a necessary native rebuild would test old code instead of the change under discussion.

7. Compilation has its own resource-estimation problem

Native scientific C++ builds can use substantial memory per concurrent compilation job. Increasing parallel jobs because many CPU cores are available can exhaust RAM rather than make the build faster.

The repository includes memory-aware build guidance. Its conservative policy budgets roughly eight gigabytes of available RAM per concurrent compile job, limits parallelism by both CPU count and memory, and falls back to one job when memory information is unavailable.

For an illustrative machine with 32 GB available and sixteen logical CPUs, that rule chooses at most four compile jobs. This is development guidance, not a universal measured peak-memory theorem for every translation unit.

The selected job count is supplied through CMAKE_BUILD_PARALLEL_LEVEL, including for native builds triggered indirectly by Python packaging. An out-of-memory failure calls for less parallelism and diagnosis, not increasingly aggressive job counts.

These costs belong to the software-development and classical-compute budget. They are separate from the logical and physical quantum resource estimates in Lessons 27–29.

8. Tests need to cover scientific contracts as well as software syntax

A unit test examines a focused operation. An integration test checks that multiple components work together. A regression test protects behavior that previously worked or reproduces a previously identified failure. A reference test compares against a trusted numerical result for a specified model.

All are useful here. We need tests that a matrix has the right dimensions, but also that its index convention and energy contraction are correct. We need serialization round trips, but also preservation of spin blocks and core constants. We need a circuit to compile, but also to prepare the intended state and apply the intended phase under the relevant ordering convention.

The C++ and Python suites cover their respective boundaries, with native GoogleTest/CTest infrastructure, Python pytest, and separate bundled-dependency tests where enabled. Optional integrations and platform-specific functionality can be skipped when dependencies are absent. A skipped capability is not a passing scientific demonstration.

Numerical assertions need meaningful tolerances. Floating-point operations can differ slightly across platforms. Stochastic quantum sampling can fluctuate. A tight golden number can be useful for a fixed reference environment but misleading if presented as a universal invariant across different solver versions.

That is why tutorial verification distinguishes portable scientific invariants from version-specific numerical snapshots. The tutorial-compatibility workflow installs the specified published tutorial package on several platforms, checks the intended dependency boundary, runs the setup/examples, and enables the full tutorial checks. It does not simply assume that testing current source proves the published tutorial environment works.

Coverage measures which code is exercised by tests. High coverage is useful but does not prove that assertions capture the correct physics. Conversely, an expensive physical reference test can cover few lines while protecting a crucial numerical contract. A balanced strategy combines small exact cases, property/invariant checks, representative end-to-end references, and explicit unsupported-case tests.

9. Automated checks reduce avoidable integration mistakes

Continuous integration, or CI, runs automated checks when changes are proposed or merged. It should expose defects before users encounter them.

The build-and-test workflow coordinates native dependencies, C++ builds, Python environments, numerical tests, coverage, and documentation for selected platforms. It can also test against a different QDK branch to detect changes across the chemistry/QDK boundary. Dependency caches reduce repeated compilation, but their keys and invalidation rules must follow the inputs that matter.

The .pre-commit configuration adds a different layer of checks. Formatting tools make code style consistent. Linters identify suspicious patterns and common errors. Type checking catches some incompatible uses without executing the calculation. YAML/JSON checks protect machine-readable configuration. Other checks look for accidental private keys, missing license headers, version misalignment, or missing type stubs.

These tools are complementary. A formatter cannot validate a spin convention. A type checker cannot prove a Trotter approximation meets an energy target. Secret scanning is not a complete security assessment. They reduce specific kinds of mistakes so review effort can focus more effectively on scientific and architectural correctness.

The separate pre-commit workflow runs the configured checks in automation. Local auto-fixing tools can modify files, so a read-only investigation should not invoke them on someone else worktree simply to inspect compliance.

10. Documentation is a built and versioned product

The documentation combines several sources. Public C++ comments feed Doxygen. Python docstrings and package interfaces feed Sphinx. Breathe bridges Doxygen output into the Sphinx site. Graphviz renders certain diagrams. Tutorial pages include selected portions of executable example files, so the explanation and the code can share a source.

The documentation build guide describes the sequence, including two Sphinx passes for generated stubs and references. Warnings are treated as failures by the documented build path. A broken reference or omitted API page is therefore part of the delivery surface, not only a cosmetic issue.

On Windows, the Makefile assumes tools and a shell that are not native PowerShell behavior. The documented alternative invokes the underlying tools in order. Platform-specific instructions are necessary because a correct build recipe for one shell can be invalid in another.

Figures also have provenance. Some are generated from mathematical or molecular data; others are maintained screenshots. The figure sources and maintenance instructions help contributors know which artifact to regenerate rather than hand-editing the wrong output. A diagram explaining a workflow should remain consistent with the current executable contracts, not just look plausible.

The examples directory is a separate educational and integration surface. It contains helper modules and data needed by the notebooks, so copying one notebook without its companions may fail. The example-packaging workflow creates a version-associated archive for release distribution.

11. Release engineering turns a tested revision into something users can install

The wheel pipeline describes builds across operating systems, architectures, and Python versions. It coordinates dependency builds, wheel creation, repair/packaging steps, tests, and a separately controlled publishing path. Its publishing parameter is not enabled merely because a nightly build ran successfully.

Wheel repair in this context means arranging native shared-library dependencies so the wheel is suitable for its target platform. It does not mean fixing a scientific algorithm after publication. Toolchain and installation scripts encode platform-specific details that a pure-Python package might not need.

Version information comes from the repository VERSION file, with corresponding package and native configuration. The Python metadata helper can append +local to untagged source builds; a matching release tag or explicit release-build path has different behavior. This helps distinguish a development checkout from a clean published version.

Release notes, the changelog, and schema migration documentation communicate changes to users. A successful release process must consider whether saved artifacts remain loadable, whether examples still match their environment, whether optional dependencies changed support, and whether defaults alter scientific or numerical behavior.

That is a broader definition of compatibility than keeping one function name unchanged.

12. Logging, telemetry, support, and licensing are explicit responsibilities

Logging records information useful for understanding a local execution, such as progress, warnings, errors, or detailed traces. Native logging has configurable severity levels and build options. Excessively detailed logs can be expensive or noisy; missing error context makes scientific failures hard to diagnose.

Telemetry sends selected usage/performance information to an external collection system. The Python telemetry implementation batches metrics, and event definitions include import/use events, algorithm identity, durations, outcomes, and bucketed basis-size information.

The enabled state follows environment overrides; absent an override, development/local-version markers disable the inspected default path, while ordinary stable builds enable it. Users can disable it by setting QSHARP_PYTHON_TELEMETRY to false, 0, none, or disabled before importing. Describing this mechanism is not a reason to send any telemetry during a read-only course investigation. The existence of metric code should also not be generalized into an unsupported promise about every external dependency policy.

The security policy directs vulnerability reports through the designated private reporting channel rather than public issues. It also warns against unpickling untrusted data and recommends the explicit JSON/HDF5 persistence interfaces. Supporting Python pickle for internal use is not permission to deserialize arbitrary files from strangers.

The support policy points users to repository issues and discussions. That is not the same as promising a commercial support agreement or service-level guarantee. Contribution guidance includes a contributor license agreement process and a code of conduct.

The top-level project is MIT-licensed. Bundled and external dependencies have their own licenses and notices; MACIS, for example, documents its own modified BSD terms. A top-level license does not erase dependency obligations or grant trademark endorsement. Scientific publications also need appropriate citations for the numerical methods and data sources actually used.

13. The engineering definition of a trustworthy handoff

A scientific feature is not complete merely because an algorithm produces an attractive number on one developer machine. It needs a clear interface, correct representation conversions, documented assumptions, meaningful failure behavior, suitable numerical checks, installable dependencies, compatible serialization, usable examples, and an honest release/support story.

These responsibilities explain why this repository contains build scripts, generated bindings, test data, diagrams, plugin configuration, migration tools, and release workflows alongside electronic-structure kernels and quantum algorithms. They are the mechanisms that make the scientific work reusable by another person and another team.

The final lesson brings these responsibilities back to one complete scientific and engineering review: what we can claim about a result, what we must still measure, and how each stage affects the others.