31Running work elsewhere and reusing results

The chemistry should not depend on the computer beside your desk

Our nitrogen example is small enough to explain locally. A research campaign can be very different: many geometries, larger basis sets, expensive correlated calculations, substantial integral storage, and many quantum-circuit simulations. The person designing that workflow may not have enough memory or compute capacity on their laptop.

Moving a calculation elsewhere sounds simple: send the inputs, run the program, retrieve the results. In practice, the engineering questions are just as important as the numerical ones. Which software is installed at the destination? What if the client disconnects? Was the job submitted twice? Where are its inputs and outputs? Can we reuse an identical result rather than recompute it? Who may cancel or clean up the job?

This lesson follows the remote execution package. Remote execution here means executing a registered QDK Chemistry algorithm through a job protocol. It does not by itself mean submitting a quantum circuit to a quantum processor. The remote algorithm may be a classical SCF calculation, a circuit-construction step, or another serializable operation.

The scientific method is chosen as before. The backend determines where and how that chosen operation is executed.

1. Three execution choices that must stay distinct

A direct algorithm.run(...) normally executes in the current process. A blocking remote call submits elsewhere and waits for the result. An asynchronous remote call returns a handle before the calculation finishes.

Blocking means the caller waits before continuing. Asynchronous means the caller can do other work while the job progresses. A job handle identifies and manages the submitted work; it is not the result itself.

Algorithms created through the registry accept remote and cache controls. A source-backed blocking example is:

from qdk_chemistry.algorithms import create

scf = create("scf_solver", "qdk", method="hf")
energy, wavefunction = scf.run(
    structure, 0, 1, "cc-pvdz",
    remote="local",
)

The backend named local deliberately uses a separate process on the same computer. It exercises the transfer and job protocol without requiring an external service. It is not a faster quantum device or a cloud account.

For asynchronous work, the module-level submit returns a Job:

from qdk_chemistry.remote import submit

job = submit(
    scf,
    structure, 0, 1, "cc-pvdz",
    remote="local",
    job_dir="job-records",
)

Here job_dir asks the SDK to persist the job record automatically. Without it, submit can return an in-memory job that the caller must explicitly save if it should survive the current process. These snippets explain the API; they were not run during course preparation.

The proxy implementation also exposes a module-level run with blocking behavior. Merely writing remote= does not turn the normal result pair into a job object. That distinction prevents code from trying to interpret a submission identifier as an energy.

2. Build a portable execution request

The proxy constructs a payload containing the algorithm type and name, configured settings, positional and keyword arguments, and identifying hashes where available. It does not transmit the live C++ solver object from Python memory.

The serializer from Lesson 30 writes a manifest and separate numerical artifacts. The compute-side worker reconstructs the algorithm through the registry and reapplies its settings. A Wavefunction travels as supported serialized data, not as an arbitrary pointer into another process.

The compute environment therefore needs compatible QDK Chemistry code, the same required algorithm registrations, suitable data loaders, and the external dependencies used by that algorithm. A local Python object that relies on an uninstalled plugin does not become usable remotely just because it can be named in a manifest.

Nested AlgorithmRef settings preserve the composition of a workflow. For example, a geometry optimizer may delegate energy calculations to a configured derivative algorithm, which in turn delegates to an electronic solver. The destination must be able to reconstruct all those layers.

The worker entry point is python -m qdk_chemistry.remote.worker. It loads the request, checks configured cache information, deserializes inputs when needed, creates the algorithm, applies settings, executes it, and either serializes outputs or commits them through a shared cache transport.

This is an explicit protocol, not arbitrary Python pickling. Its accepted values include the data classes, numerical arrays, primitive values, supported sequences, and nested algorithm references discussed previously. An arbitrary callback or unsupported dictionary value may not cross the boundary even when some in-process API can accept it.

3. A backend owns placement and transport, not the chemistry

The RemoteBackend interface defines the responsibilities needed to run that protocol:

The asynchronous submission, status, and result-retrieval hooks are required abstract operations in the inspected base class. Cancellation and cleanup have default implementations that raise NotImplementedError. Some older example prose calls more of these hooks optional; the current interface is the authority.

The backend returns a persistent description of its job, such as an operation identifier or process identity and artifact locations. That state must be JSON-compatible after path normalization. The framework validates the metadata rather than silently accepting live connection objects that could not be restored.

A backend can be registered through a plugin. Registering a backend gives it a discoverable name; it does not provision a cluster, grant credentials, or install chemistry dependencies at the destination.

A custom SSH example demonstrates direct execution on an SSH-accessible machine. SSH is a secure remote-login protocol; SCP transfers files through that ecosystem. This example assumes suitable remote commands and Python software. It does not implement submission to a queue scheduler such as Slurm merely because an HPC machine might use one.

A scheduler allocates resources among competing jobs. Integrating such a scheduler is a separate backend responsibility, not an automatic capability of the SSH transport example.

4. Follow the local subprocess implementation

The local backend makes the mechanics especially visible.

On connection, it creates a temporary working area. For a submitted job, it creates separate input and output directories, writes the serialized request, and launches the worker with the selected Python executable. Standard output and error streams go to log files.

A process is a running program with its own execution state. Launching a subprocess tests that the request really can be reconstructed independently of the client object. It is useful for debugging serialization, plugin availability, and job recovery even though no network is involved.

The backend retains a process identifier and, on supported operating systems, additional process-creation identity. Process identifiers can eventually be reused, so cancellation after reconnecting must not blindly terminate whatever program currently has the old number. The local implementation checks identity before terminating a process whose original handle is no longer available.

When the original process handle exists, a successful status requires both a zero exit code and an output manifest. When reconstructing status later, the presence of the output manifest is important evidence that results are available. Missing outputs and a dead process indicate failure rather than a fabricated result.

Disconnecting the backend does not mean canceling submitted work or deleting its artifacts. It only removes the temporary connection directory if it is empty. This separation is what allows persisted jobs to remain inspectable after the original client closes its connection.

Subprocess isolation is not a claim of a hardened security sandbox. The worker still runs trusted installed code with the permissions and environment provided by the host.

5. A job record is a durable lifecycle record

The Job class stores the backend name and reconstructible configuration, backend-specific state, algorithm information, submission time, last-known status, input and output hashes, and whether the original output was a tuple. MCP-managed jobs can also carry workspace/project ownership information.

That last output-shape field matters. A scalar energy and a pair containing energy and wavefunction must not become indistinguishable merely because results were stored as separate items.

The main operations are check(), wait(), fetch(), cancel(), save(), load(), and cleanup(). A normal lifecycle proceeds from submission through running to success, failure, or cancellation. After output retrieval the local record can have status retrieved.

Terminal means that execution has reached a final state; it does not mean success. failed and canceled are terminal too. The helper named is_successful specifically recognizes succeeded, not every terminal status, so application logic should not reinterpret every completed handle as a usable result. Retrieval is its own state transition.

wait() polls until terminal state or the configured timeout. Polling means checking status periodically, rather than continuously occupying a CPU. Default backend polling and wait timeout are five seconds and 3600 seconds. A wait timeout describes how long this client waits; it does not universally impose an execution deadline on the remote service and does not automatically cancel the job. A persisted identifier lets the caller inspect the work again later.

fetch() obtains the result and records output information. Backend artifacts are retained by default. fetch(cleanup=True) requests backend cleanup after successful retrieval and persistence; cleanup() explicitly removes terminal-job artifacts when the backend supports it. Neither is a license to remove unrelated directories or all shared cache data.

Preserving a job JSON file alone does not preserve every output forever. Large outputs may be separate artifacts or cache entries. Reproducibility requires those payloads and their loaders to remain available too.

6. Cache reuse is a different optimization from remote execution

A cache stores a result so an equivalent identified request can avoid repeated computation. Remote execution changes where work runs. The two can be used together or independently.

A cache-backed call can look like this:

energy, wavefunction = scf.run(
    structure, 0, 1, "cc-pvdz",
    cache="calculation-cache",
    remote="local",
)

A path resolves to a folder cache. On a complete cache hit, the result can be returned without launching the worker. On a miss, the operation runs and the result is stored.

The FolderCache separates job metadata from content-addressed artifacts. Small primitive outputs, such as an energy, can live in the job record. Numerical objects and arrays have separate files. Supported nested sequences use manifests that reference their components.

A cache entry is complete only if its output descriptors can reconstruct the expected result. A job record pointing to a missing wavefunction is not a usable hit. Some corrupted metadata and numerical-array files are treated as misses; a data-class loader can still raise a schema error. Cache behavior should not be described as a universal silent-recovery guarantee.

Writes use temporary files and replacement to avoid exposing some partially written artifacts. That does not make the whole system a distributed transaction manager or guarantee exactly-once computation under arbitrary simultaneous clients. Reuse of an already recorded in-flight job reduces duplicate submissions in the supported flow; concurrent requests and external service failures still need careful operational handling.

force_rerun=True bypasses cache lookup and requests a new computation. This is useful when deliberately obtaining a new stochastic realization or investigating changed external behavior. A cache hit is an old result, not another independent sample. If random seeds or external dependencies are absent from the identity, the operator must account for that limitation rather than interpret every reuse as independent evidence.

7. Layer local and shared caches without pretending they are the same place

A fast local cache is convenient for repeated analysis. A shared filesystem can let a compute node and several clients access the same large Hamiltonian without repeated uploads.

The TieredCache combines several backends in priority order:

from qdk_chemistry.remote.cache import FolderCache, TieredCache

cache = TieredCache([
    FolderCache("local-cache"),
    FolderCache("shared-team-cache", is_shared=True),
])

The example path marked shared is illustrative. A real deployment must ensure that both the client and worker can reach the intended storage. Setting a flag does not create a network mount or grant access.

On reads, the tiered cache tries earlier tiers first. A hit in a later tier is backfilled into earlier tiers: copied nearer to the caller for future access. On writes, it writes through to every eligible tier. The remote view retains only the tiers declared reachable by the compute side.

This yields two different savings. A complete result can avoid computation altogether. Shared input artifacts can avoid transfer even when a new calculation is still needed. Reusing a Hamiltonian for several trial-state studies is a typical example of the second case.

The proxy can also encounter a recorded remote job still in flight. It resumes waiting on that identifier rather than immediately submitting the same identified request again. After success, it can reconstruct results directly from a shared cache or fetch them from the backend.

Job records managed through the assistant interface are additionally scoped by workspace/project ownership. That is an application-level management boundary. It is not a replacement for operating-system permissions, cloud authorization, or careful handling of shared numerical data.

8. The compute worker can reuse results, but cache failures are not all equivalent

The worker checks a declared remote cache before reconstructing and executing the algorithm. It can also write completed output data and a job record back to that cache.

Ordinary cache read or write failures are logged on the inspected worker path. Where normal output-file transport is available, execution and result serialization can proceed even when the optimization failed. That is appropriate when a cache is optional.

A cache-only transport is different. If the remote cache is the only supported way to return results, failing to persist the result there must be an error. The worker explicitly rejects successful completion of that transport without a retained result. Otherwise the service could report completion while the client had no recoverable output.

The remote and serialization cases exercise result-shape preservation, cache bypass, staged manifests, and ownership-scoped records. These source-level scenarios help explain the intended lifecycle; no jobs or tests were run for this lesson.

9. The Discovery adapter is an integration, not a cloud entitlement

The repository also contains a Microsoft Discovery backend, registered when the optional Discovery SDK is available through the plugin loader.

It requires deployment-specific information including a workspace endpoint, project, tool identifier, and node pool. A node pool describes the compute resources on which work may be placed. Optional configuration includes the execution image, Python path, requested CPU/GPU resources and memory, storage choices, polling, and artifact-retrieval retries.

An image here describes the packaged execution environment. Requesting GPUs does not make a CPU-only algorithm use them or ensure that the selected package was compiled with suitable acceleration. Compute provisioning and algorithm capability remain separate questions.

The adapter supports auto, blob, and cache transport modes. Blob transport uses configured Azure storage linked to the Discovery storage asset; blob storage stores named binary objects remotely. Cache transport requires a shared cache accessible to the compute node and a deterministic request identity. Auto mode chooses a usable configured path; it does not invent missing storage.

Authentication is delegated through Azure credential helpers, which support Azure CLI credentials or the configured default credential chain. The SDK reports an operation identifier that is retained for later status, cancellation, and retrieval. These mechanisms depend on an authorized external deployment. Source code availability is not evidence that a particular user has that service, quota, image, or permission.

The backend can obtain defaults from QDK_DISCOVERY_* environment variables, with explicit constructor values taking precedence. A persisted backend configuration needs enough information for reconnection, so operational design should avoid placing raw secrets in fields intended for ordinary job JSON. Credentials should be supplied through the supported secure authentication mechanisms.

MCP clients can override only the configuration fields the backend explicitly marks safe. For this adapter those are retry and polling/timeout controls, not arbitrary endpoints, credentials, images, or node pools. A backend subclass must make its own declaration; permission is not automatically inherited as a blanket trust decision.

10. Evaluate the entire operational cost

Moving a computation does not erase its cost. An illustrative wall-time model is

Each term names time spent preparing artifacts, moving them, waiting for resources, calculating, recovering outputs, or interpreting results. This is an accounting model, not a benchmark measured here.

For a tiny SCF problem, transfer and startup may dominate. For a large integral model, memory and data movement may be the limiting factor. For a campaign of related calculations, caching shared inputs or prior results may save more than changing where the worker runs.

Remote execution therefore creates a contract between scientific workflow authors, backend/platform engineers, environment and storage operators, and users who supply authorized access. The chemistry team specifies a portable operation. The platform provides placement and lifecycle management. Serialization preserves its data meaning. Caching avoids eligible repetition. The caller remains responsible for interpreting success, preserving needed artifacts, and understanding whether the result was newly computed or reused.

The next lesson adds the human and AI-assistant interfaces that make those same operations available without writing the Python orchestration by hand.