Skip to content

Python environments

Each Peach codops declares the Python environments its endpoints and tasks run in, under a py_environments key in any peach.conf/*.yaml file. An environment is just a name and a list of dependencies. The CI builds one uv project per environment, and every endpoint or task picks exactly one to run in.

This is the only way to declare dependencies. The old inline dependencies: keys (repo-wide, per-block and per-endpoint/task) have been removed and are now rejected by the config schema.

Declaring environments

codops: zzebu

py_environments:
  base:                       # special: inherited into every environment
    dependencies:
      - orjson==3.10.0
      - httpx
  ml:                         # a shared, reusable environment
    dependencies:
      - torch==2.0
      - transformers

endpoints:
  recommend:
    url: /recommend
    py_env: ml                # run this endpoint in the shared 'ml' environment
    components:
      main:
        notebook: endpoint.ipynb
        method: recommend

tasks:
  reindex:
    notebook: nb.ipynb
    method: run
    py_env: ml                # run this task in the shared 'ml' environment

Each entry under py_environments is an object with a single dependencies list. Dependency strings are PyPI or private-PyPI package names, optionally pinned (orjson==3.10.0) or ranged (numpy>=2).

The rules

base is inherited everywhere

base is special. Its dependencies are inherited into every environment built for the codops, including every named environment and the internal CodopsGateway. Put anything every endpoint and task needs in base.

base is also the default environment. An endpoint or task with no py_env runs in base.

A base environment is always built, even if you do not declare one. In that case it still contains the auto-injected runtime libraries and the framework stack described below.

Named environments are opt-in

Every other entry (for example ml) is built as its own environment. An endpoint or task selects it with py_env: <name>, and then runs with that environment's dependencies plus the inherited base dependencies and the framework stack.

  • py_env works the same on both endpoints and tasks.
  • Many endpoints and tasks can share one named environment; it is built once.
  • Every declared environment is built even if nothing references it.
  • There are no per-file, per-endpoint or per-task inline dependency lists anymore: an endpoint or task runs in exactly base or one named environment.

Non-base takes precedence

base is the lowest precedence. If the same package is pinned in both base and a named environment (or the framework stack), the non-base pin wins.

py_environments:
  base:
    dependencies:
      - tqdm==4.68.3
  ml:
    dependencies:
      - tqdm==4.68.2     # 'ml' runs tqdm 4.68.2, not the base 4.68.3

Mechanically, base dependencies are listed first and the last occurrence of a package wins during de-duplication, so a more specific environment overrides the base pin.

A bare pkg and a pinned pkg==X for the same package never both end up in an environment: the bare form is dropped, so you do not get a spurious conflict.

Auto-injected runtime libraries

The CI adds two libraries to base once, so they reach every environment through inheritance:

  • pipe-algorithms-lib, pinned to the latest private-PyPI release (the recsys and task helpers).
  • implicit==0.7.2 (its collaborative-filtering dependency).

You do not need to list these yourself. If you do pin one of them in base, your pin is respected instead of the auto-injected one.

Do not re-pin framework libraries

A serve and framework stack (ray[serve], fastapi, pydantic, starlette, protobuf and friends) is pinned into every environment automatically and must stay in sync with the image. Do not declare these yourself. Your py_environments entries should be ordinary PyPI or private-PyPI package names, not raw git URLs.

How it fits together

  1. The CI unions the py_environments declarations across the repo's peach.conf files (filtered to the codops being built).
  2. For each declared environment it builds one uv project, seeded with that environment's dependencies, the inherited base dependencies, the auto-injected runtime libraries and the framework stack.
  3. Each endpoint or task is wired to its resolved environment: the one named by py_env, or base when py_env is absent.

At run time, code submitted with the peach decorator can target any of these same environments through its py_env, task or endpoint_component shorthands.