Skip to content

Decorator tasks (@peach.task)

The peach decorators let you run plain Python functions on your codops' Ray cluster — as a remote task, or as an HTTP endpoint — straight from your own code (a PEACH Lab notebook cell, an exploratory script, or a file in your codops repo). You write a normal function, decorate it, and the work runs on the cluster inside a pre-built Python environment, with the CPU, GPU and memory you ask for.

They live in pipe_algorithms_lib.compute:

from pipe_algorithms_lib.compute import peach

peach is a namespace with two decorators:

Decorator What it does
@peach.task runs the function as a Ray remote task, and returns its result
@peach.endpoint turns the function into an HTTP endpoint (a Ray Serve deployment)

This page covers @peach.task. @peach.endpoint has its own page: Decorator endpoints.

@peach(...) is now @peach.task(...)

The decorator used to be called directly — @peach(num_gpus=0.3). It is now a namespace, so the same call has to be written @peach.task(num_gpus=0.3). The import is unchanged; only the decorator line moves.

Quick start

from pipe_algorithms_lib.compute import peach

@peach.task(num_gpus=0.3, py_env="ml")
def double(x):
    return x * 2

result = double(21)   # runs on the cluster, returns 42

Calling the decorated function is synchronous: it dispatches the task to Ray and blocks until the result comes back. You get the return value directly, not a future, so there is no ray.get(...) to call yourself.

When the task is submitted, the decorator prints a link to that individual task in the cluster dashboard, so you can follow its progress and logs:

Ray task submitted | task_id=cb230... | dashboard: https://ray.<codops>.peach.ebu.io/#/jobs/01000000/tasks/cb230...

The link points at the task, not the job. The job is the whole ray:// connection and is shared by every task you submit from the same session, so its page cannot tell you anything about this particular call.

Seeing your task's output

Both print() and logging work out of the box, and by default their output is streamed back into your notebook as well as written to the worker log the dashboard link shows.

import logging

@peach.task(py_env="ml")
def train():
    print("loading data")            # always visible
    logging.info("epoch 1 done")     # visible because log_level defaults to INFO

A Ray worker is a fresh interpreter with logging unconfigured — the root logger sits at WARNING with no handler — so without help every logger.info() in a task would be dropped before it could reach the log. @peach.task therefore raises the worker's logging to log_level ("INFO" by default) for the duration of the call, and restores it afterwards so the next task scheduled onto that worker is unaffected.

log_level value Effect
"INFO" (default) logger.info() and above are visible
"WARNING" quieter — use when a third-party library in your py_env is chatty at INFO, since raising the root logger raises those too
None leave Python's defaults alone

Routine records go to stdout and WARNING or worse to stderr, so a notebook only paints the genuine problems red.

To keep a noisy task's output in the dashboard but out of your notebook, set RAY_LOG_TO_DRIVER=0 before your first peach.task call (it is read by Ray when the connection is opened). PEACH_RAY_LOG_LEVEL does the same for Ray's own connection chatter, which is turned down to WARNING by default.

Choosing the environment

The function runs inside a Python environment that has already been built for your codops. You pick which one with the py_env keyword argument:

@peach.task(py_env="ml")
def train(x):
    ...

py_env resolves to a uv project folder on the cluster (/home/ray/uv_projects/<name>). It targets any environment declared under py_environments in your peach.conf. If you do not pass it, the task runs in base.

A decorator can select an environment, but not declare one

Environments are collected from peach.conf files only. py_env therefore names an environment that already exists — if nothing declares it, the build cannot find it. A codops with only decorator code has base and nothing else, so a decorated function needing heavier dependencies still needs a peach.conf file somewhere in the same codops declaring them under py_environments.

Listing the available environments

To see which py_env values exist on your codops' cluster, call list_py_envs():

from pipe_algorithms_lib.compute import list_py_envs

list_py_envs()   # -> ["base", "ml", ...]

It returns the names of the uv projects built for your codops; any name in the list can be passed as py_env=. The lookup runs as a lightweight task on the cluster (the environments live on the cluster nodes, not on the machine calling it), so it needs the same CODOPS / RAY_URL setup as the decorators (see Environment variables below).

Installing extra packages on the fly

For quick experiments you can add packages that are not in any pre-built environment:

@peach.task(num_gpus=0.3, packages=["pandas", "numpy>=2"])
def crunch(df_bytes):
    import pandas as pd
    ...

packages installs the listed dependencies with uv --with at run time.

Exploratory use only

packages cannot be combined with py_env. uv re-resolves the whole environment at run time, which can conflict with the pre-built project environments. For anything heading to production, add the dependency to the relevant environment in your peach.conf instead (see Python environments) and let the image rebuild. Treat packages= as a prototyping shortcut, not a deployment path.

Because packages pulls from the private PyPI registry, the GITLAB_PYPI_REGISTRY_TOKEN environment variable must be set when you use it (see below).

Passing resources to Ray

Any keyword argument other than py_env, packages, cron and log_level is passed straight through to @ray.remote. Common ones:

@peach.task(num_cpus=2, num_gpus=0.5, memory=4 * 1024**3, py_env="ml")
def heavy(x):
    ...

num_cpus, num_gpus and memory are the usual Ray resource requests; Ray uses them to place the task on a suitable node.

Note

Do not pass runtime_env yourself together with py_env or packages: the decorator builds the runtime_env for you from those shorthands, and combining them raises a ValueError.

Running a task on a schedule

A @peach.task in a peach/*.py file in your codops repo can be given a cron schedule:

@peach.task(py_env="ml", cron="0 * * * *")
def hourly_reindex():
    ...

cron has no effect at run time — the decorator ignores it, and calling hourly_reindex() yourself still just runs the task once, immediately. It is read statically by the CI at build time, which generates a scheduled Prefect flow that submits the task as a Ray job every hour, the same mechanism the peach.conf task pipeline uses.

A scheduled task is called with no arguments — the generated wrapper invokes hourly_reindex() and nothing else — so anything it needs has to come from inside the function or its defaults, not from parameters.

Omit cron for a task that is only ever called from other code — for example a helper called from inside a @peach.endpoint function. Such a task gets no scheduled wrapper generated at all.

See Decorator endpoints for the peach/ folder convention, and Tasks and scheduling for the notebook-based tasks declared in peach.conf.

Running from outside the cluster

Inside PEACH Lab and on the cluster, the decorators reach Ray over the in-cluster service address and there is nothing to set up.

From a laptop or a CI runner, that address does not resolve, so the decorators go through the codops' public ray-grpc ingress instead, authenticated with the token cached by the peach CLI:

peach auth login

If you are not logged in, the call falls back to the in-cluster address anyway — which only works if you actually have network access into the VPC (VPN with split DNS, Tailscale, or running inside the cluster). Set PEACH_ALLOW_CLUSTER_FALLBACK=0 to turn that fallback off and get a clear "not logged in" error instead of an opaque DNS or connection failure.

Module-scope imports

Your task is unpickled on the head node by a per-connection Ray Client server before any worker sees it, and unpickling a module-scope import numpy as np re-runs that import there. That process runs in the base environment by default, so a task whose imports sit at module scope fails with ModuleNotFoundError even though the worker's py_env has the package. Either move the import inside the function body, or point the client server at the right environment with PEACH_CLIENT_PY_ENV=<py_env>.

Environment variables

Variable Purpose
CODOPS Your codops id (for example sesr). Used to find the cluster and to build the dashboard link. Required.
RAY_URL Override the cluster address. Defaults to the in-cluster service on-cluster, and to the public ray-grpc ingress off-cluster.
PEACH_ENV_DOMAIN The env domain of the cluster to reach when off-cluster. Defaults to peach.ebu.io.
PEACH_ALLOW_CLUSTER_FALLBACK When not logged in, whether to fall back to the in-cluster address instead of raising. Enabled by default; set to 0 for the strict behaviour.
PEACH_CLIENT_PY_ENV The environment the head node's Ray Client server unpickles your task in. Defaults to base.
GITLAB_PYPI_REGISTRY_TOKEN Grants uv access to the private PyPI registry. Required only when you use packages=.
RAY_LOG_TO_DRIVER Set to 0 before the first call to keep task output in the worker log only, instead of streaming it back to you.
PEACH_RAY_LOG_LEVEL Level for Ray's own logger. Defaults to WARNING; raise it when debugging a connection.

Inside PEACH Lab and on the cluster, CODOPS is already set for you. You mainly need the connection variables when working from outside the cluster, and the registry token only for ad-hoc packages= installs.