> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nikaplanet.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Build Geoprocessing Workers and Batch Jobs on Kubernetes

> Write a Python or R worker with a short manifest, run single and batch jobs that fan out across pods, and restrict sensitive workers to cleared roles.

A worker is your processing code plus a short manifest. NIKA GeoEngine handles the environment, hardware and scaling.

> [card] You write
>
> * A Python or R script with command-line arguments
> * A manifest: name, version, typed inputs
> * Any extra packages it needs
> * Local tests, run with GeoEngine CLI

> [card] NIKA GeoEngine handles
>
> * GDAL, PROJ, CUDA and core geo libraries
> * Mounting inputs and collecting outputs
> * Placement, scaling, access and metrics

Start from a **preset worker** (conversion, cloud removal, resampling) with no code, or package your own.

## The worker manifest

Workers are built with [GeoEngine CLI](https://docs.nikaplanet.com/nika-geoengine/overview.md), the companion command-line tool, in its `geoengine.yaml` format. A worker you build and test on a laptop runs unchanged in the cluster.

```yaml geoengine.yaml theme={null}
name: land-cover-classify
version: "2.0.0"              # MAJOR.MINOR.PATCH, immutable once published
description: Classify land cover from a multispectral scene
command:
  program: python
  script: classify.py
  inputs:
    - name: scene
      type: file
      required: true
      filetypes: [".tif", ".tiff"]
      description: Input scene
    - name: model
      type: enum
      enum_values: ["unet", "resnet50"]
      default: "unet"
      required: false
    - name: output-dir
      type: folder
      output: true              # writable; everything else is read-only
      required: true
```

### Input types

| Type | Your script receives |
| - | - |
| `file` | `--scene /inputs/scene/<filename>`, mounted read-only |
| `folder` | `--name /mnt/input_<name>`, read-only unless `output: true` |
| `string`, `number`, `boolean`, `datetime`, `enum` | `--name value` |

Every input becomes a `--name value` argument, so the script is an ordinary command-line program you can run anywhere.

### Dependencies

The geo runtime already includes GDAL with `osgeo`, fiona, shapely, pyproj, geopandas, numpy, scipy, pandas, xarray and zarr for Python, and sf, terra and stars for R. List anything else, from conda-forge or PyPI, in the worker's `pixi.toml`. For GPU work choose the CUDA runtime when you create the worker; never add CUDA packages yourself.

### Machine class

| If the worker… | Use |
| - | - |
| Converts, reprojects or tiles | **CPU** |
| Mosaics large scenes or holds a data cube in memory | **High-memory** |
| Runs a model or CUDA code | **GPU** |

Start small and move up on evidence from [real runs](https://docs.nikaplanet.com/geoengine/observability.md).

### Versions and publishing

**Patch** for fixes, **minor** for a new optional input, **major** for removed or renamed inputs or changed outputs. Build and test locally with GeoEngine CLI, then push the version to your cluster's worker catalogue.

## Run a job

A job names a worker, an exact version, a machine class and the inputs, and returns an ID at once. It moves `queued` → `running` → `completed` or `failed`.

A task **completes** when the script exits 0 and **fails** on any other exit code, with its log kept. A stopped task is sent `SIGTERM`, and its metrics are saved before it exits. Worker containers run as a non-root user.

## Batch jobs

One request covers many items. The orchestrator fans it out into parallel tasks, the pool scales to fit, and you track one job.

![A batch job fans out into parallel tasks that each write one result to shared storage](https://mintcdn.com/nika/joaiHPqbHOs4tvpR/images/geoengine/batch-fanout-light.svg?fit=max\&auto=format\&n=joaiHPqbHOs4tvpR\&q=85\&s=0108449320d43ef86869a44461b0dff9)

![](https://mintcdn.com/nika/joaiHPqbHOs4tvpR/images/geoengine/batch-fanout-dark.svg?fit=max\&auto=format\&n=joaiHPqbHOs4tvpR\&q=85\&s=f1a9056bee2437a0ed5e0bba3146ff36)

![](https://mintcdn.com/nika/joaiHPqbHOs4tvpR/images/geoengine/batch-fanout-tall-light.svg?fit=max\&auto=format\&n=joaiHPqbHOs4tvpR\&q=85\&s=9fdb85f15c41efb4e1b80dc215eba150)

![](https://mintcdn.com/nika/joaiHPqbHOs4tvpR/images/geoengine/batch-fanout-tall-dark.svg?fit=max\&auto=format\&n=joaiHPqbHOs4tvpR\&q=85\&s=03fd17e5f6aaf9f785874a4560118d35)

| Situation | Use |
| - | - |
| One input, one result | A single job |
| Same processing over many inputs | **A batch job** |
| Different steps in sequence | Your pipeline tool, one job per step |

### Four habits for fan-out

**1. One item in, one result out**

A worker that loops over a folder cannot be parallelised.

**2. Unique output paths**

Name results after the input (`scene_001_ndvi.tif`). Overwriting one fixed name is the most common fan-out bug.

**3. Safe to rerun**

Overwrite or skip existing outputs, never append.

**4. Tasks sized for one machine**

Tile big mosaics. Many small tasks schedule and recover better.

The machine class applies per task: size it for the **largest single item**, then let fan-out supply throughput. If a pool allows 40 concurrent tasks, a 200-scene batch runs in about five waves. Resubmit only failed items.

## Control who can run what

| Layer | Question | Managed with |
| - | - | - |
| **Platform administration** | Who installs, upgrades and sees pods and nodes? | Kubernetes RBAC |
| **Worker execution** | Who may run each worker? | Roles in NIKA GeoEngine |

Workers are **open** by default. Mark one **restricted** and only roles you grant can run it. Others are refused at the gateway, before any pod exists.

| | Conversion | Cloud removal | Resampling | Your Python or R | Sensitive analysis (restricted) |
| - | - | - | - | - | - |
| **Cleared analysts** | Run | Run | Run | Run | ✅ Run |
| **All other users** | Run | Run | Run | Run | ⛔ Blocked |

> [!NOTE]
>
> Role-based access to workers is part of the **full licence**.

* **Restrict by data sensitivity, not by team,** and name roles after the capability (`sensitive-analysis`, not `team-b`).
* **Grant service accounts like people.** A pipeline that runs a restricted worker needs the role too.
* **Review grants on a schedule,** like any privileged group.

**Data stays in your cluster.** Workers read and write the storage you provide; results are not copied elsewhere to run a job.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.