Recipes API

Ellf’s recipes API is designed to make it easy to extend the functionality and develop custom workflows for your use cases. The recipes SDK package includes all Python utilities you need to implement a custom recipe, as well as helpers to configure the auto-generated UI forms and command-line interfaces that let you and your team start new tasks, actions and agents using your recipes.

Recipe decorator

The recipe decorator turns a Python function into a recipe and lets you provide basic information like titles and descriptions, as well as meta data used for the auto-generated form fields and command-line interface. Ellf provides three recipe decorators you can use to create and configure recipe functions:

1. @task_recipe for annotation tasks
2. @action_recipe for actions that perform any logic
3. @agent_recipe for autonomous agents

@task_recipe decorator

Recipe decorator for annotation tasks that start the Prodigy server and annotation app.

from typing import Literal
from ellf_recipes_sdk import task_recipe, props, TextProps, Dataset, Input

@task_recipe(
    title="My Recipe",
    description="This is a task recipe",
    field_props={
        "dataset": props.dataset_choice,
        "input": TextProps(title="Input data")
    },
)
def recipe(*, dataset: Dataset[Literal["text"]], input: Input):
    stream = input.load()
    return {
        "dataset": dataset.name,
        "stream": stream,
        "view_id": "text",
    }

@action_recipe decorator

Recipe decorator for actions that don’t start the annotation server and can execute any arbitrary Python logic like model training, data analysis, data conversion etc.

from ellf_recipes_sdk import action_recipe, TextProps, BoolProps, Input

@action_recipe(
    title="My Recipe",
    description="This is an action recipe",
    field_props={
        "input": TextProps(title="Input data"),
        "verbose": BoolProps(title="Print examples")
    },
)
def recipe(*, input: Input, verbose: bool = False):
    stream = list(input.load())
    print("Examples count:", len(stream))
    if verbose:
        print(stream)

@agent_recipe decorator

Recipe decorator for autonomous agents that run independently on the cluster and can be assigned to a task to create data and annotations. Agent recipes return an Annotator callable – a function that takes a task and config dict and returns an annotated task. The SDK handles server discovery, polling, and answer submission automatically.

from ellf_recipes_sdk import agent_recipe, Annotator, TextProps, FloatProps
from ellf_recipes_sdk import detect_task_type, format_prompt, parse_response

@agent_recipe(
    title="My Annotation Agent",
    description="Autonomous agent that annotates tasks using an LLM",
    field_props={
        "model": TextProps(title="Model name"),
        "poll_interval": FloatProps(title="Poll interval (seconds)", min=0.1),
    },
)
def my_agent(*, model: str = "my-model", poll_interval: float = 1.0) -> Annotator:
    # Set up any resources (LLM client, spaCy model, etc.)
    client = setup_llm(model)

    async def annotate(task: dict, config: dict) -> dict:
        task_type = detect_task_type(task)
        prompt = format_prompt(task, task_type, config)
        raw = await client.ask(prompt)
        return parse_response(raw, task, task_type, model_name=model)

    return annotate

Annotator type alias

The type expected as the return value of an @agent_recipe function. An Annotator is a callable that receives a task example dictionary and a config dictionary, and returns an annotated example dictionary. It can be either sync or async.

Annotator = Callable[
    [dict[str, Any], dict[str, Any]],  # example dict and config dict
    dict[str, Any] | Awaitable[dict[str, Any]],  # annotated task dict
]

def annotator(eg: dict[str, Any], config: dict[str, Any]) -> dict[str, Any]:
    annotated: dict[str, Any] = annotate(eg)
    return annotated

The annotated task dictionary must follow the same JSON format that the Prodigy annotation interface expects – the exact structure depends on the task type (e.g. spans for named entities or accept for multiple choice). It should also include an answer key with a value of "accept", "reject", or "ignore". You can optionally set _annotator_id and _session_id to identify the agent in the annotation data. If you’re using the built-in agent helpers, the response format is handled for you automatically.

Agent helpers functions

The SDK provides helper functions for building agent annotators. These handle the most common annotation patterns – detecting the task type from a Prodigy task dict, formatting an LLM prompt, and parsing the LLM’s JSON response back into the expected annotation format.

from ellf_recipes_sdk import detect_task_type, format_prompt, parse_response

task_type = detect_task_type(task)  # e.g. "spans_manual"
prompt = format_prompt(task, task_type, config)
raw_response = await llm.ask(prompt)  # your LLM call
result = parse_response(raw_response, task, task_type, model_name="my-model")

Annotation task types

Annotation agents currently support the following task types, which determine how the agent should interpret the work and update the JSON.

Props

Props are Pydantic models that describe the properties of a recipe argument used for auto-generating the UI form fields and command line interface. They can include texts and descriptions that are displayed, optional UI widgets to use, as well as validations to check the input value against to display warnings and error messages. They can be defined as a dictionary keyed by argument name via the field_props argument of the recipe decorator.

TextProps BaseModel

Props for a text field for string inputs.

from ellf_recipes_sdk import TextProps, task_recipe

custom_name = TextProps(
    title="Custom name",
    description="A custom name used in the recipe",
    placeholder="Type name here..."
)

@task_recipe(title="My Recipe", field_props={"custom_name": custom_name})
def recipe(*, custom_name: str):
    ...

BoolProps BaseModel

Props for a yes-or-no field and boolean inputs.

from ellf_recipes_sdk import BoolProps, task_recipe

update_model = BoolProps(
    title="Update the model in the loop",
    description="Check this to update the model from annotations",
)

@task_recipe(title="My Recipe", field_props={"update_model": update_model})
def recipe(*, update_model: bool):
    ...

IntProps BaseModel

Props for a number field and integer inputs.

from ellf_recipes_sdk import IntProps

batch_size = IntProps(title="Batch size", min=24, step=8)

@task_recipe(title="My Recipe", field_props={"batch_size": batch_size})
def recipe(*, batch_size: int):
    ...

FloatProps BaseModel

Props for a number field and float inputs.

from ellf_recipes_sdk import FloatProps, task_recipe

learn_rate = FloatProps(title="Learning Rate", min=0.001, step=0.001)

@task_recipe(title="My Recipe", field_props={"learn_rate": learn_rate})
def recipe(*, learn_rate: float):
    ...

ListProps BaseModel

Props for a list field and inputs of type List.

from ellf_recipes_sdk import ListProps, task_recipe

labels = ListProps(
    title="Labels",
    placeholder="Type labels here...",
    min=1,
)

@task_recipe(title="My Recipe", field_props={"labels": labels})
def recipe(*, labels: List[str]):
    ...

ChoiceProps BaseModel

Props for a selection field with pre-defined options and inputs of type Union or a Literal of multiple options.

from ellf_recipes_sdk import ChoiceProps, Props, task_recipe

goal = ChoiceProps(
    title="Data collection goal",
    choice={
        "nooverlap": Props(title="Annotate all examples once"),
        "overlap": Props(title="Annotate with overlap"),
    },
)

@task_recipe(title="My Recipe", field_props={"goal": goal})
def recipe(*, goal: Literal["nooverlap", "overlap"]):
    ...

Props BaseModel

Base class that all specific props types inherit from and that can be used for all other arguments.

Validation BaseModel

Validations are used in the UI form when creating and configuring a task or action. They’re a powerful way to define custom constraints for the expected input and show messages to the user, including errors that prevent the form from being submitted, as well as warnings and info messages to help enforce best practices. Each prop can define one or more validations in order that are interpreted based on the field type, e.g. comparing the length of a list, a string or a number. See the recipe development guide for more details and examples.

validations = [
    Validation(
        op="re",
        value="^[\\w-]+$",
        message="It can contain only numbers, letters, dashes '-' and underscores '_'",
        level="error",
    ),
    Validation(
        op="gt",
        value=20,
        message="It's recommended to use a shorter name (20 characters or less)",
        level="warning",
    )
]

Pre-defined props

Some arguments are very common across recipes, and so are their props. Ellf’s recipe development SDK includes several pre-configured props as variables that you can import and use, so you don’t have to repeat titles, help texts and field settings.

from typing import List
from ellf_recipes_sdk import props, Dataset, InputDataset, Lang

@task_recipe(
    "my-recipe",
    field_props={
        "dataset": props.dataset_new,
        "input_dataset": props.dataset_existing,
        "other_dataset": props.dataset_choice,
        "label": props.label,
        "lang": props.lang,
    }
)
def recipe(
    *,
    dataset: Dataset,
    input_data: InputDataset,
    other_dataset: Dataset,
    label: List[str],
    lang: Lang
):
    ...

Types and dataclasses

Ellf includes a range of pre-defined argument types, usually dataclasses, for common recipe arguments. The recipe will then receive an instance of the dataclass. Dataclasses can also define their own methods, e.g. a load method to load a given asset or dataset.

Dataset dataclass

Type of a new dataset to create by the recipe. It’s a generic type that allows specifying the dataset kind as a parameter, e.g. Dataset[Literal["text"]] for datasets of kind "text". Specifying a kind will also limit the datasets that can be selected in the UI. When used in a recipe, the function will receive an instance of the dataclass.

from typing import Literal
from ellf_recipes_sdk import Dataset, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, dataset: Dataset[Literal["text"]]):
    return {"dataset": dataset.name, ...}

InputDataset dataclass

Type of an existing dataset that’s required to already be in the database, e.g. for datasets used as input data. It’s a generic type that allows specifying the dataset kind as a parameter, e.g. InputDataset[Literal["text"]] for datasets of kind "text". Specifying a kind will also limit the datasets that can be selected in the UI.

from typing import Literal
from ellf_recipes_sdk import InputDataset, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, input_dataset: InputDataset[Literal["text"]]):
    stream = input_dataset.load()

InputDataset.load method

Load the examples from the input dataset. It takes the same keyword arguments as Prodigy’s get_stream and returns a stream of examples.

InputDataset.export method

Export the examples from the dataset as JSONL (newline-delimited JSON).

Asset dataclass

Base class for asset. It’s a generic type that allows specifying the asset kind as a parameter, e.g. Asset[Literal["input"]] for Input. It’s typically not used directly and subclassed for specific asset types (see below). When used in a recipe, the function will receive an instance of the dataclass. See the docs on custom assets for details on how to create your own subclasses.

Input dataclass

Type of an asset used as input for a recipe, e.g. a text source. Subclass of Asset[Literal["input"]].

from ellf_recipes_sdk import Input, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, input_data: Input):
    stream = input_data.load()

Input.load method

Load the examples from the input asset. It takes the same keyword arguments as Prodigy’s get_stream and returns a stream of examples.

Input.create classmethod

Register a new input asset record with Ellf. Note that this only creates the record – you need to write the actual files to your cluster’s storage bucket first, e.g. using cloudpathlib.AnyPath. See creating assets in recipes for a full example.

from ellf_recipes_sdk import Input, action_recipe

@action_recipe(title="My Recipe")
def recipe():
    asset = Input.create(name="My asset", path="{__bucket__}/my_asset", loader="jsonl")

Input.update method

Update an existing input asset with a new version. By default, the original asset version will be incremented as a patch, e.g. 1.2.3 becomes 1.2.4.

from ellf_recipes_sdk import Input, action_recipe

@action_recipe(title="My Recipe")
def recipe(input: Input):
    input.update(path="{__bucket__}/my_updated_asset")

Patterns dataclass

Type of a patterns file, a kind of asset, for recipes that use pattern matching. When loaded, it creates a PatternMatcher instance that you can use on your stream to automatically add annotations from patterns. Subclass of Asset[Literal["patterns"]].

from ellf_recipes_sdk import Model, Input, Patterns, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, spacy_model: Model, input_data: Input, patterns: Patterns):
    nlp = spacy_model.load()
    stream = input_data.load()
    matcher = patterns.load(nlp=nlp, label="PERSON")
    stream = (eg for _, eg in matcher(stream))

Patterns.load method

Load the patterns and initialize a PatternMatcher that can be used by the recipe. This method takes the same arguments as Prodigy’s PatternMatcher to configure how the patterns should be applied.

Model dataclass

Type of a spaCy model, a kind of asset uploaded to the cluster. When loaded, it returns the initialized nlp object. Subclass of Asset[Literal["model"]].

from ellf_recipes_sdk import Model, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, spacy_model: Model):
    nlp = spacy_model.load()

Model.load method

Load a spaCy model and initialize the nlp object. Before loading, the model will be downloaded to the worker on your cluster that’s executing the recipe.

BlankModel dataclass

Create a blank nlp object for one of the available spaCy languages for tokenization or training a new pipeline. The language to use is a field of the type and can be set in the UI or on the CLI when a task or action is created from the recipe.

from ellf_recipes_sdk import BlankModel, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, spacy_model: BlankModel):
    nlp = spacy_model.load()
    print(f"Created blank model for {spacy_model.lang}")

BlankModel.load method

Initialize a blank nlp object.

UseModel dataclass

Pre-defined custom type for recipes that use a model to pre-annotate data. Combines the model name and a boolean setting whether it should be updated in the loop (which needs to be implemented in the recipe). Can also be used as part of a Union type with BlankModel so the user can choose between a pretrained pipeline or a blank model.

from typing import Union
from ellf_recipes_sdk import UseModel, BlankModel, task_recipe

@task_recipe(title="My Recipe")
def recipe(*, spacy_model: Union[UseModel, BlankModel]):
    nlp = spacy_model.load()
    if isinstance(spacy_model, UseModel) and spacy_model.update:
        # Implement logic to update the model in the loop
        ...

Lang enum

Enum of all available spaCy languages that can be initialized with spacy.blank, e.g. for tokenization or training a new pipeline.

from ellf_recipes_sdk import Lang, task_recipe
import spacy

@task_recipe(title="My Recipe")
def recipe(*, lang: Lang = Lang.en):
    nlp = spacy.blank(lang)

Secret dataclass

Secrets are pointers to API keys or other credentials and let you securely manage access across your organization. If your recipe needs to connect to an API or authenticate with a service, you shouldn’t ever have your user provide their credentials in the web form or CLI. Instead, you can use the Secret type so you’ll be able to select an existing secret, which is then passed to your recipe function.

from ellf_recipes_sdk import agent_recipe, Secret

@agent_recipe(title="My Recipe")
def recipe(*, api_key: Secret):
    gemini_key = api_key.value()
    llm = Gemini(api_key=gemini_key, model_name="gemini-2.0-flash")

Secret.value method

Get the plain-text value of a secret. This is what should be passed to API calls and other libraries that require keys to authenticate, for example in agent recipes. The value is resolved automatically based on the recipe parameter the secret was bound to – you don’t need to know the internal storage key.

@ellf_type decorator

Decorator to register dataclasses as custom types. Dataclasses can define one or more custom subfields and an instance of the dataclass will be passed to the recipe function if it’s used as a type annotation. Custom types can also be used as part of a union. The metadata defined in the decorator is propagated and merged to child classes that are also annotated with @ellf_type. See the recipe development guide and custom assets for more details and examples.

from typing import Literal, Optional
from dataclasses import dataclass
from ellf_recipes_sdk import ellf_type, IntProps

@ellf_type(
    "workflow-settings",
    title="Customize annotation workflow",
    field_props={"annotations_per_task": IntProps(optional_title="Configure annotations per task")}
)
@dataclass
class WorkflowSettings:
    feed_overlap: Literal["nooverlap", "overlap"] = "nooverlap"
    annotations_per_task: Optional[int] = None
    allow_work_stealing: bool = True
    instructions: str = ""