Query Batch Data

Controlled node

Overview

Query Batch Data fetches the slice of a project database table that belongs to the current batch run. It is designed for batch jobs: instead of using a fixed batch size, it counts the rows that match your (optional) filter and divides them evenly across the total number of batch runs, so every run receives an approximately equal share of the table.

Rows are always ordered by intellectible_id ascending before being split, which keeps the partitioning deterministic between runs.

When you test the node inside the workflow editor, it simulates batch run 0 of 1, meaning it returns every row that matches the filter.

How the batch slice is calculated

1. The node counts all rows in the selected table that match the filter.
2. The batch size is computed as ceil(row count / total batch runs).
3. The node returns rows starting at offset batch size × current batch index, limited to the batch size.

If no rows match the filter, or the current batch index falls beyond the end of the table, the node outputs null.

Errors are returned as data

If something goes wrong (for example a missing database or table selection, or running outside of a batch job), the node does not fail the workflow. Instead it outputs an object with an error field on the Data output, e.g. {error: "Table ID must be a valid string."}. Check for this shape before processing rows.

Inputs

The database, table and filter are configured directly on the node using its built-in pickers; they are not socket connections.

Input	Type	Description	Default
Database	Text	The project database to query, chosen with the database picker on the node. The open icon next to the picker opens the selected database in a new tab. Changing the database clears the table and filter selections.	-
Table	Text	The table to query, chosen with the table picker on the node. Use the refresh icon to reload the table list. Changing the table clears the filter.	-
Filter	Filter	Optional row filter built with the filter builder. Only matching rows are counted and returned.	-
Run	Event	Triggers the query.	-

Outputs

Output	Type	Description
Data	Data	The rows allocated to the current batch run as an array, null if there is nothing to return, or an {error} object if the query failed.
Done	Event	Fires when the node has finished running.

Runtime behavior and defaults

• This is a controlled node: it only runs when the Run event fires, and it emits Done when finished.
• In the workflow editor the node simulates batch index 0 out of 1 total run, so the whole (filtered) table is returned.
• In a deployed batch job, the node reads the current batch index and the total number of batch runs from the job's batch configuration. Outside of a batch job (and outside the editor) it returns an error object, since no batch index is available.
• The batch index must be a non-negative integer and the total number of batch runs must be a positive integer; otherwise an error object is returned.
• Because the slice is derived from a live row count, adding or removing rows between batch runs can shift which rows land in which batch.

Example

A typical use is a batch job that processes every row of a large table in parallel:

1. Configure the job to run as a batch job with the desired number of batch runs.
2. Add a Query Batch Data node and pick the database and table, optionally adding a filter (for example, only rows where status = 'pending').
3. Connect the workflow's start event to the node's Run input.
4. Connect the Data output to a loop that processes each row (for example, generating text with AI Write and saving it back with an update row node).
5. Use the Done event to continue the workflow once the slice has been fetched.

Each batch run automatically receives its own slice of the table, so the runs can execute in parallel without overlapping.