Validation - Ingestly

The validation node checks data against configurable criteria before passing it to downstream nodes. Use it to enforce data quality, ensure required fields are present, verify formats, or run custom validation logic. Five validation modes are available, from simple field rules to AI-powered checks.

When to use validation

You want to catch bad extractions before delivery: missing required fields, malformed totals, wrong currency.
You want to gate runs that should go to review when something looks off. Pair validation Warn with a downstream review node.
You need to enforce a strict contract with an external system. Use Schema mode with a JSON Schema document.
You want a programmable check (regex, cross-field math). Use Script mode for deterministic logic, AI mode when the check is genuinely judgment-based.
Skip validation when the criteria are simple presence checks already covered by the schema’s required field on extract.

Failure actions

Every validation mode uses a failure action to control what happens when validation errors are found. The Validation failures Review pause mode becomes available as soon as any upstream node is a validation node, with no special failure-action configuration required, so you only choose Warn and wire the downstream Review node.

Action	Behavior
Fail	The step fails and the pipeline stops. Errors are reported in the run detail.
Warn	The step succeeds and the pipeline continues, with validation errors promoted to warnings. When the rules can attribute a failure to specific field paths (Rules mode, including expression rules), those paths are recorded on the step output so a downstream Review node configured with the Validation failures pause mode pauses the run for a human to correct them. Modes that cannot attribute per-field paths (Schema, Endpoint, AI) just produce warnings.

Modes

Rules

Define field-level validation rules using a visual rule editor. Each rule specifies a field path (supports dot notation and array indexing, e.g. items[0].name), an operator, and an optional value. You can also set a custom error message per rule.

Operators

Operator	Group	Description	Value required
`required`	Presence	Field must exist	No
`not_empty`	Presence	Field must not be empty or null	No
`equals`	Comparison	Field must equal the value	Yes
`not_equals`	Comparison	Field must not equal the value	Yes
`gt`	Comparison	Field must be greater than the value	Yes
`gte`	Comparison	Field must be greater than or equal to the value	Yes
`lt`	Comparison	Field must be less than the value	Yes
`lte`	Comparison	Field must be less than or equal to the value	Yes
`contains`	String	Field must contain the substring	Yes
`not_contains`	String	Field must not contain the substring	Yes
`starts_with`	String	Field must start with the value	Yes
`ends_with`	String	Field must end with the value	Yes
`matches_regex`	String	Field must match the regular expression	Yes
`is_email`	Format	Field must be a valid email address	No
`is_number`	Format	Field must be a numeric value	No
`is_date`	Format	Field must be a valid date	No
`is_url`	Format	Field must be a valid URL	No
`in_list`	List	Field must be one of the listed values	Yes
`not_in_list`	List	Field must not be one of the listed values	Yes
`length_min`	Length	Field length must be at least the value	Yes
`length_max`	Length	Field length must be at most the value	Yes
`length_between`	Length	Field length must be between two values	Yes (two values)
`expression`	Expression	A whole-payload arithmetic expression must hold (see below)	Yes (the expression)

Expression rules

The expression operator runs a small arithmetic expression against the upstream payload, so you can enforce cross-field invariants that no single-field operator can express. The Field input is hidden in the editor; the whole rule is just one expression. Supported syntax:

Construct	Example
Comparison	`subtotal + tax = total`, `discount <= subtotal`
Operators	`=`, `==`, `!=`, `>`, `<`, `>=`, `<=`
Arithmetic	`+`, `-`, `*`, `/`, parentheses
Field reference	`subtotal`, `invoice.total`, `lineItems[0].price`
Aggregate functions	`sum({{lineItems[].amount}}) = total`, `count({{lineItems}}) > 0`, `avg({{lineItems[].price}}) < 1000`, `min(...)`, `max(...)`, `length({{customerName}}) >= 3`

sum, avg, min, and max require a list of numbers: if their argument plucks a field whose element type is text or yes/no from an array, the pipeline editor flags it as a validation error on the node at save time. count accepts a list of any element type, and length accepts a list or text. Numeric values come from JSON numbers directly. Currency-formatted strings like "$1,234.50" are also accepted; the evaluator strips common currency markers and thousand separators. If the rule violates, every field referenced by the expression is treated as the affected field (used by the Warn action to populate the failing-field list consumed by a downstream Review node’s Validation failures pause mode).

The expression syntax is independent of the {{ ... }} template syntax used elsewhere. Inside an expression rule, write subtotal + tax = total, not {{subtotal}} + {{tax}} = {{total}}. The braces are only required around aggregate-function arguments (sum({{items[*].amount}})).

Schema

Validate data against a JSON Schema document. Paste or write your schema in the editor and the node validates the incoming data against it. Useful when you need to enforce a strict contract on the data shape.

Endpoint

Call an external HTTP endpoint to validate data. The node sends the data to your URL and interprets the response to determine pass or fail.

Field	Type	Required	Description
URL	string	Yes	The endpoint URL. Supports template substitution.
Method	select	Yes	HTTP method (GET, POST, PUT, etc.)
Body	string	No	Request body template
Response mapping	object	No	Dot-path to `valid` and `errors` fields in the response JSON

By default, the node looks for valid or isValid (boolean) and errors (string array) at the root of the response body. Use response mapping to point to different paths if your endpoint returns a different shape. The endpoint request uses a fixed 45-second server timeout; it is not configurable per node. Transient errors (HTTP 429, 502, 503, 504) are retried automatically.

Script

Write JavaScript validation logic that runs in a sandboxed runtime. Each upstream node is exposed as a JS global with the same shape as {{nodeName}} in templates (success, payload, metadata, error). Reference upstream values by node name, for example extract.payload.email or vars.payload.amountLimit. Runtime limits:

Limit	Value
Execution timeout	5 seconds
Max statements	10,000
Memory	16 MB

The editor provides syntax highlighting, autocomplete, and inline error reporting. Return values:

Return type	Interpretation
`true`	Validation passes
`false`	Validation fails (generic error)
`"error message"`	Validation fails with the given message
`["error1", "error2"]`	Validation fails with multiple errors
`undefined` / `null`	Validation passes

The script must contain a return statement. Object return values are not supported.

Example:

const data = extract.payload;

if (!data.email || !data.email.includes('@')) {
  return 'Invalid email address';
}
if (data.items && data.items.length === 0) {
  return 'At least one item is required';
}
return true;

AI

Use an LLM to validate data against natural language instructions. Write a prompt describing what valid data looks like, and the model returns whether the data passes along with any errors.

Field	Type	Required	Description
Prompt	text	Yes	Natural language validation instructions
Precision	select	Yes	Model quality level: `Small`, `Medium`, or `High`

Higher precision uses a more capable model at a higher credit cost.

Output format

The validation node passes the upstream data through unchanged as its payload, and the validation diagnostics live in metadata under metadata.validation. Downstream nodes reference the data fields directly (no data wrapper), for example {{node.payload.total}}, and read the diagnostics as {{node.metadata.validation.isValid}}.

{
  "payload": { },
  "metadata": {
    "validation": {
      "isValid": true,
      "errors": [],
      "warnings": [],
      "mode": "rules",
      "failureAction": "fail"
    }
  }
}

Field	Type	Description
`payload`	object	The original upstream payload, passed through unchanged. Reference its fields as `{{node.payload.<field>}}`.
`metadata.validation.isValid`	boolean	Whether the data passed validation
`metadata.validation.errors`	string[]	Validation error messages (empty when valid)
`metadata.validation.warnings`	string[]	Warning messages (populated when failure action is `warn`)
`metadata.validation.mode`	string	The validation mode that was used
`metadata.validation.failureAction`	string	The configured failure action (`fail` or `warn`)

Inputs and outputs

Allowed inputs: Extract, transform, route, merge, review, parse. Output: The original data as the payload, with validation diagnostics in metadata.validation. When failure action is fail and validation errors exist, the step fails and downstream nodes do not execute.

Common pitfalls

Failure action set to Fail when you wanted a soft check

Fail halts the pipeline and downstream nodes never run. If you wanted the run to continue but to flag the issue, switch to Warn and consume the metadata.validation.warnings array downstream.

Endpoint mode without response mapping for a non-standard API

By default, the endpoint mode looks for valid (or isValid) and errors at the response root. If your service returns a different shape, set Response mapping to point at the correct paths or every response will be treated as invalid.

Script that doesn't return

The script must contain a return statement. A script that finishes without returning is treated as undefined (which passes validation), so a missing return silently disables the check.

AI mode used for a check that's actually deterministic

AI validation costs LLM credits per run. If the check is “amount must be a positive number” or “currency must be USD or EUR,” use Rules or Script mode for free, deterministic results. Reserve AI mode for genuinely judgment-based checks.

Validation duplicating the extract schema's required list

A required field marked on the extract schema already gates the field’s presence. Don’t repeat the same check in validation; use validation for the harder constraints (formats, ranges, cross-field rules).

Routing validation failures to a human

To send validation failures to a person, set the failure action to Warn and add a downstream Review node configured with the Validation failures pause mode. Only field-attributable failures (Rules mode) populate the field list. Schema, Endpoint, and AI failures produce warnings but no per-field paths, so route those with Fail or a manual review trigger instead.

Expression rule using template braces

Inside an expression rule, write subtotal + tax = total, not {{subtotal}} + {{tax}} = {{total}}. The pipeline editor flags template-style expressions at save time.

The pipeline editor checks rule operators and expression types against the upstream schema at save time. A rule that applies gt to a non-number, sum() to a scalar, or references a field that doesn’t exist surfaces as a validation error on the node before you can save. Applying sum(), avg(), min(), or max() to a list of non-numbers (a plucked text or yes/no field) is likewise flagged before save, whereas count() over the same list is allowed.

Extract action

Extract structured data before validating

Review action

Add human review for failed validations

Transform action

Transform validated data to a delivery format

Route action

Route data based on conditions before validating

​When to use validation

​Failure actions

​Modes

​Rules

​Operators

​Expression rules

​Schema

​Endpoint

​Script

​AI

​Output format

​Inputs and outputs

​Common pitfalls

​Related

Extract action

Review action

Transform action

Route action

When to use validation

Failure actions

Modes

Rules

Operators

Expression rules

Schema

Endpoint

Script

AI

Output format

Inputs and outputs

Common pitfalls

Related