Self-learning

Self-learning is an opt-in, per-workflow feature. When it is on, every approved review of an Extract node is kept as an example, and later extractions in the same workflow consult the most similar past documents before they read the new one. The workflow’s Learning tab shows what the workflow has learned and whether it is helping. Self-learning is not part of any plan. It is granted per organization; contact us to have it turned on for yours. Without the grant the workflow editor shows no Learning tab and no self-learning action, and the Api refuses every self-learning request. Self-learning is off per workflow until you turn it on. Open a workflow, then Enable self-learning from the workflow editor’s More actions menu. That turns capture on with the workflow’s stored settings and opens the Learning tab, where Settings holds every dial.

​

How it works

1. A reviewer approves a review task for an Extract step.
2. Ingestly keeps that extraction as a learning example: the values the model extracted, the values the reviewer approved, and every change between them, including fields the reviewer removed or added.
3. On later runs of the same workflow, the most similar past documents are handed to the model as reference data before it extracts.
4. The model still reads every value from the document in front of it. The examples show it what a document of this shape looks like after a person has corrected it; they never supply values of their own.

Examples belong to the Extract node that produced them, and they are never shared with another workflow. Corrections feed a second, slower loop as well. Ingestly distills repeated corrections into short written instructions that you approve before they reach the model, so a correction the reviewers keep making becomes a rule instead of an example.

​

What reviewers see

When self-learning is on, the review panel footer says “Self-learning is on: approved values from Extract steps become examples for later runs.” so a reviewer knows the approval trains the workflow. To keep one review out of the bank, tick Don’t learn from this review in the approve dialog. The review is approved and the run resumes exactly as usual; only the example is skipped. An approved review that was excluded reads “Excluded from learning” in the same footer, so the decision stays visible afterwards. A review whose Review node reads a custom source or limits which fields the reviewer can edit never becomes an example either, with or without the checkbox: the reviewer vetted only part of the document, so treating the rest as confirmed would teach the workflow values nobody checked.

​

Holdout

A holdout reserves a share of runs that extract with no learned context at all, so the two can be compared. Runs that received learned context are treatment runs, holdout runs are the control. Set the share with Holdout percent. It is 0, meaning off, by default. Turn it on when you want a number you can trust for whether learning is helping, and leave it off when you want every run to use everything the workflow knows.

​

The Learning tab

Once self-learning is on, the workflow editor carries a Learning tab beside Builder, Documents, Runs, and Insights. Its title line shows whether learning is on or off, a range control that sets the window every statistic covers, Settings, which opens the settings sheet, and a More actions menu with Reset Learning. Below the title line the tab has four sections, Overview, Fields, Instructions, and Examples. Reading the tab needs the Core.Workflow.Read permission; the actions and the row actions that change the bank appear only for roles holding Core.Workflow.Update (see roles and permissions). Turning learning off in Settings keeps the tab open until you leave it; it is gone from the tab bar the next time you open the workflow, and Enable self-learning is back in the More actions menu.

​

Overview

The headline numbers are: A rate never appears without its sample size. Compare the treatment and holdout figures only once both arms have enough reviewed documents to mean something. A daily chart plots reviewed and clean documents per arm across the window, so you can see the trend rather than one aggregate. Below the chart, the bank table lists one row per Extract node with its live and archived example counts, how many live examples are not embedded yet, how many vendors they cover, an Instructions count reading approved then proposed, when the node last captured, and a status: Retrieval matches on an example’s embedding, so an example counted under Not embedded is skipped until nightly maintenance embeds it from the document’s text.

​

Fields

The Fields section breaks accuracy down by field path. Each row shows how often the field was reviewed, how often it was corrected, removed, or added, its first-pass accuracy per arm with the sample sizes, and how many of its corrections are contested. Fields with the most changes come first, so the worst field is the first one you read. A contested correction is one that contradicts another: the same original value was corrected two different ways on the same field. Nightly maintenance marks them, and a contested correction stops reaching the model. Pick a node to scope the table to one Extract node, and click a row to chart that field’s daily trend across the window.

​

Instructions

An instruction is a short rule distilled from the corrections reviewers made, such as “Take the invoice number from the stamp in the header, not from the reference line.” Where an example shows the model a corrected document, an instruction tells it what to do, so a correction reviewers keep repeating is handed over once instead of every time. The Instructions section lists them. Pick a node to scope the table to one Extract node, and use the status toggle to move between Proposed, the default, Approved, Rejected, Retired, and All. Each row shows: Each row carries these actions:

• Approve puts the instruction to work on later extractions.
• Reject leaves it out. The Reject Instruction dialog takes an optional reason, so whoever reads the row later knows why.
• Retire takes an approved instruction back out again. Approved rows only.
• View evidence opens a panel with the rationale and the corrections the instruction was distilled from, so you can check it against what reviewers actually changed.

The header carries two more:

• Add Instruction writes one yourself: choose the Extract node, an optional field path, the instruction text of 12 to 400 characters, and an optional rationale. A manual instruction is approved immediately and attributed to you.
• Distill Now distills on the spot, for the selected node or for every Extract node when none is selected. It reports how many instructions were proposed, or why a node was skipped: learning is off, the node is not an Extract node, it has no schema, it has too few new corrections, or no field has enough support.

Nightly, Ingestly distills every enabled workflow’s Extract nodes that gathered at least 10 new corrections since their last proposal, with at least one field corrected 3 or more times across at least 2 different documents, so one document with a whole column wrong does not ground a rule on its own. Nodes below those thresholds are left alone until they collect more. Distillation is not billed. The gate is where an evaluation set checks an instruction before you trust it. It reads Not gated until an evaluation set covers the node, which means the gate has no verdict to offer and the judgement is yours. A proposal never reaches the model until you approve it. Approved instructions are added to the extraction prompt after the node’s Extraction hints, as a ### LEARNED INSTRUCTIONS section, and approving, rejecting, or retiring one changes that prompt exactly once, so the next run works from the new set. Only instructions written against the node’s current schema are used. Change the schema and the older ones read Stale: they stay on the page for the record, and they no longer reach the model. Ask the assistant to explain a proposed instruction. On the Instructions section it can read any instruction listed there, including the one you have open, and the corrections it was distilled from, so it answers why the instruction was proposed, what evidence backs it, and whether to approve it. It describes what reviewers changed rather than quoting values out of your documents, and the decision stays yours: the assistant never approves, rejects, or retires an instruction.

​

Examples

The Examples section lists the bank itself, live or archived, with the node, the vendor, the arm, the corrections the reviewer made, the page count, when it was captured, and its status. Click a row to open the drawer, which shows every correction alongside the extracted payload and the approved payload. A vendor that came from the email sender rather than from the Vendor key field carries a muted from sender hint, in the table and in the drawer, so you can tell the two groupings apart.

• Archive takes an example out of later extractions without deleting it.
• Restore puts an archived example back. Only examples archived by a user for the current schema can be restored: an example archived automatically, or one collected against an older schema, stays archived.

​

Settings

Reset Learning archives every live example of the workflow, or of one Extract node when you scope it, and keeps the accuracy history so the numbers you have already collected survive. Because it empties the bank, the Reset Learning dialog asks you to type the workflow name before it proceeds.

​

Schema changes

Examples are collected against the schema the Extract node had at the time. Change that schema and the examples collected against the previous one are archived automatically, because values that no longer fit the fields would mislead the next extraction rather than help it. The node starts collecting again with the next approved review, and its accuracy history is kept. A review of a run that extracted with the previous schema and is approved after the change is still recorded, for the history, but its example stays archived and never reaches the model. Learned instructions are treated the same way but are never thrown out: an instruction written against the previous schema stops reaching the model and reads Stale on the Instructions section, so the decision you made about it stays on the record.

​

Maintenance

Every night Ingestly tidies the bank of each workflow that has self-learning on, so it stays the size the settings promise and nothing misleading reaches an extraction. None of it is billed, and none of it needs you. The nightly pass runs these steps per workflow, in order:

1. Archives what a schema change orphaned. Live examples collected against a schema the Extract node no longer has, and examples whose node has left the workflow, are archived. See Schema changes.
2. Enforces the caps. Live examples beyond Max examples per vendor within a node are archived first, then those beyond Max examples per node. Examples the reviewer changed nothing in go first, then the oldest; the newest survive. An archived example reads Evicted on the Examples section.
3. Marks and clears contested corrections. When the same original value was corrected to two or more different values on one field in the last 90 days, every correction in that disagreement is marked contested. Contested corrections are hidden from retrieval and from distillation, so a contradiction never becomes a rule. The example itself stays live, and the Fields section shows the contested count. When the disagreement is gone from the last 90 days, because the conflicting examples were archived or aged out, the corrections still inside that window are cleared again; older contested corrections stay hidden.
4. Embeds examples that have none. An example captured before its text was available has no embedding, and retrieval matches on the embedding. The pass embeds it from the document’s text. An example whose document has been deleted, or whose document carries no text, is skipped for good and no longer counts as Not embedded; an embedding that fails is tried again the next night.
5. Trims field statistics. Daily field statistics older than 400 days are deleted. No surface reads past 90 days, so this changes nothing you can see.

​

Personal data

PII handling decides what a captured example is allowed to store: The policy applies at capture time, so what the example holds is what the Examples drawer shows. Changing the mode does not rewrite examples that were already captured.

​

Run detail

A run’s Extract step carries a badge saying what learned context it received: The badge appends ”, instructions applied” when the extraction also received approved instructions, so Learned: 3 similar documents, instructions applied means the run used both. Hover the badge to read what it was given. A holdout run receives neither examples nor instructions.

​

Workflow insights

A workflow’s Insights tab carries a Self-learning card with the live example count, documents reviewed, an Approved instructions row, treatment and holdout first-pass accuracy with their sample sizes, and the clean-approval rate. Open self-learning switches to the Learning tab. When the workflow has self-learning off, the card reads “Self-learning is off for this workflow.” and Open self-learning still opens the Learning tab, where the settings live. An organization without the self-learning grant sees the card without the button.

​

Related