Scheduling and triggers
Run pipelines hourly, daily, or on demand.
A pipeline that runs once is a migration. A pipeline that runs on a schedule is a product. This article covers the scheduling options and when to use each.
The three schedule types
Manual only
No schedule. The pipeline runs when a human clicks run or an automation triggers it. Use this for:
- Ad-hoc reloads.
- One-off migrations.
- Pipelines that are part of a larger automation flow (the automation handles timing).
Recurring — simple presets
The scheduler offers the common cadences as one-click presets:
- Every 15 minutes
- Every 30 minutes
- Hourly
- Daily (pick the time)
- Weekly (pick one or more days, plus the time)
- Monthly (pick the day of the month)
Most pipelines land on one of these. They cover the vast majority of real jobs.
Recurring — custom cron
When the presets don't fit, switch to advanced mode and write a cron expression. The scheduler accepts Quartz 6-field cron — the fields, in order, are second, minute, hour, day-of-month, month, day-of-week:
# Every day at 10:00
0 0 10 * * ?
# Every 15 minutes
0 0/15 * * * ?
As you type, the scheduler parses your expression back into plain English like "Daily at 10:00" so you can confirm your intent at a glance. If the summary doesn't match what you meant, the expression is wrong — fix it before saving.
Timezone
Schedules run in the timezone you choose. The scheduler has a Timezone picker that auto-detects your local zone on first use, and lists each zone with its current UTC offset so you can confirm you've got the right one. Whatever you pick is stored with the schedule and used to evaluate it — no separate cluster-timezone surprise to account for.
Daylight saving still shifts wall-clock time the way it does everywhere, so for anything sensitive, double-check the next-run time the scheduler previews.
Triggers from automations
An automation can run a pipeline as one of its steps, which is how you express orchestration the scheduler alone can't — running a pipeline, then a SQL script, then another pipeline, with later steps gated on earlier ones finishing.
See What automations do for the full picture. The short version: use schedules for simple recurring work, and automations when there's logic between multiple steps.
Pausing and resuming
Each pipeline has an active / inactive toggle. Toggling it off:
- Stops future scheduled runs.
- Does not cancel a run that's already in progress.
- Is free — you can leave a pipeline inactive indefinitely.
Turning a pipeline back on won't back-fill missed runs. A Delta artifact picks up from wherever its watermark left off; a Snapshot artifact simply reloads fresh on its next run. If you need to re-pull from a specific point on a Delta artifact, adjust its watermark before re-enabling.