One pipeline, every repo's own tests

The last post showed how one global workflow can run the same CI on every repo in your org. That solves the org’s half of the problem. This post is the other half: how each repo plugs its own tests into that shared pipeline, and how the org pipeline waits for them.

The problem: a shared pipeline is blind to each repo’s tests

The org pipeline knows how to build and deploy. What it does not know is how any one repo tests itself — a service with a Postgres integration suite, a library with a fuzz run, a frontend with a Playwright pass. Those live in the repo, and they differ from repo to repo.

Today you have two bad options. You let the repo’s own CI run independently, so the org’s deploy step and the repo’s tests fan out off the same push and race — the deploy can ship before the tests finish. Or you copy each repo’s test steps up into the org pipeline, and now the org pipeline is a pile of per-repo special cases that drifts the moment a repo changes how it tests.

The frame is right — one pipeline, every repo — but it has no extension point. The org owns the frame; each repo needs to fill in its own slot, and the frame needs to wait for it.

The move: invoke the repo’s own workflows and wait

invokeSource is that extension point. A job in the org pipeline calls it, and instead of running steps on an agent, the job emits a named event at the repo that triggered the pipeline and waits for every workflow there that opted in. Each opted-in workflow that fires becomes a job in the org run’s own graph — so you watch the repo’s tests in the same run as the deploy, and the downstream jobs gate on them.

Here is the org side. The repo-tests job hands control back to the source repo, and deploy needs it, so the deploy runs only after the repo’s tests pass:

export default workflow('org-ci-with-repo-tests', {
  on: [push({ repos: ['myorg/*'], branches: ['main'] })],
  jobs: [
    // Hand control to the source repo: emit `org.repo-tests` there and gate on
    // every workflow that subscribes. Required by default.
    job('repo-tests', { invoke: invokeSource('org.repo-tests') }),
    // Ships only after the source repo's own tests reported success.
    job('deploy', {
      needs: ['repo-tests'],
      runsOn: ['kici:os:linux'],
      run: async ({ $ }) => {
        await $`./deploy.sh`;
      },
    }),
  ],
});

And here is a repo filling the slot. This workflow lives in an application repo, not the org pipeline. It subscribes to the org’s event with kiciEvent, so when the org pipeline invokes the repo, this runs and the gate waits for it:

export default workflow('repo-tests', {
  on: [kiciEvent({ name: 'org.repo-tests' })],
  jobs: [
    job('unit', {
      runsOn: ['kici:os:linux'],
      run: async ({ $ }) => {
        await $`pnpm test`;
      },
    }),
  ],
});

That is the whole contract. The org names an event; a repo opts in by listening for it. Add the one file and your tests are in the org pipeline; the org pipeline never had to know they existed.

Required by default: a repo that forgot fails loud

Here is the part that matters at three in the morning. What happens when a repo never wired up its tests — no subscriber, nobody listening for the event?

invokeSource is required by default. Zero subscribers is a failed gate, not a pass. The deploy job needs the gate, so it does not run. A repo that quietly dropped its test workflow does not sail through the org pipeline green; it stops, and you find out because the pipeline is red, not because production is.

This is the opposite of the racing setup, where a repo with no tests looks exactly like a repo whose tests happened to pass. Here, “no tests ran” is a distinct, loud outcome.

Opt out on purpose with optional

Required-by-default is the safe default, but some repos genuinely have nothing to run — a docs-only repo, a config repo. For those, the org pipeline sets optional: true on the gate:

job('repo-tests', { invoke: invokeSource('org.repo-tests', { optional: true }) });

Now zero subscribers is a green skip, and the downstream still runs. The difference is intent: with optional, “no tests” is a decision the org made, not a repo that forgot. You choose which repos are allowed to have nothing to say.

Go dynamic: evaluate the repo, invoke what applies

The gate can also be generated at runtime. A DynamicJobFn runs with the source repo checked out, so it can read the repo and decide whether to invoke at all. This variant reads the repo’s package.json and adds the gate only for a repo that declares a ci:test script — so the org invokes each repo’s tests exactly where they exist:

const gateWhereTestsExist: DynamicJobFn = async ({ sourceRepo }) => {
  const root = sourceRepo?.path ?? '.';
  const pkg = JSON.parse(await readFile(join(root, 'package.json'), 'utf8'));
  const declaresTests = 'ci:test' in (pkg.scripts ?? {});
  // A repo that declares its own tests gets an invoke gate; one that does not is
  // left un-gated rather than failed.
  return declaresTests ? [job('repo-tests', { invoke: invokeSource('org.repo-tests') })] : [];
};

export default workflow('org-ci-matrix-repo-tests', {
  on: [push({ repos: ['myorg/*'], branches: ['main'] })],
  jobs: [gateWhereTestsExist],
});

This is plain TypeScript against the tree on disk, so the rule can be anything you can express in code: gate the repos that ship a Dockerfile, invoke a different event for a repo tagged one way versus another, read a repo-local config file and branch on it. The org pipeline decides per repo what to summon, and the repos decide what to answer with.

Reading the results

An invoked run can report outputs, and they cross back to the downstream job as ctx.needs['repo-tests'].result — coverage numbers, a build id, whatever the repo’s tests emit. Secret outputs stay masked and never cross the boundary, so a repo can report a token to its own steps without leaking it into the org pipeline. The org gets exactly what the repo chose to publish, and nothing it did not.

The whole picture

The org owns the frame and each repo fills the extension point. The org pipeline runs everywhere, calls back into each repo for the tests only that repo knows how to run, and waits — required by default, so a missing test suite is a red pipeline instead of a silent deploy. Pair it with the global-workflows model and you get one pipeline that adapts to every repo without carrying a single per-repo special case.

If you want to see how this lines up against the way other tools share CI, the comparison pages go through it point by point.