Help › Tutorials

Tutorial 4: CI and the command line

Reqlyn 0.5.1 is not released yet. This help describes it ahead of release.

Operations kept in a repository can be checked on every change, with no desktop and without calling any API: the documents checked and compiled, and their regression cases replayed. Against a server, a pipeline can run a published version once per build, and ask whether a newer upstream description would break anything published.

You will need the files exported at the end of tutorial 2 (device-check.arazzo.yaml, its two descriptions and device-check.cases.yaml), and, for the last two parts, the server of tutorial 3. Every command below was run as written; the GitHub Actions file at the end is an example of putting them together and was not itself run on GitHub.

1. A repository

repo/
  workflows/
    device-check.arazzo.yaml
    device-check.inventory.openapi.yaml
    device-check.compliance.openapi.yaml
    device-check.cases.yaml

A pipeline starts with no data folder, so give it an empty one of its own, and a secret-store name of its own, for every job:

export REQLYN_DATA_DIR=$(mktemp -d)
export REQLYN_SECRET_SERVICE=dev.reqlyn.ci

2. Check and compile

for f in workflows/*.arazzo.yaml; do reqlyn arazzo check "$f" || exit 1; done
reqlyn arazzo compile workflows/device-check.arazzo.yaml > /dev/null

check ends with 1 when a document is wrong, naming each problem at its place; compile ends with 1 when it names something Reqlyn cannot run. Neither sends anything, and neither needs a variable.

A change that reads a field the description does not document is refused here. For instance, an output role: $response.body#/status gives:

workflows[0].outputs.role: the source description does not say that "status" in the response of "device" is a string, an integer, a number or a boolean, which Reqlyn needs to know

3. Replay the cases

reqlyn arazzo save workflows/device-check.arazzo.yaml
Kept 1 case(s) from workflows\device-check.cases.yaml.
reqlyn test "Device check" --junit cases.xml
passed	pe02
1 of 1 cases hold.

save stores the document as an operation in the job's data folder, with the cases beside it. test replays each case through the engine with nothing sent, and holds the output to the one the case kept; --junit writes a report for the pipeline to show:

<testsuite name="Device check" tests="1" failures="0" errors="0" skipped="0" time="0.000">
  <testcase classname="Device check" name="pe02" time="0.000"/>
</testsuite>

To replay a pack file other than the one kept with the document: reqlyn test "Device check" --pack workflows/device-check.cases.yaml.

Now break it, as a careless change might: make the workflow's hostname output read the device's role.

    hostname: $steps.device.outputs.role

In a fresh data folder:

reqlyn arazzo save workflows/device-check.arazzo.yaml
reqlyn test "Device check" --junit cases.xml
FAILED	pe02
  hostname: "PE"; expected "pe02.pitb02-pa"
0 of 1 cases hold.

It ends with 1, and the report says where:

<testcase classname="Device check" name="pe02" time="0.000">
  <failure type="CASE_DIFFERS" message="hostname: &quot;PE&quot;; expected &quot;pe02.pitb02-pa&quot;"/>
</testcase>

A case holds the output field by field. When a field changes on every run (a time, a request id), leave it out: reqlyn test "Device check" --ignore pe02 checkedAt.

A replay also tests each step's success criteria against what the case kept: each response's body, its status, and the headers the criteria read. Tighten the device step's pattern to '^cr[0-9]+\.', and a case kept from a live run of pe02 no longer holds:

FAILED	pe02	Step "device": the response (200) does not meet its criterion "hostname" matches /^cr[0-9]+\./.

A case kept by a Reqlyn from before October 21, 2026 kept bodies only. It still replays, and says so: Criteria not tested at "device": the case kept no status or headers there …. Keep it again from a live run (reqlyn run … --keep-case pe02) to have its criteria tested.

Against a test API

A job can also run a document against an API it can reach, with nothing stored and no desktop. A description's server default says where an API is; an exported description gives none (it writes default: '', as the address is an environment's), so give each with --var for that run:

reqlyn arazzo run workflows/device-check.arazzo.yaml --input '{"deviceId": 4831, "check": "ci-1842"}' --var inventory=http://127.0.0.1:8787 --var compliance=http://127.0.0.1:8787

It prints the output and ends as any run does.

A secret is never given with --var: a name such as api_token is refused, and nothing runs.

4. Run a published version once per build

The remote commands take the server and the token from the environment; in a pipeline, both come from its secrets, never from the command line:

export REQLYN_SERVER=https://reqlyn.example
export REQLYN_TOKEN=…   # from the pipeline's secret store
reqlyn remote run <id> 1 --environment lab --input '{"deviceId": 4831}' --key ci-build-1842

It waits for the run and ends as a local run does: 0 whole, 1 failed, 2 not run, 3 partial. The key makes it safe to retry the job: asked again with the same key and input within a day, it gives the same run and makes no other. Asked with the same key and another input, it is refused:

That idempotency key was used with another input or environment. A key belongs to one submission: use a new key for a different one.

and ends with 2. Use the build's number in the key.

5. Will a newer upstream description break anything?

When the inventory team publishes a new OpenAPI description, hold it against every published version whose source names it, before it is deployed. Here inventory-v2.openapi.yaml is the tutorial's inventory description with hostname renamed name:

reqlyn remote impact inventory-v2.openapi.yaml --source inventory
01a15ecd-… 1, active, activated in lab
  used by: named NOC dashboard (network-ops); seen calling first administrator
  device (device)	hostname: no longer documented in status 200	read by this version

It ends with 1, because a run could fail or give something else. --source is the source description's name, as the operations' Arazzo sources name it. An address is fetched by the server, through its allowlist; a file is sent from the job. Nothing is kept or changed.

6. Putting it together

The CI image, with its GitHub Action and GitLab template, runs these commands with nothing installed on the runner. A GitHub Actions job with reqlyn on its path (installed from the Linux archive, say):

name: operations
on: [push, pull_request]
jobs:
  check:
    runs-on: ubuntu-latest
    env:
      REQLYN_SECRET_SERVICE: dev.reqlyn.ci
    steps:
      - uses: actions/checkout@v4
      - name: Check and compile every workflow
        run: |
          export REQLYN_DATA_DIR=$(mktemp -d)
          for f in workflows/*.arazzo.yaml; do reqlyn arazzo check "$f"; reqlyn arazzo compile "$f" > /dev/null; done
      - name: Replay the regression cases
        run: |
          export REQLYN_DATA_DIR=$(mktemp -d)
          reqlyn arazzo save workflows/device-check.arazzo.yaml
          reqlyn test "Device check" --junit cases.xml
      - name: Run the published version
        if: github.ref == 'refs/heads/main'
        env:
          REQLYN_SERVER: ${{ secrets.REQLYN_SERVER }}
          REQLYN_TOKEN: ${{ secrets.REQLYN_TOKEN }}
        run: reqlyn remote run ${{ vars.DEVICE_CHECK_ID }} 1 --environment lab --input '{"deviceId": 4831}' --key "ci-${{ github.run_id }}"

A token for a pipeline should reach only what it runs: reqlyn-server token issue "Network" "ci" operator --operations <id> --environments lab.

What you did

  • Documents checked and compiled with nothing sent, refusing what Reqlyn cannot type or run.
  • Regression cases replayed in a fresh data folder, with a JUnit report, and a broken mapping caught.
  • A published version run once per build, safe to retry with an idempotency key.
  • A newer upstream description held against everything published, before it goes out.

Every command and its options: the command line, and reqlyn help <command>.

Help for Reqlyn 0.5.1. All help topics