Help › Guide
Reqlyn in CI
Reqlyn 0.5.1 is not released yet. This help describes it ahead of release.
A pipeline runs the reqlyn command line in the Reqlyn CI image: an operation's regression cases (reqlyn test), a data-driven run (reqlyn run --data), a mock for the job to call (reqlyn mock), and runs on a server (reqlyn remote). The image holds the program and nothing else: no secret, no token and no server's address. The repository has a GitHub Action (ci/github) and a GitLab template (ci/gitlab/reqlyn.gitlab-ci.yml) that run it by a name you set.
The image
Build it on a computer that builds Reqlyn, from the repository's root:
powershell -File deploy\ci\build.ps1 -Image registry.example/reqlyn-ci:0.1.0
docker push registry.example/reqlyn-ci:0.1.0
The build vendors every dependency first from what Cargo already holds (cargo --offline vendor), so the container needs no network and no SSH key for the shared crates; build the workspace once before. The image is Debian with the program at /usr/local/bin/reqlyn, a shell, and REQLYN_DATA_DIR=/tmp/reqlyn, so each job starts with an empty data folder.
Signing it
The image is signed in its registry, by digest, with a cosign key made once on the build computer:
powershell -File deploy\ci\signing-key.ps1
powershell -File deploy\ci\sign.ps1 -Image registry.example/reqlyn-ci:0.1.0
signing-key.ps1 makes the key pair with a password of its own; the private key and the password are kept in Windows Credential Manager (dev.reqlyn.ci-signing/key and dev.reqlyn.ci-signing/password), never on disk, and the public key is deploy/ci/cosign.pub in the repository. It refuses to replace a key that is kept. sign.ps1 signs the pushed image's digest; the signature is stored beside the image in that registry and nothing is sent to a public transparency log. Anyone with the repository checks it before running the image:
cosign verify --key deploy/ci/cosign.pub --insecure-ignore-tlog=true registry.example/reqlyn-ci:0.1.0
Both scripts need cosign on the path.
Secrets in a job
A CI container has no secret store. There, a secret variable's value is read from the job's environment: REQLYN_SECRET_ and the variable's name in capitals, each character that is not a letter or a digit _. A secret variable api-key of any environment is REQLYN_SECRET_API_KEY. It is read only when the operating system's secret store cannot be reached, as in the image, and is never written anywhere. Only secret variables are given so: a credential's secret, a client certificate's key and the rest are not. Give the value from the pipeline's own secrets (a GitHub secret, a masked GitLab variable), never on a command line. The remote commands take REQLYN_SERVER and REQLYN_TOKEN the same way.
A folder workspace in the checkout (Requests) is named with REQLYN_FOLDER; its environments' files name their secret variables with no value, and the job gives each.
GitHub Actions
Copy ci/github into the repository (or refer to it where it is kept), and give it the image's name and the commands:
jobs:
cases:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./ci/github
with:
image: ${{ vars.REQLYN_IMAGE }}
run: |
reqlyn arazzo save workflows/device.arazzo.yaml
reqlyn test Device --junit reqlyn-cases.xml
| Input | For |
|---|---|
image |
The CI image, by the name it was pushed as |
run |
The commands, as a shell script run in the image; the step fails when one ends other than 0 |
folder |
A folder workspace in the checkout, relative to it, given as REQLYN_FOLDER |
The step runs the image with the checkout mounted as its working folder, as the runner's own user, so a report a command writes is in the checkout. Each REQLYN_SECRET_<NAME>, REQLYN_SERVER and REQLYN_TOKEN in the step's env is passed into the container by name. The runner needs Docker, as ubuntu-latest has.
GitLab CI
Copy ci/gitlab/reqlyn.gitlab-ci.yml into the repository, include it, set REQLYN_IMAGE, and extend .reqlyn:
include:
- local: ci/reqlyn.gitlab-ci.yml
variables:
REQLYN_IMAGE: registry.example/reqlyn-ci:0.1.0
cases:
extends: .reqlyn
script:
- reqlyn arazzo save workflows/device.arazzo.yaml
- reqlyn test Device --junit reqlyn-cases.xml
The job runs in the image; a report named reqlyn-*.xml is kept and shown as the job's tests.
A gate: rows against a mock, then load
A job can serve a mock from the repository's own description, run an operation once for each row of a data file against it, then load it, held to thresholds, with nothing else to reach:
- uses: ./ci/github
with:
image: ${{ vars.REQLYN_IMAGE }}
run: |
reqlyn mock workflows/device.inventory.openapi.yaml --port 8787 > /tmp/mock.log 2>&1 &
for i in 1 2 3 4 5 6 7 8 9 10; do grep -qs 127.0.0.1 /tmp/mock.log && break; sleep 1; done
reqlyn arazzo save workflows/device.arazzo.yaml
reqlyn run Device --data workflows/devices.csv --var inventory=http://127.0.0.1:8787 --junit reqlyn-data.xml
reqlyn load Device --callers 4 --seconds 10 --input '{"deviceId": 4831}' --var inventory=http://127.0.0.1:8787 --p95-under 250 --errors-under 1 --junit reqlyn-load.xml
The mock is served inside the job's container, so the load measures Reqlyn and the mock, not an API: it shows a threshold holds and that the job fails when it does not. Against a real test API, give its address in place of the mock's.
What is checked
bash deploy/ci/check.sh <image> runs the fixture repository (fixtures/ci) as the Action and the template do: its cases hold (0); its workflow broken so an output reads the wrong field fails (1, the report saying CASE_DIFFERS); the template's job runs in the image's shell; and a folder workspace's run whose secret variable inventory_key is not given fails (1) and, given as REQLYN_SECRET_INVENTORY_KEY, succeeds (0), its mock serving in the same job; the gate above holds (0, three rows and the load within both thresholds), and fails (1) with its mock slowed past the p95 threshold (--delay 300), saying Over the threshold p95 under 250 ms. GitHub and GitLab themselves did not run it.
Help for Reqlyn 0.5.1. All help topics