We are always working to get better. Building as quickly as Patrii has from the start is impressive, and it leaves an equally impressive pile of tests to run by hand. At the beginning we could check everything in a few minutes. But the more XaaS we added (everything as a service), the more we found ourselves slowing down in order to keep delivering quality services.

We want to keep a human side to our testing, and still, doing every click ourselves, filling in the forms, going over the same things from top to bottom, had become repetitive.

Patrii has used unit tests, and builds tested and shipped automatically through CI/CD, from day one. That already kept regressions to a minimum. But saying "the product is now stable and ready for production" took longer and longer: the front end tests and the back end tests were each fine on their own, and nothing actually tested the agreement between the two, the way our customers use our services.

So we decided to work on a tool of our own, one that would let us write routines and run tests shaped the way a person tests, while capturing evidence and checking assertions.

What that changes, concretely

A routine is the ordered list of what a person would do, with a stated purpose and a claim about what each step proves. It arrives through the menu, like a customer, because a customer does not type the address of an interior page. Every step is photographed, together with the console output it produced. And whatever the evidence cannot settle goes to a review queue, where a person decides with the picture in front of them.

A QA Wave wave on itself: 77 agents, 77 finished, each tile an agent and its latest frame. Two are red at the moment of capture, which is exactly what the tool is for.

The same estate, twice: on the left QA Wave's own 77 routines, on the right the Patrii Cloud console's 120. A zone per subject, a dot per routine, a colour per verdict. “asserts nothing” is its own colour, because a routine that passes without proving anything is not a routine that is doing well.

The repetition is gone. The judgement stayed. That was the whole point.

And one thing we had not planned for: the engine tests itself, with itself. The routines that check its own interface are generated from its own menu, so adding a page without regenerating its routines makes the check fail. That is the rule we hold ourselves to before offering it to anybody else.

630 routines, 18,591 runs, and what it costs: a wave is bound by its slowest routine, here 1,929 seconds. The month's consumed units are there too, with the month-end projection.

The side effect we did not see coming

Because every routine takes screenshots and explains what it is doing and in what order, the artifacts of a run are already an illustrated procedure. Our own customer guides are wired to it: a screenshot slot names a key rather than a path, and the key always resolves to the newest capture of that page. A picture cannot quietly go out of date either, because the thing that produces it is the thing that breaks: the frame is captured by a step that asserts something, so the moment the page changes enough to matter, the routine fails and no page is published.

The Patrii Cloud console, photographed by a routine during a wave. This is the kind of image that feeds the screenshots in our customer guides: taken by a test rather than by a person, and retaken on the next wave.

We were not trying to solve the documentation problem. It turns out an honest test suite looks a great deal like an up-to-date manual.

And where it runs, because the question comes up immediately

The moment you mention screenshots, the real question arrives: who else can see them? All of this runs on Patrii Cloud, our own sovereign cloud. And sovereign means the whole chain, from who owns it to the code that runs it: a Canadian company we own ourselves, facilities here, our hardware, our code, and open source for the rest.

The order matters. Jurisdiction follows the provider, not the rack: a server can sit in Montréal and still answer to Washington if the company operating it is American. That is why ownership comes first in that sentence. No American provider holds your screens or your evidence.

A preview, for now

A closed group is already using it. Access will open to all our customers afterwards.

We are pleased to give you a short preview of a service that will be added to our platform for our customers before long. No date, no price, no promise: just the tool we use every day, and the intention to make it available.

Every image on this page was taken by QA Wave itself, during real waves. Cropped for layout, never staged. It is the same mechanism that would feed your documentation.