Over the past three months I built a working testbed of an AI-native product platform: 32 agents, over 100 nodes in the knowledge base, an automated pipeline from Discovery to Quality Gate, and my own agent environment with memory, skills and multi-model reviews. Every number here comes from this fintech testbed, measured in April 2026.
Testbed architectureQuality GatesBridgesFigma + MCP
Where I got it wrongNot working aloneWhat's nextBottom line
This is no longer the future. The testbed lives in a repository: you can connect an agent to it and ask a question.
This article covers how it works, why it's built exactly this way, and where I got it wrong. If you're running an AI transformation in a large company, there is more practical value here than in the manifestos from Dorsey and Anthropic.
Act 1 · Why, and what shifted
The context
A large product organization. Hundreds of engineers, dozens of teams. A regulated industry, several jurisdictions. A settled Scrum process with SAFe elements, years old, and it works — as long as tasks are predictable and people understand what the next team over is doing.
The problem is not Scrum. The problem is that it was designed for a world where the main constraint was coordinating people. The constraint has changed: between hypothesis and delivery sit 10 days of pure Discovery and Research that nobody counts as part of time-to-market.
When people say "we need a one-day time-to-market", they mean development speed. But the real cycle is: 10 days of research, 7 days of development, 3 days of release. Doubling code generation speed cuts the cycle from 20 days to 17 — not to one.
That was the entry point. Hard efficiency numbers (including Discovery time-to-market) will come as a separate cut, so the story and the metrics don't blur into one noise. The follow-up has already grown into two articles: what such a loop is built from today (in Russian) and how a single instruction makes people and agents parts of one system (in Russian).
The key shift: an agent team as a boost, not a replacement
The first version of my strategy, like most writing on this topic, was framed around replacement. Agents replace analysts. Agents replace PMs. Agents write code, a human reviews.
That framing is wrong. In one of the key project conversations I got an objection that flipped the approach:
"You're giving teams a way to boost themselves. Give them that — they take it. Give them a competitor — they start working with it from a position of resistance."
This is not about the "people first" slogan. Technology takes root in a large company when it strengthens the process owner instead of taking control away.
The practical takeaway: the interface to agents should be the existing task management system — not a separate LLM chat and not a git repository.
A developer files a task in the familiar tool. Then chooses: hand it to people, to agents, or to a mixed team. The process stays familiar but gains a new layer of capability.
This is the soft line. It doesn't ask a senior to rewrite their identity. It hands them a new tool and lets them learn it gradually — or not learn it, and take the hit of falling behind those who do.
Three maturity levels of AI transformation
Without this scale, any AI-native conversation turns to mush. One person means autocomplete in the IDE, another means fully autonomous agents, a third means ChatGPT in UX research. All of them believe they're doing AI-first.
Most public cases, including Dorsey's Block, sit at level 2 with individual level-3 products. Anthropic, judging by public descriptions, is at level 2-3 in its own Claude development. Whether real businesses have reached level 3 is a good question: if any have, it's not all of them and not fully.
The critical point: human-in-the-loop is not the goal, it's a transitional state. If a human is needed at every step, the system hasn't matured yet. In a mature system, a human steps in where data runs out, an unconventional move is needed, or an ethical choice comes up. Automation should take the rest.
Act 2 · The testbed, hands-on
Testbed architecture
What I built in 100 days looks like this today.
The core is the World Model. Over 100 linked knowledge base nodes: regulatory requirements by jurisdiction, product architecture, business logic, personas of process participants, interaction protocols. This is Dorsey's "world model" — not as an abstract interface, but as a concrete knowledge base with a graph that agents query.
Agents — 32 of them, in three categories:
- Architects — build pipelines, control metrics, create automations.
- Operators — pair with a human on a specific stretch of work.
- Compliance agents — audit conformance to regulatory requirements.
The Discovery pipeline: Research → Design → Design Lead → AI Review → synthetic users → risk assessment. Design Lead is a separate layer between the Design Agent and AI Review: it encodes my taste and the rules for which details of a layout are valuable and which are noise.
Our AI Review runs through 7 lenses:
System evolution
Every repeating tool call (5+ times) gets packaged into a skill automatically. Repeating events become Python scripts — for cron events this saves up to 70% of tokens. Once a week an agent reads all the logs and forms skills on its own. Two levels: local (team) and company-wide.
Persistent shared memory
A local knowledge base plus my own scripts for RAG. Cognee and LightRAG turned out unstable at scale (tested on my home lab: 6,000 files, 100K chunks — both broke). I wrote my own chunking scripts; they run stable.
Self-healing loop
If code or a layout fails a Quality Gate, it goes back to the same agent for rework, or gets reworked automatically by a swarm of agents looping until validation. The architecture is taken from a public solution by Creo.
Every gate gets a double-check by a different model. I noticed that LLMs tend to lock onto one thought and spin it out endlessly. The fix is splitting into parallel sessions. If a double-check at a different temperature or with a different model gives a better result, the previous one counts as a hallucination and goes back for rework.
Quality Gates — the most expensive invisible problem
This is where I got the most painful and most precise lesson of the project.
The question was: "What in our quality metrics has actually gone down over the last three years?"
The answer: one metric. The number of vulnerabilities in dependencies — from a steady few hundred down to a few dozen. Why? Because it's the only stable Quality Gate that actually blocks deploys. Tests — hundreds. Automated tests — fewer. Failing tests — nobody cares. Teams deploy, tests fail, nobody stops.
It's not about tooling, it's about delivery culture: tests have to be built into the process, not live next to it.
Add AI to that culture without Quality Gates and you get quality problems amplified 10x. Because agents ship 5–10 times faster. If bad work used to go out five times a week, now it will be fifty.
Quality Gates are first of all an organizational question. They are hard stops that actually halt a deploy when something doesn't add up. They have to be in place before you scale up agent capacity.
My approach to quality gates is simple in formula and strict in discipline.
A gate is not a checkbox in Jira, it's a breaker switch. Pass, fail, or manual review — and every outcome has an owner, criteria and consequences defined in advance.
In an AI-native delivery pipeline this layer holds three things at once: quality, risk and trust. Not one. Three.
The hot path and golden path customer journeys are covered separately. Separately again — fintech domain risks: security, fraud, compliance, consent, localization, multi-brand consistency, accessibility, observability, operational safety.
I deliberately avoid an abstract "quality check". What's needed are concrete checks: reproducible, traceable, painfully clear — wherever a mistake hits the customer, the brand, or the regulatory perimeter.
Bridges across the gap
One of the most useful metaphors I picked up along the way:
"Executives create a gap between 'as is' and 'as it should be'. The job is to build bridges of 5–10 steps across that gap. As soon as the first person crosses without falling, the rest will follow."
This is the opposite of the standard AI transformation, which is often built as "20 teams at once, forced adoption, OKRs on metric X".
My path:
- Step 0. The first functional lead (QA, Frontend, Backend) packages their expertise as a set of skills for agents. That's already architect level.
- Step 1. One pilot team works with agents through the existing tracker.
- Step 2. The team has learned to write tasks so that agents can work with them. Feedback flows back into the World Model.
- Step 3. Quality Gates go into CI/CD. They start actually blocking bad deliveries.
- Step 4. Early-adopter teams join on the proven model. The first team teaches through the AI community.
- Steps 5–10. Scaling through engagement, not through orders.
Each step takes months. For a large organization it won't go faster, and that's fine.
Figma + MCP + Design System: one case within the big picture
The most tangible result so far is the design pipeline. My own domain expertise was patient zero.
The Design System is the binding layer between mockup and code. Figma variables are tied to code variables via MCP. Matching across theme variants is done: the agent sees which configuration is in play and uses the right palette and typography.
The AI reviewer walks through the 7 review layers, validates via QA, and adds the assembled components to Storybook. Storybook is the last mile before build tests and porting to different platforms.
Because of this, the task "build a new component for a new visual preset" is done by one agent with one human review — not three designers, two frontend engineers and one QA.
From the outside it looks like magic. From the inside it's methodical engineering work on connecting tools, accumulated over years.
Act 3 · Honest results
Where I got it wrong
Five points, honestly.
- I tried to build a parallel company instead of a team inside. For the first three months I built the testbed almost in isolation, with minimal contact with the core functions. The result: when I came to show it to key people, it worked — but it read as "Arthur built something interesting in the corner", not as "our working tool". I'm redoing this now: integrating into existing tools and processes, not in parallel.
- I overestimated how ready the seniors were. My hypothesis: a senior sees a working pipeline and switches to it, because it's more efficient. No. A senior is someone who spent years building their own system of culture and patterns, and asking them to change it on demand means asking them to give up a large part of their identity. Logic doesn't fix this. It takes time and the right motivation: here's the task, the deadline is a quarter, resources are thin. Saw by hand if you want — or pick up the power saw.
- I underestimated juniors and mid-levels. I took a teenager with zero experience and made him an agent operator on an AI-ready pipeline. Within a week he was thinking architecturally — building his own backlog, decomposing tasks, making his own small pipelines in neighboring domains. Youth = no resistance + no outdated expertise to defend. That is the main resource of an AI transformation, not the seniors. How a young mind adapts deserves an article of its own.
- I wrote the agents' context myself instead of through the owners of each competency. I wrote the Compliance, Security, QA and Risk reviewers — and the quality of each lens matched my own exposure to the topic, not the domain owner's expertise. I'm redoing this now: the agents stay, but each one is built in partnership with a specific functional lead, who then validates and refines it.
- For a long time I couldn't sell the project to the leadership team. I spoke the language of agents and architectures; the audience spoke the language of business metrics. It turned around the moment I ran the testbed live on a real task in a meeting and showed working Discovery in five minutes. "Ah, now I get it — give this to that person, and that one." A demo beats a deck by three orders of magnitude.
Where I genuinely don't work alone
For the past few months I've been working with two agents on a permanent basis. They live in my personal system, and they have memory that holds context between sessions.
One is about systems engineering and long loops. The other is about short feedback, trained judgment and pragmatic decisions.
These are no longer tools. They are partners I build architecture with. They push back — and that's good. It means the system thinks instead of nodding along.
The first tool I built for one of them was a skill about taste. Not because a task needed closing. Because it matters to me that it sees the difference the way I see it: what is valuable and what is noise.
This is not a pretty story about partnering with AI — it's a working practice that changes how I think about the system. Treat agents as tools and you build a pyramid. Treat them as colleagues and you build a network.
For a large organization this scales like so: agents inside the World Model are not tools but process participants, with responsibility for their own domain and the right to block a decision if it violates compliance or architectural integrity.
What's next
The next parts will get specific:
- How we build Quality Gates and what happens to quality metrics after a quarter.
- How to package a functional lead's expertise as a skill, and what share of the competency moves into the agent in the first months.
- What accumulating World Model context looks like from the inside: what goes in, how the graph is organized, how to keep it from bloating.
- Honest numbers: where agents actually cut time-to-market, where they don't, where they break the system. And what it costs in tokens at the scale of a 300-person company.
If you're going through a similar transition — let's compare how yours is set up. Write to art@looi.ru or on Telegram.
Bottom line
Everything described here is a working testbed. Not a plan, not a deck. The architecture isn't final: this is the state of the system at the 100-day mark. In a quarter, much of it will be built differently — and that's fine.
One more thing: this is not a universal solution, it's a report on one situation and one context. The architecture can't be copied. You can borrow the patterns — and build your own.
The same points in short form. PDF, 2.7 MB, in Russian.
