-
🛑 AI Agent Factory with Playwright — Class Recordings
7 Lessons-
StartGetting Started: The Agent Factory Blueprint & Environment Setup
-
StartBuilding the Playwright Tool Layer Your Agents Will Use
-
StartAgent Templates: Regression, Exploratory, API & Visual Testers
-
StartOrchestration & Memory: Running Multiple Agents Together
-
StartEvaluating Agents: Scoring, Benchmarks & Reliability
-
StartDeploying the Factory: Docker, CI and 24/7 Agent Runs
-
StartConclusion: Scaling Checklist & Your 30-Day Agent Factory Roadmap
-
This course is not open for enrollment
AI Agent Factory with Playwright
Ship a reusable factory of QA agents powered by Playwright automation.
7
in-depth lessons
4
agent templates
1
orchestration layer
24/7
agents running in CI
∞
lifetime access
tool layer · agent templates · orchestration · memory · evaluation · Docker · continuous runs
Course Summary
Build an end-to-end agent factory: reusable templates, tool layers over Playwright, orchestration, memory, evaluation and deployment. Learn to spin up new testing agents on demand for regression, exploratory, API and visual checks, then run them reliably in CI at scale.
One agent is a demo. A factory is a system.
Stop building agents one at a time
Most teams get one impressive agent working and then stall, because the second one means rewriting everything. A factory fixes that: a shared Playwright tool layer, typed templates, an orchestrator, a memory store and an evaluation harness — so a new testing agent takes an afternoon rather than a quarter.
This is the infrastructure course. You will build the layer underneath the agents: the part that makes them cheap to create, safe to run, measurable in production and possible to operate at three in the morning without anyone watching.
The second agent costs as much as the first
Nothing was designed for reuse, so every new use case duplicates tool code, prompts, retry logic and logging.
Agents work locally and fail in CI
Different browser versions, no display, missing secrets, no retry policy and no way to see what happened.
Nobody can prove they help
Without a benchmark suite and a scoring harness, every change is an opinion and every regression goes unnoticed.
Concurrent agents collide
Two agents editing the same branch, competing for the same test data, or racing on the same environment. Orchestration is not optional.
Outcomes
What you will be able to build by the last lesson
A factory, not a prototype.
1
Design a factory blueprint
The layered architecture: tools, templates, orchestrator, memory, evaluation and deployment, with clear boundaries.
2
Build a reusable tool layer
A typed, tested Playwright surface every agent shares: navigate, locate, run, assert, capture, report.
3
Instantiate agents from templates
Regression, exploratory, API and visual testers, each a small configuration rather than a new codebase.
4
Orchestrate concurrent runs
Queues, worker pools, shared memory, conflict handling and a supervisor that can stop a run.
5
Give agents useful memory
Short-term run context, long-term learned knowledge, and a clear policy on what is worth remembering.
6
Score agents objectively
A fixed benchmark suite and a scoring harness so improvement is a number, not an impression.
7
Deploy for continuous operation
Docker images, CI schedules, health checks, alerting and graceful degradation.
8
Operate it safely
Spend caps, kill switches, audit trails and a rollback path when an agent misbehaves.
The method
From blueprint to a factory running in CI, in seven lessons
Bottom-up. Each layer is built, tested and then depended on by the next.
1
Blueprint
Draw the architecture before writing code. Decide the boundaries between tools, agents, orchestration and evaluation.
2
Tool layer
Build the shared Playwright surface with types, tests and versioning. This is the foundation everything else stands on.
3
Templates
Turn the tool layer into four agent templates driven by configuration rather than new code.
4
Orchestration
Add queues, concurrency, shared memory and a supervisor so multiple agents can run together.
5
Evaluation
Build the benchmark suite and scoring harness. Nothing goes to production unmeasured.
6
Deployment
Containerise, schedule, monitor and alert. Make it survive unattended overnight runs.
7
Scale
Consolidate into a checklist and a rollout plan for growing the factory beyond four agents.
Curriculum in detail
Every lesson, topic by topic
Seven class recordings, built as one continuous system.
1
Getting Started: The Agent Factory Blueprint & Environment Setup
Lesson 1 · ArchitectureArchitecture first. We draw the whole factory, name the layers, define the contracts between them, and set up a repository that will still make sense at agent number twelve.
- ▸Why one-off agents do not scale
- ▸The six layers: tools, templates, orchestrator, memory, evaluation, deployment
- ▸Contracts between layers and why they matter
- ▸Monorepo layout and package boundaries
- ▸Configuration strategy: environment, secrets and per-agent settings
- ▸Model provider abstraction so you are not locked in
- ▸Observability from day one: structured logs and run identifiers
- ▸Setting up the development environment
You build: A repository skeleton with all six layers stubbed, typed and wired together, plus a hello-world agent that runs end to end.
2
Building the Playwright Tool Layer Your Agents Will Use
Lesson 2 · FoundationThe most important lesson in the course. Every agent inherits the quality of this layer, so we treat it like production infrastructure: typed, tested and versioned.
- ▸Designing the tool surface: what to expose and what to hide
- ▸JSON schemas, input validation and useful error messages
- ▸Navigation, locating, interaction and assertion tools
- ▸Running specs and returning structured results
- ▸Trace capture and artifact handling as tools
- ▸Screenshot and visual snapshot tooling
- ▸Test data creation and environment reset tools
- ▸Unit testing the tool layer itself
- ▸Versioning and backward compatibility for tools
- ▸Rate limiting and resource cleanup
You build: A published, versioned, unit-tested tool package that any agent in the factory can import.
3
Agent Templates: Regression, Exploratory, API & Visual Testers
Lesson 3 · TemplatesFour working agents, each defined as configuration over a shared skeleton. This is where the factory metaphor stops being a metaphor.
- ▸The shared agent skeleton: loop, tools, memory, stop conditions
- ▸Regression agent: run, detect, classify, report
- ▸Exploratory agent: charter-driven navigation and observation
- ▸API agent: contract checks, schema validation and edge cases
- ▸Visual agent: baselines, diffing and tolerance handling
- ▸Template configuration: prompts, tool subsets, budgets, limits
- ▸Composing templates for hybrid use cases
- ▸Per-template evaluation criteria
- ▸Adding a fifth template as an exercise
You build: Four production-shaped agents instantiated from one skeleton, each configured rather than hand-coded.
4
Orchestration & Memory: Running Multiple Agents Together
Lesson 4 · CoordinationOne agent is easy. Six agents sharing an environment, a branch and a budget need real coordination. We build the orchestrator and the memory store.
- ▸Job queues, priorities and worker pools
- ▸Concurrency limits and resource contention
- ▸Shared environment access and test data isolation
- ▸Short-term memory: run context and step history
- ▸Long-term memory: what is genuinely worth persisting
- ▸Retrieval strategy and keeping context small
- ▸Conflict handling when two agents touch the same file
- ▸A supervisor that can pause, resume and kill runs
- ▸Dead-letter handling and poison jobs
- ▸Global budget enforcement across concurrent runs
You build: An orchestrator running four agents concurrently against a shared environment with isolated data and a global spend cap.
5
Evaluating Agents: Scoring, Benchmarks & Reliability
Lesson 5 · MeasurementWithout this lesson the rest is theatre. We build a fixed benchmark suite and a scoring harness so every prompt change produces a number.
- ▸Designing a benchmark suite that does not rot
- ▸Golden tasks with known-correct outcomes
- ▸Scoring rubrics: correctness, cost, latency, safety
- ▸Deterministic scoring versus model-graded scoring
- ▸Handling non-determinism and running multiple trials
- ▸Regression detection across prompt and model changes
- ▸Reliability metrics: success rate, retry rate, abstain rate
- ▸Cost per successful outcome as the headline number
- ▸Dashboards and reporting the results to stakeholders
- ▸Deciding when an agent is good enough to deploy
You build: A benchmark suite plus a scoring harness that produces a comparable score for every agent on every change.
6
Deploying the Factory: Docker, CI and 24/7 Agent Runs
Lesson 6 · OperationsGetting the factory off your laptop. Containers, schedules, health checks, alerting and the failure modes that only appear at three in the morning.
- ▸Docker images with browsers and dependencies baked in
- ▸Image size, layer caching and build times
- ▸Running headless in CI and in a scheduled container
- ▸Secrets management and least-privilege credentials
- ▸Health checks, readiness and liveness
- ▸Scheduled runs, event-triggered runs and manual dispatch
- ▸Alerting: what should wake someone and what should not
- ▸Graceful degradation when a model provider is down
- ▸Log aggregation and reproducing a past run
- ▸Kill switches and rollback
You build: A containerised factory running scheduled agent jobs in CI with health checks, alerting and a working kill switch.
7
Conclusion: Scaling Checklist & Your 30-Day Agent Factory Roadmap
Lesson 7 · ConsolidationHow to grow this beyond four agents without recreating the mess you started with. Checklist, governance and a rollout plan.
- ▸The scaling checklist, layer by layer
- ▸Adding a new agent template in under a day
- ▸Governance: who can add agents and who approves them
- ▸Keeping the tool layer stable as consumers multiply
- ▸Cost review cadence and budget ownership
- ▸Onboarding a teammate into the factory
- ▸Measuring and reporting business value
- ▸A thirty-day plan to get the first three agents live
You build: A written scaling checklist and a four-week plan to take the factory from working prototype to team-owned system.
Tooling
What the factory is built from
Ordinary, dependable infrastructure. Nothing here is exotic.
| Component | Role in the factory |
|---|---|
| Playwright (Node/TypeScript) | The browser and test execution layer every tool wraps. |
| Typed tool package | The shared, versioned surface all agents import. |
| LLM APIs with provider abstraction | Model access, routed per task and swappable without touching agent code. |
| Job queue | Scheduling, priorities, concurrency limits and retries. |
| Memory store | Short-term run context and long-term persisted knowledge. |
| Evaluation harness | Benchmark tasks, scoring rubrics and regression detection. |
| Docker | Reproducible images with browsers and dependencies included. |
| GitHub Actions | Scheduled and event-triggered agent runs with secrets and artifacts. |
| Structured logging & metrics | Run identifiers, step traces, cost accounting and alerting. |
| Kill switch & budget guard | Hard stops on spend, runtime and blast radius. |
Before you start
What you need, and what you honestly do not
This is the most infrastructure-heavy course in the series. Comfort with TypeScript and CI is assumed.
What you need before lesson one
- ✓Solid TypeScript or JavaScript — you will be designing types and interfaces
- ✓Working Playwright knowledge
- ✓Node.js 18 or newer and Docker installed
- ✓An LLM API key
- ✓A CI environment you can run scheduled jobs on
Helpful, but not required
- +Prior experience building a single AI agent
- +Familiarity with job queues or message brokers
- +Some backend or platform engineering background
- +Experience owning a service in production
If you have never built an agent before, the AI Agents Masterclass is the better starting point. This course assumes you already know why an agent loop works and want to industrialise it.
Fit check
Is this the right course for you?
This is for people who have to make agents work at team scale, not just demo them.
This is for you if
- ✓You already built one agent and hit a wall building the second
- ✓You own test infrastructure and are being asked to add AI capability
- ✓You need agents running unattended in CI, not on someone laptop
- ✓You have to prove value with numbers, not anecdotes
- ✓You are responsible for cost, safety and auditability
- ✓You want architecture, not another prompt-engineering tutorial
This is probably not for you if
- ✗You have never built an AI agent — start with the AI Agents Masterclass
- ✗You want a quick win rather than infrastructure
- ✗You are not comfortable with TypeScript types and interfaces
- ✗Your team has no CI and no appetite for containerised jobs
What you get
Everything you keep after the last lesson
A complete, running system plus the documents that let a team own it.
Seven class recordings
Long-form build sessions, including the refactors. Lifetime access.
The complete factory repository
Six layers, four agent templates, orchestrator, memory and evaluation harness.
A versioned Playwright tool package
Typed, unit-tested and ready to publish internally.
A benchmark suite and scoring harness
Golden tasks, rubrics and regression detection you can extend.
Docker images and CI workflows
Containerised runs, schedules, health checks and alerting already wired.
A scaling checklist and rollout plan
Governance, cost ownership and a thirty-day path to three live agents.
The difference
What actually changes for your team
Before and after, once the factory exists.
| Today | After this course |
|---|---|
| A new agent means a new codebase | A new agent is a configuration file over an existing skeleton |
| Tool code copied and diverging across agents | One versioned, tested tool package every agent imports |
| Agents run on one engineer laptop | Containerised, scheduled runs in CI with health checks and alerts |
| Improvement is a matter of opinion | Every change produces a benchmark score you can compare |
| Concurrent runs corrupt shared test data | Orchestrated queues with isolation and conflict handling |
| Spend is discovered at the end of the month | Global budget caps enforced per run, with cost per successful outcome tracked |
| Nobody knows what an agent did last night | Structured logs, run identifiers and a reproducible audit trail |
Your instructor
Pramod Dutta
Pramod Dutta is the founder of The Testing Academy and has spent over a decade building and maintaining test infrastructure, which is exactly the lens this course takes.
The factory pattern here came from watching teams get one agent working and then stall completely. The interesting problem was never the agent — it was the layer underneath it.
The build is recorded live, refactors included, because the design decisions are the actual content.
10+ yrs
in testing and automation
1M+
learners reached through The Testing Academy
6 layers
built end to end in this course
FAQ
Everything people ask before enrolling
Ask support if your question is not answered here.
How is this different from the AI Agents Masterclass?
That course teaches you to build individual agents. This one teaches the infrastructure that makes building the fifth agent as cheap as the first — tool layer, templates, orchestration, evaluation and deployment.
Do I need to take that course first?
It is strongly recommended. This course assumes you already understand agent loops, tool calling and prompt design.
What will running this cost me?
The factory includes hard budget caps and per-task model routing precisely so cost stays predictable. The evaluation lesson makes cost per successful outcome a first-class metric.
Is TypeScript mandatory?
Effectively yes. The tool layer relies on types and schemas for safety, and the templates are typed configurations.
Can I run this without Docker?
You can develop without it, but the deployment lesson uses containers because browser dependencies in CI are otherwise painful.
Does this work with a self-hosted model?
Yes. The provider abstraction is built in lesson one specifically so you can route cheap classification tasks to a local model.
How many agents can the factory realistically run?
That depends on your budget and environment capacity. The orchestration lesson covers concurrency limits, contention and how to find your ceiling honestly.
What if my team uses a different test framework?
The architecture transfers directly. The tool layer is Playwright-specific and would need to be rewritten for another runner.
Is access limited in time?
No. Lifetime access to the recordings and updates.
Ready when you are
Build the layer that makes every future agent cheap.
Seven lessons, six layers, four agent templates and a factory that runs unattended in CI.
- ✓Seven full class recordings with lifetime access
- ✓The complete six-layer factory repository
- ✓A versioned, typed Playwright tool package
- ✓Four production-shaped agent templates
- ✓Benchmark suite, scoring harness and dashboards
- ✓Docker images, CI schedules and a scaling checklist
Course Curriculum
Pramod Dutta
Founder of The Testing Academy, a YouTube Channel with 95K subscribers where Pramod teaches about Software testing & Test Automation. With overall 10+ years of experience in Software Testing & Test Automation, he has mentored 10,000+ students in Software Testing, API Testing, Test Automation.
Pramod Dutta is working as SDET Manager at Tekion | Ex BrowserStack Employee & BrowserStack Champion & Certified Scrum Master. Pramod has a vast range of experience handling from Manual Testing, Mobile & Web Automation, Desktop & cloud services like AWS, GCP extra.
By joining this Automation Testing Course , you’ll have the opportunity to take control of your life, work in an exciting industry with infinite possibilities and live the life you want.
