-
🛑 Playwright Automation with Test Dashboards — Class Recordings
7 Lessons-
StartGetting Started: Playwright Reporters & the Built-in HTML Report
-
StartAllure Reporting for Playwright End to End
-
StartBuilding a Custom Reporter & Storing Results in a Database
-
StartLive Dashboards with Grafana and Metabase
-
StartTrends, Flake Detection & Quality KPIs That Matter
-
StartCI Integration: GitHub Actions, Slack Alerts & Published Reports
-
StartConclusion: Dashboard Checklist & Your 30-Day Reporting Roadmap
-
This course is not open for enrollment
Playwright Automation with Test Dashboards
Turn Playwright runs into live dashboards, reports and quality metrics.
7
in-depth lessons
4
dashboards you build
3
reporters wired up
12
quality KPIs defined
∞
lifetime access
HTML report · Allure · custom reporters · SQL storage · Grafana · Metabase · Slack alerts
Course Summary
Make your Playwright results visible: HTML and Allure reporting, custom reporters, historical trends, flake detection and Grafana style dashboards. Wire results into CI, Slack and databases so teams and managers can track quality in real time.
Results nobody can see are results nobody trusts
Turn a green tick into evidence your business understands
A passing pipeline tells nobody anything. This masterclass takes Playwright output all the way through to dashboards that answer the questions people actually ask — is quality improving, which tests are flaky, which areas are untested, and can we ship on Thursday.
You will start with the built-in HTML report, add Allure, write a custom reporter that persists every run to a database, and finish with live Grafana and Metabase boards, threshold-based Slack alerts and a set of KPIs that survive contact with a leadership meeting.
The report dies with the runner
The HTML report exists for ninety seconds inside a CI container and then vanishes. Nobody outside the pipeline ever sees it.
No history, so no trends
Each run is an island. You cannot answer whether things are getting better because nothing is stored.
Flaky tests are folklore
Everyone knows which tests are flaky, nobody has the numbers, and so nothing ever gets prioritised.
Managers ask questions you cannot answer
Coverage by area, mean time to detect, pass rate trend, duration creep — all reasonable questions with no data behind them.
Outcomes
What you will be able to do by the last lesson
Reporting from the source all the way to a board on a wall.
1
Choose reporters deliberately
HTML, list, line, JSON, JUnit, blob and Allure — what each is for and how to run several without slowing the suite.
2
Write a custom reporter
Hook into the runner lifecycle and emit exactly the structured data your dashboard needs.
3
Persist every run
A schema for runs, tests, attempts, durations, errors and artifacts, plus the ingestion path from CI.
4
Build boards people open
Grafana and Metabase dashboards designed separately for engineers and for managers.
5
Detect flakiness with numbers
A flake score based on attempt history, not on the loudest opinion in standup.
6
Track trends that mean something
Pass rate, duration, failure clustering, coverage by area and mean time to detect.
7
Alert without creating noise
Threshold and anomaly-based Slack alerts, so a red channel still means something.
8
Publish reports from CI
Merged multi-shard HTML and Allure reports, hosted and linked automatically on every run.
The method
From the built-in report to a live dashboard, in seven lessons
Each lesson removes a limitation of the previous one. By lesson four the data is permanent and queryable.
1
Master the built-in output
Understand what Playwright already gives you before adding anything. Most teams underuse it badly.
2
Add a richer report
Allure end to end: steps, attachments, categories, history and severity.
3
Make results permanent
A custom reporter plus a database schema, so every run is queryable forever.
4
Visualise
Grafana and Metabase boards built from that schema, with separate views for different audiences.
5
Turn data into judgement
Flake scores, trend lines, coverage by area and the KPIs worth reporting upward.
6
Automate the loop
CI publishing, hosted reports and threshold-based Slack alerts.
7
Consolidate
A dashboard checklist and a thirty-day plan for rolling this out on a real project.
Curriculum in detail
Every lesson, topic by topic
Seven class recordings, building one continuous reporting pipeline.
1
Getting Started: Playwright Reporters & the Built-in HTML Report
Lesson 1 · FoundationsMost teams use maybe a third of what Playwright already provides. We go through every built-in reporter, when each one earns its place, and how to combine them without slowing the run.
- ▸The reporter interface and how the runner drives it
- ▸list, line, dot, json, junit, html and blob compared
- ▸Running multiple reporters simultaneously
- ▸Reporter options: output folders, open behaviour, attachments
- ▸The HTML report anatomy: filters, traces, attachments
- ▸Blob reports and merge-reports for sharded runs
- ▸JUnit XML and what CI systems do with it
- ▸Annotations, tags and test metadata that show up in reports
- ▸Keeping report size under control
You build: A multi-reporter configuration producing a merged HTML report, JUnit XML and JSON output from a sharded run.
2
Allure Reporting for Playwright End to End
Lesson 2 · Rich reportingAllure done properly, not just installed. Steps, attachments, categories, severity, history and the environment block that makes a report self-explanatory.
- ▸Installing and wiring the Allure reporter
- ▸Steps and nested steps for readable narratives
- ▸Attaching screenshots, traces, logs and API payloads
- ▸Categories: classifying failures automatically
- ▸Severity, epics, features and stories for grouping
- ▸The environment block and why it prevents arguments
- ▸History and trend graphs across runs
- ▸Generating and hosting the Allure report
- ▸Allure versus the built-in report: an honest comparison
You build: A fully configured Allure report with categories, severity, attachments and cross-run history, published from CI.
3
Building a Custom Reporter & Storing Results in a Database
Lesson 3 · PersistenceThe turning point of the course. A custom reporter that captures the run lifecycle and writes structured rows to a database, so results outlive the pipeline.
- ▸The Reporter API: onBegin, onTestBegin, onTestEnd, onEnd
- ▸Capturing retries, attempts and the difference between them
- ▸Designing the schema: runs, suites, tests, attempts, errors, artifacts
- ▸Recording branch, commit, environment and trigger metadata
- ▸Durations, timings and where the time actually goes
- ▸Error fingerprinting so identical failures group together
- ▸Artifact links: traces, screenshots and video URLs
- ▸Batching writes and not slowing the test run
- ▸Handling partial runs, cancellations and CI timeouts
- ▸Idempotent ingestion and re-running the same commit
You build: A custom Playwright reporter writing every run, test and attempt into a relational database with full metadata.
4
Live Dashboards with Grafana and Metabase
Lesson 4 · VisualisationTwo tools, two audiences. Grafana for the engineering view, Metabase for the people who want to ask their own questions without writing SQL.
- ▸Connecting Grafana to the results database
- ▸Panels: pass rate, duration, failure count, flake rate
- ▸Time ranges, variables and per-branch filtering
- ▸Drill-down from a panel to an individual failing test
- ▸Metabase setup and question building for non-engineers
- ▸Designing a manager view versus an engineer view
- ▸Avoiding vanity metrics on a dashboard
- ▸Performance: indexes, materialised views and query cost
- ▸Embedding a board in a wiki or a team page
- ▸Access control and who sees what
You build: Two live dashboards — an engineering board in Grafana and a self-serve board in Metabase — reading from your results database.
5
Trends, Flake Detection & Quality KPIs That Matter
Lesson 5 · InterpretationData becomes useful when it changes a decision. This lesson is about the metrics worth tracking and, just as importantly, the ones to ignore.
- ▸Defining flakiness precisely and computing a flake score
- ▸Attempt-history analysis and rolling windows
- ▸Quarantine policy: when to isolate a test and when to delete it
- ▸Pass rate trends and why raw pass rate misleads
- ▸Duration creep and finding the tests that slow down over time
- ▸Failure clustering by error fingerprint
- ▸Coverage by product area, not by line count
- ▸Mean time to detect and mean time to fix
- ▸Choosing three KPIs and defending them
- ▸Metrics that look good and change nothing
You build: A flake scoreboard, a quarantine policy and a three-KPI quality summary generated automatically from run history.
6
CI Integration: GitHub Actions, Slack Alerts & Published Reports
Lesson 6 · AutomationClosing the loop. Reports published automatically, results ingested on every run, and alerts that fire only when something genuinely changed.
- ▸GitHub Actions workflow for sharded runs and report merging
- ▸Publishing HTML and Allure reports to a hosted location
- ▸Linking the report from the pull request automatically
- ▸Ingesting results into the database as a pipeline step
- ▸Slack notifications: format, content and links that work
- ▸Threshold alerts versus per-run alerts
- ▸Anomaly detection on pass rate and duration
- ▸Escalation rules and who gets paged
- ▸Preventing alert fatigue and keeping the channel useful
- ▸Handling ingestion failures without failing the build
You build: A complete CI pipeline that runs sharded tests, publishes a hosted report, ingests results and posts a threshold-based Slack alert.
7
Conclusion: Dashboard Checklist & Your 30-Day Reporting Roadmap
Lesson 7 · ConsolidationTurning the pipeline into something the team maintains. Checklist, ownership and a staged rollout.
- ▸The complete reporting checklist
- ▸Rollout order: what to ship in week one
- ▸Getting managers to actually use the board
- ▸Data retention, archiving and database growth
- ▸Maintaining the reporter as Playwright evolves
- ▸Ownership: who fixes the dashboard when it breaks
- ▸Extending the schema for new questions
- ▸A thirty-day plan from zero reporting to live dashboards
You build: A written dashboard checklist and a four-week rollout plan tailored to an existing project.
Tooling
Everything you set up
All open source or free tier. Nothing here requires a purchase order.
| Tool | What you use it for in this course |
|---|---|
| Playwright reporters | Built-in list, json, junit, html and blob output as the raw data source. |
| merge-reports | Combining blob reports from parallel shards into one browsable report. |
| Allure | Rich reporting with steps, attachments, categories, severity and history. |
| Custom Reporter API | Hooking the run lifecycle to emit structured, database-ready records. |
| PostgreSQL (or SQLite) | Persisting runs, tests, attempts, errors and artifact links. |
| Grafana | Engineering dashboards: pass rate, duration, flake rate and drill-down. |
| Metabase | Self-serve question building for managers and non-engineers. |
| GitHub Actions | Sharded runs, report publishing, result ingestion and alerting. |
| Slack webhooks | Threshold and anomaly alerts with links back to evidence. |
| SQL | The queries behind every panel, written out and explained. |
Before you start
What you need, and what you honestly do not
You need a Playwright suite and basic SQL comfort. Everything about reporters, dashboards and schema design is taught from scratch.
What you need before lesson one
- ✓A working Playwright suite, even a small one
- ✓Node.js 18 or newer
- ✓Basic SQL: SELECT, JOIN and GROUP BY
- ✓Docker for running the database and dashboard tools locally
- ✓A GitHub repository for the CI lesson
Helpful, but not required
- +Previous experience with Allure or another reporting tool
- +Familiarity with Grafana or any BI tool
- +Some database schema design experience
- +An existing suite with a known flakiness problem
Everything runs locally in Docker first, so you can complete the whole course without provisioning any shared infrastructure. The CI lesson then shows the hosted path.
Fit check
Is this the right course for you?
This is for people who are asked to report on quality, not just produce it.
This is for you if
- ✓Your test results disappear when the pipeline finishes
- ✓You are asked for quality metrics and have nothing to show
- ✓Flaky tests are a known problem with no numbers attached
- ✓You want dashboards engineers and managers both find useful
- ✓You are a QA lead who needs to justify investment with data
- ✓You want to build reporting infrastructure once and reuse it
This is probably not for you if
- ✗You have no Playwright suite yet — start with the CLI masterclass
- ✗You want a hosted commercial reporting product rather than something you build
- ✗You are not willing to write any SQL
- ✗You need enterprise BI governance rather than a working team dashboard
What you get
Everything you keep after the last lesson
A pipeline, a schema, dashboards and the queries behind them.
Seven class recordings
Full build sessions with the SQL and configuration written live. Lifetime access.
A custom Playwright reporter
Complete source, capturing runs, tests, attempts, errors and artifacts.
A results database schema
Tables, indexes and migrations, designed for the questions you will actually ask.
Grafana and Metabase dashboards
Exportable board definitions plus every SQL query behind them, explained.
A flake scoring and quarantine policy
The calculation, the thresholds and the process around them.
A CI workflow and Slack alerting setup
Sharded runs, hosted reports, ingestion and threshold-based notifications.
The difference
What actually changes in your week
Before and after, once reporting is in place.
| Today | After this course |
|---|---|
| The report vanishes with the CI container | A hosted report linked from every pull request, kept as long as you want |
| Every run is an isolated event | Full history in a database you can query any way you like |
| Flaky tests are a matter of opinion | A ranked flake scoreboard with attempt history behind every entry |
| Managers ask for metrics you cannot produce | A self-serve board where they answer their own questions |
| Failures are triaged one by one | Error fingerprinting groups identical failures into a single item |
| Slack alerts on every run, so nobody reads them | Threshold and anomaly alerts, so a notification means something changed |
| Nobody knows if quality is improving | Trend lines for pass rate, duration and coverage by area |
Your instructor
Pramod Dutta
Pramod Dutta is the founder of The Testing Academy, with over a decade in software testing and automation and a long-standing interest in the reporting layer that most teams neglect.
The pipeline taught here came from a recurring problem: teams with good suites and no way to show anyone what those suites were telling them.
Everything is built live — the reporter, the schema, the SQL and the dashboards — so you see the design decisions, not just the finished board.
10+ yrs
in testing and automation
1M+
learners reached through The Testing Academy
2 dashboards
built end to end from your own data
FAQ
Everything people ask before enrolling
Not covered here? Ask support before enrolling.
Which database does the course use?
PostgreSQL in the recordings, running locally in Docker. The schema is deliberately plain SQL, and a SQLite variant is included for anyone who wants zero setup.
Do I need to know SQL well?
Basic SELECT, JOIN and GROUP BY is enough. Every query behind every dashboard panel is written out and explained line by line.
Grafana or Metabase — which should I use?
Both are covered because they serve different audiences. Grafana suits engineering time-series views; Metabase suits non-engineers asking their own questions.
Will this work with an existing Allure setup?
Yes. Lesson two builds on standard Allure configuration, and the custom reporter in lesson three runs alongside it rather than replacing it.
Does the custom reporter slow down my tests?
Writes are batched and the lesson covers exactly how to avoid adding meaningful overhead, including what to do when ingestion fails.
Can I use a hosted reporting service instead?
You can, and the course discusses the trade-off. The point of building it is control over the schema and the questions you can ask.
Is this specific to GitHub Actions?
The CI lesson uses GitHub Actions, but the ingestion and publishing pattern transfers to any CI system.
Does this cover coverage measurement?
It covers coverage by product area based on test metadata, which is the metric teams actually act on. It does not cover code coverage instrumentation.
Is access limited in time?
No. Lifetime access to all recordings and updates.
Ready when you are
Make your test results impossible to ignore.
Seven lessons from the built-in report to live dashboards, permanent history and alerts that mean something.
- ✓Seven full class recordings with lifetime access
- ✓A custom reporter with full source
- ✓A results database schema and migrations
- ✓Grafana and Metabase boards plus every SQL query
- ✓A flake scoring and quarantine policy
- ✓CI publishing, ingestion and Slack alerting
Course Curriculum
Pramod Dutta
Founder of The Testing Academy, a YouTube Channel with 95K subscribers where Pramod teaches about Software testing & Test Automation. With overall 10+ years of experience in Software Testing & Test Automation, he has mentored 10,000+ students in Software Testing, API Testing, Test Automation.
Pramod Dutta is working as SDET Manager at Tekion | Ex BrowserStack Employee & BrowserStack Champion & Certified Scrum Master. Pramod has a vast range of experience handling from Manual Testing, Mobile & Web Automation, Desktop & cloud services like AWS, GCP extra.
By joining this Automation Testing Course , you’ll have the opportunity to take control of your life, work in an exciting industry with infinite possibilities and live the life you want.
