Cutting regression time 30% on an identity platform with Playwright and LLM triage

Rebuilt a manual-heavy regression process for cybersecurity and identity products into a balanced automated suite with AI-assisted failure triage.

Client details are withheld under NDA. This case study describes the domain, the problem, and the approach only.

Context

A remote QA lead role on enterprise identity and access management products, covering web, API, and mobile surfaces for a globally distributed engineering team.

Problem

Regression was mostly manual, so every release cycle depended on the same people repeating the same checks. Failures took time to classify because logs were long and the root cause was often environmental rather than a product defect.

Approach

Introduced Playwright for new end-to-end coverage and kept Selenium and Fireflink suites where they already paid for themselves. Set a deliberate 70/30 manual-to-automation balance so exploratory testing kept its place. Wired the suites into Jenkins and GitHub Actions so they ran on every merge. Added an LLM-assisted step that summarised failure logs and proposed a likely category before a human looked, and used the same models to draft test cases from requirements for review. Ran JMeter load tests ahead of releases.

Outcome

Regression cycle time fell by about 30%. Triage moved from reading raw logs to confirming a proposed classification. The team mentored into the new stack kept ownership of the suites after handover.

Tools

Playwright, Selenium, Fireflink
JMeter
Jenkins, GitHub Actions
LLM-assisted log summarisation and test drafting