AWS has published the third part of its series on QA Studio, a reference solution built on Amazon Nova Act. In the AWS Machine Learning Blog post, the authors add two capabilities aimed at organizations where many teams share one test setup: applications and team-based user management.
What the earlier parts covered
The first post introduced QA Studio, where tests are written in natural language and run through AI-driven visual navigation. The second added test suites and a command-line tool for batch regression runs in CI/CD pipelines. This part deals with what happens when hundreds of tests belong to different products and squads.
Applications and health metrics
An application represents the system under test, such as a website or web app front end. It is created with a name, a base URL and optional environments like development, staging and production. Use cases and test suites sit inside an application and inherit its owner.
Every execution is written to Amazon CloudWatch as custom metrics, split by application and environment. The application page shows pass rate, execution count and current failures, plus a health score from 0 to 100. The score starts at the pass rate and loses points for flaky tests, so a suite that flips between pass and fail does not look perfect.
Trend charts run over either 7 days at hourly granularity or 30 days at daily granularity, and can be filtered to one environment. A flaky-test view ranks use cases by how often they flipped in the last seven days. A recent-failures list links to the Nova Act trajectory logs and session recordings.
Teams and roles
An application belongs to exactly one team, and people outside that team do not see it. There are five cumulative roles:
- Admin has studio-wide access.
- Team Lead manages members and creates applications.
- Editor changes use cases and suites.
- Executor runs them without modifying them.
- Viewer has read-only access.
Permissions are set at the application level and apply to the tests inside it. An owning team can share an application with another team at view, execute or full level. A person's effective permission is the lower of their role and the grant, so a Viewer given full access stays read-only.
Under the hood
Each team maps to Amazon Cognito groups, one per role, and an AWS Lambda authorizer turns group membership into API scopes on every request. Teams, memberships, ownership and share grants live in DynamoDB. The backend checks both the caller's scope and whether their team owns or has been shared the application. A CI pipeline can authenticate as a machine identity with only the scopes it needs.
Availability
The features are part of the QA Studio reference solution, which is deployed with the AWS Cloud Development Kit in your own account. According to the post, test data, results and metrics stay within that account. Deployment steps are in the QA Studio GitHub repository. The post gives no pricing.