Blog banner
case-study

Designing TestDynamiz

UI

uipirate

2 views · 18 min read

18 min read  |  1 months ago


case-studyproduct-designux-designqa-testingtest-managementsoftware-testingui-design

We designed a testing platform for teams that couldn't afford to keep testing the way they'd always tested — manually, messily, and across too many tools.

There's a dirty secret in software development. Everyone agrees testing is important. Everyone knows that shipping untested code is how bugs reach production, how releases break, how customer trust erodes.

And almost nobody does it well.

Not because they don't want to. Because the tools make it hard.

Most development teams — especially growing ones, especially startups, especially teams without a dedicated QA department — test their software using some combination of:

  • Spreadsheets tracking what's been tested

  • Text documents describing test cases in plain English

  • Automation scripts that one engineer wrote six months ago and nobody else can maintain

  • A vague understanding that "someone checks the critical flows before we deploy"

  • Hope

This isn't a process. It's an accumulation of workarounds that works fine when you have ten test cases and collapses when you have a hundred.

TestDynamiz was built for the moment when those workarounds stop working — when a team realizes that their testing process needs to be a system, not a habit.

"Every team we talked to agreed on two things: testing is essential, and their current testing process was a mess. The gap between those two statements was the entire product opportunity."


What the client saw

TestDynamiz approached us with a clear observation: the testing tool landscape was split into two extremes.

On one side: enterprise-grade test management platforms. Powerful. Comprehensive. And completely overwhelming for a team of twelve engineers shipping a SaaS product. These tools required dedicated QA specialists to operate, weeks of onboarding, and budgets that startups couldn't justify.

On the other side: code-based testing frameworks. Flexible. Fast for developers who already knew them. And completely inaccessible to anyone who wasn't a senior engineer comfortable writing automation scripts from scratch. The learning curve was a cliff.

In between — for the team that needed structured testing but couldn't afford enterprise complexity or engineering-heavy scripting — there was almost nothing.

That's where TestDynamiz wanted to live.

A platform that was powerful enough for QA professionals but approachable enough for developers, engineering managers, and startup teams who needed testing done well without making it a full-time job.


Two audiences, one product

The hardest constraint in the entire project wasn't a feature or a workflow. It was the audience.

TestDynamiz needed to serve two fundamentally different user types:


QA Engineers

Testing professionals. They think in test strategies, coverage matrices, regression suites, and edge cases. They've used Selenium, Cypress, Playwright. They understand the underlying mechanics of automated testing.

What they wanted from TestDynamiz:

  • Advanced test organization and management

  • Fine-grained execution control

  • Detailed reporting and coverage metrics

  • The ability to build and maintain large test libraries

What would make them leave:

  • Feeling like the tool was "dumbed down" for non-testers

  • Losing control over test logic or execution parameters

  • Insufficient depth in reporting


Developers (and everyone else)

Engineers, managers, startup founders, and team leads who need testing done but don't specialize in it. They know enough to write a basic test. They don't want to maintain a testing framework.

What they wanted from TestDynamiz:

  • Fast test creation without writing scripts

  • Clear organization so tests don't become a mess

  • Simple execution — run tests, see results

  • Minimal maintenance burden

What would make them leave:

  • Feeling like the tool required QA expertise to use

  • Complex setup before they could create their first test

  • Interfaces that assumed familiarity with testing terminology


These audiences wanted the same outcome — tested software — but they spoke different languages, had different skill sets, and had different tolerances for complexity.

The product couldn't choose one and ignore the other. It had to serve both.

"QA engineers wanted depth. Developers wanted speed. The product needed to deliver both — without making either audience feel like a second-class citizen."


The workflow that organized everything

Before designing screens, we needed to design the logic of the product.

Testing has a natural progression — but most tools don't reflect it. Tests exist in flat lists, disconnected from the projects they serve, without clear relationships to each other or to the environments they run against.

We structured TestDynamiz around a progressive, project-centric workflow:

Projects → the container for everything. A project represents a product or application being tested.

Environments → where tests run. Development, staging, production. Configured once, reused everywhere.

Test Suites → organized collections of related tests. Regression, smoke testing, release validation.

Test Cases → individual tests within suites. Reusable, taggable, modular.

Execution → running tests individually, by suite, or in bulk.

Reporting → understanding what passed, what failed, what was skipped, and what it means.

This hierarchy did something subtle but important: it gave users a mental model for their testing process. Instead of managing a growing list of disconnected tests, they were building a structured testing architecture — with projects at the top, environments defining context, suites providing organization, and cases containing the actual logic.

Every level of the hierarchy answered a different question:

  • Project: What am I testing?

  • Environment: Where am I testing it?

  • Suite: What category of testing is this?

  • Test case: What specific thing am I validating?

  • Execution: Did it work?

  • Report: What's the overall health?


Project setup that doesn't feel like homework

The first interaction with TestDynamiz was creating a project. If this felt like filling out a form, we'd already lost.

Project setup was designed as a focused, minimal process:

  • Project name and description — what are we testing?

  • Assigned manager — who owns this?

  • Testing environments — development, staging, production URLs

  • Initial configuration — basic settings that could be modified later

Everything else — suites, cases, credentials, variables — came after. The project just needed to exist before the real work began.

We made a conscious decision: project creation should take under sixty seconds. Any field that wasn't essential to getting started was moved to settings, configurable later.

This was the first application of a principle we used throughout the product: don't ask for information before the user needs to provide it.


Environments: test once, run anywhere

One seemingly small feature that solved a surprisingly large problem.

Development teams test the same functionality across multiple environments — the feature works in development, but does it work in staging? Does it still work in production after deployment?

In most workflows, this means maintaining separate test configurations for each environment — different URLs, different credentials, different variables. Change an environment, and you're updating tests everywhere.

TestDynamiz separated test logic from environment configuration.

Teams configured their environments once:

  • Development — local or CI/CD URLs, dev credentials

  • Staging — pre-production URLs, staging data

  • Production — live URLs, production credentials

Tests referenced environments, not hardcoded values. Run the same test against staging today, production tomorrow. No modifications to the test itself.

This sounds like a backend architecture decision — and it was. But it was also a design decision, because the interface had to make this separation visible and intuitive. Users needed to understand that they were creating environment-agnostic tests without being taught the concept explicitly.

We communicated it through the interface itself: when creating a test, the environment was a selection — not a set of inline fields. The visual separation implied the conceptual separation.

"The best infrastructure decisions are the ones users never think about. Environment separation was invisible — until teams realized they could run the same test against three environments without changing a line."


The low-code test builder

This was the feature that defined TestDynamiz. And it was the hardest thing to design in the entire project.


The problem with test automation

Traditional test automation requires writing code. Selenium scripts. Cypress tests. Playwright flows. Each framework has its own syntax, its own setup, its own maintenance burden.

For teams with experienced QA engineers, this works. For everyone else, it's a wall.

A developer who can build a full-stack application might not know how to write a proper Page Object Model for their login flow. An engineering manager who understands testing strategy might not be able to write the Cypress command that validates their checkout process.

The skills required to understand testing and the skills required to automate testing are different. And the gap between them was keeping most teams stuck in manual testing.


Actions, not scripts

The low-code test builder closed that gap by replacing scripts with structured actions.

Instead of writing:

driver.get("https://app.example.com")
driver.find_element(By.ID, "username").send_keys("testuser")
driver.find_element(By.ID, "password").send_keys("password123")
driver.find_element(By.ID, "login-button").click()

Users assembled a visual flow:

  1. Launch URLhttps://app.example.com

  2. Enter Text → Username field → testuser

  3. Enter Text → Password field → password123

  4. Click Element → Login button

  5. Assert → Dashboard is visible

Same test. Same logic. Zero code.

Each action was a building block — a discrete, configurable step that users could add, reorder, modify, and remove. The test flow was visible as a sequence of actions, readable by anyone regardless of their automation experience.


Why this was hard to design

Making a low-code builder feel both simple and powerful is one of the most deceptive challenges in product design.

Too simple and QA engineers dismiss it as a toy. "I can't handle conditional logic. I can't manage dynamic data. I can't build anything real with this."

Too complex and developers abandon it. "This is just as hard as writing code, except now I'm fighting a visual interface instead of a text editor."

We navigated this through progressive capability:

The default view showed basic actions — navigate, type, click, assert. These covered 80% of common test scenarios. A user could build their first useful test in minutes.

Advanced actions — store variables, apply credentials, conditional steps, data-driven inputs — were available but not promoted. They appeared when users needed them, through clear expansion points, not cluttered initial interfaces.

The builder didn't pretend to replace code for every scenario. It replaced code for most scenarios — and for the rest, it made the structure visible enough that a developer could understand the test's intent before extending it programmatically.

"The low-code builder had to pass two tests: a QA engineer had to take it seriously, and a startup founder had to use it without reading documentation. Both happened — but not on the first design iteration."


Test suites: organization at scale

Individual tests are manageable. A hundred individual tests are not.

As teams grow and products evolve, the number of test cases expands rapidly. Without organization, tests become a flat list — unstructured, unnavigable, and unmaintainable. Teams stop adding tests because finding and managing existing ones takes longer than running them manually.

Test Suites solved this by providing organized, reusable collections:

Regression suites — tests that validate existing functionality hasn't broken after changes. Run before every release.

Smoke suites — critical path tests that validate core system functions. Run after every deployment.

Release validation suites — comprehensive test sets that confirm deployment readiness. Run before go-live.

Feature-specific suites — tests grouped by product area. Authentication, checkout, onboarding, API integrations.

Suites weren't just folders. They were executable units. A user could run an entire regression suite with one action — executing dozens of tests across a configured environment, with results aggregated at the suite level.

This was the difference between "we have tests" and "we have a testing strategy."


Test case reusability

One design decision that had outsized impact: test cases could belong to multiple suites.

A login test — validating that authentication works — is relevant to regression testing, smoke testing, and release validation. In most tools, this means duplicating the test across suites, creating maintenance nightmares when the login flow changes.

In TestDynamiz, test cases were modular assets. Create once, tag into multiple suites. Update the test once, and every suite that references it stays current.

This turned test management from a duplication exercise into an assembly exercise — building suites from a library of proven, maintained test components.


The Data Library: killing hardcoded values

This was one of the most impactful features from a maintainability perspective — and one of the least visually exciting.

Automated tests constantly reference shared data: login credentials, API tokens, environment variables, test user information. In most test setups, these values are hardcoded directly into test scripts.

This creates a cascade of problems:

  • Credential rotation breaks everything. When a password changes, every test that references it fails — and someone has to find and update every hardcoded instance.

  • Environment switching requires editing. Moving tests from staging to production means changing URLs, credentials, and tokens throughout the test suite.

  • Security risks multiply. Credentials scattered across test files are credentials waiting to be exposed.

  • Onboarding new team members is painful. "Where are the test credentials?" becomes a question with no clear answer.

The Data Library centralized all of this:

  • Credentials — username/password pairs for different environments and user types

  • Tokens — authentication tokens and API keys

  • Variables — reusable values referenced across tests

Tests referenced library entries, not raw values. Change a credential in the library, and every test that uses it automatically picks up the update. Switch environments, and the library serves the appropriate values.

It's the kind of feature that doesn't make demos exciting but saves teams hundreds of hours of maintenance over the life of a project.

"Nobody gets excited about a credential management feature. But every team that's spent an afternoon debugging tests that failed because someone rotated a password understands why it matters."


Test execution: from creation to validation

The execution flow was designed to be the simplest part of the platform — because by the time users reached execution, the complexity should already be managed.

Users could execute tests at multiple levels:

  • Individual test execution — run a single test case against a selected environment

  • Suite execution — run an entire suite, aggregating results across all cases

  • Bulk execution — run multiple suites or selected tests in a single action

Each execution produced structured results: pass, fail, skip, with details on what happened at each step. Failed steps included enough information to diagnose the issue without leaving the platform.

The design principle for execution was: one click to run, one screen to understand.

No configuration dialogs before execution. No separate results pages to navigate to. Click "run," watch the results populate, and understand the outcome — all in the same context.


The reporting dashboard

The dashboard answered the question that every engineering manager, every release manager, and every founder eventually asks:

"How confident should I be in this release?"

Key metrics surfaced immediately:

  • Total tests — how much coverage exists

  • Passed — what's working

  • Failed — what's broken

  • Skipped — what wasn't tested (and why)

  • Coverage trends — is testing improving or degrading over time?

  • Recent executions — what ran recently and what happened?

The dashboard was designed for glanceable confidence. A manager checking before approving a release shouldn't need to dig into individual test results. The high-level metrics should tell them: safe to ship, or not.

When the numbers said "not," the dashboard provided drill-down paths directly to the failing tests, the affected suites, and the specific execution details.

"The dashboard didn't answer 'how many tests do we have?' It answered 'should we ship?' That's a fundamentally different question — and designing for it changed every metric we displayed."


The design system

TestDynamiz's design system was built for productivity — not decoration.

Testing platforms are daily-use tools. Users spend hours creating tests, organizing suites, reviewing results. Every unnecessary element, every extra click, every unclear label costs time multiplied by frequency.

The system was deliberately minimal.


Typography: Red Hat Display

A modern, professional typeface with strong readability across data tables, code-adjacent content, and dashboard metrics. Red Hat Display provided:

  • Clean rendering at small sizes — critical for action step labels and test metadata

  • Distinctive character at large sizes — giving the platform personality without sacrificing clarity

  • Strong number rendering — essential for pass/fail counts, coverage percentages, and execution metrics


Color strategy

Primary Purple (#7828C8) The platform's identity color. Used for primary actions, navigation emphasis, and interactive elements.

Purple was a deliberate choice — differentiating TestDynamiz from the sea of blue developer tools. In a market where every testing platform, CI/CD tool, and developer dashboard defaults to blue, purple created instant visual distinction.

Functional colors:

  • Green — pass, success, healthy

  • Red — fail, error, broken

  • Yellow/Orange — skip, warning, attention

  • Grey — inactive, disabled, neutral

These weren't arbitrary. They matched the universal testing vocabulary: green means pass, red means fail. Deviating from these conventions in a testing tool would create confusion. Adhering to them made the platform feel immediately intelligible.


Components built for testing workflows

Action blocks. The building blocks of the low-code builder. Each action — navigate, type, click, assert — was a distinct visual component with clear inputs, configurable parameters, and drag-to-reorder capability.

Test case cards. Compact representations showing test name, suite membership, last execution status, and last run date. Designed for list views where users scan dozens of tests quickly.

Suite containers. Visual groupings that showed suite name, test count, last execution summary, and run action. Each container was a miniature dashboard for that suite's health.

Result indicators. Pass/fail/skip states used consistently across every context — test cases, suites, execution views, dashboards. Color-coded, icon-supported, and instantly recognizable.

Data library entries. Credential and variable cards showing type, name, and reference count (how many tests use this entry). The reference count was a subtle but important detail — it told users "if you change this, these tests are affected."


What was harder than expected


Making the low-code builder feel legitimate.

Low-code tools carry a stigma in the developer community. "Real engineers write code." We needed the builder to feel like a professional tool — not a simplified toy — while still being accessible to non-engineers. The visual design of action blocks, the precision of available actions, and the ability to handle real-world test scenarios (not just "hello world" demos) were what ultimately earned credibility.

The first version of the builder looked too much like a flowchart tool. Developers dismissed it immediately. We redesigned it as a structured sequence — closer to how code reads (top to bottom, step by step) but without requiring code to write. That shift — from flowchart to sequence — changed how users perceived it.


Deciding what's in the MVP and what's not.

TestDynamiz was an MVP — not a mature platform. Every feature we designed raised the question: "Is this essential for launch, or is this a version 2 feature?" Test scheduling, CI/CD integration, parallel execution, visual regression testing — all valuable, all requested, all deferred.

The discipline was knowing that a focused MVP that did basic testing well was more valuable than an ambitious MVP that did everything poorly. We drew the line at: create, organize, execute, report. Everything that supported that flow was in. Everything else was documented for future development.


Balancing the dashboard for different roles.

Engineering managers wanted high-level health metrics. QA engineers wanted detailed failure analysis. Developers wanted to know if their specific feature's tests passed.

One dashboard couldn't serve all three equally. We designed a layered approach: the default view showed health metrics (for managers), with drill-down paths to detailed results (for QA) and filterable views by feature area (for developers). Each audience got their answer — just at different depths.


Naming things for mixed audiences.

QA engineers understand "regression suite," "smoke test," "assertion." Developers understand some of these. Startup founders might understand none.

Every label in the platform had to be precise enough for professionals and clear enough for newcomers. We avoided dumbing down terminology — "assertion" is the correct word and using a simpler one would alienate QA engineers. Instead, we added contextual descriptions: tooltips, helper text, and onboarding hints that explained concepts without replacing established vocabulary.


Reflection

TestDynamiz was a masterclass in designing for constraint.

Constraint of audience — serve two different user types with one product. Constraint of scope — build an MVP, not a platform. Constraint of complexity — make automated testing accessible without making it simplistic.

A few things this project left us with:


The best developer tools teach while they work.

TestDynamiz didn't just let users create tests. The progressive workflow — Projects → Suites → Cases → Execution — taught users how to think about testing. The hierarchy implied a methodology. By using the tool, teams were learning testing strategy, not just testing mechanics.


Low-code isn't about removing code. It's about removing barriers.

The low-code builder didn't eliminate scripting because scripting is bad. It eliminated scripting as a prerequisite for testing. Teams that could write code still could. Teams that couldn't were no longer excluded. The builder didn't lower the ceiling — it raised the floor.


Reusability is the hidden killer feature.

Test case reuse across suites. Centralized credential management. Environment-agnostic test configuration. None of these are exciting on a feature list. But they're the difference between a testing process that scales and one that collapses under its own weight.


MVP discipline is product discipline.

Saying no to good features — scheduling, CI/CD integration, parallel execution — was painful. But shipping a focused product that did core testing well created a foundation. Shipping an unfocused product that did everything partially would have created technical debt disguised as features.


Simplicity for experts is harder than simplicity for beginners.

Making a testing tool simple for someone who's never tested before is relatively straightforward — show them less, guide them more. Making a testing tool simple for a QA engineer with ten years of experience is genuinely hard — they know what's missing, they expect specific capabilities, and they'll dismiss anything that feels like it's not respecting their expertise.


What TestDynamiz became

TestDynamiz became a structured, accessible testing workspace — a product where teams could go from "we need to test this" to "we know this works" without maintaining scripts, fighting framework complexity, or losing track of what's been tested.

Not a testing framework for senior engineers.

Not an enterprise QA management suite.

A testing platform for teams that need testing done well — built around the idea that automated testing should be as organized as the software it's testing.

Project Setup → Test Organization → Automation → Execution → Reporting.

One workflow. One platform. One less excuse not to test.


TYPOGRAPHY: RED HAT DISPLAY

PRIMARY COLOR: PURPLE (#7828C8)

Personalized Help

Struggling with Designing TestDynamiz? Let's talk about your product.

UI Pirate is a product design & development agency trusted by 50+ SaaS founders and enterprise teams across the US, UK & beyond. Tell us what you need.

More to Read

All articles →