We built for test generation. Our customers are using it for traceability, compliance readiness, and quality reporting. Here’s what we learned.

The plan vs. reality

Six months into WalnutAI’s early access program, we had a problem every product team dreams about: customers were using the product in ways we didn’t anticipate. We built the platform to answer one question: “given a requirement, generate the test cases we need to validate it.” Useful, narrow, concrete.

Our early customers took that capability and built six distinct use cases — three we predicted and three we didn’t. The unexpected ones changed how we think about what WalnutAI actually solves. Here’s the breakdown.

The three use cases we expected

These landed exactly as we imagined:

Pre-release compliance readiness check

Before a major release, a QA lead uploads the release notes and a spec. WalnutAI generates test cases covering every acceptance criterion, and generates a traceability matrix showing which test case covers which requirement. For regulated teams (healthcare, fintech, automotive), this matrix is the evidence they need to pass an audit. One customer used it to prove that every story in a release was tested before go-live — something that used to take them a full week to manually document. Now it’s a 5-minute export.

The compliance angle became a permanent feature request. We started tracking traceability links as first-class data.

Test maintenance reduction via API testing

Several teams realized they were spending more time maintaining brittle UI tests than writing new ones. Using WalnutAI, they shifted from GUI automation to API-layer testing. The tool generates test cases that could be run at the API level instead of through a browser, cutting execution time from 45 minutes to 5 minutes per suite while increasing reliability.

The shift wasn’t about WalnutAI generating API tests specifically — it was about test case generation being fast enough that it made sense to regenerate and re-scope tests after every major refactor, instead of maintaining old ones. Test as code, not as legacy debt.

Quality reporting to non-technical stakeholders

A PM started using WalnutAI’s test coverage dashboard to show her CEO how much of the product release was validated. It’s a simple metric — percentage of requirements with passing test cases — but executives understood it instantly. No jargon. The CEO could see in real time: “73% of this release is validated; we’re on track.”

We learned that the ability to say “release readiness is 73%” in a standup is more valuable than we thought. It’s decision-making data, not just QA data.

The three use cases that surprised us

ADR validation

An architecture team used WalnutAI to validate Architecture Decision Records. They uploaded their ADR for “we’re shifting to event-driven architecture” into Intelligence Hub, and got back test cases validating that the code actually implemented the event-driven model they decided on. If the test cases failed, it meant the ADR wasn’t actually being followed in the codebase. This caught a whole module that had never switched from synchronous calls.

We never expected WalnutAI to be an ADR compliance tool. But the principle is the same: spec (the ADR) + test cases (the proof) + gap analysis (what’s missing) = accountability. One team is now embedding test case validation into their ADR workflow.

Architectural consistency checks

A fintech team has several teams building microservices. They built a spec that documents their shared architectural patterns — how services talk to each other, how they handle timeouts, what the error contract looks like. They uploaded that spec to WalnutAI and used the generated test cases as acceptance tests for new microservices. If your service doesn’t pass the architectural test suite, you know you’ve drifted from the pattern.

This is probably the most sophisticated use case we’ve seen. It treats a spec as both documentation and executable contract. As they bring new engineers onto the team, the test cases are the teaching artifact: “here’s what architectural consistency looks like, here are the tests we run.”

Quality trend dashboards

A QA director running a SaaS product had been manually updating a spreadsheet every sprint: test coverage by module, pass rate, average time-to-fix defects. It was a Friday afternoon task that took 90 minutes. WalnutAI’s analytics exports fed directly into her dashboard, which now auto-updates every day. She can see trends: “our API layer test coverage is stable at 94%, but frontend coverage dipped to 68% last sprint.” She can talk to the team with data.

The value wasn’t test generation here, it was traceability + reporting. Once you know which test cases came from which requirement, you can slice and dice that data in ways you couldn’t before.

What we learned

Two patterns jumped out:

First: traceability is the load-bearing wall. Every unexpected use case started with “I have a spec, and I need to know if X is true about my code.” The test cases were the vehicle, but traceability was the infrastructure. We started prioritizing bidirectional links, immutable audit trails, and queryable spec sections.

Second: we underestimated breadth of use case. We thought WalnutAI would be a QA tool. It’s turning out to be a platform for spec-driven validation — which includes testing, but also includes compliance, architecture, and decision-making. Different audiences, different primary keywords, same underlying machinery.

How this changed the product

We built native compliance reports with audit-trail timestamps and 7-year retention.

We added ADR-specific templates and validation modes.

We invested heavily in analytics and trend dashboards.

We made traceability links immutable and queryable by section.

None of this was in the original roadmap. Early customers dictated it.

The pattern: they knew their problem before they knew the solution

Every unexpected use case came from a team that had already felt the pain. The architect who wanted to validate ADRs had already seen modules drift from architectural intent. The fintech team validating services had already onboarded an engineer who misunderstood the pattern. The QA director automating her dashboard had already spent 90 minutes on Friday that she wished she’d spent on strategy.

They didn’t discover WalnutAI’s value through marketing. They found it because they had a specific problem (spec that might not match code) and they tried using test generation as the solution.

For builders deciding on use cases

If you’re building a tool and you’re trying to prioritize customers, here’s what we learned: go after the people who already feel acute pain, not the people you think will eventually want your product. The early adopters who found ADR validation weren’t looking for a testing tool. They were looking for a way to enforce architecture.

Talk to customers who have already tried to solve the problem themselves — spreadsheets, custom scripts, manual audits. They’ve already justified the business case. Your job is to make it efficient.

What’s next

We’re building deeper integrations with the tools these customers already use: JIRA for specs and stories, Figma for design validation, GitHub for commit-to-test traceability. The pattern emerging is clear: customers want specs (in whatever form) to become automatically testable and queryable.

What’s your problem? Find your use case or tell us what you’re building. Start a free trial 
or book time with the team.