Testing Suite

OVERVIEW

Transforming a "Black Box" into a high-precision testing environment for financial products

Policy Managers are responsible for the logic that powers financial products like loans and credit cards. This isn't just one or two rules; it is a complex web of hundreds of micro-decisions—verifying identity, detecting fraud, and calculating risk.

The Problem: Previously, managers had no way to test these rules themselves. They relied on Engineers to run "Black Box" scripts to verify logic. This was slow, opaque, and created a high risk for manual errors.

The Solution: I designed a Self-Service Simulation Suite that allows managers to test their own rules in a safe sandbox—ensuring they don't make a multi-million dollar mistake before the rules go live.

PCM

THE DISCOVERY

Finding the "Ground Truth"

By shadowing the QA team and analyzing their manual testing workflow, I identified two non-negotiable requirements to bridge the gap between technical stability and business safety:

THE CHALLENGE

Beyond the "Black Box"

Strategic Conflict: Speed vs. Accuracy

To hit a tight deadline, the initial plan was to build a "dummy" form that wasn't connected to the bank's live data. I challenged this approach. I argued that proving the software works is not the same as proving the policy works. I advocated for a data-driven architecture that pulls from the live database, ensuring managers test against reality rather than a placeholder.

Managing Information Scale

A typical policy uses ~70 rules, but those are pulled from a library of 1,000+ variables. I had to design a workspace that felt lightweight for daily tasks while providing "surgical" access to the full data library. The challenge was hiding the 900+ points of "noise" without sacrificing the power of the tool.

THE ADVOCACY

Negotiating with engineering

I sat down with our lead engineers. I told them: "The generic template is a waste of time. We need to mirror the live database."

SOLUTION

Three Part Design

1. The Mirror Environment (Testing Against the "Ground Truth")

To ensure Intent-Output Alignment, I designed a workspace that mirrors the live production policy. Users don't just look at the code; they interact with a live simulation.

The Workflow: A manager can input a specific value—for example, setting a FICO score to 500—to verify that the policy correctly triggers a "Denied" status.

The Value: This provides immediate, visual proof that the policy logic is behaving exactly as intended before it is deployed to real customers.

MIRROR ENVIRONMENT INTERFACE

2. Multi-Persona Stress Testing (Solving for "The Ripple Effect")

Testing one person at a time isn't enough to prove a policy is safe. I designed a Batch Persona Engine that allows managers to run multiple "what-if" scenarios simultaneously.

The Design: Managers can create a library of diverse personas (e.g., a high-debt student, a retired veteran, a first-time flyer) and run them all against the mirrored policy with one click.

The Value: This allows the manager to see if a rule change intended for one group accidentally "breaks" the approval for another, catching logic conflicts at scale.

BATCH PERSONA ENGINE

3. Smart Sorting & AI "Shortcuts" (Solving for "Information Scale")

Navigating 1,000+ variables to find the one you want to "poke" is a major bottleneck. I implemented two design interventions to keep the workflow fast:

The "Recently Modified" Pinning: Any variable the manager edits is automatically moved to the top of the list. They no longer have to hunt for their "mock inputs"—their active workspace stays front and center.

The AI Logic Agent: I integrated a natural language command bar. Instead of manually scrolling through 1,000 variables to find "Minimum Income Requirements," a manager can simply ask the AI to find it and change the value for them.

AI COMMAND BAR & SMART SORTING

RESULTS & IMPACT

From zero to hero