Part of a research program. UXR-226 ran alongside UXR-224, which validated the checkout savings progress bar concept. Both studies addressed the same product area, and their findings were reviewed together when making the final production recommendation. This is how a research program compounds value: individual studies inform each other.
The challenge
Two approaches to communicating the same information. Which one works?
With the savings progress bar from UXR-224 moving toward production, the team needed to evaluate two competing banner designs for communicating the progressive discount and free shipping system to cart customers. Both banners communicated the same information. They just did it in structurally different ways.
Design A
Status-Forward
Show customers where they are and how close they are to the next tier.
- Highlights current discount tier prominently
- Visual progress bar toward the next tier
- Shows savings applied and shipping status simultaneously
Design B
Instruction-Forward
Tell customers what to do to unlock the next tier.
- Leads with explanatory messaging about how the system works
- Emphasizes what to add to reach the next tier
- Less visual emphasis on current status
Research questions
Five questions the study needed to answer.
1
Do users correctly understand how the savings system works?
2
Which concept better communicates discount progression?
3
Which concept more clearly communicates the free shipping threshold?
4
How much cognitive effort does each design require?
5
Which design do users prefer overall?
The results
The data was unambiguous on the metric that matters most.
Free shipping threshold clarity
Design A
90% correctly identified threshold
A 70-point gap in the metric that matters most.
When the majority of users can't correctly identify the free shipping threshold, the design is failing at its core job — regardless of how any other metric scores. That number made the primary recommendation clear.
The nuance
Design A wins. Design B has something worth keeping.
Design B's explanatory approach — telling users what to do to unlock the next tier — showed stronger discount comprehension in one study. The framing works for discount mechanics even when it fails for shipping clarity. That's not a failure to discard; it's a signal about what to borrow.
The recommendation is a hybrid, not a binary. Ship Design A's visual structure with targeted copy from Design B's instructional language applied to the discount tier sections. Get the clarity of A with the comprehension strengths of B where B actually outperformed.
Primary
Ship Design A
The 70-point gap in free-shipping clarity is the deciding factor. Design A correctly communicates the threshold to 90% of users — the core job of this banner. This is the defensible call.
Secondary
Incorporate Design B's instructional copy
Design B's explanatory framing showed stronger comprehension of discount progression mechanics. Selectively apply that copy approach to Design A's discount tier sections without disrupting its structural clarity. Get the best of both.
Reflection
What worked and what I'd do differently.
What worked well
- Testing both zero state and progress state gave a complete picture of each design across the user journey — not just how it looked at the starting point.
- Running UXR-224 and UXR-226 in parallel built a richer evidence base for the production decision than either study alone. The findings reinforced each other.
- Identifying the nuance in Design B's copy strength meant the recommendation wasn't a binary winner/loser — it was actionable guidance the team could actually use and implement.
What I'd do differently
- Add a moderated component for at least a subset of participants — seeing where users pause or re-read would surface in-the-moment confusion that post-task questions miss entirely.
- Test the hybrid design explicitly before recommending it rather than inferring it would work. The recommendation to blend Design A structure with Design B copy was logical but the hybrid itself wasn't validated. That gap should have been closed before production.
Outcome
Design A recommended for production with targeted copy elements from Design B applied to the discount tier sections. The study directly informed the production decision — and reinforced the value of running paired studies that address the same product area simultaneously.
Common questions
Comprehension studies and comparative banner design research
How do you measure comprehension of a checkout banner design?
Comprehension of a checkout banner is measured through structured tasks that require participants to correctly identify specific information from the design — the free shipping threshold, the current discount tier, how much more they need to spend to unlock the next tier. These are scored as correct or incorrect, not as ratings. The scoring is combined with rating scales for perceived ease and clarity, and open-ended questions to capture the reasoning behind responses. The combination reveals both whether users understood the design and why they did or didn't.
When should a UX recommendation be a hybrid rather than choosing one design?
A hybrid recommendation is appropriate when two competing designs each outperform the other on distinct, separable dimensions. In this study, Design A was clearly superior on the most critical metric — free shipping threshold clarity — while Design B's explanatory copy approach showed stronger comprehension of discount progression mechanics in one study. Because these strengths operate on different parts of the design, it's possible to incorporate Design B's copy strategy into Design A's structural framework without creating conflicts. A hybrid recommendation is only credible when the rationale is specific and testable, not a diplomatic compromise between stakeholder preferences.
What is a research program in UX and how is it different from a single study?
A research program is a series of studies that address the same product area from different angles, with findings that build on each other over time. UXR-224 and UXR-226 are both part of the same research program — they addressed the checkout pricing experience from different directions, and their findings were reviewed together when making the final production recommendation. A single study answers one question with one sample. A research program builds a layered evidence base where each study increases confidence in the overall picture and reduces the risk of acting on a finding that doesn't hold across contexts.
What's the difference between a status-forward and instruction-forward design approach?
A status-forward design prioritizes showing users where they are — what discount they currently have, how close they are to the next tier, what they've already earned. It emphasizes current state. An instruction-forward design prioritizes telling users what to do — what to add to their cart, how much more to spend, what they'll receive if they take action. It emphasizes next action. The tradeoff is between clarity about present status and motivation to take the next step. This study found that status-forward (Design A) was significantly better at communicating the free shipping threshold, while instruction-forward (Design B) showed some advantage in discount progression comprehension.