60-second story
From neutral payment scheduling to a validated conversion lever
Problem
The checkout offered financial flexibility but presented it as neutral scheduling information rather than a booking incentive.
Approach
I translated a psychologically grounded hypothesis into an A/B test with benefit-oriented payment communication and owned it end to end. Users were randomized 50/50.
Key insight
The same payment logic, framed as a benefit instead of neutral information, measurably changed booking behavior.
Decision
The tested communication package achieved +26% purchase conversion in the combined desktop and phone analysis, establishing a clear rollout recommendation.
Outcome
The team prioritized implementation and the production solution follows the tested direction. The test measured behavior, not the underlying psychological mechanism.
+26%
Purchase conversion, >99% reliability
+28%
Revenue per visit (approx.), >99% reliability
Now live
Implemented by the product partner; no reliable guardrail change
01 STARTING POINT
The checkout offered financial flexibility but failed to communicate it as a benefit
On an online travel platform, users could pay a partial amount at checkout and settle the remainder later. However, the existing interface presented this option as neutral scheduling information. The open question was whether a benefit-oriented framing would influence booking completion. The experiment aimed to determine if the new communication was strong enough to support a rollout decision.
BUSINESS SIGNAL
Conversion uplift was a core e-commerce objective
One specific lever lay in the presentation of the payment structure and price communication close to the point of purchase.
02 PRODUCT DECISION
CRITICAL UNCERTAINTY
Does benefit-oriented communication of the existing payment logic increase purchase conversion reliably enough to prioritize the variant for rollout?
WITHOUT EVIDENCE
Design preference instead of decision basis
WITH EVIDENCE
Rollout, iteration or discard
03 HYPOTHESIS
IF
the payment structure is communicated as a benefit rather than neutral information,
THEN
Variant B should achieve higher purchase conversion than the existing presentation,
BECAUSE
users perceive the booking as more financially accessible.
By highlighting the lower immediate payment (“Pay only X € now”) and framing the remaining amount as a later payment step, perceived short-term costs and payment-related uncertainty are reduced.
WHAT THE HYPOTHESIS DOES NOT DIRECTLY MEASURE
Whether users perceived less pressure, more trust or a different price anchor was not directly assessed. The experiment tested behavior – not the underlying psychological mechanism.
04 SOLUTION
01
Benefit card makes the advantage visible
02
Copy shifts focus to payment flexibility
03
Now/Later creates a clear hierarchy
INTERPRETATION BOUNDARY
Since multiple elements were changed simultaneously, the test demonstrates the effect of the entire variant package – not the contribution of individual components. The original screens shown have been approved for portfolio publication.
05 EXPERIMENT DESIGN & METHODOLOGY
The experiment was designed around the rollout decision
Purchase conversion was defined as the primary metric upfront. Users were randomized 50/50 between control and Variant B. The published overall analysis covers desktop and phone; device splits were reviewed as supplementary analysis.
01
50/50 traffic split
02
Fixed sample (frequentist)
03
Desktop + phone
(published overall analysis)
04
Test period Oct 29 – Nov 11, 2025
PRIMARY KPI
Purchase conversion
Decision metric
SECONDARY KPI
Revenue per visit
Business relevance
GUARDRAIL
Average cart value
No degradation
ANALYSIS NOTE
Tablet was part of the test but was excluded from the published overall analysis due to very low volume. A sensitivity check including tablet confirmed both the effect direction and the rollout decision.
06 RESULTS
RELATIVE UPLIFT
+26%
Purchase conversion
Overall result for desktop + phone. Phone showed the stronger, statistically reliable effect; desktop showed the same positive direction but below 90% reliability.
NORMALIZED INDEX · CONTROL = 100
Control
100
Variant B
126
STATISTICAL SIGNIFICANCE
Reliability >99% – statistically significant in the applied methodology.
PRACTICAL SIGNIFICANCE
Substantial relative effect on the primary conversion metric.
BUSINESS METRIC
+28%
Revenue per visit
Relative uplift; statistically reliable per experiment platform.
GUARDRAIL
No reliable difference
Average cart value
The analysis showed no statistically reliable change.
07 EVIDENCE BOUNDARIES
CONFIRMED
The tested package influenced booking behavior
approx. +26% purchase conversion (highly significant)
approx. +28% revenue per visit (highly significant)
Evidence supporting a rollout recommendation
NOT ATTRIBUTABLE FROM THIS TEST
Why the variant package worked remains open
reduced payment pressure or increased trust
contribution of individual copy or UI elements
causal long-term effect post-rollout
EVIDENCE CHAIN
01 OBSERVATION
Variant B outperformed control on purchase conversion and RPV.
02 FINDING
The relative uplifts were reported as highly statistically reliable.
03 INTERPRETATION
The combined communication package changed observed booking behavior.
04 INSIGHT
The tested communication package around the payment logic was a clearly measurable conversion lever.
05 IMPLICATION
The tested direction was strong enough for a rollout recommendation.
06 DECISION
The team prioritized implementation based on the test results.
07 PRODUCT IMPACT
The production solution follows the tested direction.
08 DECISION SHIFT & PRODUCT RESPONSE
01 MY ROLE
Executed the test end-to-end, analyzed results and synthesized decision-oriented findings
02 COLLABORATION
Recommended rollout and jointly aligned design and edge cases with the booking engine (IBE) partner
03 PRODUCT TEAM
Decided to reprioritize and incorporate into the product roadmap
04 PRODUCT PARTNER
Solution was technically implemented and is now live
DECISION SHIFT
Before the test: psychologically grounded assumption
After the test: evidence-based rollout decision
09 IMPLEMENTATION & OWNERSHIP
MY OWNERSHIP
End-to-end experiment ownership
Hypothesis and research rationale
Copy and UX/UI concept
Test setup and execution
Analysis and synthesis
Rollout recommendation and stakeholder communication
Design and edge-case support for implementation
TEAM & PARTNER
Product decision and implementation
The team made the prioritization decision. The product partner owned the technical implementation.
RECONSTRUCTION · LIVE SOLUTION
Payment schedule
YOUR BENEFIT
More time to save
Pay only a portion today and the rest later.
Travel deposit
Amount anonymized
now
Remaining payment
Amount anonymized
later
Payment dates are presented in a clear, accessible format.
The live solution stayed very close to the tested direction. An independent causal post-rollout validation is not part of this case.
REFLECTION
A bundled test accelerated the decision but did not isolate individual effects
The test answered the rollout question quickly and reliably. Should knowledge about individual contributing factors be needed later, copy, benefit card and visual hierarchy would need to be tested separately.
10 REFLECTION
MY KEY LEARNINGS
What I took away methodologically from this experiment
A variant package accelerates decisions – but limits the causal attribution of individual changes.
AI-assisted implementation can make more complex variants testable faster; QA and clean tracking remain non-negotiable.
A pre-defined analysis framework with a primary metric and segmentation logic reduces post-hoc interpretation.
End-to-end ownership keeps hypothesis, execution, analysis and rollout recommendation aligned.
TEAM LEARNINGS
What the test revealed for product development
Payment communication is a relevant area for experimentation and should be developed further on a hypothesis-driven basis.
The tested benefit-oriented communication package outperformed neutral payment scheduling; the individual contribution of copy, card and hierarchy remained open.
Rollout decisions become more robust when statistical and practical significance are considered together.
Clear evidence boundaries prevent a successful test from explaining more than it actually measured.
Close collaboration between research and product shortens the path from evidence to implementation – without blurring ownership.
KEY LEARNING
For my experiment prioritization, I take away that changes close to the binding decision moment can have high leverage potential – and therefore require outcome metrics and guardrails.
