60-second story
From a questionable pricing logic to a no-further-rollout recommendation
Problem
A crossed-out price appeared even on surcharges. The learned deal signal contradicted the actual price relationship.
Approach
I challenged the logic and tested the risky pricing display against a crossed-out price shown only on genuine savings in an A/B test.
Key insight
No evidence for a loss of trust, but a clearly measurable product risk.
Decision
The evidence supported a clear no-further-rollout recommendation: show crossed-out prices only on genuine savings.
Outcome
The risk variant was discontinued before it became a standard.
−16.6%
Purchase conversion, >99% reliability
−18.5%
Revenue per visit, >99% reliability
Discontinued
Risk variant
01 CONTEXT
A deal signal with no actual savings
A crossed-out price was meant to signal savings. In the offer box, however, it also appeared when the selected option was more expensive. The visual deal signal thus contradicted the actual price relationship.
The logic had been introduced to make price changes transparent. What remained unclear was whether a crossed-out price on surcharges negatively affected the booking decision.
To avoid basing the decision on assumptions, I set up an A/B test with clearly separated variants and an unambiguous metric hierarchy.
THE CORE DECISION
Continue showing crossed-out prices on surcharges or display them only on genuine savings?
BUSINESS SIGNAL
A misleading price signal could jeopardize both purchase conversion and revenue simultaneously.
02 HYPOTHESIS
IF
the crossed-out price is hidden on surcharges and shown only on genuine savings,
THEN
the comparison variant achieves higher purchase conversion; checkout-adjacent metrics trend in the same direction.
BECAUSE
the comparison variant is more consistent with the learned meaning of a crossed-out price. Whether this consistency changed trust or reduced irritation was not directly measured.
03 EXPERIMENT DESIGN & METHODOLOGY
An A/B test as risk control – not routine optimization
For a robust product decision, the risky pricing logic was tested against a comparison variant that showed the crossed-out price only on genuine savings. Evaluation followed a clear metric hierarchy.
01 SETUP
Randomized 50/50 allocation via feature flag
Offer page, all devices
02 EXPOSURE
19 test days;
approx. 78k visits per variant
03 METRICS
Primary: Purchase CR
Business: Revenue per Visit
SECONDARY: Click “Proceed to Booking”, Access Checkout, Bounce Rate, Device splits exploratory
CUPED increased the precision of the purchase evaluation. Reliability refers to the statistical significance, i.e. the confidence level in the method used; device splits remained exploratory.
04 SOLUTION
What was tested was behavior and pricing logic – not trust itself
The experience remained identical. The only change was whether a crossed-out price remained visible on surcharges. That is why I name the variants by their visible logic – not by the contradictory flag coding.
ORIGINAL A: Crossed-out price even on surcharges
PRICE DETAIL · ORIGINAL SCREEN
Comparison variant B: Crossed-out price only on savings
PRICE DETAIL · ORIGINAL SCREEN
Evidence boundary: The test measures behavior. Whether irritation or loss of trust caused the effect remains an interpretation. The original screens shown are approved for portfolio publication.
05 RESULTS
The risky pricing logic lost at two critical points
The variant “crossed-out price even on surcharges” reduced purchase conversion and revenue per visit.
−16.6%
PURCHASE CONVERSION
>99% reliability
−18.5%
REVENUE PER VISIT
>99% reliability
No evidence for loss of trust, but a clearly measurable product risk
SECONDARY: Click “Proceed to Booking” −5.9%, Access Checkout −6.3%, Bounce Rate without clear difference
06 DECISION SHIFT & PRODUCT RESPONSE
The UI question led to a clear no-further-rollout recommendation
The evidence was not sufficient for a trust theory, but strong enough for a clear product response.
01 EVIDENCE
Purchase and RPV declined significantly
02 RECOMMENDATION
Do not continue the always-on logic
03 PRODUCT RESPONSE
Crossed-out price only on genuine savings
The result stopped an avoidable risk before it became a standard
07 OWNERSHIP & COLLABORATION
I owned both the test and the quality of the decision
MY OWNERSHIP
Initiation, Hypothesis, Experiment Design, Variant Review, QA, Analysis, Device Splits, Stakeholder Readout, Recommendation
TEAM CONTRIBUTION
Product and Development: technical review, feature flag implementation, QA support and decision on further product logic.
End-to-end experiment owner: from critical questioning through to the decision brief
08 REFLECTION & EVIDENCE BOUNDARIES
What the experiment proved and what it did not
CONFIRMED
The always-on variant significantly reduced purchase conversion and revenue per visit.
Checkout-adjacent metrics trended in the same direction.
The no-further-rollout recommendation was data-driven.
NOT CONFIRMED
Irritation or trust as the underlying cause.
Causal device-level differences.
Long-term or annualized business impact.
Strong experimentation also means not making the narrative bigger than the evidence.
09 KEY LEARNINGS
