{% extends "layouts/base.html" %} {% block title %} A/B Test Promotion {% endblock %} {% block stylesheets %}{% endblock stylesheets %} {% block content %}
A/B Test Promotion
Dimension selection
  • A/B Test Promotions are bunch of testes which compares effect of Promotions to the business. There are two metrics which help us to measure comparison values. These are Number of orders counts per promotion and Average purchase amount per promotion. Promotion of Usage related to these metrics are tested with Before - After Analysis in order to th answer the question that is "Is there any significant increase/decrease on Order Count / Average Payment Amount per User after the Promotion is used?" While we need to see to Promotion which increases/decreases the number of orders and the average purchase amount, This Section can help us to find the right promotion.
  • Hypothesis are;
    H0 : These is no significant Difference with A - B of order count / purchase amount.
    H1 : These is significant Difference with A - B of order count / purchase amount.
  • Bootstrapping Method;
    Each test process are applied with Boostraping Method. So, Iteratively randomly selected A and B sample customers of order count / average purchase amount will be tested acceptable enough time. Each iteration A - B samples are randomly selected. Accepted ratio will show us the confidence of our test results. For instance, There are 100 tests and 75 of them are ended with H0 Accepted. So, test are % 75 our test are H0 Accepted.
  • How are Promotions Compared?
    Each combination promotions are tested individually. Average of Purchase Payment Amount per User Sample for Promotion (1) is compared to Average of Purchase Payment Amount per User Sample for Promotion (2).
  • How are Before - After Promo Usage Tests designed?
    It might be very confusion and it might seem very sophisticated A/B Test process, however it basically compares promotions related to customers of usage. Each promotion of usage is tested separately. Each promotion of used users and their used timestamp are collected for each promotion. Each customers of one week before promotion usage and one week after the promotion use time periods are compared with to metrics which are order count and average purchase amount. We aim to detect significant increase after time period while comparing to before time period about purchase amount or order count.
    related charts are;
    'Before - After Promo Usage Tests are shown at 'Order And Payment Amount Difference for Before And After Promotion Usage',
    'Before - After Time Periods Customers' Average Purchase Payment Amount Test (Test Accepted!) ',
    'Before - After Time Periods Customers' Average Purchase Payment Amount Test (Test Rejected!)',
    'Before - After Time Periods Customers' Total Purchase Count Test (Test Accepted!)',
    'Before - After Time Periods Customers' Total Purchase Count Test (Test Rejected!)'
Promotion Comparison {% if data_type['promotion_comparison'] == False %} sample data {% endif %}
- Y-axis represents total number Test Accepted Count.
- X-axis represents Test Reject Ratio.
- Each dot on scatter plot represents each promotion.
- This is a comparison of each promotion pairs. Each promotion pair is tested and Each dot represents 1st promotion of each promotion pairs.
- More valuable Promotions are the dots which have less Reject Ratio and the more number of Accept Count.
Order And Payment Amount Difference for Before And After Promotion Usage {% if data_type['promotion_usage_before_after_amount_accept'] == False %} sample data {% endif %}
- Y-axis represents difference of Average Payment Amount per customers.
- X-axis represents difference of Average Purchase Count per customers.
- Differences are calculated per user and per promotion.
- Differences are calculated between customers of purchase before and after using the promotion.
Before - After Time Periods Customers' Average Purchase Payment Amount Test (Test Accepted!) {% if data_type['promotion_comparison'] == False %} sample data {% endif %}
- A : Avg. purchase amount Before the promoted Order for each promotion of usage per customer.
- B : Avg. purchase amount After the promoted Order for each promotion of usage per customer.
- Test Accepted : Significant Increase on Avg. Purchase Amounts after the Promoted order.
- Each Test are applied individually (per promotion).
Before - After Time Periods Customers' Average Purchase Payment Amount Test (Test Rejected!) {% if data_type['promotion_usage_before_after_amount_reject'] == False %} sample data {% endif %}
- A : Avg. purchase amount Before the promoted Order for each promotion of usage per customer.
- B : Avg. purchase amount After the promoted Order for each promotion of usage per customer.
- Test Rejected : Significant Decrease on Avg. Purchase Amounts after the Promoted order.
- Each Test are applied individually (per promotion).
Before - After Time Periods Customers' Total Purchase Count Test (Test Accepted!) {% if data_type['promotion_usage_before_after_orders_accept'] == False %} sample data {% endif %}
- A : Total P. Count Before the promoted Order for each promotion of usage per customer.
- B : Total P. Count After the promoted Order for each promotion of usage per customer.
- Test Accepted : Significant Increase on Total Purchase Counts after the Promoted order.
- Each Test are applied individually (per promotion).
Before - After Time Periods Customers' Total Purchase Count Test (Test Rejected!) {% if data_type['promotion_usage_before_after_orders_reject'] == False %} sample data {% endif %}
- A : Total P. Count Before the promoted Order for each promotion of usage per customer.
- B : Total P. Count After the promoted Order for each promotion of usage per customer.
- Test Rejected : Significant Decrease on Total Purchase Counts after the Promoted order.
- Each Test are applied individually (per promotion).
{% endblock content %} {% block javascripts %} {% endblock javascripts %}