Before setting up A/B Testing, you will want to create an “A” Message and a “B” message. Essentially two counterpart messages that you want to compare. These messages can have different message content, different subject lines, and even be sent at different times.
You can create a copy of a message and simply change one thing between the two messages. However you decide to test your messages, be sure that you have two created for later comparison of open and/or click-through rates.
After creating your two counterpart messages, you can set up A/B Testing from the Filters section within your message. You can find A/B Testing under “Show Advanced Filters.” From here, you will work in the section titled “Only Include Orders Ending in These Digits” field. This is where you will set up your A/B Test.
Next, divide up your orders based off the numbers contained in your customer’s Order IDs. For example, in Message A you would enter in 12345 and that message would only be sent to orders ending in those digits. Conversely, in Message B you would enter in the remaining numbers which would be would be orders ending in 67890. You can also divide up your orders by even or odd numbers. For example, Message A could be sent to orders ending in even numbers 24680 and Message B could then be sent to orders ending in odd numbers 13579. (Please note that while you do not need spaces or commas between these numbers, they do represent the single ending digit of the order ID, and not the last 5)
After dividing up your messages and order numbers, you can check on your Message Analytics for both messages to see which message yielded higher open rates. You can check as frequently as every week, or allow the A/B to continue for at least a month to get a clearer idea of your message’s performance.