Ready to dive into the world of A/B Testing?
Start with these commonly asked questions
- When should I start my A/B test?
- How can A/B testing help with my channel growth?
- Why can’t I run an A/B test on my video before it’s been published?
- How does your A/B Testing work?
- When should I select a time period vs statistical significance?
- What is statistical significance?
- Does TubeBuddy automatically apply the winning results to the video?
- How long will it take for a test to finish when reaching Statistical Significance?
- How often does TubeBuddy swap between the original and variation during the A/B test?
- Should I test two or more elements (thumbnails, titles, tags, descriptions) at the same time?
When should I start my A/B test?
We recommend that tests be run on videos at least 2-3 weeks after they have been published to give the most helpful, insightful results since YouTube heavily promotes new videos during this time. Otherwise, impression data from the tests is then going to be skewed and you won't be able to accurately determine which metadata element you're testing is truly performing better (and not just due to YouTube notifying your subscribers and pushing out your video to browse features).
How can A/B testing help with my channel growth?
Our A/B testing tool intends to provide you with data that helps inform you of what works and what doesn’t work when it comes to thumbnails, titles, tags, and descriptions. (For instance: are viewers more drawn to a thumbnail with a person’s face in it or not? Do viewers click more on videos with titles that are on the silly side or when they’re more informative?)
Why can’t I run an A/B test on my video before it’s been published?
The reason that the scheduled video (and any videos that are unlisted or private) does not show the option to run A/B testing is that the video needs to be live to gather the data needed to calculate which title, thumbnail, tag, description, etc. is performing better. If the video is private or unlisted, it is not going to be shown to viewers at all and therefore information regarding impressions and CTR cannot be gathered.
How does your A/B Testing work?
Our A/B testing starts with having you create a variation of a video's metadata (this could include the thumbnail, title, tag, or description). Our tool then alternates between your Original and the Variation every 24 hours until the test is completed either based on a certain number of days that you picked or based on statistical significance being achieved.
When should I select a time period vs statistical significance?
This is a matter of preference, though we recommend selecting statistical significance since you might not receive enough data in a designated time period that will conclusively tell you whether or not the variation is successful.
What is statistical significance?
Statistical significance is how likely it is that the difference between your experiment’s control version (original) and test version (variation) isn’t due to error or random chance. You can learn more in this article.
Does TubeBuddy automatically apply the winning results to the video?
Your video will automatically get set to the winning version once Statistical Significance is achieved (please note that this will not be done with tests run for a set number of days).
How long will it take for a test to finish when reaching Statistical Significance?
The test length can be as quick as 2 days or last weeks depending on how many times YouTube displays it on screen and how well your variation CTR is performing against the original.
How often does TubeBuddy swap between the original and variation during the A/B test?
We alternate your video's metadata - whether that’s a change in thumbnails, title, or description - every 24 hours at midnight PST (to line up with YouTube Analytics statistics). In an ideal world, we'd be able to have each impression throughout the day alternate between the Original and Variation which would be a “true” A/B test. Since YouTube analytics are only provided in 24-hour blocks, our current system is the best that can be done taking into account YouTube's limitations.
Should I test two or more elements (thumbnails, titles, tags, descriptions) at the same time?
Although you can do this in our A/B test tool, we do not recommend testing more than one element at the same time because it will become difficult to pinpoint what was successful or not. For instance, if you test both a title and thumbnail change on the same video and the variation wins, you will be unable to know whether the impact had more to do with the title or thumbnail.