Optimizely has grown to be one of the major A/B testing tools. It’s suitable for companies of all sizes — from startups and small businesses to major corporations. Optimizely’s tools and integrations allow anyone to quickly learn more about their audience and experiment with design and content changes non-destructively through A/B testing. With A/B tests you can confirm, for example, which CTA (call to action) leads to the most clicks, or which version of a checkout leads to the most online sales.
Optimizely also integrates with mobile apps and TV apps, meaning that you can analyze metrics and improve conversions on virtually any platform.
It sounds ambitious (and maybe even a little complex), but you can start out small with their simple visual editor, which lets you experiment with fairly simple A/B tests (no code necessary). When you feel you’re getting the hang of Optimizely (it’s actually really easy to use), you can integrate Optimizely more natively using their many SDKs. In this article, I’ll summarize the tools Optimizely offers and identify a suitable scenario for when they could be used.
Since it’s an ambitious tool with a huge offering, let’s break down the features.
What You Need Before Using Optimizely
While there are no official requirements for using Optimizely, you should already have your KPIs established beforehand. A KPI (key performance indicator) is a metric that you’re tracking to measure success — such as the number of user registrations, sales, newsletter signups and so on. In short, what are your website’s goals? Have them noted down.
(If you need some background to website goals, analytics and more, check out our recent articles on using analytics to improve UX, such as our introduction to Google Analytics.)
Your website (or app) should also have a reasonable amount of traffic, so that the data collected is informative enough. I would also recommend a minimum of 1000 visitors over 1–2 days on the page you’d like to optimize, to justify the costs of Optimizely services. If you’re not quite at these levels yet, consider using Google Optimize as a free alternative.
Optimizely offers a standard 30-day free trial for all of their services, so that you can experience the benefits for yourself and learn how to use it like a pro before investing any money. Like I said, to make the most of the 30-day trial, have your KPIs established and experiments thought out beforehand, to make the most of each free day.
Okay, so let’s dive into the features now.
Web Experimentation
Web Experimentation is essentially a visual editor for creating A/B tests on your website. With the editor, you can make content, color and layout ordering changes to create one or more variations of a web page.
Then, using the audience builder, you can target specific segments of your audience based on geolocation, time, gender and more. With these powerful targeting features, you can then test your variations on a very specific audience. The results are fed into a live reporting system, which gives you detailed feedback on all of your variations, to help you identify the one with the highest conversions, which you can then implement permanently.
Changes to code are not required, and your original version is not disturbed.
Web Personalization
Optimizely’s Web Personalization tool is similar to its Web Experimentation tool, but with a focus on A/B testing content rather than visuals, where the aim is to display content that’s relevant and interesting to the audience.
For example, consider the home page of a sportswear store. Visitors with a user persona matching that of a hiker could be shown a mountain-view background with the latest hiking shoes. To take this a step further, you could show hikers in colder locations thermal clothing and a snowy scene. With Optimizely’s exceptional targeting features, the options are endless.
By tailoring content to audiences based on their location, current weather conditions and more, your business can maximize sales and stand out from the competition by giving users exactly what they’re looking for.
Web Recommendations
With the Web Recommendations tool, Optimizely uses intelligent machine learning software to show the right things at the right time, to maximize engagement and conversions by displaying the content and products that users will likely want to see, based on a variety of factors. This recommendation engine allows you to experiment with algorithms, the placement of the recommendation widget, and the visual design of it.
Considering that Amazon’s recommendations engine is responsible for over 35% of Amazon’s sales, you’d be astounded by how useful this tool is!
Optimizely for Mobile Apps
Optimizely’s mobile tools allow you to integrate your A/B experiments directly into native mobile applications, using code. It encompasses a lot of Optimizely’s web features, but the focus on code-based experiments over the use of a visual editor means that you can instantly publish app updates without having to wait for approval from the iOS/Android app store.
Experimentation is full stack, so A/B tests can span both front end and back end, using any of Optimizely’s mobile SDKs (Android, Swift, Objective-C).
Full Stack (for Deeper Experiments)
Speaking of SDKs (and code-based experiments), Optimizely allows you to create your A/B experiments (for both web and mobile) using any of the programming languages we use today, including the three mobile app languages mentioned above. Aside from those, there’s also Python, Java, Ruby, Node, PHP, C# and JavaScript SDKs, which are all open source.
Regardless of what language your app or website is built in, Optimizely can be integrated into it, giving your team more control and total customization when compared to tests made with the visual editor. As mentioned before, you’ll be able to experiment with much more than visual and content changes.
If you were building a tool or feature for SaaS startup, you could create two entirely different versions of it to see which converts better.
Full stack integrations hook into the same targeting engine that Web Personalization uses, so you can utilize that same power to target audiences at a high level while having much greater customization within your website.
OTT (for TV apps)
Optimizely has created an OTT tool to allow creators of TV apps to A/B test variations across various OTT platforms such as tvOS and Chromecast. OTT stands for “over the top”, which means delivering video over the internet without the need to subscribe to a subscription service such as Sky or HBO.
Optimizely’s new OTT tool allows designers and marketers to experiment with visuals and content for different TV platforms — for example, showing a slightly different theme or layout for Apple TV than Vimeo or YouTube, to ensure that the visual experience looks and feels like an Apple experience.
Like with the mobile SDKs, instant feature rollouts mean that users will be shown the latest variations and experiments without needing to update their TV application, meaning that you’ll get valuable feedback instantly.
Analytics
Regardless of what Optimizely tools you’re using, the Analytics Statistics Engine is there to help you evaluate results. Traditional statistic engines like Google Analytics only give relevant results upon experiment completion. This means that if you look at your results mid-way through an experiment, they may be inaccurate by up to 30%! This can lead you to preemptively make the wrong design decision and end your experiment way too early.
Optimizely’s engine uses a new algorithm to show you live statistics mid-experiment, with an error margin of only 5%. This means that you’re much less likely to make the wrong decision regarding your experiments.
Use detailed reporting to evaluate your experiments’ performance. These reports utilize the unique analytics method created by Optimizely to deliver the most accurate results as quickly as possible, even mid-experiment.
Conclusion
Optimizely is really pushing the boundaries, providing A/B testing tools to markets that previously didn’t have them (such as OTT) while revolutionizing analytics and reporting to be more informative than ever before.
It’s suitable for roles of all kinds — designers, developers, marketers, managers, and all combinations of those working together. Optimizely ensures that whatever your skill level, there’s a tool that allows you to experiment with A/B testing for your app or website.
Frequently Asked Questions (FAQs) about A/B Testing Tools
What are the key features to look for in an A/B testing tool?
When choosing an A/B testing tool, there are several key features to consider. Firstly, the tool should have a user-friendly interface that allows you to easily design and implement tests. Secondly, it should provide detailed analytics to help you understand the results of your tests. Thirdly, it should offer a variety of testing options, such as split testing, multivariate testing, and multi-page testing. Lastly, it should have a robust support system, including tutorials, guides, and customer service, to assist you in using the tool effectively.
How does Optimizely compare to other A/B testing tools?
Optimizely is a popular A/B testing tool due to its comprehensive features and user-friendly interface. It offers a variety of testing options, detailed analytics, and a robust support system. However, it may not be the best choice for all businesses, particularly those with a tight budget or specific needs. Other tools, such as VWO, Unbounce, and Google Optimize, offer similar features at different price points and may be better suited to certain businesses.
What are some alternatives to Optimizely for A/B testing?
There are several alternatives to Optimizely for A/B testing. VWO is a popular choice due to its comprehensive features and competitive pricing. Unbounce is another good option, particularly for businesses focused on landing page optimization. Google Optimize is a free tool that integrates seamlessly with other Google products, making it a good choice for businesses already using Google Analytics or Google Ads.
How can I run an A/B test using Optimizely?
Running an A/B test using Optimizely involves several steps. Firstly, you need to define your test, including the pages you want to test and the changes you want to make. Secondly, you need to set up your test in the Optimizely platform, including defining your audience and setting your traffic allocation. Thirdly, you need to implement your test, either by using the Optimizely editor or by manually coding the changes. Lastly, you need to monitor your test and analyze the results.
What is the cost of using Optimizely for A/B testing?
The cost of using Optimizely for A/B testing depends on the plan you choose. Optimizely offers several pricing tiers, each with different features and benefits. The cost can range from a few hundred dollars per month for the basic plan to several thousand dollars per month for the enterprise plan. It’s important to consider your budget and needs when choosing a plan.
How can I interpret the results of an A/B test?
Interpreting the results of an A/B test involves analyzing the data provided by your testing tool. This typically includes metrics such as conversion rate, bounce rate, and average session duration. You should compare these metrics between your control group and your test group to determine which version of your page performs better. It’s also important to consider statistical significance when interpreting your results, to ensure that your findings are not due to chance.
Can I run multivariate tests with Optimizely?
Yes, Optimizely supports multivariate testing. This allows you to test multiple changes at once, rather than just one change at a time as with A/B testing. Multivariate testing can be more complex and requires more traffic than A/B testing, but it can provide more detailed insights into how different elements of your page interact with each other.
What are the benefits of A/B testing?
A/B testing offers several benefits. It allows you to make data-driven decisions about your website, rather than relying on guesswork. It can help you improve your conversion rate, reduce your bounce rate, and increase your average session duration. It can also help you understand your audience better, by revealing what types of content and design elements they respond to.
How long should I run an A/B test?
The length of an A/B test can vary depending on several factors, including the amount of traffic your website receives, the size of the change you’re testing, and the level of confidence you want to achieve. As a general rule, you should run your test until you have enough data to reach a statistically significant result. This typically takes at least a few weeks, but it can take longer for websites with low traffic.
Can I use Optimizely for mobile app testing?
Yes, Optimizely supports mobile app testing. This allows you to test changes to your mobile app in the same way you would test changes to your website. Mobile app testing can help you improve your app’s user experience, increase your app’s engagement rate, and boost your app’s conversion rate.
Jamie MurphyJamie has over 7 years experience working for International brands as a full stack developer & e-commerce/conversion optimization expert. Outside of work he likes to hike and is trying to learn new languages.
