We primarily use this tool to run AB tests on our website, answer our own hypotheses about what messaging and elements will drive growth and speak to the right audience, and measure those tests as well.
Pros
AB testing capability.
Ability to conduct tests inside of Optimizely (vs. pure redirect testing).
Reporting on test success.
Cons
We have had some reporting inconsistencies between your platform and our BI tools.
Some of the UX elements are confusing to understand.
It is extremely expensive.
Likelihood to Recommend
It’s definitely well suited to a high-volume SaaS product that serves consumers or prosumers. I don’t recommend it for smaller-volume tests or upmarket B2B because test results are too expensive to make a difference.
We are constantly running tests with Optimizely Feature Experimentation, usually 2-3 at a time. For Optimizely Feature Experimentation, we are testing customer journeys, customization of our product cards, CTAs and mobile at any particular time. Most useful have been basic UX tests, such as the inclusion of CTAs or 'add to cart' type buttons in different areas.
Lately, we've been running shipping/delivery messaging on the product page.
Pros
Reporting is clear and quickly useful
Integration with our codebase is simple, once initial set up was complete
Ability to start/stop is quick
Cons
Documentation for initial setup was confusing and difficult to parse
Likelihood to Recommend
Our Optimizely Feature Experimentation setup has helped us test product card display across multiple areas of the site all at once, including on homepage, product page, product recommendation carousels and the shopping cart. Because Optimizely Feature Experimentation works across different areas of the codebase from the same flag, we're able to track and manage the entire experience from one place.
VU
Verified User
Professional in Product Management (501-1000 employees)
I used Optimizely to run A/B experiments on a web platform with a consumers to validate the usage of new features. The business value is that it helps us understand the product/market fit of a feature before we make it permanent for all customers. And with Optimizely we could easily feature flag the experience to groups of users and control when to turn on or turn off the experiment.
Pros
Feature flags
Audience segmentation
SDK enables complex experiments
Cons
Reporting of user funnel activity in the tool
View only mode for experiment observers
App / Web in same experiment
Likelihood to Recommend
I believe Optimizely is valuable for two scenarios
1) optimizations of a user experience to find opportunities for quick wins (Web UI) - this is something all A/B tools can do and Optimizely seems to provide all the capabilities needed<div>2) testing new features before rollout to all audiences of development of a native app experience (SDK) - this is where I think Optimizely shines as this is harder to do with A/B testing tools.</div>
VU
Verified User
Consultant in Product Management (10,001+ employees)
We use Optimizely Feature Experimentation to conduct controlled A/B and multivariate experiments across our web and backend platforms. The core business problem it addresses is the need for data-driven decision-making—enabling us to test hypotheses about feature changes, UX optimizations, and algorithmic updates before rolling them out broadly. This helps reduce the risk of deploying underperforming or harmful changes, and accelerates learning cycles across product, design, and engineering teams.
Pros
Targeted experiments to specific user cohorts
Gradual feature rollouts
Backend experimentation to test service-level logic
Cons
Provide good web-extension made in-house by Optimizely
AI based Optimizely setup where product person can enter the requirement and AI can interpret it to setup feature
Likelihood to Recommend
Optimizely is leading the industry with experimentations. All my previous organizations were using Optimizely for feature releases. I would definitely recommend to colleague.
We use it for Product AB and MVT testing, to both assist us with splitting traffic between variants and also to provide standardised reporting and calling of results on those tests. The scope is any test within our Product that can be easily implemented within Optimizely 's interface, which typically applies to all UI/UX changes, but also some different algorithm or multichannel approaches
Pros
Splitting traffic between variants and enabling you to scale up or down the amount of traffic in each one
Giving a standardised report that you can share with a huge number of users
Showing a large variety of results/metrics you can then dive into
Cons
Would be nice to able to switch variants between say an MVT to a 50:50 if one of the variants is not performing very well quickly and effectively so can still use the standardised report
Interface can feel very bare bones/not very many graphs or visuals, which other providers have to make it a bit more engaging
Doesn't show easily what each variant that is live looks like, so can be hard to remember what is actually being shown in each test
Likelihood to Recommend
Good for standard Product UI/UX type experiments, such as changing the positioning of buttons on a page or how presented to user. Useful as well for adding in additional features and seeing how these are engaged with. Less useful for very complex testing that requires custom bucketing from DS/ML type teams, which may be easier to do offline
VU
Verified User
Director in Product Management (201-500 employees)
The use case was for a client to who wanted data on which user submission got better results; one with all of the form fields on the page, or one that had only a couple and was more of a multi step form to fill out.
Pros
Captured data
Offers advanced features
Has a free trial
Cons
Hard to sign up
A lot of work to implement
Tools aren’t easy to understand
Likelihood to Recommend
If you really wanted to a/b test a lot of specific areas of the site. And had a developer that can help install it. Less useful when you want to do just a single simple test, as it’s a lot of work to do.
We use Optimizely Feature Experimentation to test new design options on our website.
As our brand and business evolve, we need a way to quickly test ideas and creative concepts. We also need data to help us inform these decisions.
Feature Experimentation allowed us to quickly test multiple hypotheses and variations, which has been key to improving our site's language and UX.
Pros
Delivers statistical data in easy to understand format
Easily allows us to create variations without involving developers
Quick to make multiple variations.
Cons
Difficult integration if your data is not front end
Costly MAU model needs to be based on experiments not on site visits
It's not easy to understand how to build an Experiment
Onboarding team is more focused on punching through their slides and not focused on your needs or understanding.
Likelihood to Recommend
It's not as well suited for app testing, based on our experiences.
If there is a need for front-end web testing this is where it excels. There are capabilities to build complex implementations but those will require a developer and that is counter to the intent of WYSWYG editor, isn't it?
We are using Optimizely Feature Experimentation for multiple purposes across the organization. Business department along other departments like Operations and Product identify opportunities and build hypothesis to test.
During the first year we have been building tests to improve:
- Delivery costs (f.e trying to recommend PUDO as a shipping method)
- Increase average order value
- Increase CTR at different levels of the user journey (HOME, PLPs, PDPs)
- How removing "aggresive pricing labels" don't decrease CR
Pros
Easy to use UI where you can change HTML and add Javascript without involving IT Team (reduce timmings and provide freedom to experimentation team)
Building custom metrics using datalayer is the best.
Custom triggers to setup audiences
Cons
Flickering could be a nightmare in a client side solution. It would be great to be able to turn your solution client or server side in the same contract/solution.
Experiment results page could help more quicker in identifying "eternal" tests (tests where you get not enough difference between original and variants and the platform always say that 100K more events remaining). Maybe a warning message saying: Your test is not going well, maybe it's time to analyze why: Here is my impressions (Ask Opal).
It would be great if we could edit events directly from a test. Sometimes you need to create or edit an event to config your test, then you have to exit the test, click implementation, new/edit event. It's save a lot of time if you can jump from these to spaces directly.
Likelihood to Recommend
I think Optimizely Feature Experimentation is an awesome platform for product , IT and Analytics teams. These is not easy because you could find a better platform for product but then a pain for IT or Analytics teams to get the perfect trigger to fire the test or add custom metrics for example.
In the other hand, It's not your tool if you want to run test based on the same page but different designs, something crucial for product teams where you would like to test a new PDP page or HOME page. If you plan to run these kind of tests then you have to jump to another server side solution.
... and these opinion match with my previous thoughts where Optimizely could mix a solution for client side and server side like Google Tag Manager did some years ago.
VU
Verified User
Board Member in Product Management (51-200 employees)
We used to switch feature flags locally and using Optimizely Feature Experimentation has been extremelly helpful to know which flags are enabled and which ones are disabled. It has been a great tool to have!
Pros
UI interaction
functionality
easy to understand
Likelihood to Recommend
It does a great job at making it easy to use and understand. The UI is very user friendly, which makes it easier for new users to pick up and understand how to use it. I think that it also separates projects very effectively which is helpful when understanding which feature flags we should enable. I also enjoy being able to set custom targets. For example, we have two code bases for 2 products. We are able to enable for 1 and not the other, or 1 fully and half of the other. This makes is super easy to roll out functionality!
As the responsible of the experimentation and optimization center of excellence, I work with my team to support product owners in launching experiments through Optimizely. We use the tool to test new features before rolling them out to all users. This allows us to make data-driven decisions, reduce risk, and improve the customer experience. Our main use cases are in product and development, focusing on A/B testing and progressive rollouts.
Pros
Works well with Adobe Analytics, making it easier to track experiment results.
Easily test features without needing "a lot" of engineering effort.
Supports feature flags and gradual rollouts, which helps reduce risk when we make the roll out within out different shops
Cons
Calculating economic impact in base to our parameters of the winner ab test
Can improve in the pre analysis before launching the tests
Could improve is in the speed of results reporting
Likelihood to Recommend
Optimizely is great for gradual rollouts, we use it to release features to a small group, monitor impact, and scale safely. It also works well for quick A/B tests, letting us validate ideas fast with minimal dev effort. It’s less ideal for content-heavy or visual experiments where non-tech users need more hands-on contro
