Contentsquare vs. Optimizely Feature Experimentation

It is well suited to businesses with a full time web analyst that will be using the tools to create actionable reports that drive action in the company. It is less appropriate where companies are just looking for some new tools and it will be forgotten soon after implementation. If a website is critical to your business and you have dedicated resources or consultants to help you understand the data, then it is invaluable. I love it.

Based on my experience with Optimizely Feature Experimentation, I can highlight several scenarios where it excels and a few where it may be less suitable. Well-suited scenarios: - Multi-Channel product launches - Complex A/B testing and feature flag management - Gradual rollout and risk mitigation Less suited scenarios: - Simple A/B tests (their Web Experimentation product is probably better for that) - Non-technical team usage -

Incentivized

• The qualitative aspects of user experience are very well captured by ClickTale. We can get solid actionable insights through the various dashboards which track mouse movements, clicks, scrolls, etc.
• The visual conversion funnels give a very good high level view of landing page performance and how they work together.
• The video recordings are especially helpful. The user behavior captured is especially helpful in making decisions about user interface - form fields, call to actions, etc.

• Splitting traffic between variants and enabling you to scale up or down the amount of traffic in each one
• Giving a standardised report that you can share with a huge number of users
• Showing a large variety of results/metrics you can then dive into

Incentivized

• Looking at isolated recordings without creating a segment, we still have to use the old interface
• User interface has been massively improved but there are a few nags here and there that still need to be addressed
• Mobile and desktop should be shown side by side and not separately

Incentivized

• Difficult integration if your data is not front end
• Costly MAU model needs to be based on experiments not on site visits
• It's not easy to understand how to build an Experiment
• Onboarding team is more focused on punching through their slides and not focused on your needs or understanding.

Incentivized

For small companies with limited user testing budgets, ClickTale serves as a useful user testing tool. When I cannot get the funds for in depth user testing, I always know that I have a baseline of information I can rely on

Competitive landscape

Incentivized

I had some issues interacting with viewing recordings of a specific page by many users but my impression was that this was going to be fixed.

Easy to navigate the UI. Once you know how to use it, it is very easy to run experiments. And when the experiment is setup, the SDK code variables are generated and available for developers to use immediately so they can quickly build the experiment code

Incentivized

Support was there but it was pretty slow at most times. Only after escalation was support really given to our teams

Incentivized

It’s straightforward. Docs are well written and I believe there must be a support. But we haven’t used it

Incentivized

ContentSquare [(Clicktale)] is going deeper on UX understanding than traditional web analytics tools. You can truly understand how a page is used (where users click or even miss click, on which part of the page they are spending most of their time, if some links are clicked but bad positioned on the page...), and that's a thing you can't really measure trough a traditional web analytics tool.

Incentivized

In previous companies I've used Monetate which is a similar A/B testing kind of feature experimentation engine that is very similar from my memory, but again, back to the point of these new features of the analytics engine and Opal, it kind of cuts it above Monetate from my experience. Obviously Monetate may have improved since when I lost use it, but from what I can see, yeah.

Incentivized

had troubles with performance for SSR and the React SDK

Incentivized

• I learned how effective some of our image carousels were. How only 10% of a page visitors were being exposed to only the first slide. People were scrolling down or leaving the page without ever being exposed to 90% of the content. Once I provided this input to stakeholders it was an easy sell to redesign this aspect of the page.
• I used the mouse-move heat map to analyze user interaction with the footer. Showing stakeholders the before and after redesign heat maps did wonders for improving my credibility as an usability analyst.
• We used Clicktale to help analyze our 404 error page effectiveness. Our redesign gave us a 14% lower bounce rate on our redesigned 404 error page. Stakeholders appreciated a quantitative measure to gauge the success of that project.

• We have a huge, noteworthy ROI case study of how we did a SaaS onboarding revamp early this year. Our A/B test on a guided setup flow improved activation rates by 20 percent, which translated to over $1.2m in retained ARR.

Incentivized