Katalon vs. ReadyAPI

Free trial version is fine but you don't get the adequate support and VFM for the licensed version. Customer service needs to be improved a lot and any QA's with less technical background or less time to troubleshoot issues in the software on their own would find it difficult to use.

As stated, we do a LOT of API testing, the swaggerhub import makes it easy to add APIs. This is very well-suited, as well as easy management of the steps/cases/suites inside of ReadyAPI. The one thing I do wish ReadyAPI was better suited for is changes to data, we have a lot of test cases in ReadyAPI and if we make a change to how the backend data is structured, one-by-one adjustments need to be made to the steps. Less appropriate, UI testing.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• SoapUI NG Pro is a prefect tool for setting up complicated test cases with many steps including parameterized Web Service requests, response assertions, data generators, data sources, data sinks (report recorders) and more
• You can use Groovy or JavaScript for more complicated automation such as validating the results, extracting data, using external Java libraries or running system processes. Groovy IDE is compatible with Java and you can easily write your code in Java with very few modifications.
• SoapUI enables you to run multiple test suites in parallel or in a loop, and provide user friendly reports including all test case results and test case coverage.
• SoapUI NG Pro lets you to perform security test against your web services with predefined scenarios such as xml bombing, xml injection, sql injection, buffer overflow tests, monkey testing and so on.
• SoapUI NG Pro, offers a simple but practical solution for low to medium load testing scenarios. You can obtain many more Load and Performance testing features by extending your license with purchasing LoadUI Pro.

Incentivized

• Katalon Studio supports only grovvy language for now. It will be nice to see support for other scripting languages.
• Performance issues; sudden crash of the tool.
• Debug button is provided only to licensed owner since Katalon 7+ version.
• New tool, few resources available to learn. So there is a mild learning curve here.

Incentivized

• REST - They have come a long way, but there could still be improvements here. I find the learning curve much higher and not as straight forward using REST vs SOAP.
• Composite Projects - I'd really like to see them implement something around saving/refreshing Test Suites when using the Composite project ability in Pro. This is currently an enhancement in feature backlog (see their forum for more details). Functionality around Projects and Test Cases seems pretty sound.

I would definitely renew the ReadyAPI as I was pretty happy while using it. But then I switched my job, and the current workplace is using Postman so we are using it for now. Plus, I don't have any issues with Postman. At the end of the day, it about selecting the tool that gets your work done more efficiently.

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

SoapUI allows us to combine multiple tests and adhere
to the sequence that they need to run in order to complete successfully.
It has an excellent GUI design and the reporting mechanism is also very
good. It does consume a lot of memory though during concurrent testing

Soap UI has managed to continuously build on it's solid foundation and keep improving by each release. It is by far the most dependable and accurate testing tool out there of its kind. Available via connecting to VM's created as SoapUI test machines give access to it anytime, anywhere practically.

It has an excellent GUI design and the reporting mechanism is also very
good. It does consume a lot of memory though during concurrent testing.

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

For my purposes, the provided help features have been sufficient. I am sure I would be better off if I were to spend more time studying the app. For now, I load a project, connect, and execute. It just works.

Incentivized

no very easy but lacks documentation

Initially, our oragnization had to integrate Katalon tests with the Jenkins to schedule the test runs. The reports were stored in company local database. To identify test flakiness or product issue one had to dig deep into the reports. This issue was all solved by Katalon TestOps. Katalon TestOps provides report history in one place and also helps in easy triage of the report by pointing out script flakiness

Incentivized

I have not used other API testing products but I am completely satisfied with the functionality and performance of ReadyAPI. It covers all required API protocols and database connections that are used in our organization. It also allows extending the functionality by adding external DB drivers and wiring custom scripts when the native assertion/data manipulation test steps are not sufficient.

Incentivized

It has an excellent GUI design and the reporting mechanism is also very
good. It does consume a lot of memory though during concurrent testing. However, I have read that added monitoring tools have been added, which if so the 7 could possibly go to a 8 or 9.

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• We were able to achieve our needs in API test automation with the tool
• In terms ROI, I would say that we have gotten what is basically needed
• Though the initial setup was tough, after that we were able to continue it with user friendly UI and reporting.
• Overall, it had [a] neutral impact in terms [of] ROI.

Incentivized