Apache JMeter vs. SonarQube Server

The key questions I would pose are: how high of a concurrent load do you need to generate against the system? How technically skilled are your personnel that will perform the load testing? How much programmable/scripting/code based control do you desire from the tool? Do you prefer more of a GUI based tool or a programmatic code/scripting based tool? JMeter will be for those that want a GUI, moderate to low scripting/code based control/programmability, moderate to low load generation, and for users experienced in the novice to expert range.

Scenarios where SonarQube is well suited:

1. Large codebase: The tool's static analysis capabilities can help teams quickly identify and fix bugs, vulnerabilities, and code smells in large codebases.
2. Compliance and security: The tool can check the code against industry standards or regulations, such as OWASP and CWE, and identify any issues that need to be addressed.
3. Agile development: SonarQube can be integrated with CI/CD pipelines allowing teams to continuously monitor and improve code quality throughout the development process.
4. Teams using multiple languages: Teams that use multiple programming languages can benefit from using SonarQube, as the tool supports a wide range of languages and can be integrated with a variety of development tools.

Scenarios where SonarQube may be less appropriate:

1. Small codebase: Organizations with a small codebase may not see the full benefits of using SonarQube, as the tool's static analysis capabilities may be overkill for a smaller codebase.
2. Limited resources: Organizations with limited resources may find it difficult to set up and configure SonarQube, as the tool can be complex and may require specialized expertise.
3. Limited integration: Organizations that use development tools or IDEs that are not supported by SonarQube may find it difficult to integrate the tool into their existing development workflow.
4. Limited scalability: Large organizations with millions of lines of code may find SonarQube's performance and scalability to be an issue. It may take longer for the analysis to finish and the results may not be as accurate.

Incentivized

• Easy to construct Webservices script - You just need to add a sampler with the endpoint, payload and header information. And boom you can run a load test on the service with varying load and capture the response time and throughput. In a commercial tool like Loadrunner or Silkperformer, this process would take at least couple of hours, while with Jmeter we can do this in 10 minutes or less.
• Modularizing the scripts - Jmeter has a very good way of organizing the scripts and the reusability of the existing script (transactions) is also as easy as copy pasting the script.
• Jmeter tests can be run from anywhere, so you can run it from the idle webserver itself if you have some firewall restrictions to test it from your local or other environment.
• Jmeter distributed tests can be spawn from the AWS and results collated to see the performance from different regions in the world.

• Generating code quality report
• Calculates junit coverage of the codebase very efficiently and precisely
• Highlights the bugs and vulnerabilities in our codebase
• Informs the user of the improvements which can be done to the code to make it cleaner
• SonarQube also suggests remediation and resolution of the problems it highlights

Incentivized

• Even though it has very easy installation, you can run into issues easily. I was ab;e to trigger a test at browser level but couldn't stop the tests. I had to kill it from task manager and then had to reinstall from begining.
• There are so many things in JMeter you can do especially with the sampler for selenium for UI tests. More documentation/webinars etc.. would help the community better.
• The jmx files should have an easy way to be integrated with ci/cd tools and cloud.

Incentivized

• It doesn't provide automatic pull request with fixes
• It doesn't provide insights about the libraries of the projects
• The administration management user interface could be simplified
• It doesn't provide an order to fix issues, like archives with more and frequent commits have top priority

Incentivized

Price, Wiki and user sharing. Having access to the information provided by the developers and other open source providers is key for me. The ability to share information and get answers directly is very important to success in software testing. And the price of this product currently is amazing. Too many companies charge way too much money for products that are far behind in their value and pertinence

I can jump right into a new test plan and start building from scratch. The natural progression from test plan to thread group and then designing the basic format of the process is very streamlined and smooth. With only slight modifications I can build out a very complex model from a very basic beginning.

It can improve in some user experience and usability parts, like the code view and the way we assign issues it's a bit hidden and not highlighted

Incentivized

I have been using JMeter for the last year. By using this tool, you can make sure the system will work under varied loads. It helps us to simulate real time scenarios by creating required virtual users and make sure the application will work under load. Perform load, stress, and stability testing using JMeter.

Incentivized

We we easily able to integrate the SonarQube steps into our TFS process via the Microsoft Marektplace, we didn't have the need to call SonarQube support. We've used their online documentation and community forum if we ran into any issues.

Incentivized

I have evaluated LOADUI (web free version) and it was a very unstable tool and I could not rely on those results completely as I was not sure how the tool was performing. It only generated the top 10 less transaction times and when Jmeter was in use, it was very effective in using plugins to accommodate more outputs.

Incentivized

SonarQube identifies significant more thing compared to the built-in suggestions in IntelliJ IDEA. The suggestions how to correct issues are also a lot better with SonarQube. IntelliJ IDEA provides great refactoring support to make it easy to refactor the code to solve issues. We use these tools together and they really complement each other.

Incentivized

• It helped to estimate resources required for a particular client requirement. For example if the client wants the application to be used by 500 users for 1 hour, based on the load test we perform for 500 users and 1 hour we recommend resources and their capabilities.
• Jmeter was used to identify bottleneck. This would lead to better performance.
• Jmeter was used to identify failures as well.

• Positive ROI from the standpoint of flagging several issues that would have otherwise likely been unaddressed and caused more time to be spent closer to launch
• Slightly positive ROI from time-saving perspective (it's an automated check which is nice, but depending on the issues it finds, can take developers time to investigate and resolve)