Input
Perform automatic code analyze is becoming increasingly popular among developers, thanks to its benefits. The most obvious, it helps developer to write more clean and understandable code. In addition, this analysis helps to trace potential bugs and problems automatically, without expending effort on writing tests. It also helps to find duplicate code.
Perform an analysis is usually simple and fast. Many of these can be integrated with software for continuous integration. To perform an analyze, it is obviously necessary an access to source code. Analyzer must scan all source code: without it, there is no way to perform test.
A tester should read, if possible, all documentations about code and its development procedure, to understand the history of its development. With this information, the tester can make predictions about the possible outcome of analyze.
Developers who make use of automatic analysis of the code will improve their coding skills, learning from their mistakes.
An automatic source code review had even benefits with DevOps. These checks allows migrating new code into production with a fast and reliable inspection. Positive effects occur even on budget. A bad-written code has a cost of maintenance that can erode software value. Usually, about 20% of the average budget is to support the development. Actual studies have indicated that well written code comes in around 5%.
Procedures
SonarQube is an open source platform for continuous inspection of code quality.
It covers lots of languages: Java, C/C++, Objective-C, C#, PHP, Flex, Groovy, JavaScript, Python, PL/SQL, COBOL, etc…
After a test, Sonar offers reports and metrics, like duplicated code, coding standards, unit tests, code coverage, code complexity, potential bugs, comments and design and architecture. It implements SQALE methodology to compute technical debt.
It even provide metrics history, evolution graphs and differential views.
Sonar can be integrates with Maven, Ant, Gradle, continuous integration tools (Like Atlassian Bamboo or Jenkins…), Eclipse development environment and other external tools (JIRA, Mantis, LDAP, Fortify, etc.). It is expandable with the use of plugins.
Sonarqube is made of three parts:
- Sonarqube Server
- Sonarqube Client
- Source code
Server sonarqube, having the database in the analysis and the web interface, must be accessible from the client machines.
All instruction for Sonarqube server installation is available here: http://docs.sonarqube.org/display/SONAR/Installing+the+Server
Sonarqube client contains bin file for execution and configuration files.
Bin file is executed from the folder containing the source code. To function, the folder where you run must be a file sonar.properties, described below.
Configuration file contains the information to access the database server sonarqube, such as address, login and password, version ...
Once Sonarqube environment is ready (client and server), the final part is to prepare sonar.properties file with project information. Besides containing information such as project name, version, and description, this file may contain paths to be excluded from test, or software language to analyze (in case you have more than one language, otherwise languages are identified all).
Ouput
Sonarqube produces a list of metrics and issues after performing a test.
Metrics are useful for understand information like how many LoC, how many lines of comments, duplications and many more.
Issues gave useful information about code. Every tag represents a defined problem in source code. It is not easy to define which issues are useful for analysis of the code and which are not.
For example, issues about variable name that does not comply with standard are not much useful information. Issues about function too long or with too many cycles are more useful, because we can understand that code is not clear or easy to understand... Work in progress
Here's a list of sonarqube tags:
| Tags | Explanation |
|---|---|
| Brain-overload | Those issues are related to difficult code, deep nested cycle |
| Bad-practice | the code likely works as designed, but the way it was designed is widely recognized as being a bad idea |
| Bug | Issues about something wrong in code and it will probably affect production (bad variable use, unit test failed…) |
| CERT | related to CERT standard |
| Clumsy | extra steps are used to accomplish something that could be done more clearly and concisely. (E.G. calling .toString() on a String) |
| Confusing | will take maintainers longer to understand than is really justified by what the code actually does |
| Convention | coding convention - typically formatting, naming, whitespace... |
| CWE | relates to Common Weakness Enumeration |
| Design | there is something questionable about the design of the code |
| Lock-in | environment-specific features are used |
| MISRA | relates to MISRA standards |
| OWASP | relates to OWASP Top Ten security standards |
| Pitfall | nothing is wrong yet, but something could go wrong in the future; a trap has been set for the next guy, & he'll probably fall into it and screw up the code |
| Sans-top25 | relates to the SANS Top 25 Coding Errors |
| Security | relates to the security of an application |
| Suspicious | it's not guaranteed that this is a bug, but it looks suspiciously like one. At the very least, the code should be re-examined & likely refactored for clarity |
| Unpredictable | the code may work fine under current conditions, but may fail erratically if conditions change |
| Unused | unused code, E.G. a private variable that is never used |
| User-experience | There's nothing technically wrong with your code, but it may make some or all of your users hate you |
Issues are divided into 5 category:
Blocker | This issue affects operations and security. These might make the whole application unstable in production. |
Critical | This issue affects operations and security. These might lead to an unexpected behavior in production without affecting the integrity of the whole application. |
Major | This issue might have a substantial impact on productivity. |
Minor | This issue might have a potential and minor impact on productivity. |
Info | Not known or yet well-defined security risk or impact on productivity. |