Static analysis as means of improving code quality

Static analysis
as means of improving code quality
Sergey Vasiliev
PVS-Studio
vasiliev@viva64.com

Density of errors (per 1 KLOC)
0
20
40
60
80
100
< 2 2-16 16-64 64-512 > 512

Vulnerabilities ~ bugs
The National Institute of Standards
and Technology (NIST) reports that
64% of software vulnerabilities
stem from programming errors
and not a lack of security features.

Number of vulnerabilities
0
3000
6000
9000
12000
15000
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Cost of fixing a security defect
0
1000
2000
3000
4000
5000
6000
7000
8000
Development Build QA Release Phase

Key points
• Error density increases non-linearly.
• Cost of fixing a problem increases with time.
• Vulnerabilities often are
simple programming errors.

Static analysis
Advantages:
• Early detection of problems.
• Full code coverage.
• Great at detecting various patterns of errors.
Disadvantages:
• False positives.
• Accurate level of error severity is unknown.

Terminology
• CWE (Common Weakness Enumeration) –
potential vulnerabilities that can become real.
• CVE (Common Vulnerabilities and Exposures) –
real vulnerabilities, discovered in applications.

Дублирующиеся подвыражения
PascalABC.Net
if (File.Exists(pdbFileName) &&
File.Exists(pdbFileName)) {
....
}

Duplicated subexpressions
PascalABC.Net
if (File.Exists(pdbFileName) &&
File.Exists(pdbFileName)) {
....
}
SonarC# warning: Identical sub-expressions on both sides of operator "&&".

Ошибка? Уязвимость!
CVE-2014-1266
iOS
if ((err = SSLHashSHA1.update(
&hashCtx, &signedParams)) != 0)
goto fail;
goto fail;
PVS-Studio warnings:
• CWE-483 V640. The code's operational logic does not correspond with its
formatting. The statement is indented to the right, but it is always executed. It is
possible that curly brackets are missing.
• CWE-561 V779 Unreachable code detected. It is possible that an error is
present.

Error? Vulnerability!
CVE-2014-1266
iOS
if ((err = SSLHashSHA1.update(
&hashCtx, &signedParams)) != 0)
goto fail;
goto fail;
PVS-Studio warnings:
• CWE-483 V640. The code's operational logic does not correspond with its
formatting. The statement is indented to the right, but it is always executed. It is
possible that curly brackets are missing.
• CWE-561 V779 Unreachable code detected. It is possible that an error is
present.

Неправильный вызов функции
Doom 3
void Sys_GetCurrentMemoryStatus(
sysMemoryStats_t &stats ) {
....
memset( &statex, sizeof( statex ), 0 );
....
}
CppCheck warning: memset() called to fill 0 bytes of '&'

Incorrect function call
Doom 3
void Sys_GetCurrentMemoryStatus(
sysMemoryStats_t &stats ) {
....
memset( &statex, sizeof( statex ), 0 );
....
}
CppCheck warning: memset() called to fill 0 bytes

Бессмысленная проверка
Jenkins
int cnt = 0;
for (R b = getLastBuild(); cnt<5 && b!=null;
b=b.getPreviousBuild()) {
FilePath ws = b.getWorkspace();
if (ws != null)
return b;
}
PVS-Studio warning: V6007 Expression 'cnt < 5' is always true.

Useless check
Jenkins
int cnt = 0;
for (R b = getLastBuild(); cnt<5 && b!=null;
b=b.getPreviousBuild()) {
FilePath ws = b.getWorkspace();
if (ws != null)
return b;
}
PVS-Studio warning: V6007 Expression 'cnt < 5' is always true.

Опасное разыменование указателя
Unreal Engine
bool FHeadMountedDisplay::IsInLowPersistenceMode() const
{
const auto frame = GetCurrentFrame();
const auto FrameSettings = frame->Settings;
return frame && FrameSettings->Flags.bLowPersistenceMode;
}
Klocwork warning: Suspicious dereference of pointer 'frame' before NULL check

Dangerous pointer dereference
Unreal Engine
bool FHeadMountedDisplay::IsInLowPersistenceMode() const
{
const auto frame = GetCurrentFrame();
const auto FrameSettings = frame->Settings;
return frame && FrameSettings->Flags.bLowPersistenceMode;
}
Klocwork warning: Suspicious dereference of pointer 'frame' before NULL check

Непроверенные входные данные
NcFTP
if (fgets(newname, sizeof(newname) - 1, stdin) == NULL)
newname[0] = '0';
newname[strlen(newname) - 1] = '0';
PVS-Studio warning: CWE-20 V1010 Unchecked tainted data is used in index:
'strlen(newname)'.

Unchecked input data
NcFTP
newname[0] = '0';
PVS-Studio warning: CWE-20 V1010 Unchecked tainted data is used in index:
'strlen(newname)'.

NcFTP
newname[0] = '0';

NcFTP
newname[0] = '0';
0???
-1

Reproducing the error
• Server connection.
• Downloading file from a server.
• Input of a string, starting with 'N'.
• Input '0'.
• ....
• PROFIT!

Reproducing the error
• ncftp.exe ftp://speedtest.tele2.net
• get 512KB.zip
• Now let's have some fun
• 0???
• ....
• PROFIT!

Static analysis as means of improving code quality

Myth:
static analysis is for newbies, professionals
don’t make mistakes

Single checks are ineffective
• ... but better than no checks at all.
• “Let’s check projects before the release!"
• Critical errors were fixed
for a higher price.

CVE-2015-8948
libidn
else if (fgets (
readbuf, BUFSIZ, stdin) == NULL) {
....
}
if (readbuf[strlen (readbuf) - 1] == 'n')
readbuf[strlen (readbuf) - 1] = '0';
PVS-Studio warning:
CWE-20 V1010 Unchecked tainted data is used in index: 'strlen(readbuf)'.

CVE-2016-6262
libidn
else if (getline (&line, &linelen, stdin) == -1) {
....
}
if (line[strlen (line) - 1] == 'n')
line[strlen (line) - 1] = '0';
PVS-Studio warning:
CWE-20 V1010 Unchecked tainted data is used in index: 'strlen(line)'.

CVE-2016-6262
libidn
else if (getline (&line, &linelen, stdin) == -1) {
....
}
if (strlen (line) > 0)
if (line[strlen (line) - 1] == 'n')
line[strlen (line) - 1] = '0';

CVE from libidn
CVE-2015-8948.
Commit, closing
this vulnerability: 10.08.2015
CVE-2016-6262.
Commit, closing
this vulnerability: 14.01.2016
Time difference – 5 months.

Use static analysis regularly.

Effective use
• Locally on developers’ computers.
• On a build server.
• Prompt fixing of errors.
• ...
• PROFIT!

Local use
• Cost of fixing an error is minimal.
• Developer is inside a context:
it’s easier to handle a warning.
• No one will find out about the problem
except you and the analyzer:)

Incremental analysis
• Analyzing only modified newly written code.
• Reducing of analysis time.
• Great for early detection of errors.

Example from personal experience
• Analyzer developers also make mistakes :)
• Integration with Visual Studio +
incremental analysis.

Using on a build server
• Detecting errors, that got into a repository.
• Various scenarios of working with
analysis results:
• Distribution by mail;
• Generation of issues;
• Using in CI-systems;
• etc.

Example from personal experience
• Nightly analysis.
• Merging of logs,
filtration, conversion.
• Handling the analysis results:
• Mails distribution;
• Publication of results in Jenkins.

Analyze code on developer
machines and on a build server.

Deploying on a project
High Medium Low Total
1350 35943 45346 82639

• How did that even work?
• What should be fixed and
how can it be done?
• How to separate old warnings from
the new ones?

False positives
• Are triggered on correct code.
• Analysis logs are cluttered.
• Are inevitable.

"Redundant" analysis
• Third-party code is analyzed.
• Irrelevant project warnings
clutter the output.

Problems solutions
Problem Solution
False positives Message suppression

Problems solutions
Problem Solution
"Explosion" after the first analysis "Freezing" of all warnings

Problems solutions
Problem Solution
Long analysis time Incremental analysis

Problems solutions
Problem Solution
Inconvenience of working with 'raw' logs Usage of auxiliary tools

Problems solutions
Problem Solution
Warnings on third-party code Excluding from the analysis

Problems solutions
Problem Solution
Warnings on third-party code Excluding from the analysis
Irrelevant diagnostics Disabling diagnostic rules

Investigate the tools that are
suggested
by developers of analyzer.

• "Freezing" of existing warnings.
• Configuring the analyzer.
• Don’t let new errors appear.
• Go back to old errors whenever spare
resources and time is available.

When deploying static analysis,
mark existing warnings as
irrelevant.

Recommended articles on the topic
"How the PVS-Studio Team
Improved Unreal Engine's Code"
https://bit.ly/2IKnnch
"Static Analysis as Part of the
Development Process in Unreal Engine"
https://bit.ly/2KL4ZAu

Reducing costs
0
1000
2000
3000
4000
5000
6000
7000
8000
Development Build QA Release Phase

No Silver Bullet
• Static analysis is not a cure-all.
• Effectively complemented
by other tools.

Sergey Vasiliev
E-mail: vasiliev@viva64.com
PVS-Studio site: https://www.viva64.com

Static analysis as means of improving code quality

More Related Content

What's hot (20)

Similar to Static analysis as means of improving code quality (20)

More from Andrey Karpov (20)

Recently uploaded (20)

Static analysis as means of improving code quality