What you should know about software measurement platforms

IT Leader’s Guide to Enterprise Software Quality Solutions What you need to know about the engine that will power your software quality initiative

CAST Copyright © 2014 - 2016
1
Contents
I.Taking the First Step
II.Software Measurement Essentials
III.Do-It-Yourself Solutions
IV.Enterprise Solutions
V.Total Cost of Ownership Analysis

2
Taking the First Step
Congratulations! Your organization, from developers up to executives, has officially recognized that software quality measurement is a key business value driver. Not only is this a demonstration of your company’s maturity, it is a strategic inflection point that, if executed properly, will propel you toward sustainable returns.
Companies who have thoughtfully deployed software quality measurement platforms have experienced:
Improved collaboration between IT and business
Reduced time-to-market
Expanded revenue generation opportunities
Increased user and customer satisfaction
Now, you just have to make sure that your new strategic focus on software quality has the platform to help you accurately measure the most relevant metrics and provide the right visibility to the right people.
Sounds easy, right? Well, it depends on what you use to power your initiative.

3
Software Quality Measurement Essentials
•Essential Capabilities
•Solution Breadth and Depth

4
Software Quality Measurement Essentials
There are many offers today in the code analysis market space. How do you choose the right one?
Here’s a quick look at the most essential capabilities of a software quality measurement solution. Solutions that fail to offer all of these capabilities will not deliver the value you are seeking.
A comprehensive software quality measurement platform must address all of your goals. A “good-enough” approach may give you “not-enough” results.
Measurement precision
Wide technology scope
Accurate analytical methods
Management visibility
Enterprise scalability
Measurements must be consistent over time, and trends must be meaningful
Ability to analyze the technologies within your most critical application
Accurate analysis must examine the entire application system and in between layers, beyond a single code unit
Easily scalable to all critical applications and accessible by 100 or more staff
Have intuitive analytics for management to quickly track performance & hotspots
Software Quality Essentials
Description
Consequences of “Good-Enough” Approach
Ambiguous measurements cannot help you make the right business decisions.
Exposes critical applications to high risk, or incur costs for custom solutions
Inaccurate and irrelevant measurements will force wrong and costly decisions
Inefficiently run several instances, reports are manually aggregated
Quality program will lose momentum without management involved
Follow industry standards for quality measurements and best practices (e.g. OMG, CISQ, SEI)
Rooted in industry standards
Program will be challenged by all involved and quickly lose credibility

5
Breadth and Depth of the Solution
A truly meaningful and valuable software quality measurement platform must have breadth and depth.
Wide technology coverage that tidily addresses your most critical applications
Capability to analyze all layers of your critical applications; from code, to component, to system
Breadth
Depth
import java.applet.*;
import java.awt.*;
public class SecondApplet extends Applet {
static final String message = "Hello World";
private Font font;
// One-time initialization for the applet
public void init() {
font = new Font("Helvetica", Font.BOLD, 48);
}
// Draw the applet whenever necessary. Do some fancy graphics.
public void paint(Graphics g) {
// The pink oval
g.setColor(Color.pink);
g.fillOval(10, 10, 330, 100);
// The red outline. java doesn't support wide lines, so we
Unit Level
•Code quality at the developer level
Component Level
•Entire technology layer
•Inter-application invocation
•Single technology
System Level
•Entire application
•Interaction between components
•Architectural cohesion

6
Do-It-Yourself Solutions
•The Lure of “Simplicity”
•Tangible Costs of DIY
•Hidden Costs of DIY

7
Beware of the Lure of “Simplicity”
Developing your own aggregation platform using statistics from single-technology code analyzers may appear to be a simple and frugal option to power your software quality program. But, they do not meet many of software quality measurement essential capabilities, and can cost you more long term.
Many technical organizations are drawn to single-technology analyzers when they begin to formulate their strategy around software quality measurement. While its low up front costs and seemingly endless customization potential make these analyzers an intriguing way to approach software quality, there are a myriad of downsides.
A Component of the Whole
To start, code analyzers are only a component of a complete software quality measurement platform. While licensing the analyzers may be cheap, many companies have quickly learned that they have to either invest development resources in creating a complex aggregation platform or purchase an commercially-available aggregator in order to effectively track and report on business-relevant metrics.
“Close-Enough” Measurements
Aggregating results from individual analyzers may lead you down the wrong path. While data form individual analyzers can be aggregated, it does not paint the whole picture. Issues at the code level, do not have the same impact as issues at the system-level, where interactions between components can cause catastrophic system failure. Only looking at measurements at the code level is a in accurate approach that may prompt you to spend resources fixing a problem that has very little business value.
Unfortunately, in this case, the whole is not greater than the sum of its parts.
You Own the Platform - Completely
Analyzer components have to be updated frequently when new versions are released. New releases can destabilize the platform, or worse – completely change the results of the measurements you have been taking.
You see, independent components do not have to follow a coherent roadmap. Product development is driven by a single technology or language, and not based on your overall enterprise needs.

8
The Tangible Costs of DIY
$1.2M
INITIAL DEVELOPMENT
5 Technical Staff
$240K PROJECT MANGEMENT 1 Project Manager
$300K
SOFTWARE QUALITY
1 Software Quality Consultant
BUILD
$340K / year
PLATFORM R&D
2 Technical Staff + Hardware
$120K / year
TRAINING & SUPPORT
1 Technical Staff
$180K / year
STRATEGY & MANGEMENT
1 Application Owner
$75K / year SOFTWARE QUALITY 0.5 Software Quality Consultant
MAINTAIN
So, what are the costs of building and operating your own code analyzer aggregator platform? We did the math for you. These estimates are based on resources located in the U.S. We also assume that the platform was built with a “good- enough” approach. A platform that satisfies the “software quality measurement essentials checklist” (p. 4) will likely cost exponentially more.
$715K / year
$1.74m over 2 years

1.Project Scope Creep – Currently available single-technology code analyzers primarily address Java. If your applications are in C, COBOL, .Net, or other commonly used languages, you may need to purchase or extend additional custom analyzers. This means a significant expansion in the scope of the project.
2.Scarce Expertise – Doing it yourself doesn’t automatically mean inexpensive. Many companies hire expensive code quality consultants in order to complete the platform.
3.Inability to Scale – Many aggregators work very well at an individual developer or small team scale. However, an organizational- wide adoption of software analysis requires sharing of information, and easily accessible visibility.
4.Opportunity Cost – We estimate that it takes five resources over two years to build an adequate software quality analysis platform (10 man-years). That means these resources have to deprioritize tasks that may have better returns.
5.Waiting for Return – Typically the value of software quality analysis and the process that implement to address it begin to materialize six to 12 months after initial implementation. Building your own means that the return is much further away.
6.Licensing Legality – Although the initial license had no cost, many companies face legal action due to mismanagement of code analyzer components—a serious drain on resources, money, and talent.
7.Component Updates – New releases of code analyzers must be researched, before upgrading. You may face several new releases within a very short time frame, misaligned with your release schedule.
8.Version Control – Some components in your application may be written using the same technology, (e.g. JDK), but in different versions (e.g. JDK 5 and JDK 6). When code analyzers are updated they may change language version support, requiring you to have more than one instance of the analyzer.
9
The Hidden Costs of DIY
Building and supporting a software quality measurement platform using single-technology analyzers is not a small task. The length and hidden costs of the projects are alarming.

10
Enterprise Software Quality Measurement Solutions
•Advantages of an Enterprise Solution
•Managed Services – A Prudent Approach

11
Advantages of an Enterprise Solution
Investing in a balanced and refined software quality measurement platform produced by a company with sound expertise and business understanding is the wiser option.
Enterprise software quality measurement solution
Software Quality Expertise
A provider who has dedicated all of its time and energy in software quality solutions will likely deliver a more refined and valuable product, than a solution that is built in-house with individual code analyzers. By buying an enterprise solution you can accelerate toward returns through their specialization in the space.
Customer Driven Road map
You have a bigger voice in your provider’s roadmap decisions than with single- technology components where thousands of voices are competing to be heard. With a larger share-of-voice you have more control overall how future versions of a product fit your needs.
Dedicated Support When you do business with a solution provider, you are the boss. Not only do you have access to expert user support, you can also enlist help for implementation, adoption, and operation of the platform from people who really know it well.
Lower Cost of Ownership
With no need for development resources and simple update packages, enterprise solutions will cost you less to own and operate in the longer term. This also allows your company to focus on its core competency, selling the products and services you know best.

12
Managed Services – A Prudent Approach
Risk has become a centerpiece in enterprise software with the growing need of companies’ to innovate through new solutions.
Managed services allow companies to quickly implement and try new software at low risk, and, for many, has become the only way to buy new software.
Implementing and operating an enterprise software quality measurement platform by using the provider’s managed services can have many benefits.
1.Shift Risk to Provider
Let your provider take on the risks of their own solution. Traditionally, to onboard a new software solution, you may need to dedicate project management resources, staff to operate and support the new application, and invest in some additional hardware.
With managed services, your provider will provide the man power and hardware to deploy the software. In the case of software quality measurement platforms, this means that they will:
Stand up physical and logical infrastructure
Define the code gathering and analysis process
Design the consumption model
Determine channels of communication
It also means that you do not have to hire more staff, or worry about what will happen to them if the project is aborted.
2.Greater Budget Control
Buying software in a managed services model means that a large portion of the funds can be sourced from the operational expenses bucket instead of capital expenses. This enables companies to make faster decisions to start or end a project.
3.Accelerate to Time to Value
Because the providers are experts in how to deploy and operate their software, they will enable to you get ramped up and start measuring your software quality in a meaningful way in much less time than if you managed it on your own. This helps you accelerate toward your payback period and vastly increases your returns.
4.Capture Opportunity Cost
A managed services model allows you to focus on your key value-adding activities. Operational distractions such as bugs, performance issues, and upgrades are all handled seamlessly by the provider without your involvement.

About CAST
“CAST’s holistic system approach, looking at the architecture, transactions, control, and data flow across multiple technologies, may be very beneficial, with numerous engineering studies showing that bad software engineering practices in the ways components are interrelated and interact…account for only 10% of total defects, but can lead to 90% of production issues.”
CAST is a pioneer and world leader in software analysis and measurement, with unique technology resulting from more than $100 million in R&D investment. CAST introduces fact-based transparency into application development and sourcing to transform it into a management discipline. More than 250 companies across all industry sectors and geographies rely on CAST to prevent business disruption while reducing hard IT costs and software risk. CAST is an integral part of software delivery and maintenance at the world's leading IT service providers. Founded in 1990, CAST is listed on NYSE-Euronext (Euronext: CAS) and serves IT intensive enterprises worldwide with offices in North America, Europe and India.
www.castsoftware.com

What you should know about software measurement platforms

More Related Content

What's hot (20)

Similar to What you should know about software measurement platforms (20)

More from CAST (20)

Recently uploaded (20)

What you should know about software measurement platforms