SlideShare a Scribd company logo

Analysis of-quality-of-pkgs-in-packagist-univ-20171024

C
Clark Everetts

The document presents an analysis of the quality of libraries available in the Packagist ecosystem, emphasizing the importance of reliable and secure packages for PHP applications. It discusses the challenges developers face in finding quality libraries and offers tools and methods for assessing library quality, including static analysis tools. The author, Clark Everetts, provides a PHP script for running several quality analysis tools on codebases to aid developers in evaluating their dependencies.

Software
1 of 33
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 1 An Analysis of the Quality of Libraries in the Packagist Universe Clark Everetts Sr. Professional Services Consultant 24 October 2017
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 2 Slides, Joind.in, and Stuff • Rate & comment: https://joind.in/talk/377ec • Slides: https://www.slideshare.net/clarkphp • Tweets: #zendcon2017 • Twitter: @clarkphp
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 3 INTRODUCTION/ BACKGROUND
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 4 As you know, Composer is… • Knows what packages your application or library depends upon • Obtains those packages, and all of their dependencies, and installs appropriate versions of them into your project (and local cache) • When requested, checks for updates compatible with your project, and downloads them into your project (and local cache) • Allows you to pin multiple applications/libraries to the same or different versions of the packages they use. Composer makes it easier to manage application dependencies. … a per-project PHP dependency manager
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 5 Packagist.org Repository Package Archivist Just a Composer (“type”) Repository… • … but it is the primary repository for open source packages • Best Practice for Open Source Projects: register it at packagist.org • Searchable / Browsable • Less work for people to find and use your package. • Many, many, many packages available. There is duplication in functionality and – I suspect - a wide range of quality.
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 6 https://packagist.org/
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 7 Questions of interest to me • Composer & Packagist are GREAT! • Easy to consume 3rd party libraries • Easy to publish libraries How to find libraries that are • Reliable • Well-built • Tested • Secure How many projects available via Packagist?
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 8 Why so important? Your Project Project == Application == Library DEPENDENCIES, PACKAGES, LIBRARIES needs A, B, C, D A B C D needs E E F needs E You are responsible for the application, and for the choice in which dependencies you use. HG needs G, H
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 9 No, really! Consumer-only. Why important? Higher quality libraries are • Easier to understand • Easier to maintain • Easier to test • Get updated sooner • Bugfixes • New features • Don’t break as easily or as often • Greater interest/participation from dev team and users *Campbell County Kentucky Public Library
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 10 SECURITY
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 11 Security – Known Vulnerabilities
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 12 "conflict": { "adodb/adodb-php": "<5.20.6", "amphp/artax": ">=2,<2.0.6|<1.0.6", "aws/aws-sdk-php": ">=3,<3.2.1", "bugsnag/bugsnag-laravel": ">=2,<2.0.2", "cakephp/cakephp": ">=3,<3.0.15|>=2,<2.4.99|>=2.5,<2.5.99|>=2.6,<2.6.12|>=1.3,<1.3.18|>=2.7,<2.7.6|>=3.1,<3.1.4", "cart2quote/module-quotation": ">=4.1.6,<=4.4.5|>=5,<5.4.4", "cartalyst/sentry": "<2.1", "codeigniter/framework": "<=3.0.6", "composer/composer": "<=1.0.0-alpha11", "contao-components/mediaelement": ">=2.14.2,<2.21.1", "contao/core": ">=2,<3.5.28", "contao/core-bundle": ">=4,<4.4.1", "doctrine/annotations": ">=1,<1.2.7", "doctrine/cache": ">=1,<1.3.2|>=1.4,<1.4.2", "doctrine/common": ">=2,<2.4.3|>=2.5,<2.5.1", "doctrine/dbal": ">=2,<2.0.8|>=2.1,<2.1.2", "doctrine/doctrine-bundle": "<1.5.2", "doctrine/doctrine-module": "<=0.7.1", "doctrine/mongodb-odm": ">=1,<1.0.2", "doctrine/mongodb-odm-bundle": ">=2,<3.0.1", "doctrine/orm": ">=2,<2.4.8|>=2.5,<2.5.1", "dompdf/dompdf": ">=0.6,<0.6.2", "drupal/core": ">=8,<8.3.7", "drupal/drupal": ">=8,<8.3.7", etc., etc. Avoiding packages with known vulnerabilities
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 13 Security – The Unknown Vulnerabilities • OWASP Discussion on Source Code Analysis Tools – https://www.owasp.org/index.php/Source_Code_Analysis_Tools • Static Application Security Testing – Security flaws are currently better than the tools – IDEs, catch potential vulnerable code during development • Strengths – Scalability – Reliable for buffer overflow, SQL injection • Weaknesses – Configuration affects security – False positives – Many flaws not amenable to automatic discovery • Authentication/Access Control • Poor use of crypto
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 14 Security – Some Tools for PHP • RIPS – http://sourceforge.net/projects/rips-scanner/ • phpcs-security-audit – PHP_CodeSniffer sniffs for core PHP and Drupal 7 – https://github.com/FloeDesignTechnologies/phpcs-security-audit • VisualCodeGrepper (VCG) – C/C++, C#, VB, PHP, Java, and PL/SQL for security issues and for comments indicative of insecure code – http://sourceforge.net/projects/visualcodegrepp/ • (Plug for KlocWork from Rogue Wave, for C, C++, Java, C#) – https://www.roguewave.com/capabilities/static-code-analysis
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 15 OTHER TOOLS
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 16 Some static analysis tools for PHP • PHPLOC – size metrics, complexity, dependencies, structure – https://github.com/sebastianbergmann/phploc • Copy/Paste Detector – finds duplicated code (exact dups) – https://github.com/sebastianbergmann/phpcpd • PHP Depend - – https://pdepend.org/ • PHP Mess Detector – https://phpmd.org/ • Phan – https://github.com/phan/phan • Exakat – https://www.exakat.io/ • PHPMetrics – http://www.phpmetrics.org/
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 17 WHAT MEAN QUALI-TAY?
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 18 What do we mean by “quality”? Concepts/words • Reliable, well-built, tested, secure What “observables” indicate higher quality? Quantifiable vs. Non- Quantifiable Download totals indicate popularity, but download trend would be more indicative. Download totals also show developers trying out a package, not necessarily continued usage. Stars or Likes or ratings
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 19 Identifying a quality package • Does it do what we think it does/expect it do to, and at what level of confidence? • How many defects are known to exist, or have existed, in it? – Relative to size of the package codebase (defect density)? • How many defects are known to have been corrected, and at what level of confidence? • What is the rate of defect discovery? Of defect removal? • What are the densities of various severity levels of defects?
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 20 Context! Critical Programming errors can lead to catastrophic outages or performance degradations that make a system unusable. Such programming errors at the system level are 90% of production issues. Such errors at the unit level, though far more numerous, are less than 10% of production issues.
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 21 A SIMPLE ONE-STOP ANALYSIS SCRIPT
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 22 A PHP script to run some QA tools $ cat analyse-codebase.php <?php // analyse-codebase.php // Runs several PHP static analysis tools; assumes *.phar names // This could be done with a shell/batch script, instead of php. // If running under Windows, use paths like 'C:desiredpathtofile' // If running under *nix, use paths like '/desired/path/to/file' // If PHP binary is not in your command path, you need to specify path to it. // const PHP_EXE = '/the/path/to/your/PHP/executable'; const PHP_EXE = 'php'; // the location where the QA tools reside (anywhere you like) // const QATOOLS = '/a/folder/containing/this/analyse-codebase.php/script'; const QATOOLS = 'C:Usersclark.ePHP-QA-Tools’;
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 23 PHP QA tools script: setup continued // the directory (folder) ENCLOSING the code you wish to analyze // const CODEBASE_ROOT = '/path/to/parent/of/your/source/code/appname'; const CODEBASE_ROOT = 'C:Usersclark.escratchpadqasamples'; // where to write the analyses OUTPUT files const OUTPUT_DIR = 'C:Usersclark.escratchpadqa'; // used to give a name to the output files $appName = 'samples'; // this is a folder containing the code to analyze. // It is APPENDED to CODEBASE_ROOT, allowing for analysis of // a particular folder within CODEBASE_ROOT), if needed. $codebaseLocation = '';
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 24 PHP QA tools script: setup continued // list of subfolders to exclude from counting (can be empty, but you often should exclude some folders) // PLEASE check this list and edit accordingly, BEFORE running the script. // Note, for PHPLOC, any directory in the tree with a name in this array is ignored, not just the top-level dirs. $excludeDirs = [ 'cache', 'config', 'documents', 'logs', 'public', 'sql_source', 'third_party', 'test', 'vendor', 'views', ];
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 25 PHP QA tools script: calling the tools // becomes part of the output file names $timestamp = date('YmdHis'); $targetPhpVersion = '7.1'; print phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; //print phpCompatibility($appName, $timestamp, $excludeDirs, $codebaseLocation, $targetPhpVersion) . PHP_EOL;
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 26 PHP QA tools script: Lines of Code function phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phploc'; $options = '--log-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --log-csv=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.csv" . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 27 PHP QA tools script: Copy/Paste Detector function copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpcpd'; $options = '--log-pmd=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --no-ansi' . ' --no-interaction' . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 28 PHP QA tools script: PHP Depend function pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'pdepend'; $options = '--dependency-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- dependencies.xml" . ' --jdepend-chart=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.svg" . ' --jdepend-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --overview-pyramid=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- overview-pyramid.svg" . ' --summary-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- summary.xml" . ' --coderank-mode=inheritance' // how is this used? . ' --coverage-report=' . OUTPUT_DIR . "$appName- $tool-$timestamp-coverage.xml" . ' --ignore=' . implode(',', $excludeDirs);
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 29 PHP QA tools script: PHP Depend $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 30 PHP QA tools script: PHP Mess Detector function messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpmd'; $ruleSets = ['cleancode', 'codesize', 'controversial', 'design', 'naming', 'unusedcode']; foreach ($ruleSets as $ruleSet) { $options = "text $ruleSet --reportfile " . OUTPUT_DIR . "/$appName-$tool-$timestamp-$ruleSet.out" . ' --exclude ' . implode(',', $excludeDirs); // --strict: also report those nodes with a @SuppressWarnings annotation shell_exec('php ' . QATOOLS . "/$tool.phar " . CODEBASE_ROOT . "/$codebaseLocation $options"); print $ruleSet . PHP_EOL; } }
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 31 Other Resources • PHP The Right Way - http://www.phptherightway.com/ • OWASP Tools - https://www.owasp.org/index.php/Source_Code_Analysis_Tools • Survive the Deep End: PHP Security (2013, but relevant) - http://phpsecurity.readthedocs.io/en/latest/ • SonarQube – “Water Leak” approach to code quality – https://www.sonarqube.org/
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 32 So, who is this guy? Clark Everetts, ZCE • Rogue Wave Software (acquired Zend October 2015) • PHP since 2005 • Professional Services Consultant – Architecture and Performance Audits – PHP, Zend Framework Training – Application Development, Best Practices, etc. – IBM i • clark.everetts@roguewave.com @clarkphp +ClarkEveretts
© 2017 Rogue Wave Software, Inc. All Rights Reserved. 33 THANK-YOU clark.everetts@roguewave.com @clarkphp +ClarkEveretts Tweet: #zendcon2017 Rate, comment, get slides https://joind.in/talk/377ec Your feedback is invaluable!

More Related Content

What's hot (20)

PDF
DevSecCon London 2017 - MacOS security, hardening and forensics 101 by Ben Hu...
DevSecCon
 
PDF
The Seven Habits of Highly Effective Puppet Users - PuppetConf 2014
Puppet
 
PDF
Getting out of the Job Jungle with Jenkins
Sonatype
 
PPTX
Prescriptive Security with InSpec - All Things Open 2019
Mandi Walls
 
KEY
Continuous Integration & Drupal
LimoenGroen
 
PPT
Getting Started With Jenkins And Drupal
Philip Norton
 
PDF
LasCon 2014 DevOoops
Chris Gates
 
PPTX
Adding Security to Your Workflow With InSpec - SCaLE17x
Mandi Walls
 
PDF
Composer - The missing package manager for PHP
Tareq Hasan
 
PDF
JUC Europe 2015: Continuous Integration and Distribution in the Cloud with DE...
CloudBees
 
PDF
Drupal Continuous Integration (European Drupal Days 2015)
Eugenio Minardi
 
PPTX
Adding Security and Compliance to Your Workflow with InSpec
Mandi Walls
 
PDF
Choosing the Right Framework for Running Docker Containers in Prod
Josh Padnick
 
PDF
Create Disposable Test Environments with Vagrant and Puppet
Gene Gotimer
 
PPTX
Drupal Continuous Integration with Jenkins - The Basics
John Smith
 
PPTX
Drupal Continuous Integration with Jenkins - Deploy
John Smith
 
PPTX
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
Chris Gates
 
PDF
Open Canary - novahackers
Chris Gates
 
PPTX
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
ZIP
Drupal Deployment
Jeff Eaton
 
DevSecCon London 2017 - MacOS security, hardening and forensics 101 by Ben Hu...
DevSecCon
 
The Seven Habits of Highly Effective Puppet Users - PuppetConf 2014
Puppet
 
Getting out of the Job Jungle with Jenkins
Sonatype
 
Prescriptive Security with InSpec - All Things Open 2019
Mandi Walls
 
Continuous Integration & Drupal
LimoenGroen
 
Getting Started With Jenkins And Drupal
Philip Norton
 
LasCon 2014 DevOoops
Chris Gates
 
Adding Security to Your Workflow With InSpec - SCaLE17x
Mandi Walls
 
Composer - The missing package manager for PHP
Tareq Hasan
 
JUC Europe 2015: Continuous Integration and Distribution in the Cloud with DE...
CloudBees
 
Drupal Continuous Integration (European Drupal Days 2015)
Eugenio Minardi
 
Adding Security and Compliance to Your Workflow with InSpec
Mandi Walls
 
Choosing the Right Framework for Running Docker Containers in Prod
Josh Padnick
 
Create Disposable Test Environments with Vagrant and Puppet
Gene Gotimer
 
Drupal Continuous Integration with Jenkins - The Basics
John Smith
 
Drupal Continuous Integration with Jenkins - Deploy
John Smith
 
DevOops Redux Ken Johnson Chris Gates - AppSec USA 2016
Chris Gates
 
Open Canary - novahackers
Chris Gates
 
DevSecCon Tel Aviv 2018 - Integrated Security Testing by Morgan Roman
DevSecCon
 
Drupal Deployment
Jeff Eaton
 

Similar to Analysis of-quality-of-pkgs-in-packagist-univ-20171024 (20)

PDF
Php Dependency Management with Composer ZendCon 2016
Clark Everetts
 
PDF
Open source software: The infrastructure impact
Rogue Wave Software
 
PPTX
How to migrate SourcePro apps from Solaris to Linux
Rogue Wave Software
 
PPT
Case study
karan saini
 
PPTX
Continuous security: Bringing agility to the secure development lifecycle
Rogue Wave Software
 
PPTX
Programming languages and techniques for today’s embedded andIoT world
Rogue Wave Software
 
PPTX
Securing Container Deployments from Build to Ship to Run - August 2017 - Ranc...
Shannon Williams
 
PDF
stackconf 2024 | How to hack and defend (your) open source by Roman Zhukov.pdf
NETWAYS
 
ODP
DevOps, CLI, APIs, Oh My! Security Gone Agile
Matt Tesauro
 
PPTX
Xebia labsperforce final
Perforce
 
PDF
Monitoring Attack Surface to Secure DevOps Pipelines
Denim Group
 
ODP
Effective DevSecOps
Pawel Krawczyk
 
PDF
Good Practices for Developing Scientific Software Frameworks: The WRENCH fram...
Rafael Ferreira da Silva
 
PDF
What is the Secure Supply Chain and the Current State of the PHP Ecosystem
sparkfabrik
 
PDF
Collaborative security : Securing open source software
Priyanka Aash
 
PDF
Shift Left Security
gjdevos
 
PPTX
AppSec DC 2019 ASVS 4.0 Final.pptx
TuynNguyn819213
 
PPTX
AppSec DC 2019 ASVS 4.0 Final.pptx
Josh Grossman
 
PDF
Open source software governance with DejaCode
nexB Inc.
 
PPTX
[Wroclaw #5] OWASP Projects: beyond Top 10
OWASP
 
Php Dependency Management with Composer ZendCon 2016
Clark Everetts
 
Open source software: The infrastructure impact
Rogue Wave Software
 
How to migrate SourcePro apps from Solaris to Linux
Rogue Wave Software
 
Case study
karan saini
 
Continuous security: Bringing agility to the secure development lifecycle
Rogue Wave Software
 
Programming languages and techniques for today’s embedded andIoT world
Rogue Wave Software
 
Securing Container Deployments from Build to Ship to Run - August 2017 - Ranc...
Shannon Williams
 
stackconf 2024 | How to hack and defend (your) open source by Roman Zhukov.pdf
NETWAYS
 
DevOps, CLI, APIs, Oh My! Security Gone Agile
Matt Tesauro
 
Xebia labsperforce final
Perforce
 
Monitoring Attack Surface to Secure DevOps Pipelines
Denim Group
 
Effective DevSecOps
Pawel Krawczyk
 
Good Practices for Developing Scientific Software Frameworks: The WRENCH fram...
Rafael Ferreira da Silva
 
What is the Secure Supply Chain and the Current State of the PHP Ecosystem
sparkfabrik
 
Collaborative security : Securing open source software
Priyanka Aash
 
Shift Left Security
gjdevos
 
AppSec DC 2019 ASVS 4.0 Final.pptx
TuynNguyn819213
 
AppSec DC 2019 ASVS 4.0 Final.pptx
Josh Grossman
 
Open source software governance with DejaCode
nexB Inc.
 
[Wroclaw #5] OWASP Projects: beyond Top 10
OWASP
 
Ad

Recently uploaded (20)

PDF
Digger Solo: Semantic search and maps for your local files
seanpedersen96
 
PPTX
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
PDF
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
PPTX
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
PDF
[Solution] Why Choose the VeryPDF DRM Protector Custom-Built Solution for You...
Lingwen1998
 
PPTX
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
PDF
MiniTool Power Data Recovery 8.8 With Crack New Latest 2025
bashirkhan333g
 
PDF
IDM Crack with Internet Download Manager 6.42 Build 43 with Patch Latest 2025
bashirkhan333g
 
PPTX
Foundations of Marketo Engage - Powering Campaigns with Marketo Personalization
bbedford2
 
PDF
IObit Driver Booster Pro 12.4.0.585 Crack Free Download
henryc1122g
 
PDF
Top Agile Project Management Tools for Teams in 2025
Orangescrum
 
PPTX
Coefficient of Variance in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PPTX
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
PPTX
Help for Correlations in IBM SPSS Statistics.pptx
Version 1 Analytics
 
PPTX
Agentic Automation Journey Session 1/5: Context Grounding and Autopilot for E...
klpathrudu
 
PDF
Add Background Images to Charts in IBM SPSS Statistics Version 31.pdf
Version 1 Analytics
 
PPTX
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
PDF
AI + DevOps = Smart Automation with devseccops.ai.pdf
Devseccops.ai
 
PDF
SciPy 2025 - Packaging a Scientific Python Project
Henry Schreiner
 
PPTX
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
Digger Solo: Semantic search and maps for your local files
seanpedersen96
 
OpenChain @ OSS NA - In From the Cold: Open Source as Part of Mainstream Soft...
Shane Coughlan
 
MiniTool Partition Wizard Free Crack + Full Free Download 2025
bashirkhan333g
 
Hardware(Central Processing Unit ) CU and ALU
RizwanaKalsoom2
 
[Solution] Why Choose the VeryPDF DRM Protector Custom-Built Solution for You...
Lingwen1998
 
AEM User Group: India Chapter Kickoff Meeting
jennaf3
 
MiniTool Power Data Recovery 8.8 With Crack New Latest 2025
bashirkhan333g
 
IDM Crack with Internet Download Manager 6.42 Build 43 with Patch Latest 2025
bashirkhan333g
 
Foundations of Marketo Engage - Powering Campaigns with Marketo Personalization
bbedford2
 
IObit Driver Booster Pro 12.4.0.585 Crack Free Download
henryc1122g
 
Top Agile Project Management Tools for Teams in 2025
Orangescrum
 
Coefficient of Variance in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
Agentic Automation: Build & Deploy Your First UiPath Agent
klpathrudu
 
Help for Correlations in IBM SPSS Statistics.pptx
Version 1 Analytics
 
Agentic Automation Journey Session 1/5: Context Grounding and Autopilot for E...
klpathrudu
 
Add Background Images to Charts in IBM SPSS Statistics Version 31.pdf
Version 1 Analytics
 
Change Common Properties in IBM SPSS Statistics Version 31.pptx
Version 1 Analytics
 
AI + DevOps = Smart Automation with devseccops.ai.pdf
Devseccops.ai
 
SciPy 2025 - Packaging a Scientific Python Project
Henry Schreiner
 
Agentic Automation Journey Series Day 2 – Prompt Engineering for UiPath Agents
klpathrudu
 
Ad

Analysis of-quality-of-pkgs-in-packagist-univ-20171024

  • 1. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 1 An Analysis of the Quality of Libraries in the Packagist Universe Clark Everetts Sr. Professional Services Consultant 24 October 2017
  • 2. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 2 Slides, Joind.in, and Stuff • Rate & comment: https://joind.in/talk/377ec • Slides: https://www.slideshare.net/clarkphp • Tweets: #zendcon2017 • Twitter: @clarkphp
  • 3. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 3 INTRODUCTION/ BACKGROUND
  • 4. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 4 As you know, Composer is… • Knows what packages your application or library depends upon • Obtains those packages, and all of their dependencies, and installs appropriate versions of them into your project (and local cache) • When requested, checks for updates compatible with your project, and downloads them into your project (and local cache) • Allows you to pin multiple applications/libraries to the same or different versions of the packages they use. Composer makes it easier to manage application dependencies. … a per-project PHP dependency manager
  • 5. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 5 Packagist.org Repository Package Archivist Just a Composer (“type”) Repository… • … but it is the primary repository for open source packages • Best Practice for Open Source Projects: register it at packagist.org • Searchable / Browsable • Less work for people to find and use your package. • Many, many, many packages available. There is duplication in functionality and – I suspect - a wide range of quality.
  • 6. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 6 https://packagist.org/
  • 7. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 7 Questions of interest to me • Composer & Packagist are GREAT! • Easy to consume 3rd party libraries • Easy to publish libraries How to find libraries that are • Reliable • Well-built • Tested • Secure How many projects available via Packagist?
  • 8. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 8 Why so important? Your Project Project == Application == Library DEPENDENCIES, PACKAGES, LIBRARIES needs A, B, C, D A B C D needs E E F needs E You are responsible for the application, and for the choice in which dependencies you use. HG needs G, H
  • 9. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 9 No, really! Consumer-only. Why important? Higher quality libraries are • Easier to understand • Easier to maintain • Easier to test • Get updated sooner • Bugfixes • New features • Don’t break as easily or as often • Greater interest/participation from dev team and users *Campbell County Kentucky Public Library
  • 10. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 10 SECURITY
  • 11. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 11 Security – Known Vulnerabilities
  • 12. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 12 "conflict": { "adodb/adodb-php": "<5.20.6", "amphp/artax": ">=2,<2.0.6|<1.0.6", "aws/aws-sdk-php": ">=3,<3.2.1", "bugsnag/bugsnag-laravel": ">=2,<2.0.2", "cakephp/cakephp": ">=3,<3.0.15|>=2,<2.4.99|>=2.5,<2.5.99|>=2.6,<2.6.12|>=1.3,<1.3.18|>=2.7,<2.7.6|>=3.1,<3.1.4", "cart2quote/module-quotation": ">=4.1.6,<=4.4.5|>=5,<5.4.4", "cartalyst/sentry": "<2.1", "codeigniter/framework": "<=3.0.6", "composer/composer": "<=1.0.0-alpha11", "contao-components/mediaelement": ">=2.14.2,<2.21.1", "contao/core": ">=2,<3.5.28", "contao/core-bundle": ">=4,<4.4.1", "doctrine/annotations": ">=1,<1.2.7", "doctrine/cache": ">=1,<1.3.2|>=1.4,<1.4.2", "doctrine/common": ">=2,<2.4.3|>=2.5,<2.5.1", "doctrine/dbal": ">=2,<2.0.8|>=2.1,<2.1.2", "doctrine/doctrine-bundle": "<1.5.2", "doctrine/doctrine-module": "<=0.7.1", "doctrine/mongodb-odm": ">=1,<1.0.2", "doctrine/mongodb-odm-bundle": ">=2,<3.0.1", "doctrine/orm": ">=2,<2.4.8|>=2.5,<2.5.1", "dompdf/dompdf": ">=0.6,<0.6.2", "drupal/core": ">=8,<8.3.7", "drupal/drupal": ">=8,<8.3.7", etc., etc. Avoiding packages with known vulnerabilities
  • 13. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 13 Security – The Unknown Vulnerabilities • OWASP Discussion on Source Code Analysis Tools – https://www.owasp.org/index.php/Source_Code_Analysis_Tools • Static Application Security Testing – Security flaws are currently better than the tools – IDEs, catch potential vulnerable code during development • Strengths – Scalability – Reliable for buffer overflow, SQL injection • Weaknesses – Configuration affects security – False positives – Many flaws not amenable to automatic discovery • Authentication/Access Control • Poor use of crypto
  • 14. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 14 Security – Some Tools for PHP • RIPS – http://sourceforge.net/projects/rips-scanner/ • phpcs-security-audit – PHP_CodeSniffer sniffs for core PHP and Drupal 7 – https://github.com/FloeDesignTechnologies/phpcs-security-audit • VisualCodeGrepper (VCG) – C/C++, C#, VB, PHP, Java, and PL/SQL for security issues and for comments indicative of insecure code – http://sourceforge.net/projects/visualcodegrepp/ • (Plug for KlocWork from Rogue Wave, for C, C++, Java, C#) – https://www.roguewave.com/capabilities/static-code-analysis
  • 15. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 15 OTHER TOOLS
  • 16. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 16 Some static analysis tools for PHP • PHPLOC – size metrics, complexity, dependencies, structure – https://github.com/sebastianbergmann/phploc • Copy/Paste Detector – finds duplicated code (exact dups) – https://github.com/sebastianbergmann/phpcpd • PHP Depend - – https://pdepend.org/ • PHP Mess Detector – https://phpmd.org/ • Phan – https://github.com/phan/phan • Exakat – https://www.exakat.io/ • PHPMetrics – http://www.phpmetrics.org/
  • 17. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 17 WHAT MEAN QUALI-TAY?
  • 18. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 18 What do we mean by “quality”? Concepts/words • Reliable, well-built, tested, secure What “observables” indicate higher quality? Quantifiable vs. Non- Quantifiable Download totals indicate popularity, but download trend would be more indicative. Download totals also show developers trying out a package, not necessarily continued usage. Stars or Likes or ratings
  • 19. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 19 Identifying a quality package • Does it do what we think it does/expect it do to, and at what level of confidence? • How many defects are known to exist, or have existed, in it? – Relative to size of the package codebase (defect density)? • How many defects are known to have been corrected, and at what level of confidence? • What is the rate of defect discovery? Of defect removal? • What are the densities of various severity levels of defects?
  • 20. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 20 Context! Critical Programming errors can lead to catastrophic outages or performance degradations that make a system unusable. Such programming errors at the system level are 90% of production issues. Such errors at the unit level, though far more numerous, are less than 10% of production issues.
  • 21. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 21 A SIMPLE ONE-STOP ANALYSIS SCRIPT
  • 22. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 22 A PHP script to run some QA tools $ cat analyse-codebase.php <?php // analyse-codebase.php // Runs several PHP static analysis tools; assumes *.phar names // This could be done with a shell/batch script, instead of php. // If running under Windows, use paths like 'C:desiredpathtofile' // If running under *nix, use paths like '/desired/path/to/file' // If PHP binary is not in your command path, you need to specify path to it. // const PHP_EXE = '/the/path/to/your/PHP/executable'; const PHP_EXE = 'php'; // the location where the QA tools reside (anywhere you like) // const QATOOLS = '/a/folder/containing/this/analyse-codebase.php/script'; const QATOOLS = 'C:Usersclark.ePHP-QA-Tools’;
  • 23. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 23 PHP QA tools script: setup continued // the directory (folder) ENCLOSING the code you wish to analyze // const CODEBASE_ROOT = '/path/to/parent/of/your/source/code/appname'; const CODEBASE_ROOT = 'C:Usersclark.escratchpadqasamples'; // where to write the analyses OUTPUT files const OUTPUT_DIR = 'C:Usersclark.escratchpadqa'; // used to give a name to the output files $appName = 'samples'; // this is a folder containing the code to analyze. // It is APPENDED to CODEBASE_ROOT, allowing for analysis of // a particular folder within CODEBASE_ROOT), if needed. $codebaseLocation = '';
  • 24. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 24 PHP QA tools script: setup continued // list of subfolders to exclude from counting (can be empty, but you often should exclude some folders) // PLEASE check this list and edit accordingly, BEFORE running the script. // Note, for PHPLOC, any directory in the tree with a name in this array is ignored, not just the top-level dirs. $excludeDirs = [ 'cache', 'config', 'documents', 'logs', 'public', 'sql_source', 'third_party', 'test', 'vendor', 'views', ];
  • 25. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 25 PHP QA tools script: calling the tools // becomes part of the output file names $timestamp = date('YmdHis'); $targetPhpVersion = '7.1'; print phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; print messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) . PHP_EOL; //print phpCompatibility($appName, $timestamp, $excludeDirs, $codebaseLocation, $targetPhpVersion) . PHP_EOL;
  • 26. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 26 PHP QA tools script: Lines of Code function phpLoc($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phploc'; $options = '--log-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --log-csv=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.csv" . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
  • 27. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 27 PHP QA tools script: Copy/Paste Detector function copyPasteDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpcpd'; $options = '--log-pmd=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --no-ansi' . ' --no-interaction' . ' --exclude ' . implode(' --exclude ', $excludeDirs); $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
  • 28. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 28 PHP QA tools script: PHP Depend function pDepend($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'pdepend'; $options = '--dependency-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- dependencies.xml" . ' --jdepend-chart=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.svg" . ' --jdepend-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp.xml" . ' --overview-pyramid=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- overview-pyramid.svg" . ' --summary-xml=' . OUTPUT_DIR . "/$appName-$tool-$timestamp- summary.xml" . ' --coderank-mode=inheritance' // how is this used? . ' --coverage-report=' . OUTPUT_DIR . "$appName- $tool-$timestamp-coverage.xml" . ' --ignore=' . implode(',', $excludeDirs);
  • 29. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 29 PHP QA tools script: PHP Depend $output = shell_exec('php ' . QATOOLS . "/$tool.phar $options " . CODEBASE_ROOT . "/$codebaseLocation" . ' > ' . OUTPUT_DIR . "/$appName-$tool-$timestamp.out"); return $output; }
  • 30. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 30 PHP QA tools script: PHP Mess Detector function messDetector($appName, $timestamp, $excludeDirs, $codebaseLocation) { $tool = 'phpmd'; $ruleSets = ['cleancode', 'codesize', 'controversial', 'design', 'naming', 'unusedcode']; foreach ($ruleSets as $ruleSet) { $options = "text $ruleSet --reportfile " . OUTPUT_DIR . "/$appName-$tool-$timestamp-$ruleSet.out" . ' --exclude ' . implode(',', $excludeDirs); // --strict: also report those nodes with a @SuppressWarnings annotation shell_exec('php ' . QATOOLS . "/$tool.phar " . CODEBASE_ROOT . "/$codebaseLocation $options"); print $ruleSet . PHP_EOL; } }
  • 31. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 31 Other Resources • PHP The Right Way - http://www.phptherightway.com/ • OWASP Tools - https://www.owasp.org/index.php/Source_Code_Analysis_Tools • Survive the Deep End: PHP Security (2013, but relevant) - http://phpsecurity.readthedocs.io/en/latest/ • SonarQube – “Water Leak” approach to code quality – https://www.sonarqube.org/
  • 32. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 32 So, who is this guy? Clark Everetts, ZCE • Rogue Wave Software (acquired Zend October 2015) • PHP since 2005 • Professional Services Consultant – Architecture and Performance Audits – PHP, Zend Framework Training – Application Development, Best Practices, etc. – IBM i • [email protected] @clarkphp +ClarkEveretts
  • 33. © 2017 Rogue Wave Software, Inc. All Rights Reserved. 33 THANK-YOU [email protected] @clarkphp +ClarkEveretts Tweet: #zendcon2017 Rate, comment, get slides https://joind.in/talk/377ec Your feedback is invaluable!