SlideShare a Scribd company logo

Unit Testing Lots of Perl

Workhorse Computing
Workhorse Computing

The document discusses the stagnation of Perl 5.8 and the challenges faced in transitioning to newer versions due to lack of testing. It emphasizes the importance of unit testing, sharing experiences from a case where 75,000 modules were tested to validate their functionality. The author proposes methods for automated testing and urges the use of testing to ensure compatibility with upcoming versions of Perl.

Technology
1 of 68
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
Getting There from Here: Unit testing a few generations of code. Steven Lembark Workhorse Computing lembark@wrkhors.com
perl 5.8 Fun isn’t it? A generation of nothing. No improvements in Perl. No advances in CPAN. No reason to upgrade. 20 years of nada... Ask all the programmers stuck with 5.8.
perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: ???
perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: Because “it worked”.
perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: Because something else “might not” work.
perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: Because we can’t show that 5.X works. (for some definition of Christmas > 5.8)
perl 5.X Q: Why can’t we show that 5.X works? A: Because Christmas came after 2002.
perl 5.X Q: Why can’t we show that 5.X works? A: Because we stopped testing our code.
perl 5.X Q: Why can’t we show that 5.X works? A: Because they stopped testing our code.
perl 5.X Q: Why can’t we show that 5.X works? A: Because we have worked around it for way too long.
perl 5.X Q: Why can’t we show that 5.X works? A: Because we have worked around it for way too long. perlbrew, all of the other approaches to dodging /usr/bin/perl.
OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing.
OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing. 75_000 modules worth of testing.
OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing. 75_000 modules worth of testing. Q: But how did you write 75_000 unit tests?
OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing. 75_000 modules worth of automated testing. A: Lazyness.
Syntax checks are the first pass. Unit tests get a bad rap. “They don’t really test anything.”
Syntax checks are the first pass. Unit tests get a bad rap. “They don’t really test anything.” Except whether 75_000 modules compile. Code that was written on 5.8. That now to run on 5.20+.
How do we test that much code? Metadata driven testing: Encode data into a test. The standard test is data-driven. Tests each validate one small part of a module.
How do we test that much code? Q: How do you write that many tests? A: Don’t.
How do we test that much code? Q: How do you write that many tests? A: Don’t. Symlinks are your friends.
Simple basic test: require_ok Run “require_ok $module” for every *.pm. Simple, basic, obvious test for syntax errors. … missing modules. … botched system paths. … all sorts of things.
Simple basic test: require_ok Run “require_ok $module” for every *.pm. Simple, basic, obvious test for syntax errors. Nice thing: $module can be a path. Even an absolute path.
Passing a path “require_ok $path” One argument. Q: How do we pass it? A: In the basename of a test.
Basic unit test use v5.30; use Test::More; use File::Basename; my $base0 = basename $0, '.t'; my $s = substr $base0, 0, 1; my $path = join ‘/’, split m{[$s]}, $base0; require_ok $path; done_testing __END__
Basic unit test Install them using shell. #!/bin/bash dir=$(cd $(dirname $0)/../..; pwd -L); cd “$dir/t/01-units”; find $dir/lib -name ‘*pm’ | while read i do base=”$(echo $i | tr ‘/’ ‘~’).t”; ln -fs ../bin/01-unit_t ./$base; done
Basic unit test Install them using shell. From ./t/bin #!/bin/bash dir=$(cd $(dirname $0)/../..; pwd -L); cd “$dir/t/01-units”; find $dir/lib -name ‘*pm’ | while read i do base=”$(echo $i | tr ‘/’ ‘~’).t”; ln -fs ../bin/01-unit_t ./$base; done
Basic unit test Install them using shell. Path to basename. #!/bin/bash dir=$(cd $(dirname $0)/../..; pwd -L); cd “$dir/t/01-units”; find $dir/lib -name ‘*pm’ | while read i do base=”$(echo $i | tr ‘/’ ‘~’).t”; ln -fs ../bin/01-unit_t ./$base; done
Basic unit test Now you know if it compiles. prove t/01-units;
Basic unit test Now you know if it compiles. A bit faster. prove --jobs=8 t/01-units;
Basic unit test Now you know if it compiles. At 3am without watching. args=(--jobs=8 --state=save ); prove ${args[*]} t/01-units;
Basic unit test Now you know if it compiles. At 8am knowing what failed. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units;
Basic unit test Now you know if it compiles. Missing module anyone? args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units Can't locate Foo/Bar.pm in @INC (you may need to install the Foo::Bar module) (@INC contains: /opt/perl/5.30/lib/site_perl/5.30.1/x8 6_64-linux
Basic unit test Now you know if it compiles. Fine: Install them. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm ;
Basic unit test Now you know if it compiles. Local mirror provides pre- verified modules. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm -M$local ;
Basic unit test Now you know if it compiles. Local library simplifies auto- install. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm -M$local -l$repo --self-contained ;
Basic unit test Start with a virgin /opt/perl. Keep testing until it’s all use- able. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm -M$local -l$repo --self-contained ;
Version control what you install git submodule add blah://.../site_perl; perl Makefile.PL INSTALL_BASE=$PWD/site_perl; cpanm --local-library=$PWD/site_perl; If anyone asks, you know what non-core modules have been installed. If anything breaks, check out the last commit.
What else can we do with a path? Ever fat-finger a package? package AcmeWigdit::Config; use v5.30;
What else can we do with a path? Ever fat-finger a package? Paths define packages. require_ok $path or skip ... ; my ($sub) = $path =~ m{^.+?/lib/(.+?) [.]pm}x; my $pkg = $sub =~ s{/}{::}g; isa_ok $pkg, ‘UNIVERSAL’ or skip “$pkg not ‘UNIVERSAL’”; $pkg->can( ‘VERSION’ ) or skip “$path missing $pkg”, 1;
What else can we do with a path? Ever fat-finger a package? Paths define packages. All defined packages are UNIVERSAL. require_ok $path or skip ... ; my ($sub) = $path =~ m{^.+?/lib/(.+?) [.]pm}x; my $pkg = $sub =~ s{/}{::}g; isa_ok $pkg, ‘UNIVERSAL’ or skip “$pkg not ‘UNIVERSAL’”; $pkg->can( ‘VERSION’ ) or skip “$path missing $pkg”, 1;
What else can we do with a path? Ever fat-finger a package? Paths define packages. VERSION is UNIVERSAL. require_ok $path or skip ... ; my ($sub) = $path =~ m{^.+?/lib/(.+?) [.]pm}x; my $pkg = $sub =~ s{/}{::}g; is_ok $pkg, ‘UNIVERSAL’ or skip “$pkg not ‘UNIVERSAL’”; $pkg->can( ‘VERSION’ ) or skip “$path missing $pkg”, 1;
Minor issue with paths Packages are related to paths. We know that now. In 2000 not everyone got that.
Minor issue with paths What about: package <basename>; use lib <every directory everywhere>; It works, but there are better ways...
Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config
Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config
Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config ./lib/AcmeWig/Acme/Config.pm Acme::Config
Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config ./lib/AcmeWig/Acme/Config.pm Acme::Config What’s the expected package?
Solving nested paths Say we figure out the paths: qw ( ./lib/AcmeWig ./lib/AcmeWig/Acme ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/AcmeWig/External/Plastic )
Solving nested paths We can obviously iterate them... qw ( ./lib/AcmeWig ./lib/AcmeWig/Acme ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/AcmeWig/External/Plastic )
Solving nested paths Q: But how do we avoid duplicate tests? qw ( ./lib/AcmeWig ./lib/AcmeWig/Acme ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/AcmeWig/External/Plastic )
Solving nested paths A: Sort them by length. qw ( ./lib/AcmeWig/AcmeWig/External/Plastic ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/Acme ./lib/AcmeWig )
Solving nested paths A: Sort them by length & exclude known paths. qw ( ./lib/AcmeWig/AcmeWig/External/Plastic ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/Acme ./lib/AcmeWig )
Solving nested paths my %prune = (); my $wanted = sub { -d && exists $prune{ $File::Find::dir } ? $File::Find::prune = 1 : $File::Find::Path->$install_test_symlink }; for my $dir ( @dirs_sorted_by_length ) { find $wanted, $dir; $prune{ $dir } = (); }
Solving nested paths my %prune = (); my $wanted = sub { -d && exists $prune{ $File::Find::dir } ? $File::Find::prune = 1 : $File::Find::Path->$install_test_symlink }; for my $dir ( @dirs_sorted_by_length ) { find $wanted, $dir; $prune{ $dir } = (); # don’t revisit }
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package.
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: Double the separator. ~path~to~repo~lib~AcmeWig~Acme~Config.pm.t
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: Double the separator. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: ‘//’ works fine with require_ok. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t /path/to/repo/lib/AcmeWig//Acme/Config.pm.t
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: Gives sub-path for module. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t Acme~Config.pm.t
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: basename with s{~}{::}. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t Acme::Config
Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Save the search dir’s length and substr in a ‘~’: my $l = length $dir; ... substr $symlink, $l, 0, ‘~’;
Skipping test groups We started with 75_000 files. Found some dirs we didn’t want to test. Fix: my @no_test = qw( … ); my %prune = (); @prune{ @no_test } = ();
Net result We were able to test 45_000+ files each night. Found missing modules. Found outdated syntax. Managed to get it all working. Largey with “require_ok”.
Knowledge is power Unit tests are useful. Provide automated, reproducable results. No excuse for “It may not work in v5.30.” We can know. And fix whatever “it” is.
For some definition of Perl5 Perl7 is coming. Adopting it requires showing that “it works.” Which means finding where it doesn’t. Unit testing all of CPAN. Unit testing all of everything.
For some definition of Perl5 Perl7 is coming. Adopting it requires showing that “it works.” Which means finding where it doesn’t. Unit testing all of CPAN. Unit testing all of everything. We just have to be lazy about it.
Units of your very own! In case you don’t like pasting from PDF: https://gitlab.com/lembark/perl5-unit-tests If anyone wants to work on this we can release it as App::Testify or Test2::Unitz … Be nice to have a migration suite for Perl7.
Units of your very own! A wider range of unit tests is shown at: https://www.slideshare.net/lembark/path-toknowlege With a little bit of work we could get them all on CPAN.

More Related Content

What's hot (20)

PDF
The $path to knowledge: What little it take to unit-test Perl.
Workhorse Computing
 
PDF
Get your teeth into Plack
Workhorse Computing
 
PDF
BSDM with BASH: Command Interpolation
Workhorse Computing
 
PDF
Shared Object images in Docker: What you need is what you want.
Workhorse Computing
 
PDF
Replacing "exec" with a type and provider: Return manifests to a declarative ...
Puppet
 
PPTX
Puppet camp chicago-automated_testing2
nottings
 
PDF
Puppet: What _not_ to do
Puppet
 
PDF
Test-Driven Puppet Development - PuppetConf 2014
Puppet
 
PPTX
Troubleshooting Puppet
Thomas Howard Uphill
 
PDF
Puppet loves RSpec, why you should, too
Dennis Rowe
 
PDF
What you need to remember when you upload to CPAN
charsbar
 
PDF
Better detection of what modules are used by some Perl 5 code
charsbar
 
PDF
Can you upgrade to Puppet 4.x?
Martin Alfke
 
PDF
Release with confidence
John Congdon
 
PDF
SDPHP - Percona Toolkit (It's Basically Magic)
Robert Swisher
 
PDF
Lies, Damn Lies, and Benchmarks
Workhorse Computing
 
PPT
Working with databases in Perl
Laurent Dami
 
PDF
Puppet modules for Fun and Profit
Alessandro Franceschi
 
PDF
Test Driven Development with Puppet - PuppetConf 2014
Puppet
 
PDF
PL/Perl - New Features in PostgreSQL 9.0
Tim Bunce
 
The $path to knowledge: What little it take to unit-test Perl.
Workhorse Computing
 
Get your teeth into Plack
Workhorse Computing
 
BSDM with BASH: Command Interpolation
Workhorse Computing
 
Shared Object images in Docker: What you need is what you want.
Workhorse Computing
 
Replacing "exec" with a type and provider: Return manifests to a declarative ...
Puppet
 
Puppet camp chicago-automated_testing2
nottings
 
Puppet: What _not_ to do
Puppet
 
Test-Driven Puppet Development - PuppetConf 2014
Puppet
 
Troubleshooting Puppet
Thomas Howard Uphill
 
Puppet loves RSpec, why you should, too
Dennis Rowe
 
What you need to remember when you upload to CPAN
charsbar
 
Better detection of what modules are used by some Perl 5 code
charsbar
 
Can you upgrade to Puppet 4.x?
Martin Alfke
 
Release with confidence
John Congdon
 
SDPHP - Percona Toolkit (It's Basically Magic)
Robert Swisher
 
Lies, Damn Lies, and Benchmarks
Workhorse Computing
 
Working with databases in Perl
Laurent Dami
 
Puppet modules for Fun and Profit
Alessandro Franceschi
 
Test Driven Development with Puppet - PuppetConf 2014
Puppet
 
PL/Perl - New Features in PostgreSQL 9.0
Tim Bunce
 

Similar to Unit Testing Lots of Perl (20)

PDF
Metadata-driven Testing
Workhorse Computing
 
PDF
Continuous Integration Testing in Django
Kevin Harvey
 
PDF
Ansible roles done right
Dan Vaida
 
KEY
How To Test Everything
noelrap
 
PDF
Ansible testing
Scott van Kalken
 
PDF
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
NETWAYS
 
PDF
Can you upgrade to Puppet 4.x? (Beginner) Can you upgrade to Puppet 4.x? (Beg...
Puppet
 
ODP
Advanced Perl Techniques
Dave Cross
 
PDF
How I hack on puppet modules
Kris Buytaert
 
PDF
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
George Boobyer
 
KEY
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
Puppet
 
PPTX
Angular JS in 2017
Ayush Sharma
 
PPTX
Autotesting rails app
Anton Naumenko
 
PPTX
Testing Ansible
Anth Courtney
 
PDF
Docker perl build
Workhorse Computing
 
PDF
IntroTestMore
tutorialsruby
 
PDF
IntroTestMore
tutorialsruby
 
PDF
Ansible with oci
DonghuKIM2
 
PDF
Automate Yo' Self
John Anderson
 
PDF
Automate Yo'self -- SeaGL
John Anderson
 
Metadata-driven Testing
Workhorse Computing
 
Continuous Integration Testing in Django
Kevin Harvey
 
Ansible roles done right
Dan Vaida
 
How To Test Everything
noelrap
 
Ansible testing
Scott van Kalken
 
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
NETWAYS
 
Can you upgrade to Puppet 4.x? (Beginner) Can you upgrade to Puppet 4.x? (Beg...
Puppet
 
Advanced Perl Techniques
Dave Cross
 
How I hack on puppet modules
Kris Buytaert
 
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
George Boobyer
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
Puppet
 
Angular JS in 2017
Ayush Sharma
 
Autotesting rails app
Anton Naumenko
 
Testing Ansible
Anth Courtney
 
Docker perl build
Workhorse Computing
 
IntroTestMore
tutorialsruby
 
IntroTestMore
tutorialsruby
 
Ansible with oci
DonghuKIM2
 
Automate Yo' Self
John Anderson
 
Automate Yo'self -- SeaGL
John Anderson
 
Ad

More from Workhorse Computing (19)

PDF
Object::Trampoline: Follow the bouncing object.
Workhorse Computing
 
PDF
Wheels we didn't re-invent: Perl's Utility Modules
Workhorse Computing
 
PDF
mro-every.pdf
Workhorse Computing
 
PDF
Paranormal statistics: Counting What Doesn't Add Up
Workhorse Computing
 
PDF
Generating & Querying Calendar Tables in Posgresql
Workhorse Computing
 
PDF
Hypers and Gathers and Takes! Oh my!
Workhorse Computing
 
PDF
Findbin libs
Workhorse Computing
 
PDF
The W-curve and its application.
Workhorse Computing
 
PDF
Perl6 Regexen: Reduce the line noise in your code.
Workhorse Computing
 
PDF
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Workhorse Computing
 
PDF
Neatly folding-a-tree
Workhorse Computing
 
PDF
Light my-fuse
Workhorse Computing
 
PDF
Paranormal stats
Workhorse Computing
 
PDF
Putting some "logic" in LVM.
Workhorse Computing
 
PDF
Selenium sandwich-3: Being where you aren't.
Workhorse Computing
 
PDF
Selenium sandwich-2
Workhorse Computing
 
PDF
Selenium Sandwich Part 1: Data driven Selenium
Workhorse Computing
 
PDF
Designing net-aws-glacier
Workhorse Computing
 
PDF
Ethiopian multiplication in Perl6
Workhorse Computing
 
Object::Trampoline: Follow the bouncing object.
Workhorse Computing
 
Wheels we didn't re-invent: Perl's Utility Modules
Workhorse Computing
 
mro-every.pdf
Workhorse Computing
 
Paranormal statistics: Counting What Doesn't Add Up
Workhorse Computing
 
Generating & Querying Calendar Tables in Posgresql
Workhorse Computing
 
Hypers and Gathers and Takes! Oh my!
Workhorse Computing
 
Findbin libs
Workhorse Computing
 
The W-curve and its application.
Workhorse Computing
 
Perl6 Regexen: Reduce the line noise in your code.
Workhorse Computing
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Workhorse Computing
 
Neatly folding-a-tree
Workhorse Computing
 
Light my-fuse
Workhorse Computing
 
Paranormal stats
Workhorse Computing
 
Putting some "logic" in LVM.
Workhorse Computing
 
Selenium sandwich-3: Being where you aren't.
Workhorse Computing
 
Selenium sandwich-2
Workhorse Computing
 
Selenium Sandwich Part 1: Data driven Selenium
Workhorse Computing
 
Designing net-aws-glacier
Workhorse Computing
 
Ethiopian multiplication in Perl6
Workhorse Computing
 
Ad

Recently uploaded (20)

PDF
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
PDF
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
PDF
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
PDF
July Patch Tuesday
Ivanti
 
PDF
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
PPTX
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
PDF
NewMind AI - Journal 100 Insights After The 100th Issue
NewMind AI
 
PDF
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
PDF
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
PDF
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
PDF
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
PPTX
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
PDF
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
PPTX
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
PDF
Smart Trailers 2025 Update with History and Overview
Paul Menig
 
PPTX
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
PPTX
AUTOMATION AND ROBOTICS IN PHARMA INDUSTRY.pptx
sameeraaabegumm
 
PDF
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
PDF
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 
"AI Transformation: Directions and Challenges", Pavlo Shaternik
Fwdays
 
HCIP-Data Center Facility Deployment V2.0 Training Material (Without Remarks ...
mcastillo49
 
Newgen Beyond Frankenstein_Build vs Buy_Digital_version.pdf
darshakparmar
 
July Patch Tuesday
Ivanti
 
"Beyond English: Navigating the Challenges of Building a Ukrainian-language R...
Fwdays
 
Building Search Using OpenSearch: Limitations and Workarounds
Sease
 
NewMind AI - Journal 100 Insights After The 100th Issue
NewMind AI
 
[Newgen] NewgenONE Marvin Brochure 1.pdf
darshakparmar
 
Building Real-Time Digital Twins with IBM Maximo & ArcGIS Indoors
Safe Software
 
HubSpot Main Hub: A Unified Growth Platform
Jaswinder Singh
 
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
OpenID AuthZEN - Analyst Briefing July 2025
David Brossard
 
New from BookNet Canada for 2025: BNC BiblioShare - Tech Forum 2025
BookNet Canada
 
AI Penetration Testing Essentials: A Cybersecurity Guide for 2025
defencerabbit Team
 
Smart Trailers 2025 Update with History and Overview
Paul Menig
 
WooCommerce Workshop: Bring Your Laptop
Laura Hartwig
 
AUTOMATION AND ROBOTICS IN PHARMA INDUSTRY.pptx
sameeraaabegumm
 
The Rise of AI and IoT in Mobile App Tech.pdf
IMG Global Infotech
 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
Empower Inclusion Through Accessible Java Applications
Ana-Maria Mihalceanu
 

Unit Testing Lots of Perl

  • 1. Getting There from Here: Unit testing a few generations of code. Steven Lembark Workhorse Computing [email protected]
  • 2. perl 5.8 Fun isn’t it? A generation of nothing. No improvements in Perl. No advances in CPAN. No reason to upgrade. 20 years of nada... Ask all the programmers stuck with 5.8.
  • 3. perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: ???
  • 4. perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: Because “it worked”.
  • 5. perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: Because something else “might not” work.
  • 6. perl 5.8 Q: Why are we stuck with 5.8? A: /usr/bin/perl is 5.8 Q: Why are we stuck with /usr/bin/perl? A: Because we can’t show that 5.X works. (for some definition of Christmas > 5.8)
  • 7. perl 5.X Q: Why can’t we show that 5.X works? A: Because Christmas came after 2002.
  • 8. perl 5.X Q: Why can’t we show that 5.X works? A: Because we stopped testing our code.
  • 9. perl 5.X Q: Why can’t we show that 5.X works? A: Because they stopped testing our code.
  • 10. perl 5.X Q: Why can’t we show that 5.X works? A: Because we have worked around it for way too long.
  • 11. perl 5.X Q: Why can’t we show that 5.X works? A: Because we have worked around it for way too long. perlbrew, all of the other approaches to dodging /usr/bin/perl.
  • 12. OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing.
  • 13. OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing. 75_000 modules worth of testing.
  • 14. OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing. 75_000 modules worth of testing. Q: But how did you write 75_000 unit tests?
  • 15. OK how do we fix it? Prove that “it works”. With testing. In one client’s case: Lots and lots of testing. 75_000 modules worth of automated testing. A: Lazyness.
  • 16. Syntax checks are the first pass. Unit tests get a bad rap. “They don’t really test anything.”
  • 17. Syntax checks are the first pass. Unit tests get a bad rap. “They don’t really test anything.” Except whether 75_000 modules compile. Code that was written on 5.8. That now to run on 5.20+.
  • 18. How do we test that much code? Metadata driven testing: Encode data into a test. The standard test is data-driven. Tests each validate one small part of a module.
  • 19. How do we test that much code? Q: How do you write that many tests? A: Don’t.
  • 20. How do we test that much code? Q: How do you write that many tests? A: Don’t. Symlinks are your friends.
  • 21. Simple basic test: require_ok Run “require_ok $module” for every *.pm. Simple, basic, obvious test for syntax errors. … missing modules. … botched system paths. … all sorts of things.
  • 22. Simple basic test: require_ok Run “require_ok $module” for every *.pm. Simple, basic, obvious test for syntax errors. Nice thing: $module can be a path. Even an absolute path.
  • 23. Passing a path “require_ok $path” One argument. Q: How do we pass it? A: In the basename of a test.
  • 24. Basic unit test use v5.30; use Test::More; use File::Basename; my $base0 = basename $0, '.t'; my $s = substr $base0, 0, 1; my $path = join ‘/’, split m{[$s]}, $base0; require_ok $path; done_testing __END__
  • 25. Basic unit test Install them using shell. #!/bin/bash dir=$(cd $(dirname $0)/../..; pwd -L); cd “$dir/t/01-units”; find $dir/lib -name ‘*pm’ | while read i do base=”$(echo $i | tr ‘/’ ‘~’).t”; ln -fs ../bin/01-unit_t ./$base; done
  • 26. Basic unit test Install them using shell. From ./t/bin #!/bin/bash dir=$(cd $(dirname $0)/../..; pwd -L); cd “$dir/t/01-units”; find $dir/lib -name ‘*pm’ | while read i do base=”$(echo $i | tr ‘/’ ‘~’).t”; ln -fs ../bin/01-unit_t ./$base; done
  • 27. Basic unit test Install them using shell. Path to basename. #!/bin/bash dir=$(cd $(dirname $0)/../..; pwd -L); cd “$dir/t/01-units”; find $dir/lib -name ‘*pm’ | while read i do base=”$(echo $i | tr ‘/’ ‘~’).t”; ln -fs ../bin/01-unit_t ./$base; done
  • 28. Basic unit test Now you know if it compiles. prove t/01-units;
  • 29. Basic unit test Now you know if it compiles. A bit faster. prove --jobs=8 t/01-units;
  • 30. Basic unit test Now you know if it compiles. At 3am without watching. args=(--jobs=8 --state=save ); prove ${args[*]} t/01-units;
  • 31. Basic unit test Now you know if it compiles. At 8am knowing what failed. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units;
  • 32. Basic unit test Now you know if it compiles. Missing module anyone? args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units Can't locate Foo/Bar.pm in @INC (you may need to install the Foo::Bar module) (@INC contains: /opt/perl/5.30/lib/site_perl/5.30.1/x8 6_64-linux
  • 33. Basic unit test Now you know if it compiles. Fine: Install them. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm ;
  • 34. Basic unit test Now you know if it compiles. Local mirror provides pre- verified modules. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm -M$local ;
  • 35. Basic unit test Now you know if it compiles. Local library simplifies auto- install. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm -M$local -l$repo --self-contained ;
  • 36. Basic unit test Start with a virgin /opt/perl. Keep testing until it’s all use- able. args=(--jobs=8 --state=save,failed ); prove ${args[*]} t/01-units | grep ‘you may need to install the’ | cut -d’(‘ -f2 | cut -d’ ‘ -f7 | sort -d | uniq | cpanm -M$local -l$repo --self-contained ;
  • 37. Version control what you install git submodule add blah://.../site_perl; perl Makefile.PL INSTALL_BASE=$PWD/site_perl; cpanm --local-library=$PWD/site_perl; If anyone asks, you know what non-core modules have been installed. If anything breaks, check out the last commit.
  • 38. What else can we do with a path? Ever fat-finger a package? package AcmeWigdit::Config; use v5.30;
  • 39. What else can we do with a path? Ever fat-finger a package? Paths define packages. require_ok $path or skip ... ; my ($sub) = $path =~ m{^.+?/lib/(.+?) [.]pm}x; my $pkg = $sub =~ s{/}{::}g; isa_ok $pkg, ‘UNIVERSAL’ or skip “$pkg not ‘UNIVERSAL’”; $pkg->can( ‘VERSION’ ) or skip “$path missing $pkg”, 1;
  • 40. What else can we do with a path? Ever fat-finger a package? Paths define packages. All defined packages are UNIVERSAL. require_ok $path or skip ... ; my ($sub) = $path =~ m{^.+?/lib/(.+?) [.]pm}x; my $pkg = $sub =~ s{/}{::}g; isa_ok $pkg, ‘UNIVERSAL’ or skip “$pkg not ‘UNIVERSAL’”; $pkg->can( ‘VERSION’ ) or skip “$path missing $pkg”, 1;
  • 41. What else can we do with a path? Ever fat-finger a package? Paths define packages. VERSION is UNIVERSAL. require_ok $path or skip ... ; my ($sub) = $path =~ m{^.+?/lib/(.+?) [.]pm}x; my $pkg = $sub =~ s{/}{::}g; is_ok $pkg, ‘UNIVERSAL’ or skip “$pkg not ‘UNIVERSAL’”; $pkg->can( ‘VERSION’ ) or skip “$path missing $pkg”, 1;
  • 42. Minor issue with paths Packages are related to paths. We know that now. In 2000 not everyone got that.
  • 43. Minor issue with paths What about: package <basename>; use lib <every directory everywhere>; It works, but there are better ways...
  • 44. Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config
  • 45. Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config
  • 46. Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config ./lib/AcmeWig/Acme/Config.pm Acme::Config
  • 47. Minor issue with paths One pattern: Overlapping product of evolution. ./lib/AcmeWig/ ./lib/AcmeWig/Config.pm AcmeWig::Config ./lib/AcmeWig/Acme/Config.pm Acme::Config What’s the expected package?
  • 48. Solving nested paths Say we figure out the paths: qw ( ./lib/AcmeWig ./lib/AcmeWig/Acme ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/AcmeWig/External/Plastic )
  • 49. Solving nested paths We can obviously iterate them... qw ( ./lib/AcmeWig ./lib/AcmeWig/Acme ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/AcmeWig/External/Plastic )
  • 50. Solving nested paths Q: But how do we avoid duplicate tests? qw ( ./lib/AcmeWig ./lib/AcmeWig/Acme ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/AcmeWig/External/Plastic )
  • 51. Solving nested paths A: Sort them by length. qw ( ./lib/AcmeWig/AcmeWig/External/Plastic ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/Acme ./lib/AcmeWig )
  • 52. Solving nested paths A: Sort them by length & exclude known paths. qw ( ./lib/AcmeWig/AcmeWig/External/Plastic ./lib/AcmeWig/Acme/Plastic/External ./lib/AcmeWig/Acme/Plastic/Internal ./lib/AcmeWig/Acme/Plastic/Wrappers ./lib/AcmeWig/Plastic ./lib/AcmeWig/Acme ./lib/AcmeWig )
  • 53. Solving nested paths my %prune = (); my $wanted = sub { -d && exists $prune{ $File::Find::dir } ? $File::Find::prune = 1 : $File::Find::Path->$install_test_symlink }; for my $dir ( @dirs_sorted_by_length ) { find $wanted, $dir; $prune{ $dir } = (); }
  • 54. Solving nested paths my %prune = (); my $wanted = sub { -d && exists $prune{ $File::Find::dir } ? $File::Find::prune = 1 : $File::Find::Path->$install_test_symlink }; for my $dir ( @dirs_sorted_by_length ) { find $wanted, $dir; $prune{ $dir } = (); # don’t revisit }
  • 55. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package.
  • 56. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: Double the separator. ~path~to~repo~lib~AcmeWig~Acme~Config.pm.t
  • 57. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: Double the separator. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t
  • 58. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: ‘//’ works fine with require_ok. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t /path/to/repo/lib/AcmeWig//Acme/Config.pm.t
  • 59. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: Gives sub-path for module. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t Acme~Config.pm.t
  • 60. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Fix: basename with s{~}{::}. ~path~to~repo~lib~AcmeWig~~Acme~Config.pm.t Acme::Config
  • 61. Testing nested paths Catch: The test needs to know its ‘root’ directory. Has to subtract that to get the expected package. Save the search dir’s length and substr in a ‘~’: my $l = length $dir; ... substr $symlink, $l, 0, ‘~’;
  • 62. Skipping test groups We started with 75_000 files. Found some dirs we didn’t want to test. Fix: my @no_test = qw( … ); my %prune = (); @prune{ @no_test } = ();
  • 63. Net result We were able to test 45_000+ files each night. Found missing modules. Found outdated syntax. Managed to get it all working. Largey with “require_ok”.
  • 64. Knowledge is power Unit tests are useful. Provide automated, reproducable results. No excuse for “It may not work in v5.30.” We can know. And fix whatever “it” is.
  • 65. For some definition of Perl5 Perl7 is coming. Adopting it requires showing that “it works.” Which means finding where it doesn’t. Unit testing all of CPAN. Unit testing all of everything.
  • 66. For some definition of Perl5 Perl7 is coming. Adopting it requires showing that “it works.” Which means finding where it doesn’t. Unit testing all of CPAN. Unit testing all of everything. We just have to be lazy about it.
  • 67. Units of your very own! In case you don’t like pasting from PDF: https://gitlab.com/lembark/perl5-unit-tests If anyone wants to work on this we can release it as App::Testify or Test2::Unitz … Be nice to have a migration suite for Perl7.
  • 68. Units of your very own! A wider range of unit tests is shown at: https://www.slideshare.net/lembark/path-toknowlege With a little bit of work we could get them all on CPAN.