PL/Perl - New Features in PostgreSQL 9.0

PL/Perl New Features
in PostgreSQL 9.0

Tim Bunce - June 2010
Creative Commons BY-NC-SA 3.0

PL/Perl Changes

‣ USER
‣ INTERNAL
‣ DBA
‣ NYTPROF

quote_...

quote_literal( "foo" ) ==> "'foo'"
quote_literal( "don't "carp"" ) ==> "'don''t "carp"'"
quote_literal( "" ) ==> "''"

quote_nullable( "foo" ) ==> "'foo'"
quote_nullable( "don't "carp"" ) ==> "'don''t "carp"'"
quote_nullable( "" ) ==> "''"

quote_ident( "foo" ) ==> "foo"
quote_ident( "don't "carp"" ) ==> ""don't ""carp""""
quote_ident( "" ) ==> """"

quote_...(undef)

quote_literal( undef ) ==> undef

quote_nullable( undef ) ==> "NULL"

quote_ident( undef ) ==> """" (warn)

{en,de}code_bytea
encode_bytea( "foo" ) ==> "x666f6f"
decode_bytea( "x666f6f" ) ==> "foo"
decode_bytea( "146157157") ==> "foo"

encode_bytea( "x{263a}" ) ==> "xe298ba" UTF8
decode_bytea( "xe298ba" ) ==> "342230272" Not UTF8

encode_bytea( "" ) ==> "x"
decode_bytea( "x" ) ==> ""
decode_bytea( "" ) ==> ""

encode_bytea( undef ) ==> "x" (warn)
decode_bytea( undef ) ==> "" (warn)

looks_like_number

looks_like_number( 1 ) ==> 1
looks_like_number( 0 ) ==> 1
looks_like_number( "+7.2e-9" ) ==> 1

looks_like_number( "foo" ) ==> 0
looks_like_number( "" ) ==> 0
looks_like_number( undef ) ==> undef

looks_like_number( " 4 " ) ==> 1

looks_like_number( "5plus" ) ==> 0 (but '5plus'+0=5)

encode_array_*

encode_array_literal( ["foo","bar"] )
==> "{"foo", "bar"}"

encode_array_constructor( ["foo","bar"] )
==> "ARRAY['foo', 'bar']"

encode_array_literal( [1,[2,[undef]]] )
==> "{"1", {"2", {NULL}}}"

encode_array_constructor( [1,[2,[undef]]] )
==> "ARRAY['1', ARRAY['2', ARRAY[NULL]]]"

encode_array_*

encode_array_literal( "foo" ) ==> "foo"
encode_array_constructor( "foo" ) ==> "'foo'"

encode_array_literal( undef ) ==> undef
encode_array_constructor( undef ) ==> "NULL"

Trusted require/use
• require/use work for already loaded modules
use strict; # old way: BEGIN { strict->import(); }

• extra pre-loaded modules
use warnings;
use Carp;
use feature qw(say); # for perl 5.10 or later
use utf8; # if server_encoding is utf8

CONTEXT: ...

• PL/Perl tracks the context of log messages
- before:
WARNING: ...some warning from perl code...

- now:
WARNING: ...some warning from perl code...
CONTEXT: PL/Perl function "..."

DO '...' LANGUAGE plperl;

• Arbitrary chunks of perl code can be executed
directly from psql, or client apps, via DO
• No need to create and run a stored procedure
each time.

DO $$
spi_exec("... $_ ...") for 'a'..'z';
$$ language plperl;

Other Changes
• Using $a and $b in sort blocks now works!
• eval { ... } and eval "..."
- can now be used in plperl
• END blocks are now run at end of session
- they can't (currently) access the database
• Warnings from perl are now WARNINGs
- they used to be NOTICE

INTERNAL

• The Safe module is no longer used for plperl
- Improved security and reduced call overheads
- Upgrade to latest security patch!
• Validates return values are in server encoding
- ERROR: invalid byte sequence for encoding
• Internal code refactoring and cleanup

New plperl.* Conﬁg
• Specify perl code to run during initialization:

plperl.on_init = '...perl...'
plperl.on_plperlu_init = '...perl...'
plperl.on_plperl_init = '...perl...'

• Can only be set by superuser or postgres.conf
• Code can't access the database

~ Timeline ~
▾ Perl interpreter created on demand
▿ Options from PERL5OPT env var are processed
▿ PL/Perl support code bootstraps
▿ plperl.on_init code runs
Above may happen in postmaster at startup if
plperl is loaded via shared_preload_libraries
▾ Interpreter is specialised for plperl (or plperlu)
▿ Modules loaded: strict, warnings, features, Carp
▿ Perl operators are restricted (require, open etc.)
▿ DynaLoader package is deleted
▿ plperl.on_plperl_init code runs
▿ Database access is enabled

plperl.on_init

• Handy to set global perl conﬁguation
plperl.on_init='use lib qw(/myapp); use ...;'
plperl.on_init='require "plperloninit.pl";'

• SECURITY RISK!
Only load modules you're happy for plperl code to use!
Also check any other modules loaded as dependencies!
Use Devel::TraceLoad to see what's actually loaded:
PERL5OPT='-MDevel::Trace=summary' pg_ctl ...

PL/Perl Best Practice

• Include explicit use statements in functions
• Don't assume modules have been pre-loaded
• For plperlu that'll actually load if needed
• For plperl it'll check that module was loaded
- so you'll get an immediate clear failure if not
- for example on a replica with old postgres.conf ﬁle

plperl.on_plperl_init

• Originally intended for things like
- PGOPTIONS="-c plperl.on_plperl_init='...'"

- to enable debug or proﬁling for a session
• But...
• Can only be set by superuser or postgres.conf
- sadly, due to SECURITY DEFINER risks.
- You shouldn't write SECURITY DEFINER
functions in plperl if untrusted users can use plperl!

Devel::NYTProf
Perl Source Code Proﬁler

PostgreSQL::PLPerl::NYTProf

Enabling NYTProf
• Via postgres.conf:
plperl.on_init='use PostgreSQL::PLPerl::NYTProf'

• Via environment variable:
PERL5OPT='-MPostgreSQL::PLPerl::NYTProf' pg_ctl ...

• Immediately active. To enable on demand:
NYTPROF=start=no PERL5OPT=... pg_ctl ...
DO 'DB::enable_profile' LANGUAGE plperl;

Reporting from NYTProf

• Writes per-backend data ﬁles:
$PGDATA/nytprof.out.$pid

• To generate a report:
nytprofhtml --file=$PGDATA/nytprof.out.4321 --open

~ Demo ~
Screencast: http://timbunce.blip.tv
Video: http://www.fosslc.org/drupal/content/plperl-new-features-90

Questions?

Tim.Bunce@pobox.com
http://blog.timbunce.org
@timbunce on twitter

http://xkcd.com/519/

PL/Perl - New Features in PostgreSQL 9.0

More Related Content

What's hot (20)

Similar to PL/Perl - New Features in PostgreSQL 9.0 (20)

More from Tim Bunce (12)

Recently uploaded (20)

PL/Perl - New Features in PostgreSQL 9.0