YAPC::EU 2015 - Perl Conferences
0 likes•334 views
The document discusses various tools and concepts related to web scraping, including applications to extract content from webpages and differences between tools like WWW::Mechanize and Web::Query. It also covers platform incompatibilities, the status of the Perl compiler, and various Perl modules for performance and monitoring. Additional topics include server management, static analysis of Perl code, and community recommendations for optimizing Perl applications.
![Compiler status
perl compiler B::C - 5.14.4 in production since 2014
@cPanel [ CentOS 5,6,7 ]
Advantages:
• 10 to 20 % less memory usage
• faster startup
• no need to ship your modules / code
( dynamic modules are loaded on demand )](https://image.slidesharecdn.com/yapc-eu-2015-151014150744-lva1-app6892/85/YAPC-EU-2015-Perl-Conferences-20-320.jpg)
![Furl
simple & fast http client [made in Japan]](https://image.slidesharecdn.com/yapc-eu-2015-151014150744-lva1-app6892/85/YAPC-EU-2015-Perl-Conferences-27-320.jpg)
![Riji - git based blog tool
publish [static] files](https://image.slidesharecdn.com/yapc-eu-2015-151014150744-lva1-app6892/85/YAPC-EU-2015-Perl-Conferences-29-320.jpg)
YAPC::EU 2015 - Perl Conferences
- 2. From: YAPC::[email protected] 1. Web Scrapers 2. Platform incompatibilites 3. Compiler status 4. WebService::Simple 5. Server::Starter 6. Starlet 7. Cookie::Baker 8. Gazelle 9. Furl 10. Riki 11. Perl::Lint 12. Carmel
- 3. Granada ?
- 7. Web Scrapers • What is it ? application to extract content from a webpage…. 1. Crawl 2. Parse 3. Extract content • But Why ? 1. Website without API… 2. Website with an API… Html code can change… but api also…
- 8. Web Scrapers Scrapers for dummies
- 9. Web Scrapers Two tools for scrapping: • WWW::Mechanize browser in an object forms, links, “back button”, “reload button” • Web::Query selector based
- 11. Web Scrapers: Web::Query by Tokuhiro Matsuno
- 12. Platform incompatibilities Talk by Mark Overmeer author of POSIX::1003 • POSIX 1003.1: attempt to standardize the operating systems since 1984: libraries: libc, libc, … OS: I/O, events, permissions, memory,… • over 1200 functions
- 13. POSIX is difficult > perldoc -f print
- 14. POSIX.pm is old > perldoc POSIX
- 15. POSIX is weak Sample with select > perldoc -f select select FILEHANDLE select RBITS, WBITS, EBITS, TIMEOUT > perldoc POSIX::1003::Events
- 16. POSIX: other issues • getpid => $$ # 12 simple rewrites • POSIX::open != open| # 57 name clashes • 20 croak “use method xxx instead" • 59 croak “xxx is C-specific use yyy"
- 17. POSIX::1003
- 18. POSIX::1003 Exporter trick: +1 Sample …what’s the trick ?
- 19. POSIX::1003
- 20. Compiler status perl compiler B::C - 5.14.4 in production since 2014 @cPanel [ CentOS 5,6,7 ] Advantages: • 10 to 20 % less memory usage • faster startup • no need to ship your modules / code ( dynamic modules are loaded on demand )
- 21. Japanese “Why does it need to be so fast ? because we are…” Junichi Ishida aka uzulla slides: http://bit.ly/uzulla_yapceu2015
- 23. Server::Starter • extra protection layer on top of your plack server • "a superdaemon for hot-deploying server programs" • Gracefull restart (hot-deploying) • Only exit old process if new process is successfully booted.(safe!) • No resource leak (known)
- 24. Starlet • "a simple, high-performance PSGI/Plack HTTP server" • Very heavily used in japan.
- 25. Cookie::Baker
- 26. Gazelle a very fast{/jp} PSGI server x2 faster than starman
- 27. Furl simple & fast http client [made in Japan]
- 28. Furl - post sample
- 29. Riji - git based blog tool publish [static] files
- 30. Perl::Lint by Kawakami Why ? • "Yet Another Perl Source Code Linter" • faster than other lint tools • fast and flexible static analyzer for Perl5 • compatibility with Perl::Critic • easy to extend/add policies
- 32. Carmel (or Carton 2) Talk from Tatsuhiko Miyazawa (>200 CPAN modules): • cpanminus • carton • Corona • PSGI • Plack • Starman • Twiggy • …
- 33. Carmel (or Carton 2) Carton with cache to speedup cache blib in a common directory $HOME/.carmel/{version}-{archname}/builds
- 34. Carmel (or Carton 2)
- 43. …and also • Benchmark::Perl::Formance • Eixo::Zone: interact with (OS) namespaces • JWT tokens - http://jwt.io • JSON::WebToken • Crypt::JWT • Monitoring graphs • GrowthForecast (needs rrdtool) • HRForecast (no rrdtool required • Test::mysqld • Test::Valgrind valgrind frontend: memory errors & leaks • mRuby is mruby binding for perl5 • Measure::Everything provides a standard measuring API for modules • Menlo: cpanminus 2 • WebService::Mackerel - client for mackerel.io (performance monitor tool on cloud)
- 44. Thank you !