Python on FreeBSD
10 likes15,785 views
This document discusses Python and FreeBSD. It begins with an introduction to FreeBSD, highlighting some of the companies and websites that use it. It then discusses the python@FreeBSD mailing list and resources for using Python on FreeBSD, including installing Python via ports or packages. The document provides guidance on running Python applications and developing Python projects on FreeBSD using tools like virtualenv, ZFS, and jails. It also covers creating Python ports and submitting them to be included in the FreeBSD ports collection.
![Install Python on FreeBSD
• Via package system (advanced)
o setenv PACKAGEROOT http://ftp.tw.freebsd.org
o setenv PACKAGESITE
http://ftp.tw.freebsd.org/pub/FreeBSD/ports/`un
ame -p`/packages-`uname -r | cut -d - -f 1,2 | tr
"[:upper:]" "[:lower:]"`/Latest/
o pkg_add -r python](https://image.slidesharecdn.com/pythononfreebsd-120617091223-phpapp02/85/Python-on-FreeBSD-24-320.jpg)
![pkg-plist
bin/coverage
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/PKG-INFO
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/SOURCES.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/dependency_links.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/entry_points.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/native_libs.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/not-zip-safe
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/top_level.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.py
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyc
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyo
[...]
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.py
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyc
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyo
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/htmlfiles
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%](https://image.slidesharecdn.com/pythononfreebsd-120617091223-phpapp02/85/Python-on-FreeBSD-48-320.jpg)
Python on FreeBSD
- 1. Python on FreeBSD Li-Wen Hsu <lwhsu@{lwhsu,FreeBSD}.org> PyCon Taiwan 2012
- 2. Who is this ↘
- 3. lwhsu: 2006 開始接觸 Python 2007 得到 FreeBSD ports commit bit 2008 Sysadm & Instructor at CS.NCTU 2009 國軍 online 2010~ 菜鳥碼農+修電腦的 另外也是管 {ftp,www,cvsup,anoncvs, portsnap,update}.tw.freebsd.org 的
- 4. </開場白>
- 5. Python on FreeBSD
- 7. The FreeBSD Project • FreeBSD is an advanced operating system derived from BSD, the version of UNIX® developed at the University of California, Berkeley. o http://www.freebsd.org/about.html
- 8. Who’s using FreeBSD? • FreeBSD is used in the devices from the world’s big companies: o Apple o Blue Coat o Cisco o F5 Networks o Juniper o NetApp o Sony’s PlayStation 3
- 9. Who’s also using FreeBSD? • FreeBSD is also used to power some of the busiest sites on Internet: o Apache o Netflix o Pixnet o WhatsApp o Yahoo! o Yandex
- 10. More? • Projects based on FreeBSD o PC-BSD o m0n0wall o psSense o FreeNAS o Debian GNU/kFreeBSD o Gentoo/FreeBSD
- 11. FreeBSD: The Good Parts • BSD License • TrustedBSD MAC • GEOM Framework • Netgraph • TrustedBSD Audit • Jail • LLVM & Clang • ZFS • Ports system • DTrace • And more… http://www.freebsd.org/features.html
- 13. <FreeBSD+Python>
- 14. Origin • /usr/ports/lang/python/Makefile: # New ports collection makefile for: python # Date created: 08 August 1995 # Whom: jkh "Director of Engineering of Unix Technologies" at Apple
- 15. [email protected] • Mailing list [email protected] o FreeBSD-specific Python issues o Founded in January 2004 o http://lists.freebsd.org/mailman/listinfo/freebsd-python o Maintaining python and python related ports, except py-qt and py-gtk • [email protected] • [email protected]
- 16. [email protected] • Wiki o http://wiki.freebsd.org/Python o Who has the “Hat” o Action items o Contact information • IRC o #freebsd-python on FreeNode
- 17. Friends of [email protected] • [email protected] o Zope for FreeBSD -- porting and maintaining o http://lists.freebsd.org/mailman/listinfo/freebsd-zope o Founded in April 2012 • M_FBSD.python at bbs.cs.nctu.edu.tw
- 18. [email protected] • People o alexbl (alumni) o perky (alumni) o clsung (Taiwan) o rm o lwhsu (Taiwan) o sunpoet (Taiwan) o miwi o wen o nivit o yzlin (Taiwan)
- 19. Install Python on FreeBSD • Via ports system o cd /usr/ports/lang/python o make install clean
- 20. Install Python on FreeBSD • Ports options
- 23. Install Python on FreeBSD • Via package system o pkg_add -r python
- 24. Install Python on FreeBSD • Via package system (advanced) o setenv PACKAGEROOT http://ftp.tw.freebsd.org o setenv PACKAGESITE http://ftp.tw.freebsd.org/pub/FreeBSD/ports/`un ame -p`/packages-`uname -r | cut -d - -f 1,2 | tr "[:upper:]" "[:lower:]"`/Latest/ o pkg_add -r python
- 25. Install Python on FreeBSD • In the (on so far) future: • Pkgng o pkg install python
- 26. Wait, I have a question! • Q: Why compile Python myself? • A: o Optimize for your system o Customize build options o Easier to hack into Python in the system
- 27. Wait, I have a question! • Q: Do I really need to compile myself on *every* machine I want to deploy? • A: Build once, install everywhere. o cd /usr/ports/lang/python o make package • /usr/ports/packages/lang/python-2.7,2.tbz o install • pkg_add python-2.7,2.tbz
- 28. Run Python on FreeBSD • In /usr/ports • Also in /usr/ports o lang/python o lang/jython • (meta package) o lang/pypy o lang/python24 o lang/ironpython o lang/python25 o lang/python26 o devel/ipython o lang/python27 o lang/python31 o lang/python32
- 29. Run many Python versions on FreeBSD • /usr/local/bin/python o A symbolic link to default version o PYTHON_DEFAULT_VERSION • Can be manually defined in /etc/make.conf • Possible values: python{2.4,2.5,2.6,2.7,3.1,3.2} • You can install as many different of Python versions you want o just install via /usr/ports/lang/python* o First installed Python version will be the default
- 30. Run many Python versions on FreeBSD • Install modules for different python versions o cd /usr/ports/foo/py-bar o make PYTHON_VERSION=python2.6 install clean o This is still work in progress, there might be some minor errors. • For modules install scripts in bin/
- 31. Wait, I have a question! • Q: there is XXX, why do I need ports to install python modules? o (XXX=distutils, setuptools, distribute, distutils2, pip) • A: o For better integration with system o Do more complex deploy o Ex: install web applications like reviewboard, trac • A system only consists with python and python module • Also apache/nginx, mod_wsgi, etc.
- 32. Develop with Python on FreeBSD • virtualenv + Jenkins o devel/py-virtualenv o devel/jenkins • Use virtualenv in jenkins, make sure clean dependency. • Have a job to build a virtualenv that is usable by other jobs.
- 33. Develop with Python on FreeBSD • ZFS + jail • Jail o Light weight virtual machine o /etc/rc.conf • jail_enable="YES” • ZFS o zpool create tank ad1 o zfs create tank/jail1
- 34. Develop with Python on FreeBSD • Create a new jail o bsdinstall jail /tank/jail1 o freebsd-update -b /tank/jail1 fetch install o Modify /etc/rc.conf • jail_jail1_rootdir="/tank/jail1” • jail_jail1_hostname="jail1” • jail_jail1_ip="192.168.xxx.xxx” • jail_jail1_interface="lo0" o /etc/rc.d/jail start jail1
- 35. Develop with Python on FreeBSD • Clone a jail o zfs snapshot tank/jail1@milestone1 o zfs clone tank/jail1@milestone1 tank/jail2 o Modify /etc/rc.conf o /etc/rc.d/jail start jail2 o Another development environment is created!
- 36. Run your Python Apps on FreeBSD • Nothing special or different with other Unix-like platform
- 37. Run your Python Web Apps • GIL problem <VirtualHost *:80> DocumentRoot "/home/www/webroot" ServerName python.example.com WSGIProcessGroup pythonexample WSGIDaemonProcess pythonexample processes=16 threads=128 maximum-requests=1024 display-name=%{GROUP} </VirtualHost>
- 39. FreeBSD Ports • Each ``port’’ contains any patches necessary to make the original application source code compile and run on FreeBSD. Installing an application is as simple as typing make in the port directory. o http://www.freebsd.org/ports/
- 40. Inside a port • ls -1 /usr/ports/devel/py-coverage Makefile distinfo pkg-descr pkg-plist
- 41. Create a new Python port • Use porttools to help: o cd /usr/ports/ports-mgmt/porttools o make install clean • Create a new port o cd /your/workspace o port create py-newport
- 42. Makefile # New ports collection makefile for: py-coverage # Date created: Jun. 22, 2009 # Whom: Li-Wen Hsu <[email protected]> # # $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $ # PORTNAME= coverage PORTVERSION= 3.5.2 CATEGORIES= devel python MASTER_SITES= CHEESESHOP PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} MAINTAINER= [email protected] COMMENT= Code coverage measurement for Python LICENSE= BSD USE_PYTHON= yes USE_PYDISTUTILS= easy_install PYEASYINSTALL_ARCHDEP= yes .include <bsd.port.mk>
- 43. Makefile # New ports collection makefile for: py-coverage # Date created: Jun. 22, 2009 # Whom: Li-Wen Hsu <[email protected]> # # $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $ #
- 44. Makefile PORTNAME= coverage PORTVERSION= 3.5.2 CATEGORIES= devel python MASTER_SITES= CHEESESHOP PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
- 45. Makefile LICENSE= BSD USE_PYTHON= yes # 2.6 2.6+ -2.7 3.1+ USE_PYDISTUTILS= easy_install PYEASYINSTALL_ARCHDEP= yes .include <bsd.port.mk>
- 46. distinfo SHA256 (coverage-3.5.2.tar.gz) = b9e0dc65e42236a3c9b8978701bc9e7298ed7e24d39f99 ea2ba4b945d537fe55 SIZE (coverage-3.5.2.tar.gz) = 115497
- 47. pkg-descr Coverage measures code coverage, typically during test execution. It uses the code analysis tools and tracing hooks provided in the Python standard library to determine which lines are executable, and which have been executed. WWW: http://nedbatchelder.com/code/coverage
- 49. Before submitting • Check o Use portlint (ports-mgmt/portlint) o Command:portint –abct • Test o Use porttools (ports-mgmt/porttools) o Command: port test
- 50. Before submitting • More test o Use Tinderbox (ports-mgmt/tinderbox) o A comprehensive way to test ports • And builds packages!
- 52. Wait, I have a question! • Q: Setting up a tinderbox is too complex • A: Use RedPorts http://redports.org/
- 53. Submit! • In your port directory • Command: o port submit • Quit editor: o s)end, e)dit or a)bort? s
- 54. After submitting • A PR (problem report) is created • A committer will handle it • A new port is in the ports tree! o See on http://freshports.org/foo/bar • To update your ports tree: o portsnap fetch update • Install the new port: o cd /usr/ports/foo/bar; make install clean
- 55. Future Work of [email protected] • Remove 2.4 and 2.5 in the ports tree o 2012-06-30 • Handle packages with pip/distribute • Debug versions of lang/python* • Install only unzipped eggs • Truly modules for multiple python versions co-exist
- 56. Q&A