![C implementations cause bad APIsC implementations cause bad APIs
In [ ]: import numpy
class myarray(numpy.ndarray):
pass
In [ ]: import pandas
help(pandas.read_csv)](https://image.slidesharecdn.com/pypymeetsscipyslides-191028180914/85/Pythonic-doesn-t-mean-slow-11-320.jpg)
![PyPy - DemoPyPy - Demo
In [2]: %%time
class Quantity:
def __init__(self, value, unit):
self.value = value
self.unit = unit
def __add__(self, other):
if isinstance(other, Quantity):
if other.unit != self.unit: raise ValueError("units must match")
else: return Quantity(self.value + other.value, self.unit)
else: return NotImplemented
def __str__(self):
return f"{self.value} {self.unit}"
def compute(n):
total = Quantity(0, 'm')
increment = Quantity(1., 'm')
for i in range(n):
total += increment
return total
N = 1_000_000_000
print(compute(N))
3.6.9 (default, Jul 3 2019, 15:36:16)
[GCC 5.4.0 20160609]
1000000000.0 m
CPU times: user 29min 4s, sys: 112 ms, total: 29min 4s
Wall time: 29min 4s](https://image.slidesharecdn.com/pypymeetsscipyslides-191028180914/85/Pythonic-doesn-t-mean-slow-14-320.jpg)
Pythonic doesn't mean slow!
- 1. Pythonic doesn't mean slow! Ronan Lamy
- 2. About meAbout me PyPy core dev Python consultant and freelance developer Contact: [email protected] @ronanlamy
- 3. We love PythonWe love Python
- 4. Why Python?Why Python? Easy to learn Easy to read Simplicity Community Many libraries
- 5. Technical reasonsTechnical reasons "Readability counts" Uniform object model Simple & expressive APIs Powerful abstractions Modularity/extensibility
- 6. What about performance?What about performance? Who cares about performance?Who cares about performance?
- 7. Workaround #1: Using high-performance librariesWorkaround #1: Using high-performance libraries
- 8. GoodGood Feels like Python Also includes domain knowledge Actually the default option BadBad Somewhat slow Limited customisation options Can only use what's implemented
- 9. Workaround #2: Rewrite in CWorkaround #2: Rewrite in C Cython C + cf Write a C extension 😱 Numba ...
- 10. GoodGood Fast Developer has full control BadBad Different, less convenient, language Unoptimised parts can become signi cant again Python/C boundary is a bottleneck
- 11. C implementations cause bad APIsC implementations cause bad APIs In [ ]: import numpy class myarray(numpy.ndarray): pass In [ ]: import pandas help(pandas.read_csv)
- 12. Fast vs PythonicFast vs Pythonic
- 13. PyPyPyPy Fast and compliant implementation of Python PyPy v7.2 released on 14 October Supports 2.7 and 3.6 JIT compiler GC tuned for Python
- 14. PyPy - DemoPyPy - Demo In [2]: %%time class Quantity: def __init__(self, value, unit): self.value = value self.unit = unit def __add__(self, other): if isinstance(other, Quantity): if other.unit != self.unit: raise ValueError("units must match") else: return Quantity(self.value + other.value, self.unit) else: return NotImplemented def __str__(self): return f"{self.value} {self.unit}" def compute(n): total = Quantity(0, 'm') increment = Quantity(1., 'm') for i in range(n): total += increment return total N = 1_000_000_000 print(compute(N)) 3.6.9 (default, Jul 3 2019, 15:36:16) [GCC 5.4.0 20160609] 1000000000.0 m CPU times: user 29min 4s, sys: 112 ms, total: 29min 4s Wall time: 29min 4s
- 15. How fast is PyPy?How fast is PyPy? It dependsIt depends Up to 1000x faster than CPython in extreme cases Up to 50x in more realistic cases 2-5x on "typical" applications Can be slower (short scripts, tests, ...)
- 16. C extension supportC extension support cpyext = "Python.h for PyPy" numpy, scipy, pandas, scikit-learn, lxml, ... Cython + most extensions written in Cython 'pip install' works Wheels available at https://github.com/antocuni/pypy-wheels
- 17. PyPy qualitiesPyPy qualities Make Pythonic code fast Abstractions for free Make Python a 1st-class language for HPC
- 18. PyPy issuesPyPy issues Lags behind CPython Implementation details are different Need to understand GC and JIT to get best performance No conda Small community
- 19. cpyext issuescpyext issues CPython API is huge Grey areas Patchy library support High overhead
- 20. HPyHPy New C API No refcounting Replace PyObject* with opaque handles Don't expose implementation details https://github.com/pyhandle/hpy (https://github.com/pyhandle/hpy)
- 21. HPy: plansHPy: plans In one yearIn one year CPython: implemented on top of existing API PyPy: much more ef cient than cpyext Used by Cython Long-termLong-term Replaces current C-API Used by all relevant implementations CPython free to experiment and modify internals
- 22. SummarySummary PyPy makes pure-Python fast Compatible with PyData ecosystem Computing in Python becomes viable Complements traditional approaches to performance
- 23. ContactContact IRC: #pypy on Freenode IRC pypy-dev @ python.org Questions? http://pypy.org (http://pypy.org)
- 24. The endThe end