Cython compiler
- 1. CYTHON COMPILER BY : TANIKELLA SAI ABHIJYAN
- 2. INDEX PART A: CYTHON COMPILER 1.INTRODUCTION AND HISTORY OF CYTHON COMPILER 1.1. HISTORY OF CYTHON COMPILER 1.2. INTRODUCTION TO CYTHON COMPILER 2.USES OF CYTHON COMPILER 2.1. EXAMPLE OF CYTHON 3.ADVANTAGES AND LIMITATIONS OF CYTHON 3.1. ADVANTAGES OF CYTHON 3.2. LIMITATIONS OF CYTHON 4.CYTHON PROFILING AND PERFORMANCE
- 3. 1.INTRODUCTION AND HISTORY OF PYTHON COMPILER: 1.1 HISTORY OF CYTHON COMPILER: Cython is a derivative of the Pyrex language, and supports more features and optimizations than Pyrex. Cython was forked from Pyrex in 2007 by developers of the Sage computer algebra package, because they were unhappy with Pyrex's limitations and could not get patches accepted by Pyrex's maintainer Greg Ewing, who envisioned a much smaller scope for his tool than the Sage developers had in mind. They then forked Pyrex as SageX. When they found people were downloading Sage just to get SageX, and developers of other packages (including Stefan Behnel, who maintains the XML library LXML) were also maintaining forks of Pyrex, SageX was split off the Sage project and merged with cython-lxml to become Cython. Cython files have a .pyx extension. At its most basic, Cython code looks exactly like Python code. However, whereas standard Python is dynamically typed, in Cython, types can optionally be provided, allowing for improved performance, allowing loops to be converted into C loops where possible. 1.2 INTRODUCTION TO CYTHON COMPILER: Cython works by producing a standard Python module. However, the behavior differs from standard Python in that the module code, originally written in Python, is translated into C. While the resulting code is fast, it makes many calls into the CPython interpreter and CPython standard libraries to perform actual work. Choosing this arrangement saved considerably on Cython's development time, but modules have a dependency on the Python interpreter and standard library. Although most of the code is C-based, a small stub loader written in interpreted Python is usually required (unless the goal is to create a loader written entirely in C, which may involve work with the undocumented internals of CPython). However, this is not a major problem due to the presence of the Python interpreter. Cython has a foreign function interface for invoking C/C++ routines and the ability to declare the static type of subroutine parameters and results, local variables, and class attributes. A Cython program that implements the same algorithm as a corresponding Python program may consume fewer computing resources such as core memory and processing cycles due to differences between the CPython and Cython execution models. A basic Python program is loaded and executed by the CPython virtual machine, so both the runtime and the program itself consume computing resources. A Cython program is compiled to C code, which is further compiled to machine code, so the virtual machine is used only briefly when the program is loaded. Cython employs: • Optimistic optimizations • Type inference (optional) • Low overhead in control structures • Low function call overhead Performance depends both on what C code is generated by Cython and how that code is compiled by the C compiler. FUN FACTS ABOUT CYTHON: Despite being a superset of Python, Cython is much faster than Python. It improves Python code execution speed significantly by compiling Python code into C code.
- 4. 2.USES OF CYTHON: Cython is a programming language that aims to be a superset of the Python programming language, designed to give C-like performance with code that is written mostly in Python with optional additional C- inspired syntax. Cython is a compiled language that is typically used to generate CPython extension modules. Cython is particularly popular among scientific users of Python, where it has "the perfect audience" according to Python creator Guido van Rossum. Of particular note: • The free software SageMath computer algebra system depends on Cython, both for performance and to interface with other libraries. • Significant parts of the scientific computing libraries SciPy, pandas and scikit-learn are written in Cython. • Some high-traffic websites such as Quora use Cython. Cython's domain is not limited to just numerical computing. For example, the lxml XML toolkit is written mostly in Cython, and like its predecessor Pyrex, Cython is used to provide Python bindings for many C and C++ libraries such as the messaging library ZeroMQ. Cython can also be used to develop parallel programs for multi-core processor machines; this feature makes use of the OpenMP library. 2.1 EXAMPLE OF CYTHON: A more straightforward way to start with Cython is through command-line IPython (or through in-browser python console called Jupyter notebook). which gives a 95 times improvement over the pure-python version. FUN FACTS ABOUT CYTHON: You can use NumPy from Cython exactly the same as in regular Python, but by doing so you are losing potentially high speedups because Cython has support for fast access to NumPy arrays.
- 5. 3.ADVANTAGES AND LIMITATIONS OF CYTHON COMPILER: 3.1 ADVANTAGES OF CYTHON COMPILER: 1.Working with external C libraries can be faster Python packages like NumPy wrap C libraries in Python interfaces to make them easy to work with. However, going back and forth between Python and C through those wrappers can slow things down. Cython lets you talk to the underlying libraries directly, without Python in the way. (C++ libraries are also supported.) 2.You can use both C and Python memory management If you use Python objects, they’re memory-managed and garbage-collected the same as in regular Python. But if you want to create and manage your own C-level structures, and use malloc/free to work with them, you can do so. Just remember to clean up after yourself. 3.You can opt for safety or speed as needed Cython automatically performs runtime checks for common problems that pop up in C, such as out-of-bounds access on an array, by way of decorators and compiler directives (e.g., @boundscheck(False)). 4.Cython can use Python type hinting syntax Python has a type-hinting syntax that is used mainly by linters and code checkers, rather than the CPython interpreter. Cython has its own custom syntax for code decorations, but with recent revisions of Cython you can use Python type-hinting syntax to provide basic type hints to Cython as well. 5.Cython can be used to obscure sensitive Python code Python modules are trivially easy to decompile and inspect, but compiled binaries are not. When distributing a Python application to end users, if you want to protect some of its modules from casual snooping, you can do so by compiling them with Cython. Note, though, this is a side effect of Cython’s capabilities, not one of its intended functions. 6. Cython C code can benefit from releasing the GIL Python’s Global Interpreter Lock, or GIL, synchronizes threads within the interpreter, protecting access to Python objects and managing contention for resources. But the GIL has been widely criticized as a stumbling block to a better-performing Python, especially on multicore systems.
- 6. 3.2 LIMITATION OF CYTHON COMPILER: 1. Little speedup for conventional Python code When Cython encounters Python code it can’t translate completely into C, it transforms that code into a series of C calls to Python’s internals. This amounts to taking Python’s interpreter out of the execution loop, which gives code a modest 15 to 20 percent speedup by default. Note that this is a best-case scenario; in some situations, you might see no performance improvement, or even a performance degradation. 2.Little speedup for native Python data structures Python provides a slew of data structures—strings, lists, tuples, dictionaries, and so on. They’re hugely convenient for developers, and they come with their own automatic memory management. But they’re slower than pure C. 3. Cython code runs fastest when “pure C” If you have a function in C labeled with the cdef keyword, with all of its variables and inline function calls to other things that are pure C, it will run as fast as C can go. But if that function references any Python- native code, like a Python data structure or a call to an internal Python API, that call will be a performance bottleneck. Fortunately, Cython provides a way to spot these bottlenecks: a source code report that shows at a glance which parts of your Cython app are pure C and which parts interact with Python. The better optimized the app, the less interaction there will be with Python. 4. Cython profiling and performance: You get the best performance from any piece of code by profiling it and seeing firsthand where the bottlenecks are. Cython provides hooks for Python’s cProfile module, so you can use Python’s own profiling tools, like cProfile, to see how your Cython code performs. It helps to remember in all cases that Cython isn’t magic—that sensible real-world performance practices still apply. The less you shuttle back and forth between Python and Cython, the faster your app will run. For instance, if you have a collection of objects you want to process in Cython, don’t iterate over it in Python and invoke a Cython function at each step. Pass the entire collection to your Cython module and iterate there. This technique is used often in libraries that manage data, so it’s a good model to emulate in your own code. We use Python because it provides programmer convenience and enables fast development. Sometimes that programmer productivity comes at the cost of performance. With Cython, just a little extra effort can give you the best of both worlds.