Pycon11: Python threads: Dive into GIL!
Download as PPTX, PDF3 likes2,575 views
This document summarizes a presentation about Python threads and the Global Interpreter Lock (GIL). It discusses how the GIL provides synchronization for Python's memory management but prevents true concurrency. It describes how the GIL is implemented and how it is managed through "checks" and tick counts. It also covers how the new GIL in Python 3.2 aims to improve fairness on multicore systems through a timeout mechanism. Finally, it discusses alternatives like Jython, multiprocessing, and Stackless Python that can better utilize multiple CPU cores.
![GIL battleThe ‘Priority inversion’In a [CPU, I/O]-bound mixed application, I/O bound thread may starve! “cache-hotness” may influence the new GIL owner; usually the recent owner!Preferring CPU thread over I/O threadPython presents a priority inversion on multi-core systems.](https://image.slidesharecdn.com/pycon11chetanvishal-110919031922-phpapp02/85/Pycon11-Python-threads-Dive-into-GIL-19-320.jpg)
Pycon11: Python threads: Dive into GIL!
- 1. Python threads: Dive into GIL!PyCon India 2011Pune, Sept 16-18Vishal Kanaujia & Chetan Giridhar
- 2. Python: A multithreading example
- 3. Setting up the context!!Hmm…My threads should be twice as faster on dual cores!57 % dip in Execution Time on dual core!!
- 4. Getting into the problem space!!Python v2.7Getting in to the Problem Space!!GIL
- 5. Python ThreadsReal system threads (POSIX/ Windows threads)Python VM has no intelligence of thread management No thread priorities, pre-emptionNative operative system supervises thread schedulingPython interpreter just does the per-thread bookkeeping.
- 6. What’s wrong with Py threads?Each ‘running’ thread requires exclusive access to data structures in Python interpreterGlobal interpreter lock (GIL) provides the synchronization (bookkeeping) GIL is necessary, mainly because CPython's memory management is not thread-safe.
- 7. GIL: Code DetailsA thread create request in Python is just a pthread_create() callThe function Py_Initialize() creates the GILGIL is simply a synchronization primitive, and can be implemented with a semaphore/ mutex.../Python/ceval.cstatic PyThread_type_lockinterpreter_lock = 0; /* This is the GIL */A “runnable” thread acquires this lock and start execution
- 8. GIL ManagementHow does Python manages GIL?Python interpreter regularly performs a check on the running threadAccomplishes thread switching and signal handlingWhat is a “check”?A counter of ticks; ticks decrement as a thread runs
- 9. A tick maps to a Python VM’s byte-code instructions
- 10. Check interval can be set with sys.setcheckinterval (interval)
- 11. Check dictate CPU time-slice available to a threadGIL Management: ImplementationInvolves two global varibales:PyAPI_DATA(volatile int) _Py_Ticker;PyAPI_DATA(int) _Py_CheckInterval;As soon as ticks reach zero:Refill the ticker
- 12. active thread releases the GIL
- 13. Signals sleeping threads to wake up
- 14. Everyone competes for GILFile: ../ceval.cPyEval_EvalFrameEx () { --_Py_Ticker; /* Decrement ticker */ /* Refill it */ _Py_Ticker = _Py_CheckInterval; if (interpreter_lock) {PyThread_release_lock(interpreter_lock); /* Other threads may run now */PyThread_acquire_lock(interpreter_lock, 1); } }
- 15. Two CPU bound threads on single core machineGIL releasedwaiting for GILthread1RunningSuspendedSignals thread2GIL acquiredwaiting for GILthread2SuspendedWakeupRunningtimecheck1check2
- 16. GIL impactThere is considerable time lag with Communication (Signaling)Thread wake-upGIL acquisitionGIL Management: Independent of host/native OS schedulingResultSignificant overheadThread waits if GIL in unavailableThreads run sequentially, rather than concurrentlyTry Ctrl + C. Does it stop execution?
- 17. Curious case of multicore systemthread1, core0threadthread2, core1time Conflicting goals of OS scheduler and Python interpreter
- 18. Host OS can schedule threads concurrently on multi-core
- 19. GIL battleThe ‘Priority inversion’In a [CPU, I/O]-bound mixed application, I/O bound thread may starve! “cache-hotness” may influence the new GIL owner; usually the recent owner!Preferring CPU thread over I/O threadPython presents a priority inversion on multi-core systems.
- 20. Understanding the new story!!Python v3.2Getting in to the Problem Space!!GIL
- 21. New GIL: Python v3.2Regular “check” are discontinuedWe have new time-out mechanism.Default time-out= 5msConfigurable through sys.setswitchinterval()For every time-out, the current GIL holder is forced to release GIL It then signals the other waiting threadsWaits for a signal from new GIL owner (acknowledgement).A sleeping thread wakes up, acquires the GIL, and signals the last owner.
- 22. Curious Case of Multicore: Python v3.2CPU Thread core0timeWaiting for the GILGIL releasedRunningWaitSuspendedlangissignalTime Outwaiting for GILGIL acquiredSuspendedWakeupRunningCPU Thread core1
- 23. Positive impact with new GILBetter GIL arbitrationEnsures that a thread runs only for 5msLess context switching and fewer signalsMulticore perspective: GIL battle eliminated!More responsive threads (fair scheduling)All iz well
- 24. I/O threads in PythonAn interesting optimization by interpreterI/O calls are assumed blockingPython I/O extensively exercise this optimization with file, socket ops (e.g. read, write, send, recv calls)./Python3.2.1/Include/ceval.hPy_BEGIN_ALLOW_THREADS Do some blocking I/O operation ...Py_END_ALLOW_THREADSI/O thread always releases the GIL
- 25. Convoy effect: Fallout of I/O optimizationWhen an I/O thread releases the GIL, another ‘runnable’ CPU bound thread can acquire it (remember we are on multiple cores).It leaves the I/O thread waiting for another time-out (default: 5ms)!Once CPU thread releases GIL, I/O thread acquires and releases it again This cycle goes on => performance suffers
- 26. Convoy “in” effect Convoy effect- observed in an application comprising I/O-bound and CPU-bound threadsGIL releasedGIL releasedI/O Thread core0GIL acquiredTime OutRunningWaitRunningWaitSuspendedwaiting for GILGIL acquiredSuspendedRunningWaitSuspendedCPU Thread core1time
- 27. Performance measurements!Curious to know how convoy effect translates into performance numbersWe performed following tests with Python3.2:An application with a CPU and a I/O threadExecuted on a dual core machineCPU thread spends less than few seconds (<10s)!
- 28. Comparing: Python 2.7 & Python 3.2Performance dip still observed in dual cores
- 29. Getting into the problem space!!Python v2.7 / v3.2Getting in to the Problem Space!!GILSolution space!!
- 30. GIL free world: JythonJython is free of GIL It can fully exploit multiple cores, as per our experimentsExperiments with Jython2.5Run with two CPU threads in tandemExperiment shows performance improvement on a multi-core system
- 31. Avoiding GIL impact with multiprocessingmultiprocessing — Process-based “threading” interface“multiprocessing” module spawns a new Python interpreter instance for a process.Each process is independent, and GIL is irrelevant; Utilizes multiple cores better than threads.Shares API with “threading” module.Cool! 40 % improvement in Execution Time on dual core!!
- 32. ConclusionMulti-core systems are becoming ubiquitousPython applications should exploit this abundant powerCPython inherently suffers the GIL limitationAn intelligent awareness of Python interpreter behavior is helpful in developing multi-threaded applicationsUnderstand and use
- 33. QuestionsThank you for your time and attention Please share your feedback/ comments/ suggestions to us at:[email protected] , http://[email protected], http://freethreads.wordpress.com
- 34. ReferencesUnderstanding the Python GIL, http://dabeaz.com/talks.htmlGlobalInterpreterLock, http://wiki.python.org/moin/GlobalInterpreterLockThread State and the Global Interpreter Lock, http://docs.python.org/c-api/init.html#threadsPython v3.2.2 and v2.7.2 documentation, http://docs.python.org/Concurrency and Python, http://drdobbs.com/open-source/206103078?pgno=3
- 35. Backup slides
- 36. Python: GILA thread needs GIL before updating Python objects, calling C/Python API functionsConcurrency is emulated with regular ‘checks’ to switch threadsApplicable to only CPU bound threadA blocking I/O operation implies relinquishing the GIL./Python2.7.5/Include/ceval.hPy_BEGIN_ALLOW_THREADS Do some blocking I/O operation ...Py_END_ALLOW_THREADSPython file I/O extensively exercise this optimization
- 37. GIL: InternalsThe function Py_Initialize() creates the GILA thread create request in Python is just a pthread_create() call../Python/ceval.cstatic PyThread_type_lock interpreter_lock = 0; /* This is the GIL */o) thread_PyThread_start_new_thread: we call it for "each" user defined thread. calls PyEval_InitThreads() -> PyThread_acquire_lock() {}
- 38. GIL: in actionEach CPU bound thread requires GIL
- 39. ‘ticks count’ determine duration of GIL hold
- 41. We keep a list of Python threads and each thread-state has its tick_counter value
- 42. As soon as tick decrements to zero, the thread release the GIL.GIL: Detailsthread_PyThread_start_new_thread() ->void PyEval_InitThreads(void){ if (interpreter_lock) return; interpreter_lock = PyThread_allocate_lock(); PyThread_acquire_lock(interpreter_lock, 1); main_thread = PyThread_get_thread_ident();}
- 43. Convoy effect: Python v2?Convoy effect holds true for Python v2 alsoThe smaller interval of ‘check’ saves the day!I/O threads don’t have to wait for a longer time (5 m) for CPU threads to finishShould choose the setswitchinterval() wiselyThe effect is not so visible in Python v2.0
- 44. Stackless PythonA different way of creating threads: Microthreads!No improvement from multi-core perspectiveRound-robin scheduling for “tasklets”Sequential execution
Editor's Notes
- #16: Okokokokkokokokok