![numerical algorithms
data analysis
math building blocks
data structures
utils
Linear algebra
(dense & sparse)
Fourier transforms
Special functions
Av = λv
|A| =
∑
j
(−1)i+j
aijMij
y[k] =
N−1
∑
n=0
e−2πj kn
N x[n]](https://image.slidesharecdn.com/scipy2018keynoteralfgommers-180716025504/85/SciPy-1-0-and-Beyond-a-Story-of-Community-and-Code-4-320.jpg)
![User-friendly interfaces
New Zealand forest bio-security
sampling locations 2017
from scipy.interpolate import griddata
row_idx = griddata(weather[['X', 'Y']].values,
weather.index.values,
latlon, method='nearest')
stations = weather.iloc[row_idx, :]
Mapping to available
weather data](https://image.slidesharecdn.com/scipy2018keynoteralfgommers-180716025504/85/SciPy-1-0-and-Beyond-a-Story-of-Community-and-Code-27-320.jpg)
![from scipy.integrate import solve_ivp
def exponential_decay(t, y):
return -0.5 * y
def solution(t, y0):
return y0 * np.exp(-0.5*t)
TMAX = 10
y0 = 8
t_eval = np.linspace(0, TMAX)
analytical_solution = solution(t_eval, y0)
for method in ('RK45', 'RK23', 'Radau', 'BDF', 'LSODA'):
sol = solve_ivp(exponential_decay, t_span=[0, TMAX], y0=[y0],
t_eval=t_eval, method=method)
ax.plot(sol.t, analytical_solution - sol.y[0, :], label=method)
User-friendly interfaces
New interface for solving initial value problems](https://image.slidesharecdn.com/scipy2018keynoteralfgommers-180716025504/85/SciPy-1-0-and-Beyond-a-Story-of-Community-and-Code-28-320.jpg)
![LowLevelCallable
footprint = np.array([[0, 1, 0],
[1, 1, 1],
[0, 1, 0]], dtype=bool)
image = misc.ascent()
@llc.jit_filter_function
def fmin(values):
"""
Return minimum of values; values are pixel values in footprint pattern.
Accelerated with Numba, via `scipy.LowLevelCallable`.
"""
result = np.inf
for v in values:
if v < result:
result = v
return result
>>> %timeit ndimage.generic_filter(image, np.min, footprint=footprint)
427 ms ± 2.15 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)
>>> %timeit ndimage.generic_filter(image, fmin, footprint=footprint)
2.81 ms ± 40.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
Credits: @jni](https://image.slidesharecdn.com/scipy2018keynoteralfgommers-180716025504/85/SciPy-1-0-and-Beyond-a-Story-of-Community-and-Code-31-320.jpg)
SciPy 1.0 and Beyond - a Story of Community and Code
- 1. SciPy 1.0 and beyond A story of community and code
- 2. –Travis Oliphant SciPy was a distribution masquerading as a library
- 3. SciPy — fundamental tools for scientific computing
- 4. numerical algorithms data analysis math building blocks data structures utils Linear algebra (dense & sparse) Fourier transforms Special functions Av = λv |A| = ∑ j (−1)i+j aijMij y[k] = N−1 ∑ n=0 e−2πj kn N x[n]
- 5. Θ(t) dt = ω(t) ω(t) dt = − bω(t) − c sin(Θ(t)) numerical algorithms data analysis math building blocks data structures utils Optimization Integration Interpolation
- 6. numerical algorithms data analysis math building blocks data structures utils Clustering Graph algorithms Computational geometry
- 7. numerical algorithms data analysis math building blocks data structures utils Sparse matrices k-D Tree
- 8. numerical algorithms data analysis math building blocks data structures utils Statistics Signal processing Image processing
- 9. numerical algorithms data analysis math building blocks data structures utils Scientific data formats Physical constants • NetCDF • Matlab • IDL • Matrix Market • Fortran • Wav • Harwell-Boeing • ARFF π ℏ ∘ C −∘ F parsec angstrom units …
- 10. A short history • 2001: the first SciPy release • 2005: transition to NumPy • 2007: creation of SciKits • 2008: the Documentation Marathon • 2010: moving to a 6-monthly release cycle • 2011: development moves to GitHub • 2011: Python 3 support • 2013: continuous integration with TravisCI • 2017: SciPy 1.0 release
- 11. Why 1.0 now?
- 12. setup.py CLASSIFIERS = """ Development Status :: 5 - Production/Stable
- 13. We have a governance structure
- 14. We have a Code of Conduct
- 15. We have a roadmap
- 16. Windows wheels, at last
- 17. Community
- 18. 100 — 120 contributors per release * M.J. Nichol * Juan Nunez-Iglesias * Arno Onken + * Nick Papior + * Dima Pasechnik + * Ashwin Pathak + * Oleksandr Pavlyk + * Stefan Peterson * Ilhan Polat * Andrey Portnoy + * Ravi Kumar Prasad + * Aman Pratik * Eric Quintero * Vedant Rathore + * Tyler Reddy * Joscha Reimer * Philipp Rentzsch + * Antonio Horta Ribeiro * Ned Richards + * Kevin Rose + * Benoit Rostykus + * Matt Ruffalo + * Eli Sadoff + * Pim Schellart * Nico Schlömer + * Klaus Sembritzki + * Nikolay Shebanov + * Jonathan Tammo Siebert * Scott Sievert * Max Silbiger + * Mandeep Singh + * Michael Stewart + * Jonathan Sutton + * Deep Tavker + * Martin Thoma * James Tocknell + * Aleksandar Trifunovic + * Paul van Mulbregt + * Jacob Vanderplas * Aditya Vijaykumar * Pauli Virtanen * James Webber * Warren Weckesser * Eric Wieser + * Josh Wilson * Zhiqing Xiao + * Evgeny Zhurko * Nikolay Zinov + * Zé Vinícius + A total of 121 people contributed to this release. People with a "+" by their names contributed a patch for the first time.
- 19. ~15 active maintainers $ git shortlog --grep="Merge pull request" -sn v0.19.0..upstream/master 216 Ralf Gommers 209 Pauli Virtanen 75 Evgeni Burovski 39 Josh Wilson 19 Ilhan Polat 15 Andrew Nelson 14 Eric Larson 6 Jaime Fernandez del Rio 6 Nikolay Mayorov 6 Warren Weckesser 4 Denis Laxalde 4 Tyler Reddy 3 Matt Haberland 2 Antonio Horta Ribeiro $ python summary_pr_reviews.py v0.19.0..upstream/master # >100 PR comments @rgommers @pv @larsoner @antonior92 @ilayn @perimosocordiae @nmayorov @ev-br @person142 @andyfaff @WarrenWeckesser @mdhaber @tylerjereddy @jaimefrio @ghost @endolith @lagru @mikofski @matthew-brett @chrisb83 @sgubianpm @pvanmulbregt @josef-pkt @eric-wieser
- 20. –Nadia Eghbal Maintainers are the keystone species of software development
- 22. Matthew Rocklin Wed May 23 19:08:30 EDT 2018 Hi All, .. My gut reaction here is that if removing masked array allows Numpy to evolve more quickly then this excites me. What do “we” want?
- 23. What do “we” want?
- 26. … and code
- 27. User-friendly interfaces New Zealand forest bio-security sampling locations 2017 from scipy.interpolate import griddata row_idx = griddata(weather[['X', 'Y']].values, weather.index.values, latlon, method='nearest') stations = weather.iloc[row_idx, :] Mapping to available weather data
- 28. from scipy.integrate import solve_ivp def exponential_decay(t, y): return -0.5 * y def solution(t, y0): return y0 * np.exp(-0.5*t) TMAX = 10 y0 = 8 t_eval = np.linspace(0, TMAX) analytical_solution = solution(t_eval, y0) for method in ('RK45', 'RK23', 'Radau', 'BDF', 'LSODA'): sol = solve_ivp(exponential_decay, t_span=[0, TMAX], y0=[y0], t_eval=t_eval, method=method) ax.plot(sol.t, analytical_solution - sol.y[0, :], label=method) User-friendly interfaces New interface for solving initial value problems
- 29. Cython bindings cimport scipy.special.cython_special as csc cdef: double x = 1 double rgam rgam = csc.gamma(x)
- 30. Cython bindings from numba import jit a = np.random.randn(30, 20) @jit(nopython=True) def qr(a): return np.linalg.qr(a) >>> %timeit np.linalg.qr(a) 47.5 µs ± 2.53 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each) >>> %timeit qr(a) 10.3 µs ± 174 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
- 31. LowLevelCallable footprint = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]], dtype=bool) image = misc.ascent() @llc.jit_filter_function def fmin(values): """ Return minimum of values; values are pixel values in footprint pattern. Accelerated with Numba, via `scipy.LowLevelCallable`. """ result = np.inf for v in values: if v < result: result = v return result >>> %timeit ndimage.generic_filter(image, np.min, footprint=footprint) 427 ms ± 2.15 µs per loop (mean ± std. dev. of 7 runs, 1 loops each) >>> %timeit ndimage.generic_filter(image, fmin, footprint=footprint) 2.81 ms ± 40.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) Credits: @jni
- 32. What is next for SciPy?
- 33. Languages SciPy is written in: Python, Cython, C, C++, Fortran If Python is not fast enough, we prefer Cython right now. What about in the future?
- 34. Cython Numba Pythran Portability ++ - - Runtime dependency ++ — ++ Maturity ++ + - Maintenance status + ++ - Features ++ + - Performance + ++ ++ Ease of use - ++ + Debugging & optimization 0 + 0 Size of binaries - ++ +
- 35. To grow or not to grow? Submodules removed since 2012: Sub(sub)modules added since 2012: scipy.maxentropy scipy.lib scipy.weave scipy.misc # removal in process scipy.sparse.csgraph scipy.linalg.cython_blas/lapack scipy.special.cython_special scipy.linalg.interpolative
- 36. SciPy will grow in feature completeness and quality, not in scope
- 37. In the pipeline • Support for newer LAPACK versions • Sparse arrays (talk Hameer Abbasi today) • Better global optimizers • Rotation matrices (GSoC’18)
- 42. @FormerPhysicist @NKrvavica @Tokixix @awakenting @axiru @cel4 @chemelnucfin @endolith @gaulinmp @hugovk @ksemb @kshitij12345 @luzpaz @mamrehn @rafalalgo @samyak0210 @soluwalana @sudheerachary @tttthomasssss @vkk800 @wirew0rm @xoviat @yanxun827 @ybeltukov @ziejcow Aakash Jain Aaron Nielsen Abject Abraham Escalante Adam Cox Adam Geitgey Adam Kurkiewicz Aditya Bharti Aditya Vijaykumar Adrian Kretz Akash Goel Alain Leufroy Alan McIntyre Aldrian Obaja Aleksandar Trifunovic Ales Erjavec Alessandro Pietro Bardelli Alex Conley Alex Griffing Alex Loew Alex Reinhart Alex Rothberg Alex Seewald Alex Stewart Alexander Eberspächer Alexander Goncearenco Alexander Grigorievskiy Alexander J. Dunlap Alexey Umnov Alexis Tabary Alistair Muldal Alvaro Sanchez-Gonzalez Aman Pratik Aman Singh Aman Thakral Amato Kasahara Anant Prakash Anastasiia Tsyplia Anders Bech Borchersen Andras Deak Andreas Hilboll Andreas Kloeckner Andreas Kopecky Andreas Mayer Andreas Sorge Andreea Georgescu Andrew Fowlie Andrew Nelson Andrew Schein Andrew Sczesnak Andrey Golovizin Andrey Portnoy Andrey Smirnov Andriy Gelman André Gaul Ankit Agrawal Anne Archibald Anthony Scopatz Anton Akhmerov Antonio H Ribeiro Antonio Horta Ribeiro Antony Lee Anubhav Patel Ariel Rokem Arno Onken Ashwin Pathak Astrofysicus Axl West Baptiste Fontaine Bastian Venthur Behzad Nouri Ben FrantzDale Ben Jude Bence Bagi Benda Xu Benjamin Rose Benjamin Trendelkamp-Schroer Benny Malengier Benoit Rostykus Bernardo Sulzbach Bhavika Tekwani Bill Sacks Björn Dahlgren Bjørn Forsman Blair Azzopardi Blake Griffith Bob Helmbold Bradley M. Froehle Bram Vandekerckhove Branden Rolston Brandon Beacher Brandon Carter Brandon Liu Brett M. Morris Brett R. Murphy Brian Hawthorne Brian Newsom Brianna Laugher Brigitta Sipocz Bruno Beltran Bruno Jiménez Byron Smith CJ Carey Callum Jacob Hays Carl Sandrock Cathy Douglass Chad Baker Chad Fulton Charles Harris Charles Masson Chris Burns Chris Jordan-Squire Chris Kerr Chris Mutel Christian Christian Brodbeck Christian Brueffer Christian Häggström Christian Sachs Christoph Baumgarten Christoph Deil Christoph Gohlke Christoph Paulik Christoph Weidemann Christopher Kuster Christopher Lee Cimarron Mittelsteadt Ciro Duran Santillli Clancy Rowley Clark Fitzgerald Clemens Novak Cody Collin RM Stocks Collin Tokheim Colm Ryan Corey Farwell Daan Wynen Daisuke Oyama Damian Eads Damon McDougall Daniel B. Smith Daniel Bunting Daniel Jensen Daniel Velkov Daniel da Silva Danilo Horta Danny Hermes David Cournapeau David Ellis David Freese David Haberthür David Hagen David Huard David M Cooke David Menéndez Hurtado David Nicholson David Warde-Farley David Wolever Deep Tavker Deepak Kumar Gouda Denis Laxalde Derek Homeier Dezmond Goff Dieter Werthmüller Dima Pasechnik Diogo Aguiam Dirk Gorissen Divakar Roy Dmitrey Kroshko Dominic Antonacci Dominic Else Dorota Jarecka Douglas Lessa Graciosa Dražen Lučanin Dávid Bodnár Ed Schofield Edward Richards Egor Panfilov Eli Sadoff Elliott Sales de Andrade Emanuele Olivetti Eric Firing Eric Jones Eric Larson Eric Martin Eric Moore Eric Quintero Eric Soroos Eric Stansifer Eric Wieser Eugene Krokhalev Evan Limanto Evgeni Burovski Evgeny Zhurko FI$H 2000 Fabian Paul Fabian Pedregosa Fabian Rost Fabrice Silva Fazlul Shahriar Felix Berkenkamp Felix Lenders Fernando Perez Florian Weimer Florian Wilhelm Francis T. O'Donovan Francisco de la Peña Franziska Horn François Boulogne François Garillot François Magimel Frederik Rietdijk Fredrik Wallner Fukumu Tsutsumi G Young Gabriele Farina Gael Varoquaux Garrett Reynolds Gaute Hope Gavin Parnaby Gavin Price Gaël Varoquaux Geordie McBain George Castillo George Lewis Gerrit Ansmann Gerrit Holl Gert-Ludwig Ingold Giftlin Rajaiah Gilles Aouizerate Gilles Rochefort Giorgio Patrini Golnaz Irannejad Graham Clenaghan Greg Caporaso Greg Dooper Gregory Allen Gregory R. Lee Grey Christoforo Guillaume Horel Guo Fei Gustav Larsson Haochen Wu Helder Cesar Helmut Toplitzer Henrik Bengtsson Henry Lin Hervé Audren Hiroki IKEDA Ian Henriksen Ien Cheng Ilan Schnell Ilhan Polat Illia Polosukhin InSuk Joung Ion Elberdin Irvin Probst J.J. Green J.L. Lanfranchi Jaakko Luttinen Jackie Leng Jacob Carey Jacob Silterra Jacob Stevenson Jacob Vanderplas Jacques Gaudin Jacques Kvam Jaime Fernandez del Rio Jakob Jakobson Jakub Wilk James Gerity James T. Webber James Tocknell James Tomlinson James Webber Jamie Morton Jan Lehky Jan Schlueter Jan Schlüter Janani Padmanabhan Janko Slavič Jarrod Millman Jason King Jean Helie Jean-François B Jean-François B. Jed Frey Jeff Armstrong Jerome Kieffer Jerry Li Jesse Engel Jim Garrison Jim Radford Jin-Guo Liu Jinhyok Heo Joaquin Derrac Rus Jochen Garcke Joel Nothman Johann Cohen-Tanugi Johannes Ballé Johannes Kulick Johannes Schmitz Johannes Schönberger John David Reaver John Draper John Travers Johnnie Gray Jon Haitz Legarreta Gorroño Jona Sassenhagen Jonas Hahnfeld Jonas Rauber Jonathan Helmus Jonathan Hunt Jonathan Sutton Jonathan Tammo Siebert Jonathan Taylor Joonas Paalasmaa Jordan Heemskerk Jordi Montes Jorge Cañardo Alastuey Joris Vankerschaver Joscha Reimer Josef Perktold Joseph Albert Joseph Fox-Rabinovitz Joseph Jon Booker Josh Lawrence Josh Lefler Josh Levy-Kramer Josh Wilson Joshua L. Adelman Josue Melka Juan Luis Cano Rodríguez Juan M. Bello-Rivas Juan Nunez-Iglesias Juha Remes Julian Lukwata Julian Taylor Julien Lhermitte Julius Bier Kirkegaard Jussi Leinonen Justin Lavoie Jyotirmoy Bhattacharya Jérôme Roy Jörg Dietrich Jörn Hees K.-Michael Aye Kai Kai Striega Kai-Striega KangWon Lee Kari Schoonbee Kasper Primdal Lauritzen Kat Huang Katrin Leinweber Keith Clawson Kevin Davies Kevin Rose Klaus Sembritzki Klesk Chonkin Kolja Glogowski Konrad0 Kornel Kielczewski Kyle Oman Lam Yuen Hei Lars Buitinck Lars G Lawrence Chan Lei Ma Leo Singer Liam Damewood Lilian Besson Liming Wang Lindsey Hiltner Lorenzo Luengo Louis Thibault Louis Tiao Loïc Estève Luca Citi Luis Pedro Coelho Luke Zoltan Kelley M.J. Nichol MYheavyGo Mads Jensen Mandeep Singh Maniteja Nandana Manuel Reinhardt Marc Abramowitz Marc Honnorat Marcello Seri Marcos Duarte Marek Jacob Maria Knorps Mark Campanelli Mark Mikofski Mark Wiebe Markus Meister Marti Nito Martin Manns Martin Spacek Martin Teichmann Martin Thoma Martin Ø. Christensen Martino Sorbaro Martín Gaitán Marvin Kastner Matt Dzugan Matt Haberland Matt Hickford Matt Knox Matt Newville Matt Ruffalo Matt Terry Matteo Visconti Matthew Brett Matthias Feurer Matthias Kümmerer Matthieu Dartiailh Matthieu Melot Matty G Matěj Kocián Max Argus Max Bolingbroke Max Linke Max Mikhaylov Max Silbiger Maximilian Singh Meet Udeshi Mher Kazandjian Michael Benfield Michael Boyle Michael Danilov Michael Hirsch Michael James Bedford Michael Stewart Michael Wimmer Miguel de Val-Borro Mihai Capotă Mike Romberg Mike Toews Mikhail Pak Mikkel Kristensen MinRK Naoto Mizuno Nat Wilson Nathan Bell Nathan Crock Nathan Musoke Nathan Woods Ned Richards Neil Girdhar Nelson Liu Nick Papior Nicky van Foreest Nico Schlömer Nicola Montecchio Nicolas Del Piano Niklas Hambüchen Niklas K Nikolai Nowaczyk Nikolas Moya Nikolay Mayorov Nikolay Shebanov Nikolay Zinov Nils Werner Noel Kippers Oleksandr (Sasha) Huziy Oleksandr Pavlyk Olga Botvinnik Olivier Grisel Orestis Floros Osvaldo Martin P. L. Lim Pablo Winant Patrick Callier Patrick Snape Patrick Varilly Paul Ivanov Paul Nation Paul Ortyl Paul van Mulbregt Pauli Virtanen Pawel Chojnacki Peadar Coyle Pearu Peterson Pedro López-Adeva Fernández-Layos Per Brodtkorb Perry Lee Pete Bunch Peter Cock Peter Yin Petr Baudis Phil Tooley Philip DeBoer Philipp Rentzsch Phillip Cloud Phillip J. Wolfram Phillip Weinberg Pierre GM Pierre de Buyl Pim Schellart Piotr Uchwat Przemek Porebski Raden Muhammad Radoslaw Guzinski Rafael Rossi Rafał Byczek Ralf Gommers Ramon Viñas Randy Heydon Raoul Bourquin Raphael Wettinger Ravi Kumar Prasad Ray Bell Regina Ongowarsito Richard Gowers Richard Tsai Rob Falck Robert Cimrman Robert David Grant Robert Gantner Robert Kern Robert McGibbon Robert Pollak Rohit Jamuar Rohit Sivaprasad Roman Feldbauer Roman Mirochnik Roman Ring Ronan Lamy Ronny Pfannschmidt Rupak Das Rémy Léone Sam Lewis Sam Mason Sam Tygier Sami Salonen Samuel St-Jean Santi Villalba Saurabh Agarwal Scott Sievert Scott Sinclair Sean Gillies Sean Quinn Sebastian Berg Sebastian Gassner Sebastian Pucilowski Sebastian Pölsterl Sebastian Skoupý Sebastian Werk Sebastiano Vigna Sebastián Vanrell Sergey B Kirpichev Sergio Oller Shauna Shinya SUZUKI Siddhartha Gandhi Simon Gibbons Skipper Seabold Sourav Singh Srikiran Stefan Otte Stefan Peterson Stefan van der Walt Stefano Costa Stefano Martina Stephan Hoyer Stephen McQuay Steve Richardson Steven Byrnes Strahinja Lukić Sturla Molden Sumit Binnani Surhud More Svend Vanderveken Sylvain Bellemare Syrtis Major Sytse Knypstra Søren Fuglede Jørgensen Takuya Oshima Tavi Nathanson Ted Pudlik Ted Ying Terry Jones Tetsuo Koyama Theodore Hu Thomas A Caswell Thomas Etherington Thomas Haslwanter Thomas Hisch Thomas Keck Thomas Kluyver Thomas Pingel Thomas Robitaille Thomas Spura Thouis (Ray) Jones Tiago M.D. Pereira Tim Cera Tim Hochberg Tim Leslie Tiziano Zito Tobias Megies Tobias Schmidt Todd Goodall Todd Jennings Tom Aldcroft Tom Augspurger Tom Donoghue Tom Flannaghan Tom Waite Tomas Tomecek Tony S. Yu Tony Xiang Toshiki Kataoka Travis Oliphant Travis Vaught Trent Hauck Tyler Reddy Uri Goren Utkarsh Upadhyay Uwe Schmitt Vahan Babayan Valentine Svensson Vasily Kokorev Ved Basu Vedant Rathore Vicky Close Vikram Natarajan Vincent Arel-Bundock Vincent Barrielle Vladislav Iakovlev Víctor Zabalza Warren Weckesser Wendy Liu Wes McKinney Will Monroe Wim Glenn Yaroslav Halchenko Yevhenii Hyzyla Yoav Ram Yoni Teitelbaum Yoshiki Vázquez Baeza Yotam Doron Yu Feng Yurii Shevchuk Yusuke Watanabe Yuxiang Wang Yves Delley Yves-Rémi Van Eycke Zach Ploskey Zaz Brown Ze Vinicius Zhiqing Xiao Zé Vinícius abaecker annesylvie arcady ashish bsdz chanley dmorrill drlvk emmi474 fcady fo40225 fred.mailhot fullung hagberg hm jfinkels jmiller joe jswhit jtaylor jtravs keuj6 ktritz lemonlaug martin michaelvmartin15 mohmmadd mszep nmarais nrnrk ondrej pjwerneck poolio prabhu puenka roberto solarjoe swalton thorstenkranz timbalam tosh1ki trueprice tzito Åsmund Hjulstad A big thank you to • The SciPy core team • All SciPy contributors • Our users • The wider scientific Python community