Intro to Python Data Analysis in Wakari
- 1. Intro to Python Data Analysis in Wakari Karissa McKelvey Software Developer Continuum Analytics @karissamck November 8, 2013 PyData NYC
- 4. More Tweets, Mote Votes
- 5. MY GOALS Get you excited about data analysis in Wakari Walk through some basic analysis packages and wakari workflows Kick-start your journey
- 6. WHO ARE YOU?
- 14. Putting Science back in Comp Sci • Much of the software stack is for systems programming --- C++, Java, .NET, ObjC, web - Complex numbers? - Vectorized primitives? • Software stack for scientists is not as helpful as it should be • Fortran is still where many scientists end up
- 16. Why Python?
- 17. High Performance with BIG DATA
- 18. Packages for data analysis and visualization
- 19. Syntax – Gets out of your way
- 20. Community Driven
- 21. Ready for web applications, too.
- 23. • “Python is good for data cleanup, R for statistical models” “Which is the better Data Analysis language? R or Python?” Quora. http://www.quora.com/Data-Analysis/Which-is-the-better-Data-analysis-language-R-or-Python
- 24. • “Python is good for data cleanup, R for statistical models” • “R is quirky and weird but the statisticians love it and there really isn’t any compelling reason to switch” “Which is the better Data Analysis language? R or Python?” Quora. http://www.quora.com/Data-Analysis/Which-is-the-better-Data-analysis-language-R-or-Python
- 25. • “Python is good for data cleanup, R for statistical models” • “R is quirky and weird but the statisticians love it and there really isn’t any compelling reason to switch” • “You’re running an MCMC simulation on a laptop? Perhaps you should write it in C++/FORTRAN” “Which is the better Data Analysis language? R or Python?” Quora. http://www.quora.com/Data-Analysis/Which-is-the-better-Data-analysis-language-R-or-Python
- 26. “You’re running an MCMC simulation on a laptop? Perhaps you should write it in C++/FORTRAN” Ready for DATA, and then some
- 27. Numba: just-in-time compiler to LLVM through @decorators numba.pydata.org
- 28. Numba: just-in-time compiler to LLVM through @decorators* numba.pydata.org *aka, fast. easy.
- 30. Basic packages for data analysis and visualization
- 31. NumPy: The foundation of the Python Data Analysis stack
- 36. Pandas: Builds upon NumPy
- 37. Matplotlib: 2D plotting library
- 38. IPython: Interactive Python (+ in the Web) tab completion magic %-commands Inline plots
- 39. Anaconda: pulls it all together
- 41. wakari.io Browser-based Python & Linux environment
- 42. IPython Notebook Scientific Packages Terminal Share files, IPython notebooks, and plots with pay-as-you-go compute
- 43. Sharing in Wakari • Packages IPython notebooks, files, folders, data, and environment • Get a link • Share that link.
- 46. “A rule of thumb among biotechnology venture capitalists is that half of published research cannot be replicated”
- 47. How do we replicate research today?
- 48. collaborate on How do we replicate research today?
- 49. collaborate on How do we replicate research today? data analysis
- 50. How do we collaborate today?
- 51. How do we collaborate today?
- 52. How do we collaborate today?
- 53. How do we collaborate today?
- 54. ????????
- 55. How do we replicate research today?
- 56. wakari.io Browser-based Python & Linux environment
- 57. Enterprise or Cloud Online at wakari.io or install locally for access to your hardware and data
- 58. wakari.io Browser-based Python & Linux environment
- 59. Coming Soon
- 60. Project-based interaction user Projects starting at 10$/month with unlimited team members
- 61. Interactive Plotting Next-generation collaborative data manipulation, analysis, and presentation
- 62. Talks to see • Jack Vanderplas (Washington) – Efficient computing with Numpy • 29th Floor combo 3pm (Right now, next door!) • Julia Evans (N/A) – A practical introduction to IPython Notebook & pandas • Here, 4:45pm.
- 63. Talks to see • Sarah Guido (Michigan) – A Beginner’s Guide to Machine Learning with scikit-learn • Imram Haque (Counsyl) – Beyond the dict • Peter Wang (Continuum) – Bokeh Workshop
- 64. Special Thanks Ben Zaitlin Mark Florisson Clayton Davis Bryan Van de Ven Travis Oliphant
Editor's Notes
- #3: I do web programming
- #7: How many of you use python on a daily basis for data analysis?In the past year, raise your hand if you’ve worked primarily in python.
- #19: Domain-specific librariesStatsmodels => statistical computingScikit-image => image manipulationOpenCV => Image processing with interface that can accept NumPy arraysPyTables => HDF5 integrationNumexpr => you can write expressions on your data with cache-aware expressions, it’s very efficient.There are more packages in the python scientific stack than just these. But, it’s good to know numpy so you can get down and dirty with your data and manipulate it if need be.
- #20: PACKAGES!Occasional programmers can jump on
- #21: PACKAGES!
- #27: THIS SHOULD NEVER HAPPEN.At continuum analytics, we never want these words to be uttered again.
- #31: Python in 60 secondsNumPyScipyPandasMatplotlibScikit-learn
- #33: Homogenous
- #41: We’re going to pull it all together in Wakari.
- #57: And this is why sharing in wakari is so important