20160708 データ処理のプラットフォームとしてのpython 札幌
5 likes•3,291 views
The document is a comprehensive overview of Python programming, detailing various libraries and tools such as NumPy, SciPy, pandas, and TensorFlow. It highlights different development environments like Jupyter Notebook and Spyder, as well as data manipulation techniques and APIs related to data analysis. The content references multiple resources and includes code snippets illustrating the use of Python for data management and visualization.
![1 1.2 1000000L Python2
‘abc’ u’ ’ Python2
[1, 2, 3,‘foo’,‘bar’,‘foo’]
(1, 2, 3,‘foo’,‘bar’,‘foo’)
{‘k1’:‘value1’,‘k2’:‘value2’}
set(1, 2, 3,‘foo’,‘bar’)](https://image.slidesharecdn.com/20160708python-160709173450/85/20160708-python-27-320.jpg)
![•
•
• split
s = ‘foo, bar, baz’
items = s.split(‘,’)
print items[0]
print items[-1]
print items[0][-2:]](https://image.slidesharecdn.com/20160708python-160709173450/85/20160708-python-28-320.jpg)
![•
list comprehension
•
dictionary comprehension
• lambda map, reduce, filter
sList = [‘foo’, ‘bar’, ‘baz’]
lList = [len(s) for s in sList]
lList = map(lambda s:len(s),
sList)
lDict = {s:len(s) for s in sList}](https://image.slidesharecdn.com/20160708python-160709173450/85/20160708-python-29-320.jpg)
![• http://sinhrks.hatenablog.com/entry/2015/01/28/073327
0 1
import pandas as pd
df[‘nValue’] = df[‘value’] / sum(df[‘value’])
id value color
sapporo 43 red
osaka 42 pink
matsumoto 40 green
id value color nValue
sapporo 43 red 0.344
osaka 42 pink 0.336
matsumoto 40 green 0.32
Python](https://image.slidesharecdn.com/20160708python-160709173450/85/20160708-python-33-320.jpg)
20160708 データ処理のプラットフォームとしてのpython 札幌
- 1. Python Sky
- 3. • • Python 2000 (**) • db tech showcase MongoDB • • FB: Ryuji Tamagawa • Twitter : tamagawa_ryuji
- 5. 2015
- 6. 2016
- 8. • Python • • Python • NumPy, SciPy, matplotlib, Pandas • Python • scikit-learn • TensorFlow • Python IPython, Jupyter notebook, Spyder, VisualStudio • Python • Python • Pandas • Spark - PySpark DataFrame API • matplotlib
- 9. Part 1 : Python
- 10. Python • • Google Guido Google Google 1 • NumPy, SciPy, matplotlib → Pandas • • -2000 Linux -2010 Web Trac Google
- 12. Python • • pyODBC • Web WSGI
- 13. Python • 2.x 3.x 32bit 64bit 64bit • 2.x • 3.x 3 • 2.x 3.x
- 14. • Ruby? • R? • Java? • Scala?
- 15. Python • Python ’CPython’ JIT PyPy JVM Jython .Net IronPython • CPython • CPython 2 • C • processing PySpark
- 16. Python • Python • 1 Linux Mac OS Python Python Mac • Python pip 3.x Python 2.7.9 2.x Python pip Linux Python pip yum apt • Python Anaconda Python conda • python 2016 http://qiita.com/y__sama/items/5b62d31cb7e6ed50f02c
- 17. NumPy, SciPy, matplotlib, Pandas • • NumPy SciPy • Pandas Pandas Pandas NumPy • Anaconda Python
- 20. Python IPython Jupyter, … IDE Spyder, Rodeo Visual Studio, PyCharm, PyDev
- 22. • IPython • • • Anaconda • pip
- 23. • Jupyter Notebook • Python • IPython Notebook Python • Apache Zeppelin http:// zeppelin.apache.org
- 24. IDE • R RStudio • IDE • • 2 Spyder Rodeo • Spyder
- 25. • • Visual Studio • Eclipse PyDev • PyCharm •
- 26. Part 2 : Python
- 27. 1 1.2 1000000L Python2 ‘abc’ u’ ’ Python2 [1, 2, 3,‘foo’,‘bar’,‘foo’] (1, 2, 3,‘foo’,‘bar’,‘foo’) {‘k1’:‘value1’,‘k2’:‘value2’} set(1, 2, 3,‘foo’,‘bar’)
- 28. • • • split s = ‘foo, bar, baz’ items = s.split(‘,’) print items[0] print items[-1] print items[0][-2:]
- 29. • list comprehension • dictionary comprehension • lambda map, reduce, filter sList = [‘foo’, ‘bar’, ‘baz’] lList = [len(s) for s in sList] lList = map(lambda s:len(s), sList) lDict = {s:len(s) for s in sList}
- 30. Pandas • Pandas • matplotlib / seaborn • NumPy SciPy Python • Pandas + matplotlib OK Pandas NumPy NumPy / SciPy
- 31. Pandas • Pandas DataFrame • R • RDB 2 • index Series Columns Columns Series Series SeriesIndex
- 32. Pandas I/O • CSV JSON RDB Excel • column • RDB • import pandas as pd pd.read_csv(<filename>) pd.read_json(<filename>) pd.to_csv(<filename>) pd.to_excel(<filename>) # pd.to_clipboard()
- 33. • http://sinhrks.hatenablog.com/entry/2015/01/28/073327 0 1 import pandas as pd df[‘nValue’] = df[‘value’] / sum(df[‘value’]) id value color sapporo 43 red osaka 42 pink matsumoto 40 green id value color nValue sapporo 43 red 0.344 osaka 42 pink 0.336 matsumoto 40 green 0.32 Python
- 34. Spark - PySpark DataFrame API • Python • Spark PySpark findSpark Spark • Python Spark API DataFrame API • Spark Pandas Spark PySpark Spark node Spark node Spark node Spark node driver
- 35. matplotlib / seaborn • • Python NumPy / Pandas • Jupyter Notebook Spyder
- 36. Questions ?