IPython Notebooks - Hacia los papers ejecutables
1 like•775 views
The document discusses the digital lifecycle of astronomy research, emphasizing the need for reproducible research and efficient data infrastructure. It highlights the importance of making methodologies and tools accessible for reuse to address the reproducibility crisis. The emergence of executable papers and tools like Jupyter Notebooks is presented as a solution to enhance visibility, efficiency, and collaboration in research.
IPython Notebooks - Hacia los papers ejecutables
- 1. José Enrique Ruiz Segundo Encuentro Python Granada Centro de Enseñanzas Virtuales de la Universidad de Granada – CEVUG 26th June 2015
- 2. Astronomy research lifecycle is entirely digital » Observation proposals » Data reduction pipelines » Analysis of science ready data » Catalogs of objects and data archives » Publish process - ADS/arXiv › Materials and Methods › Results › Discussion Reproducible research is still not possible in a digital world A rich infrastructure of data is not efficiently used A normalized preservation of methodology is needed Tools Digital Astronomy
- 4. Optimize return on investments made on big facilities » Avoid duplication of efforts and reinvention » How to discover and not duplicate ? » How to re-use and not duplicate ? » How to make use of best practices ? » How to use the rich infrastructure of data ? » Intellectual contributions encoded in software More data in archives do not imply more knowledge » Expose complete scientific process, not the story » Allow easy discovery of methods and tools Visibility, Efficiency and Reuse The Reproducibility Crisis
- 5. More R’s than Pirates ! Prof. David de Roure http://www.scilogs.com/eresearch/replacing-the-paper-the-twelve-rs-of-the-e-research-record/ http://www.scilogs.com/eresearch/more-rs-than-pirates/
- 6. Open Science is much more than Open Data Repeatable Tools and methodology available I could repeat the experiment Reproducible Clear methodology and all resources available I could reproduce the results Reusable I know how it could be useful for my needs I could use all or some parts as it is I could modify and adapt it even for other purposes
- 7. Story Capture Going beyond automation Organization
- 8. # CIG Vhel e_Vhel r_Vhel Dist MType e_MType OptAssym r_MType Bmag e_Bmag 1 7299.0 3.0 1 96.9 5.0 1.5 1 1 14.167 0.271 0.173 0.571 0.040 13.383 2 6983.0 6.0 2 94.7 6.0 1.5 0 1 15.722 0.324 0.255 0.278 0.031 15.157 3 4.0 1.5 0 1 16.057 0.507 0.246 0.354 15.457 4 2310.0 1.0 3 31.9 3.0 1.5 0 1 12.818 0.424 0.252 0.863 0.017 11.685 5 7865.0 10.0 3 105.9 0.0 1.5 0 1 15.602 0.364 0.225 0.131 0.118 15.128 72 5164.0 9.0 2 68.5 5.0 1.5 1 1 14.445 0.325 0.315 0.367 0.028 13.735 Story Capture
- 9. # CIG Vhel e_Vhel r_Vhel Dist MType e_MType OptAssym r_MType Bmag e_Bmag 1 7299.0 3.0 1 96.9 5.0 1.5 1 1 14.167 0.271 0.173 0.571 0.040 13.383 2 6983.0 6.0 2 94.7 6.0 1.5 0 1 15.722 0.324 0.255 0.278 0.031 15.157 3 4.0 1.5 0 1 16.057 0.507 0.246 0.354 15.457 4 2310.0 1.0 3 31.9 3.0 1.5 0 1 12.818 0.424 0.252 0.863 0.017 11.685 5 7865.0 10.0 3 105.9 0.0 1.5 0 1 15.602 0.364 0.225 0.131 0.118 15.128 72 5164.0 9.0 2 68.5 5.0 1.5 1 1 14.445 0.325 0.315 0.367 0.028 13.735 Capture Actions, Tasks, Dependencies, Provenance Improve Clarity / Documentation Story Capture
- 10. Research Objects Distributed Technical Objects Social Objects Expose experimental context in a structured way in order to be understood http://www.researchobject.org/
- 11. The Social Dimension Visibility and Discovery
- 12. The Executable Paper Time has come to go beyond the PDF
- 13. IPython Notebooks All started as a browser-wrapper for iPython
- 14. A working methodology » A Tool – web browser as the working desktop » Code, plots and data, living with rich-text documentation » Capture exploratory and analysis tasks » Multi-language in the very same script / recipe » Fully extensible and customizable » Desktop and server-side working environment » Cloud-based adaptive to scalable computing environment » Wiki-like scripts fully shareable, re-usable and executable » Training - executable tutorials that reduce learning curve » Published in social platforms with Git versioning » From scripts/ recipes towards bigger modular projects /books IPython Jupyter Notebooks
- 15. Try it ! Local desktop: ipython notebook in the terminal In the Cloud: http://try.jupyter.org in the browser
- 16. The Library Visualization of static notebooks previously published as gists shareable via URL link, may be downloaded and executed locally http://nbviewer.ipython.org Cloud Solutions http://lmgtfy.com/?q=ipython+notebook+examples#
- 17. 17 The Library Visualization of static notebooks previously published as gists shareable via URL link, may be downloaded and executed locally http://nbviewer.ipython.org Cloud Solutions http://lmgtfy.com/?q=ipython+notebook+examples#
- 22. https://github.com/ipython-contrib/IPython-notebook-extensions Javascript extensions for added UI functionality Customizing a Notebook Server
- 23. Customizing a Notebook Server » Access to common global functions and variables » Functions as wrappers to access CLI server-side software » Integrated access to server-side databases » Customized execution framework for a research group » Internal library of executable recipes » Access restricted content
- 26. Seamless work with databases
- 28. The Executable Paper Time has come to go beyond the PDF
- 31. Publishing Research Objects Digital Science - Towards the Executable Paper http://youtu.be/1dXkmgkYuEg http://youtu.be/FQ5FaeHcdwo
- 32. Collaborative Writing and Versioning http://youtu.be/ObJG3i3jjPg