PyVideo.org - SciPy 2016

3D Drawing in Python: Reviving Visual

Catherine Holloway — Fri, 15 Jul 2016 00:00:00 +0000

Description

In 2000 David Scherer created Visual, a python package with a simple interface for drawing 3D objects to the screen. Visual abstracted away calls to OpenGL vertex drawing, textures, and transformations, and allowed primitive geometric objects to be placed on the screen in an intuitive manner. Visual was subsequently adopted by many researchers and university instructors, primarily in physics, to visualize scientific results and simulation assignments. The original Visual module used C++ to make OpenGL calls, and compatibility with Linux was never implemented. Python and OpenGL have come a long way in the past sixteen years, and now Visual can be made cross-platform by removing the C++ backend. I will present my attempt to re-implement Visual using pyglet, as well as demonstrate its API and usage in education and research visualization.

A BLAS for Tensors with Portable High Performance

Devin Matthews — Fri, 15 Jul 2016 00:00:00 +0000

A Whirlwind Tour of UC Berkeley’s Data Science Education Program

Cathryn Carson, Sam — Fri, 15 Jul 2016 00:00:00 +0000

Automatic Machine Learning?

Andreas Mueller — Fri, 15 Jul 2016 00:00:00 +0000

Community-Powered Packaging with conda-forge (BoF Session)

Phillip Elson — Fri, 15 Jul 2016 00:00:00 +0000

Computational Supply Chain Risk Management for Open Source Software

Sebastian Benthall — Fri, 15 Jul 2016 00:00:00 +0000

Description

We address the cybersecurity problems of supply chain risk management in open source software. How does one detect high-risk components in a deployed software system that includes many open source components? As a complement to software assurance approaches based on static source code analysis, we propose a technique based on an analysis of the entire open source ecosystem, inclusive of its technical products and contributor activity. we show how dependency topology, community activity, and exogenous vulnerability and exposure information can be integrated to detect high risk "hot spots" requiring additional investment. We demonstrate this technique using the Python dependency topology extracted from PyPi and data from GitHub. We will dicuss how our analysis prototype has been implemented with SciPy tools.

Conduit: A Scientific Data Exchange Library for HPC Simulations

Cyrus Harrison — Fri, 15 Jul 2016 00:00:00 +0000

Description

Conduit (http://software.llnl.gov/conduit) is a new open source project from Lawrence Livermore National Laboratory. It provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. Conduit supports in-core data coupling between packages, serialization, and I/O tasks.

Conduit leverages ideas from JSON and NumPy to provide a cross-language data access API that simplifies sharing data in the HPC ecosystem. For SciPy 2016, an important focus of our talk will be Python support in Conduit and how positive experiences using Python motivated our approach to build a sane cross-language data description solution.

Constructing Models to Deal with Missing Data

Deborah Hanus — Fri, 15 Jul 2016 00:00:00 +0000

Democratizing Geostats

Nicholas Ronnei — Fri, 15 Jul 2016 00:00:00 +0000

Description

Error and uncertainty are a fact of life, however they often go unaccounted for in geospatial models that rely on Digital Elevation Models (DEMs). Our work focuses on mitigating that problem as it relates to SRTM and ASTER GDEM. This is a discussion about how Python works within a broader system that enables researchers to work around this uncertainty without being geostatistics experts. The talk will cover how we use Python for advanced raster processing (including a vertical datum correction), calculating error statistics, and interacting with our spatial database as well as what we're doing with the end product, including a live demo.

Diffing and Merging Jupyter Notebooks with nbdime

Min Ragan Kelley — Fri, 15 Jul 2016 00:00:00 +0000

DyND Callables

Mark Wiebe — Fri, 15 Jul 2016 00:00:00 +0000

Emperor, Interactive Beta diversity Exploration

Yoshiki Vazquez Baeza — Fri, 15 Jul 2016 00:00:00 +0000

Generalized Earthquake Focal Mechanism Classification

Ben Lasscock — Fri, 15 Jul 2016 00:00:00 +0000

Description

We present a novel classification technique for identifying earthquake focal mechanism type and fault plane orientation using a robust classification technique rather than the least squares based (HASH) algorithm. The goal was to support a system capable of automatically classifying earthquakes, for applications such as microseismic monitoring. In this context, classification of both shear or/and tensile failure (mixed double-couple and CLVD sources) was required, so a generalized system was developed. More generally though, we see applications of this algorithm in hazard monitoring, particularly for early classification of tsunamigenic. The project was implemented in Python, the classification was made easy using scikit-learn and SciPy special functions, and 3-D visualization was done using Mayavi.

GR: Plotting with Python or Julia

Josef Heinen — Fri, 15 Jul 2016 00:00:00 +0000

Description

GR is a plotting package for the creation of two- and three-dimensional graphics in Python or Julia, offering unique plotting functions to visualize static or dynamic data with minimal overhead. In addition, GR can be used as a backend for other plotting interfaces or wrappers, in particular when being used in interactive notebooks. This presentation shows how visualization applications with special performance requirements can be designed on the basis of simple and easy-to-use functions as known from the MATLAB plotting library. Using quick practical examples, this talk is going to present the special features and capabilities provided by the GR framework both as a self-contained graphics library or as a fast backend for other packages. Slides may be found here: http://pgi-jcns.fz-juelich.de/pub/doc/SciPy_2016/GR-Plotting_with_Python_or_Julia.pdf

Hacking the CPython Interpreter

James Powell — Fri, 15 Jul 2016 00:00:00 +0000

HistomicsTK: Seamless Analytics for Biomedical Microscopy

Brian Helba — Fri, 15 Jul 2016 00:00:00 +0000

Description

We present the architecture of HistomicsTK, an open-source library for the processing and analysis of biomedical microscopy images. HistomicsTK leverages multiple SciPy libraries, and contains an innovative parameter serialization model to expose arbitrary CLI algorithms via a web GUI.

HyperSpy: How to Easily Bend Multi-dimensional Data to your Analytical Will

Tomas Osta — Fri, 15 Jul 2016 00:00:00 +0000

Description

HyperSpy is an open-source Python library that aims at easing the task of visualizing, analyzing, accessing and storing multi-dimensional signals. Such data structures arise in many scientific and engineering fields, from astronomy to electron microscopy. In addition, common non-linear optimization problems and our suggested new solutions will be presented.

Integrating Scripting into Commercial Applications

Eric Jones — Fri, 15 Jul 2016 00:00:00 +0000

JupyterLab: Building Blocks for Interactive Computing

Brian Granger — Fri, 15 Jul 2016 00:00:00 +0000

Description

Project Jupyter provides building blocks for interactive and exploratory computing. These building blocks make science and data science reproducible across over 40 programming language (Python, Julia, R, etc.). Central to the project is the Jupyter Notebook, a web-based interactive computing platform that allows users to author data- and code-driven narratives - computational narratives - that combine live code, equations, narrative text, visualizations, interactive dashboards and other media.

While the Jupyter Notebook has proved to be an incredibly productive way of working with code and data interactively, it is helpful to decompose notebooks into more primitive building blocks: kernels for code execution, input areas for typing code, markdown cells for composing narrative content, output areas for showing results, terminals, etc. The fundamental idea of JupyterLab is to offer a user interface that allows users to assemble these building blocks in different ways to support interactive workflows that include, but go far beyond, Jupyter Notebooks.

JupyterLab accomplishes this by providing a modular and extensible user interface that exposes these building blocks in the context of a powerful work space. Users can arrange multiple notebooks, text editors, terminals, output areas, etc. on a single page with multiple panels, tabs, splitters, and collapsible sidebars with a file browser, command palette and integrated help system. The codebase and UI of JupyterLab is based on a flexible plugin system that makes it easy to extend with new components.

In this talk, we will demonstrate the JupyterLab interface, its codebase, and describe how it fits within the overall roadmap of the project.

Keynote: Machine Learning for Social Science

Hanna Wallach — Fri, 15 Jul 2016 00:00:00 +0000

Description

In this talk, I will introduce the audience to the emerging area of computational social science, focusing on how machine learning for social science differs from machine learning in other contexts. I will present two related models -- both based on Bayesian Poisson tensor decomposition -- for uncovering latent structure from count data. The first is for uncovering topics in previously classified government documents, while the second is for uncovering multilateral relations from country-to-country interaction data. Finally, I will talk briefly about the broader ethical implications of analyzing social data.

Hanna Wallach is a Senior Researcher at Microsoft Research New York City and an Adjunct Associate Professor in the College of Information and Computer Sciences at the University of Massachusetts Amherst. She is also a member of UMass's Computational Social Science Institute. Hanna develops machine learning methods for analyzing the structure, content, and dynamics of social processes. Her work is inherently interdisciplinary: she collaborates with political scientists, sociologists, and journalists to understand how organizations work by analyzing publicly available interaction data, such as email networks, document collections, press releases, meeting transcripts, and news articles. To complement this agenda, she also studies issues of fairness, accountability, and transparency as they relate to machine learning. Hanna's research has had broad impact in machine learning, natural language processing, and computational social science. In 2010, her work on infinite belief networks won the best paper award at the Artificial Intelligence and Statistics conference; in 2014, she was named one of Glamour magazine's "35 Women Under 35 Who Are Changing the Tech Industry"; in 2015, she was elected to the International Machine Learning Society's Board of Trustees; and in 2016, she was named co-winner of the 2016 Borg Early Career Award. She is the recipient of several National Science Foundation grants, an Intelligence Advanced Research Projects Activity grant, and a grant from the Office of Juvenile Justice and Delinquency Prevention. Hanna is committed to increasing diversity and has worked for over a decade to address the underrepresentation of women in computing. She co-founded two projects---the first of their kind---to increase women's involvement in free and open source software development: Debian Women and the GNOME Women's Summer Outreach Program. She also co-founded the annual Women in Machine Learning Workshop, which is now in its eleventh year. Hanna holds a BA in computer science from the University of Cambridge, an MSc in cognitive science and machine learning from the University of Edinburgh, and a PhD in machine learning from the University of Cambridge.

Keynote: Project Jupyter

Brian Granger — Fri, 15 Jul 2016 00:00:00 +0000

Description

Brian Granger is an Associate Professor of Physics at Cal Poly State University in San Luis Obispo, CA. He has a background in theoretical physics, with a Ph.D from the University of Colorado. His current research interests include quantum computing, parallel and distributed computing and interactive computing environments for scientific computing and data science. He is a leader of the IPython project, co-founder of Project Jupyter and is an active contributor to a number of other open source projects focused on data science in Python. He is a board member of the NumFocus Foundation and a fellow at Cal Poly’s Center for Innovation and Entrepreneurship. He is @ellisonbg on Twitter and GitHub.

Announcement of Altair, Altair is a declarative statistical visualization library for Python. Altair is developed by Brian Granger and Jake Vanderplas in close collaboration with the UW Interactive Data Lab.

With Altair, you can spend more time understanding your data and its meaning. Altair's API is simple, friendly and consistent and built on top of the powerful Vega-Lite JSON specification. This elegant simplicity produces beautiful and effective visualizations with a minimal amount of code.

Labs in the Wild: Teaching Signal Processing Using Wearables & Jupyter Notebooks

Demba Ba — Fri, 15 Jul 2016 00:00:00 +0000

Description

Jupyter notebooks and the Python ecosystem provide a unique opportunity for interactive, web-based, teaching of content that has not traditionally leveraged scientific computing resources. We discuss the design and implementation of a new biological signal processing course at Harvard, ES155, which fuses Wearable technology and cloud-based analysis of data. ES155 bridges the gap that has traditionally existed between Electrical Engineering and Computer Science education, in a framework that we term “Labs in the Wild”. In the process of designing the course, we have had to solve the problem of serving Jupyter notebooks on the cloud reliably using AWS EC2 instances. This is a challenging problem because a successful approach must be scalable, cost-effective, reliable, and address the privacy concerns associated with cloud-based technologies. We describe our system in this talk, and perform a live demo of how students in our class interact with the system, and give examples of ingenious final projects put together by students. Being cloud-based, our system lowers the barrier of entry for students to begin using Python for scientific computing.

Launching Python Applications on Peta scale Massively Parallel Systems

Yu Feng — Fri, 15 Jul 2016 00:00:00 +0000

Machine Learning for Time Series Data in Python

Brett Naul — Fri, 15 Jul 2016 00:00:00 +0000

Description

The analysis of time series data is a fundamental part of many scientific disciplines, but there are few resources meant to help domain scientists to easily explore time course datasets: traditional statistical models of time series are often too rigid to explain complex time domain behavior, while popular machine learning packages deal almost exclusively with 'fixed-width' datasets containing a uniform number of features. Cesium is a time series analysis framework, consisting of a Python library as well as a web front-end interface, that allows researchers to apply modern machine learning techniques to time series data in a way that is simple, easily reproducible, and extensible.

Modeling Rate and State Friction with Python

John Leeman — Fri, 15 Jul 2016 00:00:00 +0000

Description

Friction plays a crucial role in a broad spectrum of natural and technological applications ranging from earthquakes to materials handling. Researchers working to understand frictional dynamics often develop their own software to solve specific problems with constitutive laws that include history and strain rate dependence, which has limited interdisciplinary comparison and community standards. We address these shortcomings with a Python implementation of the rate-and-state friction constitutive laws, including tools to handle multiple state variables, dynamic instability, and variations in friction rate dependence with slip velocity.

Proselint: The Linting of Science Prose, and the Science of Prose Linting

Michael Pacer — Fri, 15 Jul 2016 00:00:00 +0000

Description

Writing is notoriously hard, even for the best writers, and it's not for lack of good advice — a tremendous amount of knowledge is strewn across usage guides, dictionaries, technical manuals, essays, pamphlets, websites, and the hearts and minds of great authors and editors. But this knowledge is trapped, waiting to be extracted and transformed.

We built Proselint, a Python-based linter for prose. Proselint identifies violations of expert style and usage guidelines. Proselint is open-source software released under the BSD license and works with Python 2 and 3. It runs as a command-line utility or editor plugin (e.g., Sublime Text, Atom, Vim, Emacs) and outputs advice in standard formats (e.g., JSON). Though in its infancy – perhaps 2% of what it could be – Proselint already includes modules addressing: redundancy, jargon, illogic, clichés, sexism, misspelling, inconsistency, misuse of symbols, malapropisms, oxymorons, security gaffes, hedging, apologizing, pretension. Proselint can be seen as both a language tool for scientists and a tool for language science. On the one hand, it includes modules that promote clear and consistent prose in science writing. On the other, it measures language usage and explores the factors relevant to creating a useful linter.

QCDB Database Tools for Managing and Harmonizing Quantum Chemistry

Lori Burns — Fri, 15 Jul 2016 00:00:00 +0000

Description

Workflows in computational chemistry must manage large quantities of expensively computed and metadata-laden data, each to be accessed in its own right or recycled into complex methodologies. QCDB manages such datasets in collection, application of recognized and exploratory work-up procedures through Pandas, visualization through matplotlib, and facilitation of open-access. It is demonstrated applied to a set of nonbonded structures from the protein databank (PDB).

QIIME 2: Self-documenting, Extensible, and Reproducible Microbiome Analysis in Python 3.

Greg Caporaso — Fri, 15 Jul 2016 00:00:00 +0000

Description

We present the initial alpha release of QIIME 2, a Python 3 framework supporting interactive analysis and visualization of microbiomes on diverse high-performance computing resources; arbitrary interface development and platform integration; and a plugin system with automatic decentralized provenance tracking.

Reinventing the whl: New Developments in the Upstream Python Packaging Ecosystem

Nath — Fri, 15 Jul 2016 00:00:00 +0000

Description

Pip, wheels, and setuptools are the standard tools for installing, distributing, and building Python packages -- which means that if you're a user or package author then you're probably using them at least some of the time, even though when it comes to handling scientific packages, they've traditionally been a major source of pain. Fortunately, things have been getting better! In this talk, I'll describe how members of the scientific Python community have been working with upstream Python to solve some of the worst issues, and show you how to build and distribute binary wheels for Linux users, build Windows packages without MSVC, use wheels to handle dependencies on non-Python libraries like BLAS or libhdf5, plus give the latest updates on our effort to drive a stake through the heart of setup.py files and replace them with something better.

Scaling Up and Out Programming GPU Clusters with Numba and Dask

Siu Kwan Lam — Fri, 15 Jul 2016 00:00:00 +0000

Description

In this talk, we show how Python, Numba, and Dask can be used for GPU programming that easily scales from your workstation to a cluster, and can be controlled entirely from a Jupyter notebook. We will describe how the Numba JIT compiler can be used to create and compile GPU calculations entirely from the Python interpreter, and how the Dask task scheduling system can be used to farm these calculations out to a GPU cluster. Using an image processing example application, we will show how these two projects make it easy to iterate and experiment with algorithms on large data sets. Finally, we will conclude with tips and tricks for working with GPUs and distributed computing.

SymEngine A Fast Symbolic Manipulation Library

Ondřej Čertík — Fri, 15 Jul 2016 00:00:00 +0000

Description

The goal of SymEngine is to be the fastest C++ symbolic manipulation library (opensource or commercial), compatible with SymPy, that can be used from many languages (Python, Ruby, Julia, ...). We will present the current status of development, how things are implemented internally, why we chose C++, benchmarks, and examples of usage from Python (SymPy and Sage), Ruby and Julia.

SymPy Code Generation

Aaron Meurer — Fri, 15 Jul 2016 00:00:00 +0000

Tell Me Something I Don't Know: Analyzing OkCupid Profiles

Matar Haller — Fri, 15 Jul 2016 00:00:00 +0000

Description

In this talk, we present an approach for combining natural language processing with machine learning in order to explore the relationship between free text self-descriptions and demographics in OkCupid profile data. We discuss feature representation, clustering and topic modeling approaches, as well as feature selection and modeling strategies. We find that we can predict a user's demographic makeup based on their user essays, and we conclude by sharing some unexpected insights into deception.

Additional talk materials here:

https://github.com/juanshishido/okcupid

https://github.com/juanshishido/scipy_proceedings

The FOSSEE Python Project

Prabhu Ramachandran — Fri, 15 Jul 2016 00:00:00 +0000

What's new in Spyder 3.0

Carlos Cordoba — Fri, 15 Jul 2016 00:00:00 +0000

Description

Our aim in this talk is to present the new features available in the next major version of Spyder, the Scientific PYthon DEvelopment EnviRonment. This version (Spyder 3.0) represents almost two years of development and brings important characteristics that we would like to introduce to the SciPy community. Among them we can find: the ability to create and install third-party plugins, improved projects support, syntax highlighting and code completion for all programming languages supported by Pygments, a new file switcher (similar to the one present in Sublime Text), code folding for the Editor, Emacs keybindings for the entire application, a Numpy array graphical builder, and a fully dark theme for the interface.

Working towards all the Geophysics, but Backwards

Rowan Cockett — Fri, 15 Jul 2016 00:00:00 +0000

Description

Geophysical inversions are tools for constructing models of the subsurface (images) given a finite amount of data. SimPEG (http://simpeg.xyz) is an effort to synthesize geophysical forward and inverse methodologies into a consistent framework. We will show seven geophysical methods based around a diamond exploration case study, combining the results to drive a more informed decision. Slides may be found here: https://docs.google.com/presentation/d/1O6A85QwnnAibm7CsV2_VZ95HcRsnLyvPQIRYIz_K5xg/edit#slide=id.g15d5208fb1_2_246

A Simulation Framework for Studying the Code Acquisition and Tracking Functions of a Global Positioning System (GPS) Receiver

Mark Wickert — Thu, 14 Jul 2016 00:00:00 +0000

Description

In this talk a Python-based simulation framework is described that implements the waveform level signal processing needed to acquire and track the ranging signal of a global positioning system (GPS) satellite. This framework was developed Fall 2015 as an end-of-semester project for a digital signal processing course taken by electrical engineers. By design, GPS signals lie on top of one another, but are separable by virtue of a unique code and nearly orthogonal code assigned to each satellite. The key to position determination is the time difference of arrival (TDOA) of each of the satellite signals at the user receiver. A high precision clock maintains timing accuracy among the satellites. One of the most important tasks of the user receiver is to acquire and track the ranging code of three or more satellites in view at a given time. The framework allows the user to first explore a receiver for a single satellite signal. Object oriented Python then makes it easy to extend the receiver to processing multiple satellite signals in parallel. The source of signals used in the framework is either simulation or a low-cost (~$20) software defined radio dongle known as the RTL-SDR. With the RTL-SDR signals are captured from a GPS patch antenna, fed to the RTL-SDR, and then via USB captured into Python as a complex ndarray. The computer simulation project that utilizes the framework has the students performing a variety of simulation tasks, start from a single channel receiver building up to a four channel receiver with signal impairments present. As developed Fall 2015 the project utilizing this framework is entirely computer simulation based, but the ability to use real signals captured from the RTL-SDR, opens additional capability options. Making use of these signals is non-trival, as additional signal processing is needed to estimate the Doppler frequency error and if the data bits are to be recovered, the L1 signal carrier phase needs to be tracked. These aspects of the framework as currently under development (mid Spring 2016) for a communications theory course.

A String Theorist's Journey with Python

Chan Park — Thu, 14 Jul 2016 00:00:00 +0000

Analysis and Visualization of 3D Data with yt

Matthew Turk — Thu, 14 Jul 2016 00:00:00 +0000

Description

yt is a Python package designed for domain-specific inquiry of volumetric data, licensed under the BSD license and available at yt-project.org. Utilizing numerous components of the scientific Python ecosystem, it is able to ingest data from numerous different sources from domains such as astrophysics, nuclear engineering, weather and climate, oceanography, and seismology. Building on top of a parallelized framework for data selection, analysis, processing and visualization, inquiry can be driven based on relevant, physical quantities rather than those specific to data formats. I will describe recent advances in the yt 3.0 series, including support for particle, octree, patch and unstructured mesh datasets; interactive and batch volume rendering using both software and OpenGL backends; semantically-rich ontologies of fields, derived quantities and affiliated units (powered by sympy); user-defined kernel estimates for density; support for visualization in non-Cartesian domains; and a flexible chunking system for data IO. I will describe some of the non-astrophysics domains that yt has been applied to, and the infrastructure implemented to support that. Finally, I will describe the community-driven approach taken to designing, developing and implementing new features, and describe some of the challenges this has presented in the context of scientific software developers.

Analyzing and Manipulating Data with Pandas Beginner

Jonathan Rocher — Thu, 14 Jul 2016 00:00:00 +0000

Description

Notebooks and other materials for this tutorial may be found here: https://github.com/jonathanrocher/pandas_tutorial

Bootstrapping an Open Source Library: How MetPy Got Up and Running with Lazy Developers

Ryan May — Thu, 14 Jul 2016 00:00:00 +0000

Description

MetPy is an open-source Python package for meteorology, providing domain-specific tools for reading data, performing calculations, and visualizing data that we have recently started developing actively. In order to keep code working and in good shape, and minimize the accrual of technical debt, the project developers have focused on making use of many cloud services to automate the development of the project. These services accomplish: static code analysis, continuous integration testing, automated documentation builds, and automated releases. We will present our experiences of how these services have helped with development, as well as challenges that presented themselves, with the goal to encourage others to use those to support their own development efforts.

Communicating Model Results

Bargava Subramanian — Thu, 14 Jul 2016 00:00:00 +0000

Description

For a data scientist building predictive models, the following are important:

How good is the model ?
How good is it compared to competing/alternate models?
Is there a way to identify what worked in the models built so far, to leverage it to build something even better?

The stakeholder/end-user who finally uses the output from the model, for whom the ML process is mostly black-box, is concerned with the following:

How to trust the model output?
How to understand the drivers?
How to do what-if analysis?

The unifying theme that could answer most of the above questions is visualization. The biggest challenge is to find a way to visualize the model, the model fitting process and the impact of drivers. This talk summarizes the learnings and key takeaways when communicating model results.

Composable Multi Threading for Python Libraries

Anton Malakhov — Thu, 14 Jul 2016 00:00:00 +0000

Description

Multi-processing parallelism in Python might be unacceptable due to cache-inefficiency and memory overhead. On the other hand, multi-threaded parallelism with Python suffers from the GIL but when it comes to numeric computations, most of the time is spent in native codes where the GIL can easily be released. This is why modules such as Dask and Numba use multi-threading to greatly speed up the computations. But being used together in a nested way, e.g. when a Dask task calls Numba's threaded ufunc, it leads to the situation where there are more active software threads than available hardware resources. This situation is called over-subscription and it leads to inefficient execution due to frequent context switches, thread migration, broken cache-efficiency, and finally to a load imbalance when some threads finished their work but others are stuck along with the overall progress.

Another example is Numpy/Scipy when they are accelerated using Intel Math Kernels Library (MKL) like the ones shipped as part of Intel Distribution for Python. MKL is usually threaded using OpenMP which is known for not easily co-existing even with itself. In particular, OpenMP threads keep spin-waiting after the work is done -- which is usually necessary to reduce work distribution overhead for the next possible parallel region. But it plays badly with another thread pool because while OpenMP worker threads keep consuming CPU time in spin-waiting, the other parallel work like Numba's ufunc cannot start until OpenMP threads stop spinning or are preempted by the OS.

And the worst case is also connected to usage of OpenMP when a program starts multiple parallel tasks and each of these tasks ends up executing an OpenMP parallel region. This is quadratic over-subscription which ruins multi-threaded performance.

Our approach to solve these co-existence problems is to share one thread pool among all the necessary modules and native libraries so that one task scheduler will take care of this composability issue. Intel Threading Building Blocks (TBB) library works as such a task scheduler in our solution. TBB is a wide-spread and recognized C++ library for enabling multi-core parallelism. It was designed for composability, nested parallelism support, and avoidance of over-subscription from its early days. Thus we implemented a Python module which integrates TBB with Python, it is already available as part of Intel Distribution for Python and on Intel channel for conda users. I will show how to enable it for Numpy/Scipy, Dask, Numba, Joblib, and other threaded modules and demonstrate the performance benefits it brings.

Dask Parallel and Distributed Computing

Matthew Rocklin — Thu, 14 Jul 2016 00:00:00 +0000

Description

Dask is a pure Python library for parallel and distributed computing. Last year Dask parallelized NumPy and Pandas computations on multi-core workstations. This year we discuss using Dask to design custom algorithms and execute those algorithms efficiently on a cluster. This talk discusses Pythonic APIs for parallel algorithm development as well as strategies for intuitive and efficient distributed computing. We discuss recent results in machine learning and novel scientific applications.

Data Science is Software

Peter Bull — Thu, 14 Jul 2016 00:00:00 +0000

Description

Materials for this tutorial may be found here: https://github.com/drivendata/data-science-is-software

datreant: Persistent, Pythonic Trees for Heterogeneous Data

Sean Seyler — Thu, 14 Jul 2016 00:00:00 +0000

Description

In science the file system often serves as a de facto database, with directory trees being the zeroth-order scientific data structure. But it can be tedious and error prone to work directly with the file system to retrieve and store heterogeneous data sets. datreant makes working with directory structures and files Pythonic with Treants: specially marked directories with distinguishing characteristics that can be discovered, queried, and filtered. Treants can be manipulated individually and in aggregate, with mechanisms for granular access to the directories and files in their trees. Disparate data sets stored in any format (CSV, HDF5, NetCDF, Feater, etc.) scattered throughout a file system can thus be manipulated as meta-data sets of Treants. datreant is modular and extensible by design to allow specialized applications to be built on top of it, with MDSynthesis as an example for working with molecular dynamics simulation data. http://datreant.org/

Experiments as Iterators asyncio in Science

Daniel Allan — Thu, 14 Jul 2016 00:00:00 +0000

Description

"nA key challenge to reproducible data collection is capturing and organizing metadata without constraining flexibility and improvisation. u201cBlueskyu201d is a data collection framework designed to solve this problem. It communicates with hardware through a high-level interface and performs data collection, capturing data and rich metadata for streaming, live analysis.n n The project is developed at the National Synchrotron Light Source II --- a Dept. of Energy X-ray user facility. The X-ray beam is used by internal scientific staff and external visitors from academia and industry. These users employ bluesky in a broad range of experiments, ranging from well-defined, established techniques to ad hoc, improvised experiments.n n Bluesky expresses an experimental procedure as an iterable. Each element in the iterable specifies a granular step: u201cMove motor X; read detector Y; ...u201d. Bluesky supervises the execution of each step, handling common supervisory tasks: monitoring for problems, recovering from interruptions, safely cleaning up. In hardware control, the devil is in the details. Bluesky handles many of these details for the user, separating them from the scientific logic of the experimental procedure.n n While executing the procedure, bluesky collates all measurements and metadata into Python dictionaries with a flexible schema. These data u201cdocumentsu201d are created in a streaming fashion during the experiment and dispatched to user-defined functions. They can be printed, plotted, written into a database, broadcast as JSON documents, or fed into a real-time processing pipeline.n n Bluesky employs Python language features not as commonly used in the scipy community: generators, coroutines, and the asyncio event loop. For example, generators provide a parsimonious syntax for expressing sequential steps of an experiment. Coroutines can express adaptive logic, such as spacing measurements adaptively in response to the local slope, concentrating on regions of high variability. The asyncio event loop manages hardware control, data collection, visualization, and light-weight analysis in a single process. Wherever possible, bluesky relies on core language features and built-in data structures, avoiding a proliferation of special classes or a sprawling vocabulary.n n By integrating cleanly with the scipy stack, bluesky empowers scientists to build sophisticated experimental control logic. While currently deployed for X-ray experiments at NSLS-II, bluesky is developed in the open and should be useful for scientific experiment control in any context. It is thoroughly documented at nsls-ii.github.io/bluesky. The source code is available at github.com/NSLS-II/bluesky.n nTalk co-authers: Thomas Caswell, Brookhaven National Lab, Kenneth Lauer, Brookaven National Lab."

Getting More from Your Core: Processing & Analysis of Well Core CT Data

Brendon Hall — Thu, 14 Jul 2016 00:00:00 +0000

Description

A Python-based platform for processing and analyzing data from core CT scans will be presented. This dataset is a high resolution 3D dataset of compositional and textural information. In raw form, this data contains artifacts and is in a form unsuitable for analysis. Once the data has been cleaned, it can be processed to detect features such as beds, laminae and dip angle. It can be combined with high resolution core photographs and well logs. Machine learning algorithms can use the CT data as a feature set to perform facies classification.

Give Your Data an Entrance Exam: Tools from Psychometrics for Data Quality Evaluation

Katie Malone — Thu, 14 Jul 2016 00:00:00 +0000

Description

We discuss a means by which item response theory (IRT), originally developed as a psychometric tool for assessing a person's intellectual or academic ability given their performance on a standardized test, can be used as a data quality tool. Assuming that a dataset has an underlying "ability" to train predictive models (where the ability is specific to the type of dependent variable being predicted), we build many models on top of a variety of datasets to simultaneously assess the best dataset for a given dependent variable as well as which cases are the most "difficult" for a dataset to predict correctly. The product of this work is an understanding of both which predictions are the "hardest" to get correct for any dataset, as well as which dataset is expected to give the best predictions on a new dependent variable.

The first step in this study is to build a laboratory in which many related models can be trained and validated, reproducibly and in a self-documenting way. By running many models that look at related dependent variables, for example, a number of variables meant to predict different aspects of political behavior, we can characterize a baseline expected performance for any new model similar to those already built. We call this suite of related models a market basket, after the terminology and methodology used by economists to summarize the state of a market.

Then, when we investigate new data sources or formats, we have a well-defined process for determining whether the new data makes the models better--we re-build our market basket, and compare the results with the new data to the results without it (performance, model build time, data storage constraints) to assess the quality of our data in a way that is driven by the models and data itself.

An interesting question is how to assess whether a given dataset or feature is "better" for a given basket of models. An interesting idea comes to us from the field of psychometrics, which uses a set of tools called item response theory to assess exams (such as the SAT and GRE) and use exams to rank students by intellectual or academic ability.

Borrowing the terminology of IRT, we draw the analogy that a dataset is like a student (it has an inherent capability to accomplish certain tasks, like building good models), a single model prediction is a test question, and full set of test predictions is an exam. IRT parameterizes both the (unknown) student ability and the (also unknown) test question difficulty, and uses the EM algorithm to simultaneously solve for the parameters of both quantities at once. This allows a researcher to know both how "smart" a dataset is for solving a given basket of models, as well as rank-order "exam questions" (model predictions) by difficulty. The result is a single methodology with applications for both data quality and assessing the difficulty of making a given prediction (useful for e.g. outlier identification).

Governing Open Source Projects at Scale: Lessons from Wikipedia's Growing Pains

Stuart Geiger — Thu, 14 Jul 2016 00:00:00 +0000

GT Py : Accelerating NumPy programs on CPU&GPU w/ Minimal Programming Effort

Chi Luk — Thu, 14 Jul 2016 00:00:00 +0000

Description

GT-Py is a newly developed just-in-time compiler that can offload NumPy code to hardware accelerators with relatively little programming effort. It lets programmers add pragmas to a Python program to specify what need to be offloaded, without writing the actual offloading code. By generating OpenCL code, GT-Py can run on a variety of accelerators including GPUs from different vendors, multicore CPUs, and potentially FPGAs. Experimental results demonstrate that significant performance gains, as much as over 9000x faster than the Python interpreter execution, can be obtained by adding only a couple of pragmas to the NumPy program. GT-Py supports both Python 2.7 and Python 3.4+. It will be available to public use for free.

High Quality, High Performance Clustering with HDBSCAN

Leland McInnes — Thu, 14 Jul 2016 00:00:00 +0000

Description

Data clustering is a powerful tool for data analysis. It can be particularly useful in exploratory data analysis for helping to summarize and give intuition about a dataset. Despite it's power clustering is used for this task far less frequently than it could be. A plethora of options for clustering algorithms exist, and we will provide a survey of some of the more popular options, discussing their strengths and weaknesses, particularly with regard to exploratory data analysis. Our focus, however, is on a relatively new algorithm that appears to be the best equipped to meet the needs of exploratory data analysis: HDBSCAN* has the strengths of density based algorithms, has a small robust set of parameters, and with suitable implementation can be made highly scalable to large datasets. We will discuss how the algorithm works, taking a few different perspectives, and explain the techniques used for a high performance implementation. Finally we'll discuss ways to extend the algorithm, drawing on ideas from topological data analysis.

HoloViews Let your Data Reveal Itself

Phllip Rudiger — Thu, 14 Jul 2016 00:00:00 +0000

HOPE: A Python Just-In-Time Compiler for Astrophysical Computations

Joel Akeret — Thu, 14 Jul 2016 00:00:00 +0000

How to “Scrape” Together a Great Dataset Using Things You Find on the Internet Using Python & SciP

Deborah Hanus — Thu, 14 Jul 2016 00:00:00 +0000

Description

Using Jupyter notebooks and scikit-learn, you’ll predict whether a movie is likely to win an Oscar or be a box office hit. I’ll walk through the most important steps of creating an effective dataset using information that you find on the Internet: asking a question your data can answer, writing a web scraper, and answering those questions using nothing but Python libraries and data from the Internet. To illustrate how these steps fit together, I walk through building a dataset from IMDB data and use it to predict what makes a winning Oscar movie. To preview our findings, check out this trailer and the github repository!

JupyterHub as an Interactive Supercomputing Gateway

Michael Milligan — Thu, 14 Jul 2016 00:00:00 +0000

Description

At the Minnesota Supercomputing Institute we are exploring ways to provide the immediacy and flexibility of interactive computing within the batch-scheduled, tightly controlled world of traditional cluster supercomputing. As Jupyter Notebook has gained in popularity, the steps needed to use it within such an environment have proven to be a barrier to entry even as increasingly powerful Python tools have developed to take advantage of large computational resources. JupyterHub to the rescue! Except out of the box, it doesn't know anything about resource types, job submission, and so on. We developed BatchSpawner and friends as a general JupyterHub backend for batch-scheduled environments. In this talk I will walk through how we have deployed JupyterHub to provide a user-friendly gateway to interactive supercomputing.

Keynote: Domain - Specific Languages to Modern Processors

Andreas Klöckner — Thu, 14 Jul 2016 00:00:00 +0000

Large Scale Geospatial Analytics with Python, Spark, and Impala

Evan Wyse — Thu, 14 Jul 2016 00:00:00 +0000

Description

We harnessed the power of three different computing platforms, Spark, Impala, and scientific python, to perform geospatial analysis on mobile phone users. We will discuss data processing techniques for comparing billions of user locations per day with millions of places of interest, easily extractible insights, and methodologies for estimating impacts of treatment on these movement patterns. Our workflow has potential for application for other use cases involving geospatial movement of populations.

Lightning Talks 2016-07-13

Various speakers — Thu, 14 Jul 2016 00:00:00 +0000

MDAnalysis: A Python Package for the Rapid Analysis of Molecular Dynamics Simulations

Oliver Beckstein — Thu, 14 Jul 2016 00:00:00 +0000

Description

MDAnalysis (http://mdanalysis.org) is an object-oriented library for structural and temporal analysis of molecular dynamics (MD) simulation trajectories and individual protein structures. MD simulations of biological molecules have become an important tool to elucidate the relationship between molecular structure and physiological function. Simulations are performed with highly optimized software packages on HPC resources but most codes generate output trajectories in their own formats so that the development of new trajectory analysis algorithms is confined to specific user communities and widespread adoption and further development is delayed. The MDAnalysis library addresses this problem by abstracting access to the raw simulation data and presenting a uniform object-oriented Python interface to the user. It thus enables users to rapidly write code that is portable and immediately usable in virtually all biomolecular simulation communities. The user interface and modular design work equally well in complex scripted workflows, as foundations for other packages, and for interactive and rapid prototyping work in IPython/Jupyter notebooks, especially together with molecular visualization provided by nglview [1] and time series analysis with pandas [2]. MDAnalysis is written in Python and Cython and uses NumPy arrays for easy interoperability with the wider scientific Python ecosystem. It is widely used and forms the foundation for more specialized biomolecular simulation tools. MDAnalysis is available under the GNU General Public License v2.

[1] https://github.com/arose/nglview

[2] http://pandas.pydata.org/

MONTE Python for Deep Space Navigation

Jonathon Smith — Thu, 14 Jul 2016 00:00:00 +0000

Description

The Mission Analysis, Operations, and Navigation Toolkit Environment (MONTE) is the Jet Propulsion Laboratory's (JPL) signature astrodynamic computing platform. It was built to support JPL's deep space exploration program, and has been used to fly robotic spacecraft to Mars, Jupiter, Saturn, Ceres, and many solar system small bodies. At its core, MONTE consists of low-level astrodynamic libraries that are written in C++ and presented to the end user as an importable Python language module. These libraries form the basis on which Python-language applications are built for specific astrodynamic applications, like trajectory design and optimization, orbit determination, flight path control, and more. This talk gives a brief history of the project, shows some examples of MONTE in action, and relates the stories of its greatest successes.

MPCite: Continuous and High-throughput Allocation of Digital Object Identifiers for Calculated and Contributed Data in the Materials Project

Patrick Huck — Thu, 14 Jul 2016 00:00:00 +0000

Description

We introduce 'MPCite' which enables the continuous request, validation, and dissemination of Digital Object Identifiers (DOIs) for all inorganic materials currently available in the Materials Project (https://www.materialsproject.org/). It provides our users with the necessary software infrastructure to achieve a new level of reproducibility in their research: It allows for the convenient and persistent citation of our materials data in online and print publications and facilitates sharing amongst collaborators. We also demonstrate how we extend the use of MPCite to non-core database entries such as theoretical and experimental data contributed through 'MPContribs' or suggested by the user for calculation via the “MPComplete' service. We expect MPCite to be easily extendable to other scientific domains where the number of data records demands high-throughput and continuous allocation of DOIs.

NumPy Beginner

Alexandre Chabot LeClerc — Thu, 14 Jul 2016 00:00:00 +0000

Description

Materials for this tutorial may be found here: https://github.com/enthought/Numpy-Tutorial-SciPyConf-2016

Out with the Old and in with the New: Embedding Python in Old Fortran HPC Code

Brendan Smithyman — Thu, 14 Jul 2016 00:00:00 +0000

Description

Seismic Full-Waveform Inversion (FWI) is a field with decades-old academic codes. The program Fullwv started life in 1989, written in Fortran 77 by numerous researchers. We detail how we extended Fullwv and embedded a new Python-based FWI package called Zephyr. We override or replace portions of the Fortran source code with Python, to enable rapid development of new algorithms and methods.

Processing a Petabyte of Planetary Pixels with Python

Samuel Skillman — Thu, 14 Jul 2016 00:00:00 +0000

Description

In the last 40 years over a petabyte of publicly available earth observation imagery has been produced. In the near future, many petabytes of imagery per year will become available from a combination of public satellite missions and private satellite constellations. At the same time, commercial cloud providers are competing to provide the lowest cost alternative to on-premise compute capabilities. By combining the dramatic rise in available imagery with low cost of high performance storage, network, and compute capabilities, we have a unique opportunity to combine analysis techniques from remote sensing, machine learning algorithms, and scalable compute infrastructure. Combined, they allow for global scale investigations into how our planet is changing.

Here we will report on how we leverage the commercial cloud to generate a tiled spatio-temporal mosaic of the Earth and how it enables fast iteration for the development of both traditional model based predictions and machine learning algorithms. As part of our effort, we have processed, in less than 24 hours, over a petabyte of compressed raw data from the combination of the US Landsat and MODIS programs, totalling nearly 3 petapixels. We will detail the challenges and benefits to moving from traditional remote sensing workbenches to the commercial cloud, with particular emphasis on the benefits for researchers.

PySPH: A Reproducible and High performance Framework for Smoothed Particle Hydrodynamics

Prabhu Ramachandran — Thu, 14 Jul 2016 00:00:00 +0000

Description

SPH (Smoothed Particle Hydrodynamics) is a general purpose technique to numerically compute the solutions to partial differential equations. The method is grid-free and uses particles to discretize the various properties of interest. The method is Lagrangian and particles are moved with the local velocity. The method was originally developed for astrophysical problems (compressible gas-dynamics) but has since been extended to simulate incompressible fluids, solid mechanics, free-surface problems and a variety of other problems. The SPH method is relatively easy to implement. This has resulted in a large number of schemes and implementations proposed by various researchers. It is often difficult to reproduce published results due to the variety of implementations. While a few standard packages like (SPHysics, DualSPHysics, JOSEPHINE etc.) exist, they are usually tailor-made for particular applications and are not general purpose. Our group has been developing PySPH (http://pysph.bitbucket.org) over 5 years. PySPH is open source, and distributed under the new BSD license. Our initial implementation was based on Cython (http://cython.org) and also featured some parallelization using MPI. Unfortunately, this proved difficult to use as users were forced to implement most of their code in Cython. It was felt that we might as well have implemented it all in C++ and exposed a Python interface to that. In early 2013, we redesigned PySPH so that users were able to implement an entire simulation using pure Python. This was done by auto-generating HPC code from the pure Python code that users provided. This version ended up being faster than our original Cython implementation. Since we were auto-generating code, with a bit of additional effort it was possible to support OpenMP. PySPH has thus matured into an easy to use, yet high-performance framework where users can develop their schemes in pure Python and yet obtain performance close to that of a lower-level language implementation. PySPH also supports running on a cluster of machines via MPI. This is seamless and a serial script using PySPH can be run with almost no changes using MPI. PySPH features a reasonable test-suite and we use continuous integration servers to test it on Linux and Windows. Our documentation is hosted on http://pysph.readthedocs.org. The framework supports several of the standard SPH algorithms. A suite of about 30 examples are provided and are shipped as part of the sources and installed when a user does a pip install for example. The examples are written in a way that makes it easy to extend and also perform comparisons between schemes. These features make PySPH well suited for reproducible numerical work. In fact one of our recent papers was written such that every figure in the paper is automatically generated using PySPH. In this talk we discuss the use, design, and implementation of PySPH. We believe that this talk would be suitable for either the HPC track or the general track or even the engineering minisymposium.

PyTeCK: A Python-based automatic testing package for chemical kinetic models

Kyle Nie — Thu, 14 Jul 2016 00:00:00 +0000

Description

Combustion simulations require detailed chemical kinetic models to predict fuel oxidation, heat release, and pollutant emissions.

These models are typically validated using qualitative rather than quantitative comparisons with limited sets of experimental data.

This work introduces PyTeCK, an open-source Python-based package for automatic testing of chemical kinetic models. Given a model of interest, PyTeCK automatically parses experimental datasets encoded in an XML format, validates the self-consistency of each dataset, and performs simulations for each experimental datapoint. It then reports a quantitative metric of the model's performance, based on the discrepancy between experimental and simulated values and weighted by experimental variance. The initial version of PyTeCK supports shock tube and rapid compression machine experiments that measure autoignition delay. PyTeCK relies on several packages in the SciPy stack and greater scientific Python ecosystem. In addition to providing an easy-to-use, automated tool for evaluating chemical kinetic model performance, a secondary objective of PyTeCK is to encourage greater openness and reproducibility in combustion research.

Python and R Together at Last: Writing Cross Language Tools

Bill Lattner — Thu, 14 Jul 2016 00:00:00 +0000

Description

Both Python and R boast large data science communities. Each have developed a fantastic collection of packages from reading/writing data to plotting and visualization. Unfortunately, some tools are only available in one language or the other, but not both. Python and R provide relatively simple mechanisms for interacting with C, C++, and Fortran. There are many tools that take advantage of this interoperability. While not a simple matter, developing data science tools in these low level languages and providing Python and R wrappers allows code reuse between languages, speed benefits notwithstanding. In this talk we will discuss strategies and lessons learned from porting existing packages to Python from R and writing cross language tools from scratch.

Python at the Intersection of Data Science, Machine Learning & Cyber Anomaly Detection

Randy Paffenroth — Thu, 14 Jul 2016 00:00:00 +0000

Description

This talk will focus on the use of Python, scikit-learn, NumPy, SciPy, and pandas in Data Science and machine learning with a focus on cyber anomaly detection. The presentation will focus on how Python facilitates all stages of such analysis including data gathering, analytics, and scaling to large data sets.

Reproducible, One Button Workflows with the Jupyter Notebook & Scons

Jessica Hamrick — Thu, 14 Jul 2016 00:00:00 +0000

Description

What is the best way to develop analysis code in the Jupyter notebook, while managing complex dependencies between analyses? In this talk, I will introduce nbflow, which is a project that integrates a Python-based build system (SCons) with the Jupyter notebook, enabling researchers to easily build sophisticated, complex analysis pipelines entirely within notebooks while still maintaining a "one-button workflow" in which all analyses can be executed, in the correct order, from a single command. I will show how nbflow can be applied to existing analyses and how it can be used to construct an analysis pipeline stretching the entire way from data cleaning, to computing statistics, to generating figures, and even to automatically generating LaTeX commands that can be used in publications to format results without the risk of copy-and-paste error.

Running Python Apps in the Browser

Almar Klein — Thu, 14 Jul 2016 00:00:00 +0000

Description

The web is becoming an increasingly important place to publish research findings, but JavaScript is a language that is broken by design, and Pythonistas seem particularly repelled by the language.

Flexx is a tool to create web apps, for which the client-side is completely implemented in Python and transpiled to JavaScript. It’s easy to extend Flexx’ functionality by writing Python classes, which will be demonstrated in this talk.

Sharing Reproducible Environments with Binder

Andrew Osheroff — Thu, 14 Jul 2016 00:00:00 +0000

Description

Binder (http://mybinder.org) is a service that bundles GitHub repositories with code, Jupyter notebooks, and data into reproducible, executable environments that can be launched instantaneously in the browser with the click of a button. Under the hood, Binder uses simple and flexible dependency specifications to build Docker images on demand, and then launches and schedules them across a public Kubernetes cluster. In this talk, I’ll describe in detail how Binder works, and highlight some exciting use cases. I’ll then describe several future directions for the project, including handling larger datasets, lowering barriers for environment specification, and supporting custom deployments with user-provided computing resources.

Time Series Analysis with Python Intermediate

Aileen Nielsen — Thu, 14 Jul 2016 00:00:00 +0000

Description

Tutorial materials for the Time Series Analysis tutorial including notebooks may be found here: https://github.com/AileenNielsen/TimeSeriesAnalysisWithPython

UConnRCMPy: Python-based Data Analysis for Rapid Compression Machines

Bryan Weber — Thu, 14 Jul 2016 00:00:00 +0000

Description

The ignition delay of a fuel/air mixture is an important quantity in designing combustion devices, and these data are also used to validate computational kinetic models for combustion. One of the typical experimental devices used to measure the ignition delay is called a Rapid Compression Machine (RCM). This work presents UConnRCMPy, an open-source Python package to process experimental data from the RCM at the University of Connecticut. Given an experimental measurement, UConnRCMPy computes the thermodynamic conditions in the reactor of the RCM during an experiment along with the ignition delay. UConnRCMPy relies on several packages from the SciPy stack and the broader scientific Python community. UConnRCMPy implements an extensible framework, so that alternative experimental data formats can be incorporated easily. In this way, UConnRCMPy improves the consistency of RCM data processing and enables reproducible analysis of the data.

Using Open Source Tools to Refactor Geoscience Education

Lindsey Heagy — Thu, 14 Jul 2016 00:00:00 +0000

Description

How do we communicate fundamental concepts in a reproducible, actionable form? How do we put numerical simulation tools in the hands of undergraduate students? These are questions we have been exploring in the development of http://geosci.xyz/, a web-based resource in geophysics that leverages the geophysical software package SimPEG, Sphinx documentation, Jupyter notebooks and Binders to make examples and explanations that are reproducible and interactive.

Psi4: A Case Study on Modernizing & Modularizing Quantum Chemistry w/ Python & C++

Daniel Smith — Wed, 13 Jul 2016 00:00:00 +0000

Description

The Psi4 open-source quantum chemistry project was written from the ground up with Python and C++ and will be used as an example on how to modernize and modularize programs that are typically decades old HPC Fortran programs. The Python interface allows novice users to quickly create complex instructions through the common Python syntax. In addition, developers gain access to tailored C++ libraries that allow entirely new methodologies to be written using only Python while simultaneously having the ability to run the process efficiently on tens to hundreds of thousands of processors.

Simulating Robot, Vehicle, Spacecraft, and Animal Motion w/ Python Advanced

Jason K. Moore — Mon, 11 Jul 2016 00:00:00 +0000

Description

In this tutorial, attendees will learn how to derive, simulate, control, and visualize the motion of a multibody dynamic system with Python tools. These methods and techniques play an important role in the design and understanding of robots, vehicles, spacecraft, manufacturing machines, human motion, etc. In particular, the attendees will develop code to simulate the motion of a human balancing while standing. This is an advanced tutorial and domain specific but we have found that a broad audience enjoys the topic. Attendees should be familiar with the basics of the SciPy stack, in particular NumPy, SciPy, SymPy, and IPython and have some familiarity with classical mechanics. Details In this tutorial, attendees will learn how to derive, simulate, and visualize the motion of a multibody dynamic system with Python tools. The tutorial will demonstrate an advanced symbolic and numeric pipeline for a typical multibody simulation problem. By the end, the attendees will have developed code to simulate the uncontrolled and controlled motion of a human balancing while standing. We will highlight the derivation of realistic models of motion with the SymPy Mechanics package. Then we will cover code generation to create fast numerical functions that can be used to simulate the system. The simulation results will be plotted and visualized with a 3D WebGL browser based tool. Finally, we will use packages for optimal control to develop a controller that mimics human standing and visualize these results.

Materials for this tutorial may be found here: https://github.com/pydy/pydy-tutorial-human-standing

Symbolic Computation with Python using SymPy (Beginner)

Ondřej Čertík — Mon, 11 Jul 2016 00:00:00 +0000

Description

Materials for this tutorial are found here: https://github.com/sympy/scipy-2016-tutorial