Comet Webinar: Distributed Parallel Computing with Python

Tuesday, May 14, 2019

11:00 AM - 12:00 PM PDT

This event will be held remotely.

This webinar provides an introduction to distributed computing with Python, we will show how to modify a standard Python script to use multiple CPU cores using the concurrent.futures module from the Python standard library and then the dask package. Then we will leverage dask workers running on other machines to distribute a data processing task and monitor its execution through the live dask dashboard. You will understand the difference between threads and processes, how the Global Interpreter Lock works and principles of distributed computing. All material will be available as Jupyter Notebooks.

Instructor

Andrea Zonca, Ph.D.

Senior Computational Scientist, SDSC

Andrea Zonca has a background in Cosmology, he has been working on analyzing Cosmic Microwave Background data from the Planck Satellite. In order to manage and analyze large datasets, he developed expertise in parallel programming in Python and C++. At SDSC he helps research groups in any field of science to port their data analysis pipelines to XSEDE supercomputers. Andrea is also a certified Software Carpentry instructor.