Parallel Programming in Python

The workshop will take place at Science Park 402, 1098 XH Amsterdam. Please note that lunch and drinks at the end of the workshop are included.

This is a pilot of the new version of the Parallel Programming Workshop. The Netherlands eScience worked with SURF to align the content with the high-performance computing training provided by SURF.

In this two-day intermediate-level course, we will continue to teach the basics of Parallel Programming in Python using Numba and Dask. However, the focus will now be on getting the most out of a single core in a high-performance computing facility, all from the comfort of a familiar Python interface.

Python is one of most widely used languages to do scientific data analysis, visualization, and even modelling and simulation. The popularity of Python is mainly due to the two pillars of a friendly syntax together with the availability of many high-quality libraries. The flexibility that Python offers comes with a few downsides though: code typically doesn’t perform as fast as lower-level implementations in C/C++ or Fortran, and it is not trivial to parallelize Python code to work efficiently on many-core architectures. This workshop addresses both these issues, with an emphasis on being able to run Python code efficiently (in parallel) on multiple cores.

We’ll start with learning to recognize problems that are suitable for parallel processing, looking at dependency diagrams and kitchen recipes. From then on, the workshop is highly interactive, diving straight into the first parallel programs. This workshop teaches the principles of parallel programming in Python using Dask, Numba and asyncio. More importantly, we try to give insight in how these different methods perform and when they should be used.

The workshop is based on the teaching style of the Carpentries, and learners will follow along while the instructors write the code on screen. More information can be found on the workshop website.

Cancellation and No-Show Policy

Please be advised that by signing up, you agree to our Cancellation and No-Show Policy, which states that cancellations made less than 2 workings days prior to the event will incur a no-show fee. Please read the full policy here for more details.

If you won’t be able to attend, please cancel your registration (by following the instructions here) so that we can offer your seat to someone on the waiting list.

Audience

The workshop is aimed at PhD candidates, postdocs, (assistant-)professors and other researchers and also open for research support staff and research software engineers. We offer tickets for researchers who are affiliated with Dutch research institutions. We also offer industry tickets for attendees who are not affiliated with Dutch research institutions. We do not accept registrations by Master students.

Prerequisites

The participant should be:

• familiar with basic Python: control flow, functions, NumPy
• comfortable working in Jupyter

Recommended: understand how NumPy and/or Pandas work

Syllabus

• Recognizing potential for parallelism
• Dependency diagrams
• Measuring performance
• Working with Dask arrays
• Working with Numba
• Parallel design patterns
• Delayed evaluation
• Parallel programming using asyncio