Spark at Scale: Engineering Strategies for Data Science Workflows

September 4, 2023 | 10:43 am
Neil McCulloch, Data Science Engineer at dunnhumby
Share

This is a talk by Neil McCulloch, Data Science Engineer at dunnhumby.

Improving the performance of problematic PySpark applications can often seem like a daunting task. In this talk, I will outline a strategy for tackling these projects, delving into a case study on the performance our in-store availability reporting science, and how we have slashed runtimes in half.

This session was part of the Data Science Festival Summer School in 2023. Find out more at https://datasciencefestival.com/event…

The Data Science Festival is the place for data-driven people to come together, share cutting-edge ideas and solve real-world problems. We run monthly events, meetups and the biggest free-to-attend data festivals in the UK. Join the community at https://datasciencefestival.com/

Let’s
chat

Leave us a message using our contact form and we’ll get back to you straight away.

If you’re eager to get started, give us a call now on 01908 465 570

Thanks

for reaching out, 🙏

A member of our team will be in touch shortly to arrange our chat.

Mmm 🍪cookies!

We use cookies to make your experience on this website better, and we have a variety to choose from. Use the toggles below to customise your selection or click 'Save my cookies' to get straight to the content.