Scalable Machine Learning on Big Data using Apache Spark (Coursera)

After completing this course, you will be able to:

- gain a practical understanding of Apache Spark, and apply it to solve machine learning problems involving both small and big data

- understand how parallel code is written, capable of running on thousands of CPUs.

- make use of large scale compute clusters to apply machine learning algorithms on Petabytes of data using Apache SparkML Pipelines.

- eliminate out-of-memory errors generated by traditional machine learning frameworks when data doesn’t fit in a computer's main memory

- test thousands of different ML models in parallel to find the best performing one – a technique used by many successful Kagglers

- (Optional) run SQL statements on very large data sets using Apache SparkSQL and the Apache Spark DataFrame API.

Enrol now to learn the machine learning techniques for working with Big Data that have been successfully applied by companies like Alibaba, Apple, Amazon, Baidu, eBay, IBM, NASA, Samsung, SAP, TripAdvisor, Yahoo!, Zalando and many others.

NOTE: You will practice running machine learning tasks hands-on on an Apache Spark cluster provided by IBM at no charge during the course which you can continue to use afterwards.

Course 2 of 6 in the IBM AI Engineering Professional Certificate.

Prerequisites:

- basic python programming

- basic machine learning (optional introduction videos are provided in this course as well)

- basic SQL skills for optional content

Syllabus

WEEK 1

Introduction

This is an introduction to Apache Spark. You'll learn how Apache Spark internally works and how to use it for data processing. RDD, the low level API is introduced in conjunction with parallel programming / functional programming. Then, different types of data storage solutions are contrasted. Finally, Apache Spark SQL and the optimizer Tungsten and Catalyst are explained.

WEEK 2

Scaling Math for Statistics on Apache Spark

Applying basic statistical calculations using the Apache Spark RDD API in order to experience how parallelization in Apache Spark works

WEEK 3

Introduction to Apache SparkML

Understand the concept of machine learning pipelines in order to understand how Apache SparkML works programmatically

WEEK 4

Supervised and Unsupervised learning with SparkML

Apply Supervised and Unsupervised Machine Learning tasks using SparkML