EdX

Data Engineering with Databricks (edX)

Offered by AI (Pragmatic AI Labs),
Data Engineering with Databricks (edX)

Become an expert in modern data engineering on Databricks' unified lakehouse platform. Master ETL pipelines, data transformations with Apache Spark, and Delta Lake for reliable data management.

Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.

Master Data Engineering on Databricks Lakehouse Platform

  • Learn Databricks architecture, cluster management & notebook analysis
  • Build reliable ETL pipelines with Delta Lake for data transformation
  • Implement advanced data processing techniques with Apache Spark

Course Highlights:

  • Create & scale Databricks clusters for workloads
  • Load data from diverse sources into notebooks
  • Explore, visualize & profile datasets with notebooks
  • Version control & share notebooks via Git integration
  • Read & ingest data in various file formats
  • Transform data with SQL & DataFrame operations
  • Handle complex data types like arrays, structs, timestamps
  • Deduplicate, join & flatten nested data structures
  • Identify & fix data quality issues with UDFs
  • Load cleansed data into Delta Lake for reliability
  • Build production-ready pipelines with Delta Live Tables
  • Schedule & monitor workloads using Databricks Jobs
  • Secure data access with Unity Catalog

Gain comprehensive skills in data engineering on Databricks through hands-on labs, real-world projects and best practices for the modern data lakehouse.
This course is part of the Large Language Model Operations (LLMOps) Professional Certificate.

What you'll learn

  • Use Databricks for data engineering and ML workloads
  • Create and design ML pipelines
  • Use Llamafile and other local LLMs like Mixtral

Syllabus

Module 1: Databricks Lakehouse Platform Fundamentals

  • Introduction to the Databricks Lakehouse Platform and its architecture
  • Creating, managing, and configuring clusters
  • Setting up and using Databricks with IntelliJ, RStudio, and the Databricks CLI
  • Introduction to notebooks, including execution, sharing, and multi-language support
  • Efficient data transformation with Spark SQL and the Catalog Explorer
  • Creating tables from files and querying external data sources
  • Reliable data pipelines with Delta Lake, ACID transactions, and Z-Ordering optimization

Module 2: Data Transformation and Pipelines
Automated pipelines with Delta Live Tables
Delta Live Tables components
Continuous vs triggered pipelines
Configuring Auto Loader
Querying pipeline events
End-to-end example of Delta Live
Vacuum and garbage collection
Orchestrating workloads with Databricks Jobs
Multi-task workflows and task dependencies
Viewing job history
Using dashboards
Handling failures and configuring retries
Unified data access with Unity Catalog
Catalogs vs metastores
Unity Catalog quickstart in Python
Applying object security
Best practices for catalogs, connections, and business units

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Machine Learning With Big Data (Coursera) Coursera
University of California, San Diego

Machine Learning With Big Data (Coursera)

Statistics & Data Analysis Data Science

Want to make sense of the volumes of data you have collected? Need to incorporate data-driven decisions into your process? This course provides an overview of machine learning techniques to explore, analyze, and leverage data. You will be introduced to tools and algorithms you can use to create machine learning models that learn from data, and to scale those models up to big data problems.

Nov 9th 2026
5-12 Weeks
Algorithms Machine Learning Big Data
Details
Business Intelligence with Databricks (Coursera) Coursera
Edureka

Business Intelligence with Databricks (Coursera)

Statistics & Data Analysis Data Science

Welcome to the Introduction to Business Intelligence with Databricks course. This short course provides a introduction to business intelligence (BI) and the powerful Databricks platform. In today's data-driven business landscape, understanding BI and utilizing cutting-edge tools like Databricks is essential for optimizing decision-making processes and gaining valuable insights from data.

Nov 23rd 2026
1 Week
Business Intelligence Coursera Plus Databricks
Details
Introduction to Apache Spark (edX) EdX
University of California, Berkeley

Introduction to Apache Spark (edX)

CS: Software Engineering Statistics & Data Analysis

Learn the fundamentals and architecture of Apache Spark, the leading cluster-computing framework among professionals. Spark is rapidly becoming the compute engine of choice for big data. Spark programs are more concise and often run 10-100 times faster than Hadoop MapReduce jobs. As companies realize this, Spark developers are becoming increasingly valued.

Not Available
Course Not Available
Programming Big Data Apache Spark
Details
Data Warehouse Concepts, Design, and Data Integration (Coursera) Coursera
University of Colorado System

Data Warehouse Concepts, Design, and Data Integration (Coursera)

CS: Design & Product

This is the second course in the Data Warehousing for Business Intelligence specialization. Ideally, the courses should be taken in sequence. In this course, you will learn exciting concepts and skills for designing data warehouses and creating data integration workflows. These are fundamental skills for data warehouse developers and administrators. You will have hands-on experience for data warehouse design and use open source products for manipulating pivot tables and creating data integration workflows.

Nov 9th 2026
5-12 Weeks
Business Intelligence Data Warehousing Data Warehouse
Details
Distributed Machine Learning with Apache Spark (edX) EdX
University of California, Berkeley,BerkeleyX

Distributed Machine Learning with Apache Spark (edX)

CS: Software Engineering Statistics & Data Analysis

Learn the underlying principles required to develop scalable machine learning pipelines and gain hands-on experience using Apache Spark. Machine learning aims to extract knowledge from data, relying on fundamental concepts in computer science, statistics, probability and optimization.

No sessions available
4 Weeks
Algorithms Machine Learning Exploratory Data Analysis
Details
Power Query Fundamentals (Coursera) Coursera
Corporate Finance Institute

Power Query Fundamentals (Coursera)

Economics & Finance Business

In this online Power Query Fundamentals course, we’ll explore the world of data transformation and automation. You’ll learn to extract data from multiple different sources, and transform it into layouts more suited to analysis. We will show you how to automate data connections and transformations, as well as how to extract and consolidate data from multiple files. Finally, we’ll end by looking at how to deal with common errors.

Nov 23rd 2026
5-12 Weeks
Excel Power Query Automation
Details