Use Rust's power for high-performance, concurrent data pipelines - from core principles to real-world deployments.
Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.
Rust for Data Engineering: Efficient, Safe, and Concurrent Data Processing
- Learn to build robust data processing systems using Rust
- Explore Rust's performance, safety, and concurrency for data tasks
This 4-week course dives deep into leveraging Rust for efficient and reliable data engineering workflows:
- Mastering Rust data structures and collections for data processing
- Leveraging Rust's safety/security features in data engineering context
- Using Rust libraries like Diesel, async, Polars, Apache Arrow
- Interfacing with data stores, REST/gRPC APIs, AWS SDK
- Designing full-fledged data pipelines and processing systems in Rust
- Hands-on projects for building data ingestion tools, ETL pipelines
- Best practices for handling large datasets, optimizing performance
- Techniques for writing safe, concurrent, and lock-free code
- Deploying and maintaining Rust-based data engineering solutions
By the end, you'll gain practical experience building high-performance, secure data systems using Rust - preparing you for real-world data challenges.
This course is part of the following programs:
What you'll learn
- Leverage Rust's robust data structures and collections for efficient data manipulation.
- Use Rust's safety and security features to build reliable and secure data engineering solutions.
- Utilize Rust's libraries and tools specific to data engineering, such as Diesel, async, Polars, and Apache Arrow.
- Interface effectively with databases, data processing systems, REST and gRPC protocols, and leverage AWS SDK for cloud-based data operations in Rust.
- Design and implement comprehensive data processing systems in Rust.
- Apply the principles of concurrent programming in Rust to build high-performance data processing applications.
- Identify and mitigate common data engineering problems using Rust's unique features, like its strong type system and memory safety guarantees.
- Develop command-line applications and multi-threaded servers in Rust, focusing on efficient, safe, and concurrent processing of data.
- Create practical projects, gaining hands-on experience in Rust for data engineering.
Syllabus
\- Module 1: Rust Data Structures: Collections (18 hours)
\- 24 videos (Total 111 minutes)
\- 29 readings (Total 290 minutes)
\- 1 quiz: Rust Collections (Quiz, 30 minutes)
\- 2 assignments
\- Quiz-Getting Started With The Modern Rust Development Ecosystem (Assignment, 180 minutes)
\- Quiz-Rust Sequences and Maps (Assignment, 180 minutes)
\- 5 ungraded labs (60 minutes each)
\- Module 2: Safety, Security and Concurrency with Rust (17 hours)
\- 22 videos (Total 93 minutes)
\- 21 readings (Total 210 minutes)
\- 1 quiz: Safety, Security and Concurrency with Rust (Quiz, 30 minutes)
\- 3 assignments
\- Quiz-Rust Safety and Security Features (Assignment, 180 minutes)
\- Quiz-Security Programming with Rust (Assignment, 180 minutes)
\- Quiz-Rust Concurrency (Assignment, 180 minutes)
\- 3 ungraded labs (60 minutes each)
\- Module 3: Rust Data Engineering Libraries and Tools (13 hours)
\- 21 videos (Total 127 minutes)
\- 14 readings (Total 140 minutes)
\- 1 quiz: Quiz3: Rust Data Engineering Libraries and Tools (Quiz, 30 minutes)
\- 2 assignments
\- Quiz-Using Rust to Manage Data, Files and Network Storage (Assignment, 180 minutes)
\- Quiz-DataFrames with Rust, Python and Notebooks (Assignment, 180 minutes)
\- 3 ungraded labs (60 minutes each)
\- Module 4: Designing Data Processing Systems in Rust (13 hours)
\- 22 videos (Total 124 minutes)
\- 17 readings (Total 170 minutes)
\- 2 quizzes
\- Designing Data Processing Systems in Rust (Quiz, 30 minutes)
\- Final Course Quiz (Quiz, 30 minutes)
\- 2 assignments
\- Quiz-Getting Started with Rust Data Pipelines (Including ETL) (Assignment, 180 minutes)
\- Quiz-Using Rust and Python for LLMs, ONNX, Hugging Face, and PyTorch Pipelines (Assignment, 180 minutes)
\- 2 ungraded labs (60 minutes each)
