Rust Data Engineering (edX)

Rust for Data Engineering: Efficient, Safe, and Concurrent Data Processing

• Learn to build robust data processing systems using Rust
• Explore Rust's performance, safety, and concurrency for data tasks

This 4-week course dives deep into leveraging Rust for efficient and reliable data engineering workflows:

• Mastering Rust data structures and collections for data processing
• Leveraging Rust's safety/security features in data engineering context
• Using Rust libraries like Diesel, async, Polars, Apache Arrow
• Interfacing with data stores, REST/gRPC APIs, AWS SDK
• Designing full-fledged data pipelines and processing systems in Rust
• Hands-on projects for building data ingestion tools, ETL pipelines
• Best practices for handling large datasets, optimizing performance
• Techniques for writing safe, concurrent, and lock-free code
• Deploying and maintaining Rust-based data engineering solutions

By the end, you'll gain practical experience building high-performance, secure data systems using Rust - preparing you for real-world data challenges.
This course is part of the following programs:

• Rust Programming Professional Certificate
• Data Engineering Foundations Professional Certificate.

What you'll learn

• Leverage Rust's robust data structures and collections for efficient data manipulation.
• Use Rust's safety and security features to build reliable and secure data engineering solutions.
• Utilize Rust's libraries and tools specific to data engineering, such as Diesel, async, Polars, and Apache Arrow.
• Interface effectively with databases, data processing systems, REST and gRPC protocols, and leverage AWS SDK for cloud-based data operations in Rust.
• Design and implement comprehensive data processing systems in Rust.
• Apply the principles of concurrent programming in Rust to build high-performance data processing applications.
• Identify and mitigate common data engineering problems using Rust's unique features, like its strong type system and memory safety guarantees.
• Develop command-line applications and multi-threaded servers in Rust, focusing on efficient, safe, and concurrent processing of data.
• Create practical projects, gaining hands-on experience in Rust for data engineering.

Syllabus

\- Module 1: Rust Data Structures: Collections (18 hours)
\- 24 videos (Total 111 minutes)
\- 29 readings (Total 290 minutes)
\- 1 quiz: Rust Collections (Quiz, 30 minutes)
\- 2 assignments
\- Quiz-Getting Started With The Modern Rust Development Ecosystem (Assignment, 180 minutes)
\- Quiz-Rust Sequences and Maps (Assignment, 180 minutes)
\- 5 ungraded labs (60 minutes each)
\- Module 2: Safety, Security and Concurrency with Rust (17 hours)
\- 22 videos (Total 93 minutes)
\- 21 readings (Total 210 minutes)
\- 1 quiz: Safety, Security and Concurrency with Rust (Quiz, 30 minutes)
\- 3 assignments
\- Quiz-Rust Safety and Security Features (Assignment, 180 minutes)
\- Quiz-Security Programming with Rust (Assignment, 180 minutes)
\- Quiz-Rust Concurrency (Assignment, 180 minutes)
\- 3 ungraded labs (60 minutes each)
\- Module 3: Rust Data Engineering Libraries and Tools (13 hours)
\- 21 videos (Total 127 minutes)
\- 14 readings (Total 140 minutes)
\- 1 quiz: Quiz3: Rust Data Engineering Libraries and Tools (Quiz, 30 minutes)
\- 2 assignments
\- Quiz-Using Rust to Manage Data, Files and Network Storage (Assignment, 180 minutes)
\- Quiz-DataFrames with Rust, Python and Notebooks (Assignment, 180 minutes)
\- 3 ungraded labs (60 minutes each)
\- Module 4: Designing Data Processing Systems in Rust (13 hours)
\- 22 videos (Total 124 minutes)
\- 17 readings (Total 170 minutes)
\- 2 quizzes
\- Designing Data Processing Systems in Rust (Quiz, 30 minutes)
\- Final Course Quiz (Quiz, 30 minutes)
\- 2 assignments
\- Quiz-Getting Started with Rust Data Pipelines (Including ETL) (Assignment, 180 minutes)
\- Quiz-Using Rust and Python for LLMs, ONNX, Hugging Face, and PyTorch Pipelines (Assignment, 180 minutes)
\- 2 ungraded labs (60 minutes each)