EdX

Introduction to Designing Data Lakes on AWS (edX)

Offered by AWS,
Introduction to Designing Data Lakes on AWS (edX)

In this class, we will help you understand how to create and operate a data lake in a secure and scalable way, without previous knowledge of data science! Designing a data lake is challenging because of the scale and growth of data. This course is for professionals (Architects, System Administrators and DevOps) who need to design and build an architecture for secure and scalable Data Lake components. Students will learn about the use cases for a Data Lake and, contrast that with a traditional infrastructure of servers and storage.

Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.

Developers need to understand best practices to avoid common mistakes that could be hard to rectify. In this course we will cover the foundations of what a Data Lake is, how to ingest and organize data into the Data Lake, and dive into the data processing that can be done to optimize performance and costs when consuming the data at scale.
This course is part of the Cloud Solutions Architecture Professional Certificate.

What you'll learn

  • Where to start with a Data Lake?
  • How to build a secure and scalable Data Lake?
  • What are the common components of a Data Lake?
  • Why do you need a Data Lake and what it's value?

Syllabus

Week 1: Hello World, I mean, Hello Data Lakes!
Video: Meet the Instructors
Video: Introduction to Week 1
Video: Why Data Lakes?
Video: Characteristics of a Data Lake
Video: Data Lake Components
Reading: Data Lake Characteristics and Components
Video: Comparison of a Data Lake to a Data Warehouse
Reading: Data Lakes and Data Warehouses
Video: Discussing sample Data Lake Architectures
Quiz/Assessment: Week 1 quiz

Week 2: AWS data related services
Video: Introduction to Week 2
Video: AWS Data Lake related services
Video: Amazon S3
Video: AWS Glue Data Catalog
Reading: S3 and Glue Data Catalog
Video: AWS Services used for data movement
Reading: Kinesis, API Gateway, etc
Video: AWS Services for Data processing
Video: AWS Services for Analytics
Video: AWS Services used for Predictive Analytics and Machine Learning
Reading: EMR, Glue Jobs, Lambda, Kinesis Analytics, Redshift
Video: Introduction to AWS LakeFormation
Reading: LakeFormation
Lab: Get familiar with AWS Services and create your first simple data lake

Week 3: Ingesting the rivers
Video: Introduction to Week 3
Video: Use the right tool for the job
Video: Understanding Data Structure and when to process data
Video: Data Streaming ingestion with Amazon Kinesis Services
Video: Diving Deep on Amazon Kinesis
Demo: Batch Data Ingestion with AWS Transfer Family
Reading: Batch Data Ingestion with AWS Services
Video: Data Cataloging
Demo: Using Glue Crawlers
Reading: The importance of data cataloging
Video: Reviewing the ingestion part of some Data Lake architectures
Lab: Ingesting Web Logs

Week 4: Processing and Analyzing data that sits in the Data Lake
Video: Introduction to Week 4
Video: Data prep and AWS Glue jobs
Video: File optimizations
Demo: Using S3, Glue and Athena to get insights about NYC Taxi data
Reading: Glue Jobs, Data Prep, Athena? Columnar Data Formats and Amazon Athena Optimizations
Video: Introduction to Data Lake security
Reading: Security and compliance
Video: The power of data visualization
Video: Introduction to Amazon QuickSight
Demo: Amazon Quicksight
Reading: Data visualization, Amazon QuickSight
Video: Registry of Open Data on AWS
Lab: Create an end-to-end Data Lake with AWS Services
Video: Course wrap-up!

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Data Analytics Basics for Everyone (edX) EdX
IBM

Data Analytics Basics for Everyone (edX)

Statistics & Data Analysis

Learn the fundamentals of Data Analytics and gain an understanding of the data ecosystem, the process and lifecycle of data analytics, career opportunities, and the different learning paths you can take to be a Data Analyst. In this course, you will learn about the various components of a modern data ecosystem and the role Data Analysts, Data Scientists, and Data Engineers play in this ecosystem.

Self Paced
Self-Paced
Big Data Data Science Data Scientist
Details
Making Evidence-Based Strategic Decisions (edX) EdX
University of Maryland, College Park,University System of Maryland - USM,USMx,UMD

Making Evidence-Based Strategic Decisions (edX)

Statistics & Data Analysis

Drive alignment among managers, employees and the organizational goals through data analytics and data products. This course on digital transformation will show you how to turn your organization into a decision-making factory. What makes a good business decision? How can we combine effective data analytics and feed robust foresight and scenario planning processes?

Self Paced
Self-Paced
Artificial Intelligence Decision Making Data Products
Details
Data Warehousing and BI Analytics (edX) EdX
IBM

Data Warehousing and BI Analytics (edX)

Engineering

This course introduces you to designing, implementing and populating a data warehouse and analyzing its data using SQL & Business Intelligence (BI) tools. Today’s businesses are investing significantly in capabilities to harness the massive amounts of data that fuel Business Intelligence (BI). Working knowledge of Data Warehouses and BI Analytics tools are a crucial skill for Data Engineers, Data Warehousing Specialists and BI Analysts, making who are amongst, the most valued resources for organizations.

Self Paced
Self-Paced
SQL Analytics Business Intelligence
Details
Introduction to Transforming with Data Analytics and the Digital Organization (edX) EdX
University of Maryland, College Park,University System of Maryland - USM,USMx,UMD

Introduction to Transforming with Data Analytics and the Digital Organization (edX)

Statistics & Data Analysis

Learn how data analytics powers the digital organization and gives it a competitive edge. Every modern organization is a digital organization or will rapidly become digital. Artificial intelligence, Google/Amazon/Facebook/Uber, and big data have dramatically raised customer expectations and demand.

Self Paced
Self-Paced
Digital Transformation Transforming Data Analytics
Details
Digital Marketing Analytics: Tools and Techniques (edX) EdX
University of Maryland, College Park,University System of Maryland - USM,USMx,UMD

Digital Marketing Analytics: Tools and Techniques (edX)

Marketing & Communication Business

Learn how to leverage leading tools and approaches to digital marketing data analysis. Dive into SEO and SEM strategies including web analytics, machine learning and AI/Big Data applications to strengthen your digital marketing efforts and leverage your resources most effectively.

Self Paced
Self-Paced
Machine Learning SEO Digital Marketing
Details
Big Data Capstone Project (edX) EdX
University of Adelaide,AdelaideX

Big Data Capstone Project (edX)

Statistics & Data Analysis Data Science

Further develop your knowledge of big data by applying the skills you have learned to a real-world data science project. This project will give you the opportunity to deepen your learning by giving you valuable experience in evaluating, selecting and applying relevant data science techniques, principles and theory to a data science problem. This project will see you plan and execute a reasonably substantial project and demonstrate autonomy, initiative and accountability.

Self Paced
Self-Paced
Big Data Data Science Data Analytics
Details
Big Data sin misterios (edX) EdX
Inter-American Development Bank - IDB,IDBx

Big Data sin misterios (edX)

Statistics & Data Analysis

Transforma los datos en valor. Aprende el potencial de Big Data para aumentar la eficiencia y eficacia de tu organización en este MOOC desarrollado con Telefónica. 51.5% de los estudiantes que participaron a la encuesta dicen que el curso les servirá para mejorar la capacidad para formular, implementar, y/o evaluar políticas públicas.

Self Paced
Self-Paced
Big Data Data Visualization Data Analytics
Details
Generative AI and LLMs on AWS (edX) EdX
AI (Pragmatic AI Labs)

Generative AI and LLMs on AWS (edX)

Computer Science

Unlock scalable generative AI with expert training on deploying and optimizing large language models on AWS for peak performance and compliance. Master deploying generative AI models like GPT on AWS through hands-on labs. Learn architecture selection, cost optimization, monitoring, CI/CD pipelines, and compliance best practices. Gain skills in operationalizing LLMs using Amazon Bedrock, auto-scaling, spot instances, and differential privacy techniques. Ideal for ML engineers, data scientists, and technical leaders.

Self Paced
Self-Paced
AWS CI/CD Large Language Models
Details
Foundations of Data Analytics (edX) EdX
The Hong Kong University of Science and Technology - HKUST,HKUSTx

Foundations of Data Analytics (edX)

Statistics & Data Analysis Computer Science

Learn the fundamental techniques for data analytics and to be prepared for learning and applying more advanced big data technologies. Foundations of Data Analytics: This course will provide fundamental techniques for data analytics, including data collection, data extraction, data integration, data cleansing, and basic machine learning techniques.

Self Paced
Self-Paced
Machine Learning Big Data Data Privacy
Details