EdX

Data Creation and Collection for Artificial Intelligence via Crowdsourcing (edX)

Data Creation and Collection for Artificial Intelligence via Crowdsourcing (edX)

A one-stop shop to get started on the key considerations about data for AI! Learn how crowdsourcing offers a viable means to leverage human intelligence at scale for data creation, enrichment and interpretation, demonstrating a great potential to improve both the performance of AI systems and their trustworthiness and increase the adoption of AI in general.

Class Deals by MOOC List - Click here and see EdX's Active Discounts, Deals, and Promo Codes.

Advances in Artificial Intelligence and Machine Learning have led to technological revolutions. Yet, AI systems at the forefront of such innovations have been the center of growing concerns. These involve reports of system failure when conditions are only slightly different from the training phase and they also trigger ethical and societal considerations that arise as a result of their use.
Machine learning models have been criticized for lacking robustness, fairness and transparency. Such model-related problems can generally be attributed to a large extent to issues with data. In order to learn comprehensive, fine-grained and unbiased patterns, models have to be trained on a large number of high-quality data instances with distribution that accurately represents real application scenarios. Creating such data is not only a long, laborious and expensive process, but sometimes even impossible when the data is extremely imbalanced, or the distribution constantly evolves over time.
This course will introduce an important method that can be used to gather data for training machine learning models and building AI systems. Crowdsourcing offers a viable means of leveraging human intelligence at scale for data creation, enrichment and interpretation with great potential to improve the performance of AI systems and increase the wider adoption of AI in general.
By the end of this course you will be able to understand and apply crowdsourcing methods to elicit human input as a means of gathering high-quality data for machine learning. You will be able to identify biases in datasets as a result of how they are gathered or created and select from task design choices that can optimize data quality. These learnings will contribute to an important set of skills that are essential for career trajectories in the field of Data Science, Machine Learning, and the broader realms of Artificial Intelligence.

What you'll learn
At the end of this course you will be able to:

  • Examine the use of crowdsourcing for gathering data
  • Explain how cognitive biases and other human factors influence data quality
  • Describe the use of active learning in the creation of crowdsourced training data
  • Demonstrate the design of crowdsourcing tasks with quality control mechanisms
  • Discuss the evaluation of ML models with humans in the loop

Syllabus

Week 1: Crowdsourcing for High-quality Data Collection and The ImageNet Story
Artificial Intelligence is at the center of many recent advancements across areas such as transportation and finance. One of the reasons for this is that in the past decade we have designed methods to harness human intelligence at scale.
We will introduce and discuss the crowdsourcing paradigm and the importance of high-quality data.
Topics we will cover this week:
The intuition behind crowdsourcing
The role of crowdsourcing platforms
The need for high-quality data for AI models
What is ImageNet, the gap it filled, and how it was built

Week 2: Quality Control Mechanisms for Crowdsourcing
The quality of crowdsourced human input is one of the most crucial aspects affecting the overall value of the paradigm. In this week we will discuss the challenges that make quality control difficult to guarantee.
Topics we will cover this week:
Workers' motives and behaviors
Quality control mechanisms in crowdsourcing
Incentives in crowdsourcing (like gamification)
Cognitive aspects and psychometric methods

Week 3: Factors Affecting Quality in Crowdsourcing
Researchers and practitioners in human computation and crowdsourcing have identified several factors that affect the quality of crowdsourced data. In this week we will discuss some of the recent works in this regard.
Topics we will cover this week:
Tradeoff between task pricing and quality of output
The role of workers' demographics, qualifications and skills
The importance of task clarity and work environments
The concepts of task packaging, task framing and task priming

Week 4: Human Input for Data Creation and Model Evaluation in AI
In this week, we will cover the importance of data collection, annotation and engineering.
Topics we will cover this week:
The importance of data collection
Data generation
The role of crowdsourcing in advanced machine learning
Taxonomy of microtasks

Week 5: Reducing Worker Effort: Active Learning
In this week we explore the challenges of collecting large scale data and how to overcome them.
Topics we will cover this week:
Approaches to reducing worker effort
The implications of reducing labeling effort
The key idea of active learning
Query strategies for selecting informative instances

Week 6: Interpreting, Evaluating, and Debugging ML models
In this week, we discuss strategies for evaluating, debugging, and interpreting machine learning models.
Topics we will cover this week:
The notion of model interpretability
The role of humans in the interpretability process
Debugging ML pipelines and related challenges

Go to Class
MOOC List is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Related Courses

Generating Vision: Long-Term Big Ideas That Motivate Employees and Stakeholders (edX) EdX
University of Maryland, College Park,University System of Maryland - USM,USMx,UMD

Generating Vision: Long-Term Big Ideas That Motivate Employees and Stakeholders (edX)

The “why movement” linked to crafting purpose and mission statements is now being supplanted by the emerging “where movement.” This movement goes beyond brainstorming towards declaring the “big ideas” that will inspire stakeholders. “Where are we going?” as an organization is the primal question posed by employees that CEOs and product managers must smartly address. This course will teach you how.

Self Paced
Self-Paced
Tech for Good: The Role of ICT in Achieving the SDGs (edX) EdX
SDGAcademyX,SDG Academy

Tech for Good: The Role of ICT in Achieving the SDGs (edX)

What opportunities and challenges do digital technologies present for the development of our society? Tech for Good was developed by UNESCO and Cetic, the Brazilian Network Information Center’s Regional Center for Studies on the Development of the Information Society. It brings together thought leaders and changemakers in the fields of information and communication technologies (ICT) and sustainable development to show how digital technologies are empowering billions of people around the world by providing access to education, healthcare, banking, and government services; and how “big data” is being used to inform smarter, evidence-based policies to improve people’s lives in fundamental ways.

Self Paced
Self-Paced
AI skills: Introduction to Unsupervised, Deep and Reinforcement Learning (edX) EdX
Delft University of Technology,DelftX

AI skills: Introduction to Unsupervised, Deep and Reinforcement Learning (edX)

Learn the fundamentals and principal AI concepts about clustering, dimensionality reduction, reinforcement learning and deep learning to solve real-life problems. In this course you will learn the basics of several machine learning topics to help you solve real life challenges. Unsupervised learning techniques such as clustering and dimensionality reduction are useful to make sense of large and/or high dimensional datasets that are not annotated. Deep learning is a supervised learning technique that is useful to train neural networks to solve more complicated classification and regression tasks. Finally, reinforcement learning techniques can be used to train AI agents that interact with an environment.

Self Paced
Self-Paced
Essentials of Genomics and Biomedical Informatics (edX) EdX
IsraelX

Essentials of Genomics and Biomedical Informatics (edX)

This course presents clinicians and digital health enthusiasts with an overview of the data revolution in medicine and how to exploit it for research and in the clinic. The course will not make you a bioinformatician but will introduce the main concepts, tools, algorithms, and databases in this field.

Self Paced
5-12 Weeks
Smart Analytics, Machine Learning, and AI on Google Cloud (edX) EdX
Google Cloud

Smart Analytics, Machine Learning, and AI on Google Cloud (edX)

This course covers several ways machine learning can be included in data pipelines on Google Cloud depending on the level of customization required. Incorporating machine learning into data pipelines increases the ability of businesses to extract insights from their data. This course covers several ways machine learning can be included in data pipelines on Google Cloud depending on the level of customization required.

Self Paced
Self-Paced
Digital Transformation Strategy: Discovering and Reaching New Markets (edX) EdX
University of Maryland, College Park,University System of Maryland - USM,USMx,UMD

Digital Transformation Strategy: Discovering and Reaching New Markets (edX)

Learn key concepts of strategy, IT, and Innovation to bring emerging technologies to market through digital channels. This course is different – it’s a sample of the most important key concepts and techniques needed to develop digital strategies and bring them to market.

Self Paced
Self-Paced
Humanitarian Action in the Digital Age (edX) EdX
École Polytechnique Fédérale de Lausanne,EPFLx

Humanitarian Action in the Digital Age (edX)

The first MOOC about responsible use of technology for humanitarians. Learn about technology and identify risks and opportunities when designing digital solutions. As humanitarian practitioner, you are interested in technology but you feel it should be used more responsibly? You are worried people tend to jump on opportunities without properly considering risks? This MOOC is for you!

Self Paced
Self-Paced
Introduction to Computer Science and Programming (edX) EdX
Tokyo Institute of Technology,TokyoTechX

Introduction to Computer Science and Programming (edX)

The term “Computation” refers to the action performed by a computer. A computation can be a basic operation and it can also be a sophisticated computer simultation requiring a large amount of data and substantial resources. This course aims at introducing learners with no prior knowledge to basics and key concepts of computer science. By following the lectures and exercises of this course you will have an understanding of algorithms and you will get a real experience of programming using the language Ruby.

Self Paced
Self-Paced
Gobernanza de datos personales en la era digital (edX) EdX
The Pontificia Universidad Javeriana,JaverianaX

Gobernanza de datos personales en la era digital (edX)

Law

Aprende qué es la gobernanza de datos personales y desarrolla habilidades para diseñar e implementar leyes y políticas públicas en materia de datos personales en la era digital. Este curso en línea te ayudará a comprender qué es la gobernanza de datos, los criterios que deben tener en cuenta quienes formulan política públicas al momento de redactar o desarrollar leyes, regulaciones o políticas en materia de protección de datos y privacidad, así como aspectos prácticos de los programas de gobernanza de datos.

Self Paced
Self-Paced
Introduction to ChatGPT. Boost your productivity to the maximum using artificial intelligence! (edX) EdX
Universitat Politècnica de València,UPValenciaX

Introduction to ChatGPT. Boost your productivity to the maximum using artificial intelligence! (edX)

Have you heard about ChatGPT and want to learn how to use it to increase your productivity at work radically? Then, this is the course for you. Here, you will learn about its advantages and disadvantages, and different ways to create effective prompts. You will also see examples of its use in tasks as diverse as drafting an instance to a City Council, summarizing a document, enhancing your writing for emails, translating documents, or using it to create Excel formulas and macros.

Self Paced
Self-Paced