Machine Learning labeling tools
Labeling tools are essential to train and test the data pipeline for machine learning models powering autonomous vehicles. I worked on creating a learning management system that allows labelers to view their task history and get trained on labeling better.
Role
Design Lead
Responsibilities
Visual Design,
UI & UX Design,
Product strategy
Timeline
Jan - Feb 2024
Problem
Data labelers struggle to keep track of their labeling tasks and feedback.
Labeling tasks are a critical part of any ML model that is powering an autonomous vehicle. As our labeling workforce was generating labels, they used spreadsheets and manual tracking of their tasks. This led them to spend about 10 hours per week on tracking labels and scoring, instead of actual labeling tasks.
Research & Insights
Labeling tasks lacked real-time tracking and feedback, impacting throughput.
Operators spend time on manual trackers- which impacts and skews throughput tracking.
Understanding the steps in the user journey
Mapped the labelers' workflow and labeling (human-in-the-loop) steps in the ML pipeline
Documented the steps in the ML pipeline to account for all possible steps.
Solution
An easy way for operators to track their tasks and feedback.
We incorporated a way to view tasks and all the labels for all the ML data pipelines and steps.
Initial brainstorms
Explored the relationship between tasks and individual labels.
I designed various concepts illustrating possible ways to layout the information and the steps within the task for the ML data pipeline (side-drawer versus table view).
MVP iteration
Table views to allow for easy sorting, filtering and exporting.
We incorporated a way to view tasks and all the labels for all the ML data pipelines and steps.
Concept testing and iteration
Partnered with customers and stakeholders to iterate on concepts
We learned that the information about the labels themselves scoring and feedback was missing. I did a second pass on designs to incorporate a variety of details.
Launch
Post MVP, our labeling workforce is using this to track their tasks successfully!
We had a successful iteration, and the interface has been used by our labeling workforces for almost a quarter to improve their throughput.