Data Lineage: from overlooked feature to primary SaaS converter.
Rebuilding Data Lineage's user journey for the upcoming SaaS release, under a tight deadline.
Elementary Data
YC-backed, open-source data observability platform serving 10,000+ data engineers and analysts to monitor, trace, and resolve data pipeline issues.
A well-defined scope is the only way to ship quality results fast.
All company focus shifted to aligning major features for the upcoming SaaS release. Data Lineage was the first feature I chose to work on, a node canvas for tracing pipeline issues and downstream impact.
Positioned Data Lineage as the primary OSS to SaaS conversion driver, cited by 81% as the main reason.
Reduced cloud compute costs through optimizing Data Lineage core user flow.
3 design partners converted to paid versions within weeks of private launch.
Repeatedly voted all-time favorite feature in monthly Slack polls.
This is how Data Lineage looked when I joined.
With a two-week timeline to deliver and move to the next feature, I prioritized a focused strategy over a broad one to maintain quality.
Non-traditional discovery cycle, validated on the go.
Posthog showed strong page-load volume but sessions were short and visibly confused. With no time for a full discovery cycle, I validated on the go through quick prototypes, demos, and direct sessions with data professionals and the open-source community.
The investigation mental model
12 data professionals interviews at Elementor & Fiverr:
Main pain points at scale
Open-source Slack community feedback, ~5K members:
Data Lineage lacked an opinionated user experience.
According to research, success meant a user should be able to start from a known model, trace a path to the root cause, and never feel lost or slowed down by the tool itself.
Investigations are progressive
Usually, analysts trace paths from a known table to identify the root cause.
Too much visibility
Loading the full DAG by default is unhelpful and causes performance issues.
Remaining within context
Current experience lacked tools to refocus starting points during investigations.
Designing for progressive investigation, not full visibility.
Model-first journey
Users select a model first, then see a focused DAG by default.
File tree with search & filters
Enables quick model location, matching the database structure.
Node filters, depth controller & direction stepper
Users can group by model type, choose levels to display, and toggle upstream/downstream.
Node-level actions
Added contextual actions so users can trace issues without losing context.
Column-level lineage
Added visibility layer to follow column connections, a SaaS feature.
Beyond the scope
These features were designed and validated extensions of Column-level lineage, a SaaS-only feature. They were deprioritized to focus on the best possible open-source experience first.
Column name search
Test results on columns
Visible column types
Personal note.
Working on a true open-source changed how I approach to research. With 5K+ users giving unfiltered feedback in hours, I learned that speed and quality are not a tradeoff. They are both a result of knowing exactly what problem you are solving.
Explore other projects
Intent-based trading: driving growth and performance through behavioral design.
Operational efficiency: integrating conversational AI to streamline HR-to-IT workflows.
