Simplifying Data Pipeline with AI
As Lead Product Designer, I tackled key user pain points in Amazon AppFlow and AWS Glue - complex setup flows, high drop-off rates, limited guidance, and lack of visibility into performance and data quality. Users struggled to configure connectors, map fields, and optimize flows without deep technical expertise, often abandoning the process entirely.
To solve this, I designed an AI-powered experience that recommends personalized, pre-configured pipelines based on user intent. I led UX strategy, partnered with ML teams, and introduced natural language inputs, dynamic templates, and proactive flow insights. The solution became an AWS strategic initiative, improving adoption, reducing setup time, and setting the foundation for an intelligent, unified data integration platform.
My Role - Lead UX Designer
Shaped early product direction, uncovered user pain points through research, led usability testing, conducted competitive analysis, streamlined design processes, and influenced a cross-functional team to deliver intuitive, scalable solutions for complex data integration workflows.
Oct 2022 - Aug 2023
AWS AI Integration Builder
Amazon AppFlow, AWS Glue
Challenge
A comprehensive user research revealed that AWS lacks a fully managed, low-code data integration platform for seamless, bi-directional data transfers between SaaS apps and AWS. Closing this gap is critical to meet market trends: real-time decision-making, empowering less-technical users, and leveraging Generative AI.
Results
By integrating AppFlow with Glue and launching AI-assisted flow creation, we enabled data workers to perform bi-directional data pipelines using natural language. The AI Integration Builder simplifies setup with personalized flow definitions, boosting workflow completion by 15% in beta. We enhanced productivity with GenAI-powered bots, NLP-driven Spark code generation, and accelerated connector development using CodeWhisperer, making data integration faster, simpler, and more accessible for users of all skill levels.
58%
Faster time on creating pipelines
80%
User retention at six months
70%
Tasks completed without coding
AI Prompt and Recommendations
The system interprets the user’s goal and recommends personalized, high-quality flow definitions - preconfigured with optimal sources, destinations, mappings, and transformations.
Auto-filled Configurations
Automatically populates key integration settings - such as source, destination, data mappings, and transformations - based on user intent. This reduces manual input, minimizes errors, and accelerates flow creation.
Integration Dashboard
Offers real-time monitoring, optimization insights, and troubleshooting support by integrating with CloudWatch. It helps users validate flows, detect failures, analyze performance, and track resource utilization and costs across teams—all in one unified view.
Process
User Research: To improve the AWS Data Integration experience, I partnered with two UX researchers to study internal and external user journeys, identify key friction points, and uncover opportunities for integrating Generative AI at the most impactful entry points.
Usability Testing: I led two usability testing workshops with AppFlow and Glue UX teams to identify gaps in AWS Data Integration services. Using heuristic evaluation and complex UX guidelines, we analyzed flows and shared findings with leadership—resulting in three cross-service enhancements being prioritized.
Usability Testing Snapshot
Competitive Analysis: The Data Integration market is mature, with competitors far ahead. To identify opportunities, I analyzed four major competitors across personas, use cases, and strengths, then led two discovery workshops. Key opportunities uncovered: simplify UX with guided UIs, ease setup with pre-built connectors, and improve orchestration and parallel processing.
Process Mapping: In early design workshops, I presented UX proposals and co-created a POC for AI-based Integration Recommendations with engineers. After strong stakeholder support and leadership approval as a 2023 AWS priority, I partnered with PMs and engineers in weekly sessions to shape the end-to-end UX.
End-to-end Journey Map
User Flows
Wireframing : To visualize interactions and connect the information architecture, I created low-fidelity wireframes and collaborated with stakeholders on UI direction. After three iterations, we finalized the design and aligned it with Amazon RDS and DynamoDB for consistency across services.
Low-fid Wireframes
Prototyping & User Testing: I created hi-fidelity mockups based on wireframes and led weekly workshops to align layout and interactions. After internal prototype testing, 5 UX reviews, and 2 demo sessions, we secured leadership sign-off on the “Integration Recommendation” screen and two “Integration Creation” interfaces for A/B testing in the Beta launch.
Selected Screen: Pre-filled Integration Creation
A/B Testing
Selected Screen: Field Mapping Enhancements
Reflections & Next Steps
This project highlighted the importance of designing flexible, intelligent tools that empower expert users. It set new design patterns now adopted across three AWS services. If I could do it again, I’d involve data scientists and PMs earlier to better shape low-code solutions and align roadmaps.
Looking ahead, we’ll keep refining the recommendation engine with smarter, context-aware learning. In two years, we’ll pursue a unified Managed Ingestion product to support enterprise-grade requirements and all data sources. Within three years, we’ll evolve this into a smart business process orchestration engine. In the long run, our goal is to offer a single-pane-of-glass across AWS data services, providing customers with a modern data stack in a box - from ingestion to transformation, orchestration, and observability - all while enhancing data worker productivity.