A case study on using LLMs to transform thousands of unstructured reviews into an actionable product discovery pipeline.

The Problem

Product teams spend countless hours manually parsing through thousands of App Store and Play Store reviews. The sheer volume makes it impossible to read every piece of feedback.

As a result, critical early signals about feature requests, recurring bugs, and shifts in user sentiment are often missed or only recognized weeks later.

App reviews are a goldmine of unprompted user truth, but they are buried beneath layers of noise, spam, and unhelpful rants.

Research Objective

The goal of this research project was to understand the gap between raw user feedback and prioritized product roadmaps, and to explore how modern NLP and LLMs could automate insight extraction without losing human context.

Research Method

To understand the problem space before building, I focused on three discovery tracks:

• Workflow Observation: Interviewing PMs to see how they currently aggregate, read, and tag user reviews in spreadsheets.
• Data Profiling: Analyzing a raw dataset of 10,000+ app reviews to identify noise ratios (spam, emojis, single-word reviews).
• Prototyping: Running early LLM tests to see if models could accurately cluster complaints vs. feature requests.

Data Collection

The foundation of the project required building a scalable data ingestion pipeline. I set up automated scraping from app stores, followed by a rigorous cleaning process to normalize the text, remove spam, and filter out non-actionable reviews (e.g., "Good app").

Insight Generation Pipeline

The core of the product relies on an LLM-powered pipeline that performs semantic clustering. Instead of relying on rigid keyword searches (which miss context), the pipeline asks the LLM to:

• Extract the primary user intent (Bug, Feature Request, UX Complaint, Praise).
• Identify the specific product area (e.g., "Login", "Portfolio View", "Customer Support").
• Assign a nuanced sentiment score.

Key Insights

After running thousands of historical reviews through the prototype, three surprising patterns emerged:

Opportunity Areas

The research highlighted clear opportunities for a product solution:

• Automated Alerting: Notify teams immediately when sentiment for a specific feature drops below a threshold.
• Epic Mapping: Automatically group related user complaints and map them directly to backlog epics.
• Trend Visualization: Create a visual dashboard that separates the signal from the noise.

Solution Concepts

I designed Groww Pulse: a centralized dashboard that aggregates sentiment over time, shows top emerging themes, and allows product teams to filter insights by app version, date, or specific keyword, turning raw text into a prioritized roadmap.

Technical Approach

The solution was built as an end-to-end data product:

• Ingestion: Python scripts for continuous app store scraping.
• Inference: Integration with Groq (running Llama 3) for high-speed, low-cost semantic clustering and sentiment analysis.
• Storage & UI: PostgreSQL for structured data storage, visualized through a responsive dashboard.

Business Impact

Groww Pulse reduces the time spent parsing reviews from hours per week to minutes. More importantly, it ensures that critical production bugs and emerging user frustrations are surfaced immediately, enabling faster, data-driven product decisions.

Key Learnings

The most important lesson from this project was about the nature of feedback:

Customer feedback becomes valuable only when it is converted into a decision. An insight without an owner is just noise.

Reflection

The true value of AI in product discovery isn't just in reading text faster. It is in the ability to synthesize macro-patterns across thousands of data points—patterns that human product managers simply miss due to cognitive overload and sheer volume. Groww Pulse proved that LLMs can bridge the gap between user voices and engineering backlogs.