Feedback Analyzer
The Feedback Analyzer uses AI to analyze user feedback comments, identify content issues, and update source document quality scores.
What It Does
The Feedback Analyzer:
- Collects feedback from conversations
- Uses AI to analyze comment sentiment
- Identifies specific issues (broken links, outdated info, etc.)
- Calculates source document performance scores
- Updates source rankings for future searches
- Syncs feedback counts to cache entries
Why It Matters
Improved Search Quality
When the analyzer identifies high-performing and low-performing sources, it adjusts search rankings so:
- Good sources appear higher in results
- Poor sources are deprioritized
- Overall answer quality improves
Issue Identification
AI analysis can identify:
- Broken links
- Outdated information
- Incorrect answers
- Unclear content
- Missing information
Actionable Insights
The analyzer tells you which documents need attention, so you can prioritize updates.
When to Use
Recommended Schedule
| Timeframe | When to Run |
|---|---|
| 30-day analysis | Weekly review |
| 90-day analysis | Monthly deep dive |
Good Times to Run
- After FAQ generation: Update quality metrics
- Weekly maintenance: Keep scores current
- After content updates: See impact of changes
- When quality seems declining: Diagnose issues
Analysis Options
30-Day Analysis
- Faster to run
- Recent feedback focus
- Good for weekly reviews
- Detects recent changes
90-Day Analysis
- More comprehensive
- Identifies long-term patterns
- Better statistical significance
- Good for monthly deep dives
How to Use
Running the Analysis
- Go to the Maintenance page
- Find the Analyze Feedback with AI section
- Choose Analyze 30 Days or Analyze 90 Days
- Wait for the analysis to complete (may take a few moments)
Understanding Results
The output panel shows:
📊 Summary:
Total Feedback: 250
Sources Analyzed: 45
Query Patterns: 12
LLM Analyzed: 180
🌟 Top Performing Sources:
HFIM_Handbook.pdf... (Score: 0.89)
Course_Catalog.pdf... (Score: 0.85)
⚠️ Low Performing Sources:
Old_FAQ.pdf... (Score: 0.42)
Outdated_Info.pdf... (Score: 0.38)
Result Categories
| Metric | Meaning |
|---|---|
| Total Feedback | Number of feedback entries analyzed |
| Sources Analyzed | Documents that received feedback |
| Query Patterns | Common question patterns identified |
| LLM Analyzed | Entries processed with AI analysis |
Source Scores
- 0.8-1.0: Excellent - Users find this helpful
- 0.6-0.8: Good - Generally positive
- 0.4-0.6: Mixed - Needs review
- 0.0-0.4: Poor - Consider updating or removing
After Analysis
Review Low Performers
For sources with low scores:
- Review the actual documents
- Check if information is outdated
- Consider updating or replacing
- Re-ingest after updates
Celebrate High Performers
High-scoring sources:
- Validate your content quality
- Can be models for other content
- Should be maintained and updated
View Detailed Results
For deeper analysis:
- Go to the Analytics page
- See source performance over time
- Track improvement trends
What the AI Analyzes
Sentiment Categories
| Sentiment | Meaning |
|---|---|
| Positive | User found answer helpful |
| Negative | User found answer unhelpful |
| Neutral | User gave feedback without clear sentiment |
Issue Types Detected
| Issue | Description |
|---|---|
| Broken links | Links in response don't work |
| Outdated info | Information is no longer current |
| Wrong answer | Response didn't match the question |
| Unclear content | Answer was confusing |
| Missing info | Expected information wasn't provided |
Best Practices
Regular Analysis
- Run 30-day analysis weekly
- Run 90-day analysis monthly
- Track trends over time
Act on Insights
- Don't just run analysis - act on it
- Update low-performing sources
- Create cache entries for problem areas
Combine with Other Tools
Use feedback analysis with:
- Cache Management - Improve cached responses
- Popular Questions - Block problematic questions
- Conversations - Review specific issues
Troubleshooting
Analysis Takes Too Long
90-day analysis with many conversations may take time:
- Be patient (several minutes is normal)
- Don't close the page
- Try 30-day if 90-day times out
No Results
If analysis shows few results:
- Check that conversations exist
- Verify feedback has been given
- Wait for more user interaction data
Source Scores Seem Wrong
Scores are based on available feedback:
- Limited feedback = less reliable scores
- Wait for more data before acting
- Focus on sources with significant feedback
Related: