- Identify error sources
- Consider background errors
- Consider context errors
- Consider mislabeled or misclassified results
- Consider poor inference or incorrect model
- Account for negative impact
- Auto-detect and display errors
- Disambiguate systems hierarchy errors
- Diagnose errors that users don’t perceive
- Check output quality for relevance errors
- Fail gracefully
- Discover prediction and training data errors
- Cue the correct interactions
- Categorize user-perceived errors
- Provide paths forward from failure
- Understand the significance of a mistake’s consequences
- As you work on reducing mistakes in one area, always consider the effect your work has on other areas and overall accuracy
- When possible, address mistakes without complicating the UI
- Learn from corrections when it makes sense
- When possible, use guided corrections instead of freeform corrections
- Let people correct their corrections
- Provide immediate value when people make a correction
- Give people familiar easy ways to make corrections
- Immediately provide assistance if progress stalls
- Be especially careful to avoid mistakes in proactive features
- Support efficient dismissal
- Assume subversive use
- Imagine potential pitfalls
- Gauge the risk for potential errors
- Identify user, system, and context errors
- Weigh situational stakes and error risk
- Avoid compounding errors from other ML models
- Define “errors” and “failure”
- Predict or plan for input errors
- Never rely on corrections to make up for low-quality results
- Always balance the benefits of a feature with the effort required to make a correction
- Collect explicit feedback.
- Monitor over time.
- Allow for opting out.
- Plan for co-learning.
- Connect feedback with personalization.
- Create opportunities for feedback.
- Provide editability.
- Be prepared for changes in implicit feedback when you make changes to your app’s UI.
- Don’t ask for both positive and negative feedback.
- Make it easy for people to correct frequent or predictable mistakes.
- Always make providing explicit feedback a voluntary task.
- Request explicit feedback only when necessary.
- Consider using explicit feedback to help improve when and where you show results.
- Remember recent interactions.
- Encourage granular feedback.
- Support efficient correction.
- Learn from user behavior.
- Review implicit feedback.
- Adapt to the evolving user journey.
- Remind, reinforce, and adjust.
- Communicate value and time to impact.
- Align feedback with model improvement.
- Manage influence on user decisions.
- Connect feedback to user experience changes.
- When possible, use multiple feedback signals to improve suggestions - and mitigatie mistakes.
- Prioritize recent feedback.
- Learn from selections when it makes sense.
- Update and adapt cautiously.
- Use feedback to update predictions on a cadence that matches the - user's mental model of the feature.
- Act immediately when you receive explicit feedback and persist the resulting changes.
- Don’t let implicit feedback decrease people’s opportunities to - explore.
- Continuously update your feature to reflect people’s evolving - interests and preferences.