Identifying Bad Data Sources (When Data Does Not ROCCC)
1. Why Bad Data Matters
Not all data is useful or trustworthy. Data that fails basic quality checks can lead to incorrect conclusions, poor decisions, and real-world harm. Understanding how to recognize bad data is just as important as knowing how to find good data.
A helpful way to evaluate data quality is the ROCCC framework:
- Reliable
- Original
- Comprehensive
- Current
- Cited
When a dataset fails these criteria, it should be treated with caution—or avoided entirely.
2. R — Not Reliable
Unreliable data cannot be trusted because it is inaccurate, incomplete, or biased.
Common signs of unreliable data
- Sampling bias that does not reflect the full population
- Measurement errors or inconsistent data collection
- Misleading visualizations
Example: Misleading charts
- A bar chart with a y-axis that does not start at zero can exaggerate small changes
- Interest rates may appear to “skyrocket” even when changes are minimal
Unreliable data distorts reality and misleads interpretation.
3. O — Not Original
Non-original data comes from secondary or tertiary sources without access to the original dataset.
Why this is a problem
- Data may have been altered, summarized, or misinterpreted
- Important context may be missing
- Errors are harder to verify
If the original source cannot be identified or validated, extra scrutiny is required.
4. C — Not Comprehensive
Non-comprehensive data is missing critical information needed to answer the question at hand.
Issues with incomplete data
- Key variables are excluded
- Results provide only part of the story
- Human error may be present
Incomplete data leads to partial or misleading conclusions.
5. C — Not Current
Outdated data may no longer reflect current conditions.
Why recency matters
- Trends, populations, and behaviors change over time
- Old data can result in irrelevant or incorrect insights
Trusted sources refresh their datasets regularly, increasing confidence in their accuracy.
6. C — Not Cited
Uncited data lacks transparency and credibility.
Red flags
- No clear information about who created the data
- No indication of when the data was collected or updated
- No evidence of review or validation
Data that cannot be traced back to a credible source should not be used.
7. Summary: What Makes Data “Bad”
Data is considered poor quality if it is:
- Not reliable
- Not original
- Not comprehensive
- Not current
- Not cited
If a dataset does not ROCCC, it should not be trusted for analysis.
8. Consequences of Using Bad Data
Using bad data can result in:
- Incorrect business decisions
- Failed processes
- Financial losses
- Risks to public health or safety
Strong analysis starts by avoiding bad data.
9. Recommended Sources for Good Data
To reduce risk, analysts often rely on:
- Vetted public datasets
- Government agency data (e.g., census data)
- Academic research
- Financial and economic data from trusted institutions
These sources are more likely to meet ROCCC standards.
10. Key Takeaways
- Bad data fails the ROCCC criteria
- Unreliable data may be biased or misleading
- Non-original data lacks verifiability
- Incomplete data weakens conclusions
- Outdated data reduces relevance
- Uncited data lacks credibility
- Avoiding bad data is essential for good analysis
One-sentence summary
Data that does not meet the ROCCC standard—Reliable, Original, Comprehensive, Current, and Cited—can mislead analysis and should be avoided to ensure accurate and trustworthy decisions.