Publication Trust Assessment

Evaluate the scientific integrity and reliability of publications before building analyses on their findings.

When to Use

• User wants to evaluate the reliability and reproducibility of a genomics publication
• User asks about "publication quality", "reproducibility", "trust assessment", or "paper evaluation"
• User needs to check if a paper follows ENCODE data standards and best practices
• User wants to verify that cited datasets, tools, and methods meet community standards
• Example queries: "is this paper's ChIP-seq analysis trustworthy?", "evaluate the methods in this genomics paper", "check if this study follows ENCODE standards"

Why This Matters

Not all published findings are reliable. Problems range from formal retractions (data fabrication, image manipulation) to informal contradictions where independent groups fail to reproduce key claims. In genomics and computational biology, building pipelines or analyses on unreliable findings wastes resources and propagates errors.

The problem is not always obvious. Formal retractions are rare compared to the number of problematic papers. More commonly:

• A subsequent study contradicts the key finding without triggering a retraction
• An erratum quietly corrects a critical result
• The original authors publish a "correction" that substantially changes conclusions
• Independent groups fail to replicate and publish their negative results

This skill provides a systematic approach to detecting these issues.

Trust Assessment Framework

Trust Levels

Default trust level is 4 (standard) — absence of evidence is not evidence of absence. A paper starts at "standard" and moves up with independent replication or down with identified issues.

Step 1: Gather Publication Metadata

For any publication being assessed, retrieve full metadata:

get_article_metadata(pmids=["PMID"])

Record:

• article_types: Check for "Retracted Publication", "Published Erratum", "Expression of Concern"
• authors: Note corresponding author and senior author (last position)
• journal: Impact factor context (high-profile journals ≠ automatic reliability)
• publication_date: Older papers have had more time for replication/refutation

If starting from a DOI:

convert_article_ids(ids=["DOI"], id_type="doi")

Step 2: Check Formal Integrity Markers

2a. Retraction Status

Search PubMed for retraction notices linked to this paper:

search_articles(query="PMID[PMID] AND (Retracted Publication[pt] OR Retraction of Publication[pt])")

Also check the article_types field from Step 1 — if it contains "Retracted Publication", the paper has been formally retracted.

If retracted → Trust Level 1 (Compromised)

2b. Errata and Corrections

Search for corrections:

search_articles(query="PMID[PMID] AND (Published Erratum[pt] OR Correction[pt])")

Errata can be minor (typo in a table) or major (recalculated results that change conclusions). Read the erratum to distinguish:

• Minor: Trust level unchanged
• Major (changes key results or conclusions): Trust Level 3 (Caution advised)

2c. Expression of Concern

search_articles(query="PMID[PMID] AND Expression of Concern[pt]")

An Expression of Concern from a journal editor indicates an active investigation. Trust Level 2 (Reliability concerns) until resolved.

Step 3: Check for Contradicting Publications

This is the most important and most nuanced step. Many problematic findings are never formally retracted — they are contradicted by subsequent independent work.

3a. Find Citing Articles

find_related_articles(pmids=["PMID"], link_type="pubmed_pubmed", max_results=50)

This returns computationally similar articles. From these, search for contradiction signals.

3b. Search for Contradiction Signals

Search PubMed for articles that cite the original AND contain contradiction language:

search_articles(query="\"[key claim from title]\" AND (\"fail to replicate\" OR \"unable to reproduce\" OR \"does not\" OR \"do not support\" OR \"contradicts\" OR \"challenges\" OR \"reanalysis\" OR \"re-analysis\" OR \"not reproducible\" OR \"could not confirm\")")

Also search with the first author's last name + key topic terms:

search_articles(query="[FirstAuthor] [KeyTopic] AND (Comment[pt] OR Letter[pt] OR \"fail\" OR \"does not\")")

3c. Search Academic Databases

Use Consensus to find contradicting evidence:

consensus_search(query="does [key claim from paper] replicate? contradiction evidence")

3d. Check bioRxiv for Preprints

Contradictions sometimes appear first as preprints:

search_preprints(category="[relevant category]", date_from="[pub date]", date_to="[today]")

3e. Evaluate Contradictions

Not every disagreement is a refutation. Assess:

If key finding is contradicted by independent group with stronger methodology → Trust Level 2

Step 4: Author and Group Assessment

4a. Check Author Track Record

If a contradiction or retraction is found, check whether the authors have other problematic publications:

search_articles(query="[SeniorAuthor][Author] AND Retracted Publication[pt]")

A pattern of retractions from the same group is a stronger signal than a single incident.

4b. Contextual Notes

When flagging author concerns, use measured language:

• "Publications from this group have been subject to independent contradiction"
• "A previous study from this corresponding author was contradicted by [citation]"
• "This group's findings in [area] have not been independently replicated"

Never use: fabrication, fraud, doctored, liar, shady, dishonest — unless a formal investigation has published findings using those terms. Stick to what the published record shows.

Step 5: Generate Trust Report

Present findings in a structured format:

## Publication Trust Assessment

**Paper**: [Title] ([Journal], [Year])
**PMID**: [PMID] | **DOI**: [DOI]
**Authors**: [First Author] ... [Senior Author]

### Trust Level: [X/5] — [Label]

### Formal Markers
- Retraction: None / Yes (date, reason)
- Corrections: None / [count] ([minor/major])
- Expression of Concern: None / Yes (date)

### Contradicting Evidence
- [Citation of contradicting paper] — [brief description of contradiction]
- Independent replication: Yes/No/Unknown

### Author Context
- [Any relevant notes about author track record]

### Recommendation
- [Clear guidance on whether to rely on this paper's findings]

Step 6: Integration with Other Skills

When this skill identifies a trust issue, the finding should propagate:

• cite-encode: Flag compromised papers in citation lists
• quality-assessment: Downweight methods from compromised sources
• variant-annotation: Note if v