vcf-variant-filtering

VCF Variant Filtering Guide

Overview

Raw VCF files produced by variant callers (GATK HaplotypeCaller, bcftools mpileup, DeepVariant, etc.) contain a mixture of true variants and artifacts from sequencing errors, alignment issues, and low-coverage regions. Computing summary statistics -- Ts/Tv ratio, variant counts, allele frequency distributions -- on unfiltered data yields unreliable results because false-positive calls disproportionately inflate transversion counts and depress the Ts/Tv ratio. This guide covers how to detect whether a VCF is raw, how to apply appropriate quality filters, when filtering is not appropriate, and how to interpret the resulting statistics correctly.

Key Concepts

VCF Quality Scores (QUAL Field)

The QUAL column in a VCF file represents the Phred-scaled probability that the variant site is polymorphic. A QUAL score of 30 means a 1-in-1000 chance the call is wrong; a QUAL of 20 means 1-in-100. Variant callers assign QUAL scores based on read evidence, base qualities, and mapping qualities. Low-QUAL variants (below 20-30) are enriched for sequencing errors and alignment artifacts. Filtering on QUAL is the simplest and most widely used first-pass quality control step.

Common QUAL thresholds and their interpretation:

| QUAL Score | Error Probability | Typical Use | |------------|------------------|-------------| | 10 | 1 in 10 | Very permissive; rarely appropriate for final calls | | 20 | 1 in 100 | Lenient filtering; useful for somatic calling with supporting evidence | | 30 | 1 in 1,000 | Standard default for germline variant filtering | | 50 | 1 in 100,000 | Stringent; used in clinical or high-confidence applications | | 100+ | Extremely low | Highly supported variants; may over-filter low-coverage regions |

For GATK-based workflows, VQSR or hard filtering on INFO annotations (QD, FS, MQ, etc.) may supplement or replace QUAL filtering. GATK's recommended hard filters for SNPs include QD < 2.0, FS > 60.0, MQ < 40.0, MQRankSum < -12.5, and ReadPosRankSum < -8.0.

Ts/Tv Ratio Significance

The transition-to-transversion (Ts/Tv) ratio is a key quality metric for variant call sets. Transitions (A<->G, C<->T) are chemically favored over transversions (all other substitutions) due to the molecular structure of nucleotide bases. Expected Ts/Tv values serve as benchmarks:

• Whole-genome sequencing (WGS): approximately 2.0-2.1
• Whole-exome sequencing (WES): approximately 2.8-3.3 (higher due to CpG enrichment in coding regions)
• Raw/unfiltered call sets: often 1.5-1.8 or lower

A Ts/Tv ratio significantly below the expected range indicates contamination by false-positive transversion calls, which are the hallmark of sequencing errors. After proper quality filtering, the Ts/Tv ratio should rise to the expected range for the assay type.

Raw vs Filtered VCFs

A "raw" VCF is the direct output of a variant caller before any quality filtering has been applied. A "filtered" VCF has had quality thresholds applied, either by hard filtering (QUAL, DP, QD, etc.) or by model-based filtering (GATK VQSR, CNN). Distinguishing between the two is critical because computing statistics on raw data without disclosure leads to incorrect conclusions.

Indicators that a VCF is raw or unfiltered:

• The filename contains "raw" (e.g., sample_raw_variants.vcf)
• The FILTER column contains only . (missing) for all records
• A large fraction of variants have QUAL scores below 30
• The Ts/Tv ratio is well below the expected range for the assay type

Indicators that a VCF has already been filtered:

• The FILTER column contains meaningful values (PASS, LowQual, VQSRTrancheSNP99.90to100.00)
• A filter command is recorded in the VCF header (##FILTER= and ##bcftools_viewCommand= lines)

FILTER Column Semantics

The FILTER column in VCF format has specific semantics defined by the VCF specification:

• . (dot) -- filter status has not been applied; the variant is unassessed
• PASS -- the variant passed all filters
• Any other value -- the variant failed the named filter(s); multiple filters are semicolon-separated

A common misconception is that . means the variant passed. In reality, . means no filter has been evaluated, so the variant's quality is unknown. Another misconception is that PASS in every row means the file is unfiltered -- some callers (e.g., DeepVariant) mark all emitted variants as PASS because they only output high-confidence calls.

To inspect the FILTER column programmatically:

# Count occurrences of each FILTER value
bcftools query -f '%FILTER\n' input.vcf | sort | uniq -c | sort -rn | head

# Check if any non-'.' FILTER values exist
bcftools query -f '%FILTER\n' input.vcf | grep -v '^\.$' | head

Understanding the FILTER column is the first step in any VCF quality assessment. Always inspect it before deciding whether additional filtering is needed.

Decision Framework

Is the VCF raw or unfiltered?
├── Yes (FILTER='.', many low-QUAL variants)
│   ├── Does the user ask for RAW statistics specifically?
│   │   ├── Yes → Do NOT filter; compute on data as-is, note it is raw
│   │   └── No → Apply QUAL>=30 filter before computing statistics
│   └── Does the user specify a custom threshold?
│       ├── Yes → Use their threshold
│       └── No → Default to QUAL>=30
├── No (FILTER has PASS/other values, filtered header present)
│   ├── Does the user ask for additional filtering?
│   │   ├── Yes → Apply requested filter
│   │   └── No → Compute statistics on existing filtered set
│   └── Does the Ts/Tv ratio look suspicious despite filtering?
│       └── Yes → Investigate; may need stricter thresholds
└── Uncertain
    └── Inspect FILTER column and QUAL distribution to determine status

| Scenario | Action | Rationale | |----------|--------|-----------| | Raw VCF, general statistics requested | Apply QUAL>=30 before computing | Low-QUAL variants are enriched for errors and bias all statistics | | Raw VCF, user explicitly asks for raw stats | Compute as-is, report that data is unfiltered | Respect the user's intent; they may be evaluating caller performance | | Raw VCF, user specifies threshold (e.g., QUAL>=50) | Use the user-specified threshold | User has domain-specific requirements | | Pre-filtered VCF (FILTER=PASS present) | Compute without additional QUAL filter | Double-filtering may remove true variants unnecessarily | | VCF with mixed FILTER values | Filter to PASS-only with bcftools view -f PASS | Non-PASS variants were flagged for a reason | | VQSR-filtered VCF | Respect existing tranche filtering | VQSR is a more sophisticated filter than simple QUAL thresholds | | Small panel or targeted sequencing | Consider lower QUAL threshold (e.g., 20) | Small panels have different error profiles and fewer variants |

Best Practices

1. Always inspect the VCF before computing statistics. Check the FILTER column distribution and QUAL score distribution before running any summary. A 30-second inspection prevents publishing misleading numbers. Use bcftools query -f '%FILTER\n' input.vcf | sort | uniq -c | sort -rn | head to get a quick overview.

1. Use established CLI tools instead of custom parsers. bcftools, vcftools, and similar domain-specific tools handle edge cases that custom parsers typically miss: multi-allelic sites, complex variants, missing genotypes, and proper transition/transversion classification. Writing a Python parser to count Ts/Tv is error-prone and unnecessary.

```bash # Preferred: use bcftools for Ts/Tv computation bcftools stats input.vcf | grep ^TSTV

# Preferred: use bcftools for variant counts bcftools stats input.vcf | grep ^SN ```

1. Report what filtering was applied (or not). Every time you present VCF summary statistics, state clearly whether the data was filtered, what threshold was used, and how many variants passed. This is essential for reproducibility and for the reader to assess the validity of the r