3.4.0 (Lion)

GoNL part

Final version GoNL pipeline on b38, reference genome is ucsc build 38 ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.gz

new protocols

• combine TrimAdapters, SpikePhiX and CheckIlluminaEncoding in one protocol (PrepareFastQ.sh)

new files

• generate_template_gonl.sh
• workflow_gonl.csv
• batch_b38_chr.csv
• workflow_gonl_validating.sh (2 rounds of FastQC and MarkDuplicates)

Version updates

• dbSNP version 149 (All_20161122.vcf.gz)

Regular pipeline updates

Version updates

• GATK 3.6 --> 3.7
• picard 1.130 --> 2.9
• FastQC 0.11.3 --> 0.11.5
• BWA 0.7.12 --> 0.7.15
• sambamba v0.6.4 -> v0.6.6

New tools

• Using SeqTk (1.2) for the converting the fastQ files from Illumina 1.5 to 1.9 encoding

bugfixes + updates

• for chr Y ploidy set to 1 (instead of 2)
• added newQual argument in VariantCalling (see https://software.broadinstitute.org/gatk/blog?id=8692)
• removed VariantAnnotator out of workflow: HaplotypeCaller can change the bam by the internally indel realignment and VariantAnnotator is updating the genotypes based on the dedup.bam, this is basically not what we want
• added summary metrics file in GCbias metrics
• removed stand_call_conf 10.0 in genotyping step, default is 10.0
• removed stand_emit_conf 20.0 in genotyping step, deprecated in GATK 3.7
• combine Bwa align with SortBam (by FIFO piping) in one step
• combine SpikePhiX and CheckIlluminaEncoding in one protocol (PrepareFastQ.sh)

3.3.3 Koala

Regular pipeline

• Gavin output name is smaller
• Fill in complete path to external samples in samplesheet when running external samples

---

GoNL first version

• gonl workflow
• gonl parameters file

new protocols:

TrimAdapters Trimming of Illumina adapter
BaseRecalibrator prepare BQSR step
Base Quality Score Recalibration rescoring base qualities
VerifyBamID Check if the bam not is contanimated or that there was a sample swap
PhaseByTransmission Trio aware phasing

3.3.2 (Jackal)

HMF workflow implementation

3.3.1 (Iguana)

New features

DecisionTree (for Convading and XHMM results)

Fixes

• fixing crash in XHMM when Gender is unknown
• removing phiX reads from ToCadd file, is producing errors

3.3.0 (Hummingbird)

New features

• XHMM
• Convading
• Made script that will generate XHMM and Convading controlsDir
• Gavin (running CADD locally)
• added genderCheck.sh to generate script
• Annotating with SnpEff, CADD, ExAC and GoNL
• VEP annotation (side step after genotyping (it will not be copied to resultsdir)
• Manta as main CNV caller (still researchy, diagnostics validation is still not done)
• copy gVCF WGS samples to prm at the end of the pipeline

Updates:

• gender column can now be used by Convading
• VariantCalling is now outputting in gzip format
• Compressing variant vcf's in resultsdirectory
• Optimizing workflow due to parallelisation of certain steps
• removing windows line breaks automatically in CreateInhouseProjects step
• Removed VariantCompressing protocol since VariantCalling will automatically compresses its output

Version updates

• Manta (0.29.5 to 1.0.1)
• SnpEff (4.1g to 4.3)
• SnpEff database change from GRCh37.75 to hg19

Minor

• added CoverageCalculations as dependency on countallfinishedfiles
• CoverageCalculations has now $panel.interval_list instead of $panel.bed
• removed unused and difficult way of making duplicate statistics
• Fix in output genderCheck calculation
• Added Convading and XHMM Controlsdir used in QCReport
• Automatic detection if data is from a panel or not (Removed diagnostics=diagnostics column check)

3.2.6 (Gibbon) (minor update)

• upgrade to Python-2.7.10-foss-2015b in order for it to work on zinc-finger
• removed printing delly and cutadapt versions in QCReport
• added printing manta version to QCReport
• removed unused python module loading in CoverageCalculations
• removed unused module load NGS_Automated in CopyToResultsDir
• removed NGS_Automated string out of parameters.csv

3.2.5 (Flamengo)

new Features

- XHMM - Convading - Manta - compressing g.vcf files

New workflows and generatescripts

- reanalysis - WGS reanalysis (including genotyping with old samples)

Differences:

- genotyping has compressed g.vcf file as input - upgrade to GATK 3.6 - updated CoverageCalculations (fixing bedfile format issue, that there were no gene names in the output)

3.2.4 (Emu)

Bugfix: updated to newer version of sambamba (0.6.1 --> 0.6.3)

added autotesting (via Jenkins)

3.2.3 (Dingo)

• Writing coverage results to seperate folders (CoveragePerBase and CoveragePerTarget)
• updated to GATK 3.5
• removed dbNSFP and snpEff annotation (cannot work with the * vcf 4.1 annotation in case of deletion in alt)
• removed old QC report scripts

3.2.2 (Caribou)

• Changed CoveragePerBase protocol into CoverageCalculations
• Added Coverage calculations for subset bed files (LEVER, NEURO, SSID and GEO)
• Changed output for coverage per target and coverage per base
• CoveragePerBase added missing make genesOnly file + VariantCalling bug (double brackets)
• increased memory for MarkDuplicates
• improved the copy prmtotmp data (when prm is not available on zinc-finger it will download it from calculon)
• bugfix in CheckIlluminaEncoding (where '++' is not used anymore, it is now '+1')
• added automating:
  • start of pipeline
  • copying to prm
  • copying to zinc-finger
• fixed the duplication rate in the QC report
• Made new QCReport (in html)
• fix issue with samples on multiple lanes and counting number of samples in generate.sh

Release notes of older updates are below (track changes are lost in moving to own repo, older versions can be downloaded from https://github.com/molgenis/molgenis-pipelines/releases)

3.2.1

Major changes:

• Replaced Picard tools SortBam, MergeBam and MarkDuplicates for Sambamba tools
• renamed CheckSex protocol to GenderCheck
• Splitted the metrics calculations
• added CramConversion (turned off in workflow)
• added CoveragePerTarget
• updated to new compute version v15.12.4
• upgraded to Java 8

Minor changes + bugfixes

• fixed GenderCheck for a WGS sample
• extended walltime for Delly and added more memory
• changed PERCENTAGE_DUPLICATES to PERCENT_DUPLICATION in dedup metrics (QC report)
• added new batch options (per chr or 1 batch)
• added #string project to every protocol
• fixed VariantCalling bug (sex is Unknown bug)

3.1.2

• UPDATED the old version numbers, newest NGS_DNA version is now in the directory NGS_DNA
• added DellyAnnotator
• fixed >200-samples issue
• added pipeline version to QCReport
• bugfixes
• removed IndelRealigment step
• splitted collect metrics step into 4 steps