#+title: Interoperable community developed variant calling with bcbio and the Common Workflow Language
#+author: Brad Chapman \newline Bioinformatics Core, Harvard Chan School \newline https://bcb.io \newline http://j.mp/bcbiolinks
#+date: 3 November 2016

#+OPTIONS: toc:nil H:2

#+startup: beamer
#+LaTeX_CLASS: beamer
#+latex_header: \usepackage{url}
#+latex_header: \usepackage{hyperref}
#+latex_header: \hypersetup{colorlinks=true}
#+BEAMER_THEME: default
#+BEAMER_COLOR_THEME: seahorse
#+BEAMER_INNER_THEME: rectangles

* Introduction
** Overview
\Large
- Open source communities
- bcbio: community developed analyses
- Value of variant validation
- Interoperable infrastructure

* Motivation

** Supporting bioinformatics

#+BEGIN_CENTER
#+ATTR_LATEX: :width 1.0\textwidth
[[./images10/biostars.png]]
#+END_CENTER

https://www.biostars.org/

** Open source community

#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.8\textwidth
[[./images10/bosc2016.png]]
#+END_CENTER

http://www.open-bio.org/wiki/BOSC_2016

** Build communities

#+BEGIN_CENTER
#+ATTR_LATEX: :width .5\textwidth
[[./images/community.png]]
#+END_CENTER

\scriptsize
[[http://www.amazon.com/Community-Structure-Belonging-Peter-Block/dp/1605092770]]
\normalsize

** Working together produces great things

#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.85\textwidth
[[./images10/gnomad.png]]
#+END_CENTER

http://gnomad.broadinstitute.org/about

** Challenge: many communities

#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.4\textwidth
[[./images10/usegalaxy.png]]
#+END_CENTER
#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.4\textwidth
[[./images10/cbioportal_logo.png]]
#+END_CENTER
#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.4\textwidth
[[./images10/synapse.png]]
#+END_CENTER

\vspace{0.5cm}

https://galaxyproject.org/ \\
http://www.cbioportal.org/ \\
https://www.synapse.org/

** Challenge: open source communities not yet optimal

[[./images10/fitness.png]]

\tiny
https://en.wikipedia.org/wiki/Fitness_landscape

* Open source bioinformatics

** Large scale infrastructure development

\Large
- Shared problems -- academic, industry, startups
- Community developed analyses
- Validation
- Scaling
- Supporting a community of users
\normalsize

** White box software

[[./images5/clear_box.jpg]]

** Overview

#+ATTR_LATEX: :width 1.0\textwidth
[[./images3/bcbio_nextgen_highlevel.png]]

\vspace{1cm}
https://github.com/chapmanb/bcbio-nextgen

** High level configuration

#+BEGIN_SRC yaml
  - analysis: variant2
    genome_build: hg38
    algorithm:
      aligner: bwa
      mark_duplicates: true
      recalibrate: false
      realign: false
      variantcaller: [gatk-haplotype, freebayes, vardict]
      ensemble:
        numpass: 2
      svcaller: [lumpy, manta]
#+END_SRC

\footnotesize
https://bcbio-nextgen.readthedocs.org/en/latest/contents/configuration.html

** Uses

\Large
- Aligners: bwa, novoalign, bowtie2, HISAT2
- Variantion: FreeBayes, GATK, VarDict, MuTecT2, Scalpel, SnpEff, VEP, GEMINI,
  Lumpy, Manta, CNVkit, WHAM
- RNA-seq: Tophat, STAR, Cufflinks, Sailfish
- Quality control: FastQC, samtools, Qualimap, MultiQC
- Manipulation: bedtools, bcftools, biobambam, picard, sambamba, samblaster, samtools,
  vcflib, vt
\normalsize

** Provides

\Large
- Community -- collected set of expertise
- Installation of tools and data
- Tool integration
- Validation -- outputs + automated evaluation
- Scaling
\normalsize

** We made a pipeline -- so what?

\tiny
#+BEGIN_QUOTE
There have been a number of previous efforts to create publicly available
analysis pipelines for high throughput sequencing data. Examples include
Omics-Pipe, bcbio-nextgen, TREVA and NGSane. These pipelines
offer a comprehensive, automated process that can analyse raw sequencing reads
and produce annotated variant calls. However, the main audience for these
pipelines is the research community. Consequently, there are many features
required by clinical pipelines that these examples do not fully address. Other
groups have focused on improving specific features of clinical pipelines. The
Churchill pipeline uses specialised techniques to achieve high performance,
while maintaining reproducibility and accuracy. However it is not freely
available to clinical centres and it does not try to improve broader clinical
aspects such as detailed quality assurance reports, robustness, reports and
specialised variant filtering. The Mercury pipeline offers a comprehensive
system that addresses many clinical needs: it uses an automated workflow system
(Valence) to ensure robustness, abstract computational resources and
simplify customisation of the pipeline. Mercury also includes detailed coverage
reports provided by ExCID, and supports compliance with US privacy laws
(HIPAA) when run on DNANexus, a cloud computing platform specialised for
biomedical users. Mercury offers a comprehensive solution for clinical users,
however it does not achieve our desired level of transparency, modularity and
simplicity in the pipeline specification and design. Further, Mercury does not
perform specialised variant filtering and prioritisation that is specifically
tuned to the needs of clinical users.
#+END_QUOTE

\scriptsize
http://www.genomemedicine.com/content/7/1/68

** Sustainability

\Large
A piece of software is being sustained if people are using it, fixing it, and
improving it rather than replacing it.

\vspace{0.5cm}

\normalsize
http://software-carpentry.org/blog/2014/08/sustainability.html

** Complex, rapidly changing baseline functionality

[[./images2/gatk_changes.png]]
** Community: sustainability

#+ATTR_LATEX: :width 1.1\textwidth
[[./images9/bcbio_commits_sep2016.png]]

\vspace{1cm}

[[https://github.com/chapmanb/bcbio-nextgen]]

** Community: support

#+ATTR_LATEX: :width 1.1\textwidth
[[./images9/bcbio_issues_sep2016.png]]

\vspace{1cm}

[[https://bcbio-nextgen.readthedocs.org]]

** Community: contribution

[[./images9/bcbio_github_sep2016.png]]

[[https://github.com/chapmanb/bcbio-nextgen]]

** Community: documentation

[[./images/community-docs.png]]

[[https://bcbio-nextgen.readthedocs.org]]

** Supported analysis types

#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.4\textwidth
[[./images9/bcbio_pipelines.png]]
#+END_CENTER

\scriptsize
https://bcbio-nextgen.readthedocs.org/en/latest/contents/pipelines.html

* Validation

** Value of validation
\LARGE
- Integration tests for pipelines
- Unbiased algorithm comparisons
- Baseline for improving methods

** Reference materials

#+BEGIN_CENTER
#+ATTR_LATEX: :width .5\textwidth
[[./images/giab.png]]

#+ATTR_LATEX: :width .7\textwidth
[[./images7/ga4gh.png]]

#+ATTR_LATEX: :width .9\textwidth
[[./images7/dream_challenge.png]]
#+END_CENTER

http://www.genomeinabottle.org/
http://ga4gh.org/\#/benchmarking-team
https://www.synapse.org/\#!Synapse:syn312572

** GATK and Genome in a Bottle
\Large
- Collaboration with GATK methods development
- Compare HaplotypeCaller to other methods
- Germline validation
- Genome in a Bottle reference materials
  - \Large NA12878 -- Caucasian
  - \Large NA24385 -- Ashkenazim Jewish
  - \Large NA24631 -- Chinese

** Validation graphs

#+ATTR_LATEX: :width 1.0\textwidth
[[./images10/grading-example.png]]

** NA12878

#+ATTR_LATEX: :width 1.0\textwidth
[[./images10/gatk-giab-NA12878.png]]

** NA24385

#+ATTR_LATEX: :width 1.0\textwidth
[[./images10/gatk-giab-NA24385.png]]

** Validation results

\Large
- Good performance for GATK HaplotypeCaller
- Other good performing callers: FreeBayes
- Consistency across diverse samples
- Identify potential problem areas for tuning
   - \Large samtools Indel false positive rates
   - \Large Platypus SNP sensitivity
- PrecisionFDA: https://precision.fda.gov/

* Interoperable infrastructure

** Infrastructure Goals
\Large
- Free, open source, community developed
- Welcoming to contributions
- Local machines
- Clusters: SLURM, SGE, Torque, PBS, LSF
- Clouds: Amazon, Google, Azure
- Clinical environments
- User interface for researchers
- Integrate with LIMS
- Accessible to the general public

** Challenge: open source communities not yet optimal

[[./images10/fitness.png]]

\tiny
https://en.wikipedia.org/wiki/Fitness_landscape

** Better abstractions = more interoperability

[[./images10/abstractions.png]]

\scriptsize
https://bcbio-nextgen.readthedocs.io/en/latest/contents/cwl.html

** Workflow Description Language (WDL)

#+BEGIN_CENTER
#+ATTR_LATEX: :width 0.4\textwidth
[[./images10/WDL-workflow.png]]

#+ATTR_LATEX: :width 0.55\textwidth
[[./images10/WDL-scatter.jpg]]
#+END_CENTER

https://software.broadinstitute.org/wdl/

** Common Workflow Language (CWL)


#+ATTR_LATEX: :width 1.0\textwidth
[[./images10/cwl_pipeline_example.png]]

#+BEGIN_CENTER
http://www.commonwl.org/ \\
\vspace{0.5cm}
\scriptsize
https://f1000research.com/slides/5-1617
#+END_CENTER

** Workflow vs Workflow

\Huge

#+BEGIN_CENTER
Abstraction > Implementation

\vspace{1cm}

$WDL \leftrightarrow CWL$
#+END_CENTER

** CWL in bcbio

\Large
- Start with high level configuration file
- Generate CWL
- Run CWL:
   - \Large Any infrastructure that supports CWL
   - \Large Generated CWL
   - \Large Docker or local bcbio installation
   - \Large Genome data

\scriptsize
https://bcbio-nextgen.readthedocs.io/en/latest/contents/cwl.html

** Why use a workflow abstraction?

\Large
- Integrate with multiple platforms
   - \Large Arvados
   - \Large Toil
   - \Large Cromwell
   - \Large Galaxy
   - \Large Nextflow
   - \Large Seven Bridges
   - \Large DNAnexus
- Stop maintaining bcbio specific infrastructure
- Focus on hard biological problems

** Arvados

[[./images10/arvados.png]]

https://arvados.org/ \\
https://cloud.curoverse.com/

** Toil

\Large
- UCSC NIH Big Data to Knowledge Center for Translational Genomics
- Supports CWL via conversion to internal Python workflow description
- Local HPC support: SLURM, SGE
- Cloud: AWS + spot instances

http://toil.readthedocs.io/en/latest/

** Cromwell via conversion to WDL

\Large
- bcbio workflow abstractions supported in WDL
   - \Large Tasks, workflows, nested workflows
   - \Large Scatter based parallelization
   - \Large Grouping/batching of samples
- Work in progress CWL to WDL converter based on cwl2wdl
- Happy to collaborate

\scriptsize
https://github.com/broadinstitute/cromwell
https://github.com/chapmanb/bcbio-nextgen/blob/master/scripts/utils/cwltool2wdl.py

** Galaxy

\Large
- CWL support in progress
- Supports subset of CWL -- tool definitions
- Needs workflow support

\normalsize
https://github.com/galaxyproject/planemo \\
https://f1000research.com/posters/5-2567

** Nextflow

#+ATTR_LATEX: :width 0.9\textwidth
[[./images10/nextflow_cwl.png]]

https://www.nextflow.io/

** Seven Bridges

\Large
- Currently supports CWL v2 + extensions
- Moving to CWL v1.0
- External runner, Bunny, supports v1.0

\normalsize
https://github.com/rabix/bunny \\
http://docs.sevenbridges.com/docs/sdk-overview

** DNAnexus

\Large
- Integration of bcbio Docker containers
- Single node runs
- Convert CWL to DNAnexus API for distributed

\normalsize
https://www.dnanexus.com/developer-resources

** Connections

#+ATTR_LATEX: :width 0.6\textwidth
[[./images10/Network_Community_Structure.png]]

\scriptsize
By jham3 - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=17125894

* Summary
** Summary
\Large
- bcbio community developed resources
- Value of validation
   - \Large Germline calling with Genome in a Bottle
- Interoperable infrastructure
   - \Large Importance of abstractions
   - \Large Common Workflow Language
   - \Large Practical interoperability

\vspace{0.5cm}
http://bcb.io
