{
 "metadata": {
  "name": ""
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Variant calling pipelines: quantification, scaling and community development\n",
      "\n",
      "<div class=\"author\">\n",
      "Brad Chapman<br>\n",
      "Bioinformatics Core, Harvard School of Public Health<br>\n",
      "<a href=\"https://github.com/chapmanb\">@chapmanb</a><br>\n",
      "20 July 2013<br>\n",
      "</div>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "### Challenges: complex and rapidly changing pipelines\n",
      "\n",
      "<img alt=\"highlevel\" src=\"files/images/gatk_changes.png\" width=\"700\"/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "### Challenges: large number of specialized dependencies\n",
      "\n",
      "<img src=\"files/images/huge_seq.png\" width=\"300\"/>\n",
      "\n",
      "[https://github.com/StanfordBioinformatics/HugeSeq](https://github.com/StanfordBioinformatics/HugeSeq)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "### Challenges: quality differences between calling methods\n",
      "\n",
      "<img alt=\"highlevel\" src=\"files/images/gcat_comparison.png\" width=\"500\"/>\n",
      "\n",
      "[http://www.bioplanet.com/gcat](http://www.bioplanet.com/gcat)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "### Challenges: generalized distributed systems that scale\n",
      "\n",
      "<img alt=\"highlevel\" src=\"files/images/schedulers.png\" width=\"1000\"/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Solution\n",
      "\n",
      "<img alt=\"highlevel\" src=\"files/images/community.png\" width=\"400\"/>\n",
      "\n",
      "[http://www.amazon.com/Community-Structure-Belonging-Peter-Block/dp/1605092770](http://www.amazon.com/Community-Structure-Belonging-Peter-Block/dp/1605092770)\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "<img alt=\"highlevel\" src=\"files/images/bcbio_nextgen_highlevel.png\" width=\"1000\"/>"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Development goals\n",
      "\n",
      "- Quantifiable: assess variant quality\n",
      "- Scalable: 1500 whole genome samples\n",
      "- Reproducible: text-configurable, provenance, version tracking\n",
      "- Community developed: open source, documented and widely deployable\n",
      "- Accessible: Usable by researchers and non-scientists"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Quantify quality\n",
      "\n",
      "<img src=\"files/images/grading-summary-prep-callerdiff.png\" width=\"900\">\n",
      "\n",
      "- Reference materials: [http://www.genomeinabottle.org](http://www.genomeinabottle.org)\n",
      "- Quantification details: [http://j.mp/bcbioeval](http://j.mp/bcbioeval)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Parallel scaling\n",
      "\n",
      "<img src=\"files/images/parallel-clustertypes.png\" width=\"900\">\n",
      "\n",
      "- Infrastructure details: [http://j.mp/bcbioscale](http://j.mp/bcbioscale)\n",
      "- IPython: [http://ipython.org/ipython-doc/dev/parallel/index.html](http://ipython.org/ipython-doc/dev/parallel/index.html)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "## Increasing parallel blocks\n",
      "\n",
      "<img src=\"files/images/parallel-genome.png\" width=\"800\">"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      " ## Reproducible configuration\n",
      " \n",
      " \n",
      "    - files: [NA12878-NGv3-LAB1360-A_1.fastq.gz, NA12878-NGv3-LAB1360-A_2.fastq.gz]\n",
      "      description: NA12878\n",
      "      analysis: variant2\n",
      "      genome_build: GRCh37\n",
      "      algorithm:\n",
      "        aligner: bwa\n",
      "        recalibrate: gatk\n",
      "        realign: gatk\n",
      "        variantcaller: [gatk, freebayes, gatk-haplotype]\n",
      "        coverage_interval: exome\n",
      "        coverage_depth: high\n",
      "        platform: illumina\n",
      "        quality_format: Standard\n",
      "        validate: NA12878-nist-v2_13-NGv3-pass.vcf\n"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}