#+BLOG: smallchangebio
#+TITLE: Notes from Arvados summit: workshop and discussion on open biomedical informatics
#+CATEGORY: conference
#+TAGS: bioinformatics, arvados, clinical, open-source, open-science
#+OPTIONS: toc:nil num:nil

I'm at [[hack-reduce][hack/reduce]] in Cambridge attending the [[arvados-summit][Arvados summit]]
focused on open-source tools and datasets for medical
informatics. It's specifically centered around [[arvados][Arvados]], an open source
infrastructure from [[clinical-future][Clinical Future]] for managing and analyzing genomic
data.

[[tom-clegg][Tom Clegg]] starts off the discussion with discussion about the goals of
Arvados. The type of problems it tries to address are: too much data,
poor structure of data in work directories, failures during large
compute jobs. General idea: turning a research pipeline into a
clinical product. Shows an example of a painful directory of shell
scripts with tons of versions: we don't want that. On the other side
are complex pipelines that are difficult to reproduce because of
complexity.

Tom starts with the [[arvados-tech-arch][technical architecture]] and naming of parts:

- [[arvados-keep][Keep]] -- Content addressable file store. Uses hashes to refer to
  datasets instead of filesystem hierarchy.
- [[arvados-crunch][Crunch]] -- Map/reduce engine
- Lightning -- In memory compact genomic databases

He then walks through some [[tutorial][Tutorial documentation]] about what it is
like to use Arvados. He shows an example of searching PGP data by
specific traits, using the Python API.

Advantages of running in Arvados and using the API:
repeatable, inspectable, searchable, identified with a content hash so
safe from accidents and de-duplicated. It scales across tasks, nodes
and data and provides detailed logs and timing statistics.

Adam followed up this discussion with the status of Arvados and the
roadmap. The goals for November and December are to expand the current
Sandbox running on EC2, and improve crunch tools and documentation. In
January hope to have a new release. Second summit in April at Bio-IT
world. Goal for a production ready release in Summer 2014. Major areas
of work are to establish best practices for metadata, refactor Keep
components and work on the display UI.

Clinical Future plans to create the Arvados Foundation, whose mission
is to enable precision medicine through transparent informatics.
Clinical Future plans to monetize based on supported installation of
Arvados, while the actual code and infrastructure will remain open
source. General idea of transparent informatics is that software and
data are available. Arvados Foundation promotes these and maintains an
infrastructure around development similar to Apache Foundation.

Sasha Zaranek presents Lightning, a real time query-engine for the
Arvados platform. The idea is to provide a system to query patients
with associated metadata. Query ideas like the goals of the [[gemini][GEMINI]] system,
with the idea of extending to biomedical specific metadata about
samples, data sharing between organizations and privacy. Immediate
goals including using the [[pgp][Personal Genome Project]] as a high quality
reference dataset to get started with. A key idea of Lightning is to
have a compact genome format that encompasses multiple formats for
comparison: [[xkcd-standards][requisite xkcd]]. The format tiles variants with unique
200-500bp + 25bp tags across a reference genome, reporting
simple variants within a single tile and complex variants across
multiple tiles. Idea of being separate from assemblies is a
nice approach.

Francisco De La Vega talked about work at [[rtg][Real Time Genomics]] to call
high quality variants in individuals, populations and pedigrees. They
have a special focus on rare early childhood diseases: autosomal
recessives, de-novo. Their algorithm takes into account pedigree
information when doing trio calling, and adds in phasing information.
Trio information also allows reduction of Mendelian inconsistency
errors. Also helps reduce de-novo false positives in difficult regions
of the genome.

[[melissa][Melissa Gymrek]] talked about her work at the Whitehead to use short
tandem repeats (STRs) for genome profiling. Her cleverly named [[lobstr][lobSTR]]
tool profiles STRs from high throughput sequencing data. It detects
STRs, handles alignment and variant detection. They used this tool to
re-identify anonymous personal genomes. Currently profiling STRs in
thousands of genomic datasets. Major challenge is downloading and
preparing data: good use case for Arvados. lobSTR + Arvados pipeline:
handles custom alignment to BAM files, then allelotype into VCF
output. Melissa has crunch scripts which run lobSTR on Arvados.

[[cypher][Cypher Genomics]] talked locally and remotely about their work spun out
of Scripps. They have a custom annotation engine and big integrated
database to annotate and query variants. Have nice results ranking
and prioritizing variants for followup. Some of their goals for
Arvados: multiple location deployment + dashboards for cluster
management.

#+LINK: arvados-summit https://arvados.org/projects/arvados/wiki/Arvados_Summit_-_Fall_2013
#+LINK: hack-reduce http://www.hackreduce.org/
#+LINK: arvados https://arvados.org/
#+LINK: clinical-future http://clinicalfuture.com/
#+LINK: tom-clegg https://github.com/tomclegg
#+LINK: arvados-tech-arch https://arvados.org/projects/arvados/wiki/Technical_Architecture
#+LINK: arvados-keep https://arvados.org/projects/arvados/wiki/Keep
#+LINK: arvados-crunch https://arvados.org/projects/arvados/wiki/Computation_and_Pipeline_Processing
#+LINK: tutorial http://doc.arvados.org/user/
#+LINK: gemini https://github.com/arq5x/gemini
#+LINK: pgp http://www.personalgenomes.org/
#+LINK: xkcd-standards http://www.xkcd.com/927/
#+LINK: rtg http://www.realtimegenomics.com/
#+LINK: melissa http://melissagymrek.com/
#+LINK: lobstr http://erlichlab.wi.mit.edu/lobSTR/
#+LINK: cypher http://www.cyphergenomics.com/
