Basic introduction to package containerit
Daniel Nüst
2019-08-20
Source:vignettes/basic.Rmd
basic.RmdIntroduction
This R extension package provides features to bundle an R analysis together with the required runtime environment in so called software containers, more specifically Docker. The intention of this package is to provide a building block to support reproducible and archivable research. Development is supported by the DFG-funded project Opening Reproducible Research (http://o2r.info).
The core functionality is to create a Dockerfile from a given R session, script, or workspace directory. This Dockerfile contains all the R packages and their system dependencies required by the R workflow to be packaged.
The Dockerfiles are based on rocker (on Docker Hub). Eventually it should/could be possible to create images from scratch?
Dockerfile generation relies on the sysreqs package.
To build images and run containers, this package integrates with the harbor package.
For nitty gritty things like reading/loading/installing the exact versions, including system dependencies, internal and external libraries etc., this project is focused on the geospatial domain.
tl;dr
Load the package, do your analysis, and create a Dockerfile.
suppressPackageStartupMessages(library("containerit"))
# do stuff, based on demo("krige")
suppressPackageStartupMessages(library("gstat"))
suppressPackageStartupMessages(library("sp"))
data(meuse)
coordinates(meuse) = ~x+y
data(meuse.grid)
gridded(meuse.grid) = ~x+y
v <- variogram(log(zinc)~1, meuse)
m <- fit.variogram(v, vgm(1, "Sph", 300, 1))
plot(v, model = m)
## INFO [2019-08-20 16:44:17] Going online? TRUE ... to retrieve system dependencies (sysreq-api)
## INFO [2019-08-20 16:44:17] Trying to determine system requirements for the package(s) 'assertthat,backports,commonmark,crayon,curl,desc,digest,evaluate,FNN,formatR,fs,futile.logger,futile.options,gstat,htmltools,httpuv,intervals,jsonlite,knitr,lambda.r,later,lattice,magrittr,MASS,memoise,mime,miniUI,pillar,pkgconfig,pkgdown,promises,R6,Rcpp,rlang,rmarkdown,roxygen2,rprojroot,rstudioapi,semver,shiny,shinyFiles,sp,spacetime,stevedore,stringi,stringr,tibble,versions,xfun,xml2,xtable,xts,yaml,zoo' from sysreqs online DB
## INFO [2019-08-20 16:44:20] Adding CRAN packages: assertthat, backports, commonmark, crayon, curl, desc, digest, evaluate, FNN, formatR, fs, futile.logger, futile.options, gstat, htmltools, httpuv, intervals, jsonlite, knitr, lambda.r, later, lattice, magrittr, MASS, memoise, mime, miniUI, pillar, pkgconfig, pkgdown, promises, R6, Rcpp, rlang, rmarkdown, roxygen2, rprojroot, rstudioapi, semver, shiny, shinyFiles, sp, spacetime, stevedore, stringi, stringr, tibble, versions, xfun, xml2, xtable, xts, yaml, zoo
## INFO [2019-08-20 16:44:20] Created Dockerfile-Object based on sessionInfoThe Dockerfile object can be saved to a file or printed out.
FALSE FROM rocker/r-ver:3.6.1
FALSE LABEL maintainer="daniel"
FALSE RUN export DEBIAN_FRONTEND=noninteractive; apt-get -y update \
FALSE && apt-get install -y git-core \
FALSE libcurl4-openssl-dev \
FALSE libssl-dev \
FALSE libxml2-dev \
FALSE make \
FALSE pandoc \
FALSE pandoc-citeproc
FALSE RUN ["install2.r", "assertthat", "backports", "commonmark", "crayon", "curl", "desc", "digest", "evaluate", "FNN", "formatR", "fs", "futile.logger", "futile.options", "gstat", "htmltools", "httpuv", "intervals", "jsonlite", "knitr", "lambda.r", "later", "lattice", "magrittr", "MASS", "memoise", "mime", "miniUI", "pillar", "pkgconfig", "pkgdown", "promises", "R6", "Rcpp", "rlang", "rmarkdown", "roxygen2", "rprojroot", "rstudioapi", "semver", "shiny", "shinyFiles", "sp", "spacetime", "stevedore", "stringi", "stringr", "tibble", "versions", "xfun", "xml2", "xtable", "xts", "yaml", "zoo"]
FALSE WORKDIR /payload/
FALSE CMD ["R"]Dockerfile examples
Create Dockerfile from session
## [1] "sessionInfo"## INFO [2019-08-20 16:44:21] Going online? TRUE ... to retrieve system dependencies (sysreq-api)
## INFO [2019-08-20 16:44:21] Trying to determine system requirements for the package(s) 'assertthat,backports,commonmark,crayon,curl,desc,digest,evaluate,FNN,formatR,fs,futile.logger,futile.options,gstat,htmltools,httpuv,intervals,jsonlite,knitr,lambda.r,later,lattice,magrittr,MASS,memoise,mime,miniUI,pillar,pkgconfig,pkgdown,promises,R6,Rcpp,rjson,rlang,rmarkdown,roxygen2,rprojroot,rstudioapi,semver,shiny,shinyFiles,sp,spacetime,stevedore,stringi,stringr,tibble,versions,xfun,xml2,xtable,xts,yaml,zoo' from sysreqs online DB
## INFO [2019-08-20 16:44:24] Adding CRAN packages: assertthat, backports, commonmark, crayon, curl, desc, digest, evaluate, FNN, formatR, fs, futile.logger, futile.options, gstat, htmltools, httpuv, intervals, jsonlite, knitr, lambda.r, later, lattice, magrittr, MASS, memoise, mime, miniUI, pillar, pkgconfig, pkgdown, promises, R6, Rcpp, rjson, rlang, rmarkdown, roxygen2, rprojroot, rstudioapi, semver, shiny, shinyFiles, sp, spacetime, stevedore, stringi, stringr, tibble, versions, xfun, xml2, xtable, xts, yaml, zoo
## INFO [2019-08-20 16:44:24] Created Dockerfile-Object based on sessionInfo## An object of class "Dockerfile"
## Slot "image":
## An object of class "From"
## Slot "image":
## [1] "rocker/r-ver"
##
## Slot "postfix":
## An object of class "Tag"
## [1] "3.6.1"
##
##
## Slot "maintainer":
## An object of class "Label"
## Slot "data":
## $maintainer
## [1] "daniel"
##
##
## Slot "multi_line":
## [1] FALSE
##
##
## Slot "instructions":
## [[1]]
## An object of class "Run_shell"
## Slot "commands":
## [1] "export DEBIAN_FRONTEND=noninteractive; apt-get -y update"
## [2] "apt-get install -y git-core \\\n\tlibcurl4-openssl-dev \\\n\tlibssl-dev \\\n\tlibxml2-dev \\\n\tmake \\\n\tpandoc \\\n\tpandoc-citeproc"
##
##
## [[2]]
## An object of class "Run"
## Slot "exec":
## [1] "install2.r"
##
## Slot "params":
## [1] "assertthat" "backports" "commonmark" "crayon"
## [5] "curl" "desc" "digest" "evaluate"
## [9] "FNN" "formatR" "fs" "futile.logger"
## [13] "futile.options" "gstat" "htmltools" "httpuv"
## [17] "intervals" "jsonlite" "knitr" "lambda.r"
## [21] "later" "lattice" "magrittr" "MASS"
## [25] "memoise" "mime" "miniUI" "pillar"
## [29] "pkgconfig" "pkgdown" "promises" "R6"
## [33] "Rcpp" "rjson" "rlang" "rmarkdown"
## [37] "roxygen2" "rprojroot" "rstudioapi" "semver"
## [41] "shiny" "shinyFiles" "sp" "spacetime"
## [45] "stevedore" "stringi" "stringr" "tibble"
## [49] "versions" "xfun" "xml2" "xtable"
## [53] "xts" "yaml" "zoo"
##
##
## [[3]]
## An object of class "Workdir"
## Slot "path":
## [1] "/payload/"
##
##
##
## Slot "entrypoint":
## NULL
##
## Slot "cmd":
## An object of class "Cmd"
## Slot "exec":
## [1] "R"
##
## Slot "params":
## [1] NA
##
## Slot "form":
## [1] "exec"Create Dockerfile from script
This example uses the rgdal package because it has system library dependencies, namely GDAL and PROJ. Code snippets are taken from the sp gallery. Here is some regular R code loading a file and plotting it.
library("rgdal")
library("maptools")
nc <- readOGR(system.file("shapes/", package = "maptools"), "sids", verbose = FALSE)
proj4string(nc) <- CRS("+proj=longlat +datum=NAD27")
plot(nc)The code is not executed but dynamically saved to a temporary file, which is then used to create a Dockerfile.
scriptFile <- tempfile(pattern = "containerit_", fileext = ".R")
writeLines(text = c('library("sp")',
'library("rgdal")',
'nc <- readOGR(system.file("shapes/", package="maptools"), "sids", verbose = FALSE)',
'proj4string(nc) <- CRS("+proj=longlat +datum=NAD27")',
'plot(nc)'),
con = scriptFile)
print(
containerit::dockerfile(from = scriptFile)
)## Warning in dockerfileFromFile(file = from, dockerfile = the_dockerfile, :
## The given file is not inside the working directory! This may lead to
## incorrect COPY instructions.## INFO [2019-08-20 16:44:25] Processing R script file '../../../../../../tmp/RtmpS9XsVy/containerit_623425967d93.R' locally.
## INFO [2019-08-20 16:44:25] Creating an R session with the following expressions:
## source(file = "/tmp/RtmpS9XsVy/containerit_623425967d93.R", echo = TRUE)
## INFO [2019-08-20 16:44:27] Going online? TRUE ... to retrieve system dependencies (sysreq-api)
## INFO [2019-08-20 16:44:27] Trying to determine system requirements for the package(s) 'assertthat,backports,crayon,curl,desc,digest,formatR,fs,futile.logger,futile.options,htmltools,httpuv,jsonlite,lambda.r,later,lattice,magrittr,mime,miniUI,pillar,pkgconfig,promises,R6,Rcpp,rgdal,rlang,rprojroot,rstudioapi,semver,shiny,shinyFiles,sp,stevedore,stringi,stringr,tibble,versions,xtable' from sysreqs online DB
## INFO [2019-08-20 16:44:29] Adding CRAN packages: assertthat, backports, crayon, curl, desc, digest, formatR, fs, futile.logger, futile.options, htmltools, httpuv, jsonlite, lambda.r, later, lattice, magrittr, mime, miniUI, pillar, pkgconfig, promises, R6, Rcpp, rgdal, rlang, rprojroot, rstudioapi, semver, shiny, shinyFiles, sp, stevedore, stringi, stringr, tibble, versions, xtable
## INFO [2019-08-20 16:44:29] Created Dockerfile-Object based on /tmp/RtmpS9XsVy/containerit_623425967d93.R
## FROM rocker/r-ver:3.6.1
## LABEL maintainer="daniel"
## RUN export DEBIAN_FRONTEND=noninteractive; apt-get -y update \
## && apt-get install -y gdal-bin \
## libcurl4-openssl-dev \
## libgdal-dev \
## libproj-dev \
## libssl-dev \
## make
## RUN ["install2.r", "assertthat", "backports", "crayon", "curl", "desc", "digest", "formatR", "fs", "futile.logger", "futile.options", "htmltools", "httpuv", "jsonlite", "lambda.r", "later", "lattice", "magrittr", "mime", "miniUI", "pillar", "pkgconfig", "promises", "R6", "Rcpp", "rgdal", "rlang", "rprojroot", "rstudioapi", "semver", "shiny", "shinyFiles", "sp", "stevedore", "stringi", "stringr", "tibble", "versions", "xtable"]
## WORKDIR /payload/
## CMD ["R"]Create Dockerfile from directory
This examples packages a workspace directory.
## [1] "cli" "containerit_config.json"
## [3] "demo_sf.html" "demo_sf.Rmd"
## [5] "demo.html" "demo.Rmd"
## [7] "docker" "label-schema_1.0.0-rc.1.txt"
## [9] "logo.svg" "rstudio"
## [11] "scripts" "simple_test_script_resources"containerit looks for files in a directory that can be executed and creates a Dockerfile based on the first document found, in this case an R Markdown document.
## Warning in dockerfileFromWorkspace(path = from, dockerfile = the_dockerfile, : Found both scripts and weaved documents (Rmd) in the given directory. Using the first document for packaging:
## ../inst/demo_sf.Rmd## INFO [2019-08-20 16:44:30] Found file for packaging in workspace: ../inst/demo_sf.Rmd## Warning in dockerfileFromFile(target_file, dockerfile = dockerfile, soft =
## soft, : The given file is not inside the working directory! This may lead
## to incorrect COPY instructions.## INFO [2019-08-20 16:44:30] Processing Rmd file '../inst/demo_sf.Rmd' locally using rmarkdown::render(...)
## INFO [2019-08-20 16:44:30] Creating an R session with the following expressions:
## rmarkdown::render("../inst/demo_sf.Rmd")
## INFO [2019-08-20 16:44:34] Going online? TRUE ... to retrieve system dependencies (sysreq-api)
## INFO [2019-08-20 16:44:34] Trying to determine system requirements for the package(s) 'assertthat,backports,class,classInt,crayon,curl,DBI,desc,digest,e1071,evaluate,formatR,fs,futile.logger,futile.options,htmltools,httpuv,jsonlite,KernSmooth,knitr,lambda.r,later,magrittr,mime,miniUI,pillar,pkgconfig,promises,R6,Rcpp,rlang,rmarkdown,rprojroot,rstudioapi,semver,sf,shiny,shinyFiles,stevedore,stringi,stringr,tibble,units,versions,xfun,xtable,yaml' from sysreqs online DB
## INFO [2019-08-20 16:44:36] Adding CRAN packages: assertthat, backports, class, classInt, crayon, curl, DBI, desc, digest, e1071, evaluate, formatR, fs, futile.logger, futile.options, htmltools, httpuv, jsonlite, KernSmooth, knitr, lambda.r, later, magrittr, mime, miniUI, pillar, pkgconfig, promises, R6, Rcpp, rlang, rmarkdown, rprojroot, rstudioapi, semver, sf, shiny, shinyFiles, stevedore, stringi, stringr, tibble, units, versions, xfun, xtable, yaml
## INFO [2019-08-20 16:44:36] Created Dockerfile-Object based on ../inst## FROM rocker/r-ver:3.6.1
## LABEL maintainer="daniel"
## RUN export DEBIAN_FRONTEND=noninteractive; apt-get -y update \
## && apt-get install -y gdal-bin \
## git-core \
## libcurl4-openssl-dev \
## libgdal-dev \
## libgeos-dev \
## libgeos++-dev \
## libssl-dev \
## libudunits2-dev \
## make \
## pandoc \
## pandoc-citeproc
## RUN ["install2.r", "assertthat", "backports", "class", "classInt", "crayon", "curl", "DBI", "desc", "digest", "e1071", "evaluate", "formatR", "fs", "futile.logger", "futile.options", "htmltools", "httpuv", "jsonlite", "KernSmooth", "knitr", "lambda.r", "later", "magrittr", "mime", "miniUI", "pillar", "pkgconfig", "promises", "R6", "Rcpp", "rlang", "rmarkdown", "rprojroot", "rstudioapi", "semver", "sf", "shiny", "shinyFiles", "stevedore", "stringi", "stringr", "tibble", "units", "versions", "xfun", "xtable", "yaml"]
## WORKDIR /payload/
## CMD ["R"]
Configure Dockerfile
You can skip available packages and choose your own base image.
df <- containerit::dockerfile(from = "../inst", image = "rocker/geospatial:3.5.2", filter_baseimage_pkgs = TRUE)## Warning in dockerfileFromWorkspace(path = from, dockerfile = the_dockerfile, : Found both scripts and weaved documents (Rmd) in the given directory. Using the first document for packaging:
## ../inst/demo_sf.Rmd## INFO [2019-08-20 16:44:38] Found file for packaging in workspace: ../inst/demo_sf.Rmd## Warning in dockerfileFromFile(target_file, dockerfile = dockerfile, soft =
## soft, : The given file is not inside the working directory! This may lead
## to incorrect COPY instructions.## INFO [2019-08-20 16:44:38] Processing Rmd file '../inst/demo_sf.Rmd' locally using rmarkdown::render(...)
## INFO [2019-08-20 16:44:38] Creating an R session with the following expressions:
## rmarkdown::render("../inst/demo_sf.Rmd")## Detected API version '1.40' is above max version '1.39'; downgrading
## Detected API version '1.40' is above max version '1.39'; downgrading## INFO [2019-08-20 16:44:43] Skipping packages for image rocker/geospatial:3.5.2 (packages are unversioned): assertthat, backports, class, classInt, crayon, curl, DBI, desc, digest, e1071, evaluate, formatR, fs, htmltools, httpuv, jsonlite, KernSmooth, knitr, later, magrittr, mime, miniUI, pillar, pkgconfig, promises, R6, Rcpp, rlang, rmarkdown, rprojroot, rstudioapi, sf, shiny, stringi, stringr, tibble, units, xfun, xtable, yaml
## INFO [2019-08-20 16:44:43] Going online? TRUE ... to retrieve system dependencies (sysreq-api)
## INFO [2019-08-20 16:44:43] Trying to determine system requirements for the package(s) 'futile.logger,futile.options,lambda.r,semver,shinyFiles,stevedore,versions' from sysreqs online DB
## INFO [2019-08-20 16:44:44] Adding CRAN packages: futile.logger, futile.options, lambda.r, semver, shinyFiles, stevedore, versions
## INFO [2019-08-20 16:44:44] Created Dockerfile-Object based on ../inst## FROM rocker/geospatial:3.5.2
## LABEL maintainer="daniel"
## # CRAN packages skipped because they are in the base image: assertthat, backports, class, classInt, crayon, curl, DBI, desc, digest, e1071, evaluate, formatR, fs, htmltools, httpuv, jsonlite, KernSmooth, knitr, later, magrittr, mime, miniUI, pillar, pkgconfig, promises, R6, Rcpp, rlang, rmarkdown, rprojroot, rstudioapi, sf, shiny, stringi, stringr, tibble, units, xfun, xtable, yaml
## RUN ["install2.r", "futile.logger", "futile.options", "lambda.r", "semver", "shinyFiles", "stevedore", "versions"]
## WORKDIR /payload/
## CMD ["R"]For extended configuration options see the vignettes online at https://o2r.info/containerit/articles/.