Below is a list of resources on using R in regulated clinical trial environments…Below some of the links is a comment or note that I thought was interesting…Feel free to send me any I may have missed.

5/9/2019 Update:

Much of this information is now in the appendix of this doc:

How do I select an R package for my clinical workflow?




Best Practices for Reproducible Package Management in R


Updated information on environments can be found here:

These pages are helpful:

For Tables, check out this package:

If you are interest in validation, keep updated with this group:

Also see the R in Pharma conference:

The key document is here:

The following presents a hirearchy to understand testing and validation for R and R packages.

  1. R Core + Base and Recommended Packages

The source for R includes 3 things: R source code, a set of base packages, and a set of recommended packages.

The source for R and the R base packages are maintained by a closed group of developers called the R core team. The recommended packages are maintained by the same group and other select contributors.

All 3 components follow the release and testing cycle outlined here:

Specifically, the /tests directory of the R source includes tests for R core, the base packages, and the recommended packages:

You can navigate up the URL directory to see the /test folder for different branches of R, eg

  1. CRAN Checks on Community Packages

In addition to the base and recommended packages, R developers can make use of an extensive set of community-built packages hosted on CRAN. Any package accepted on CRAN must pass a series of automated tests: which enforce the CRAN submission policies:

The results of these tests are available on CRAN for each package:

The same tests can be run locally against any package source tar file using a command like: R CMD check package_0.1.tar.gz

The actual test code is part of R itself, not part of the package being tested.

CRAN check are run for the package plus the latest release of R, patch releases, and the current development branch of R. The tests are run on a variety of operating systems including Linux, Windows, and Mac OS.

  1. Community Packages + Developer Contributed Unit Tests

In addition to the CRAN checks, many package authors have added their own tests which help ensure the package’s functions work as intended. Most often, these tests are built into the package in a specific way so that whenever the automatic CRAN checks are run the custom tests also run.

These additional tests are part of the package source.

More information:

  1. The Environment

As mentioned, CRAN tests packages across a matrix of R versions and operating systems. However, ideally the tests are run again for the specific, targeted validated environment. This process would include:

  1. Building R from source in the validated environment. Running the tests as described in Part 1.
  2. For each community R package, download the package source into the validated environment and then run R CMD check. Once complete, install the package.

It is NOT recommended that steps 1 and 2 happen while an analysis is developed, as both steps would introduce a burden for R developers. It is sufficient to complete the process at the end of an analysis while preparing for a submission or for “freezing” the production environment.

Further info:

Other unit tests can be executed and are usually determined by the package requirements and in the risk assessment.

Some orgs develop a test framework in which each R package in the installation is associated with a test or “validation” package containing additional tests.

These package tests are usually written using the “testthat” framework for R package testing. Here is an example:

You can also find CRAN package check results for packages as explained above.

You can find published checks/tests by package, and the link to the tests results is found on the package’s main page in CRAN.

This is the link to the latest results of dplyr:

The link above shows timing.

To know what tests are performed, you can find that here:

For packages, CRAN runs test on multiple OS versions, it links to the results of the checks for each, here is the one for Debian GCC:

The link is on the Status column. Those are the results of CRAN itself running the R CMD check in Hadley’s book -

Landscape of Architecture


Travis CI…to run your tests whenever code is updated in a controlled environment, compute code coverage

testthat package

testthat: Get Started with Testing

Testing Shiny

Validation document via R Markdown - easily updated or even run nightly

Virtual machines and docker containers can be used to freeze a particular build of software

“With a syntax that is simpler than other provisioning tools (e.g. Chef, Puppet, Ansible) or Continuous Integration (CI) platforms (e.g. Travis CI, Shippable CI); users need little more than a basic familiarity with shell scripts and a Linux distribution software environment (e.g. Debian-based apt-get) to get started writing Dockerfiles.”

For R packages, packrat and similar tools can be used to freeze analysis.

Package Info




Other tools



Other info

R and Revolution R Enterprise in regulatory environments

Validation is an essential aspect regarding the employed software in regulatory environments, in particular in the pharmaceutical industry.

The FDA summarizes validation as follows: “Establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes.”

How R is used by the FDA for regulatory compliance

Tools Info

Haven enables R to read and write various data formats used by other statistical packages by wrapping the fantastic ReadStat C library written by Evan Miller. Haven is part of the tidyverse. Currently it supports:

SAS: read_sas() reads .sas7bdat + .sas7bcat files and read_xpt() reads SAS transport files (version 5 and version 8). write_sas() writes .sas7bdat files.

If you use R in a production environment, you have most likely experienced that some circumstances change in ways that will make your R scripts run into trouble. Many things can go wrong; package updates, external data sources, daylight savings time, etc. There is a general increasing focus on this within the R community and words like “reproducibility”, “portability” and “unit testing” are buzzing big time. Many really neat solutions are already helping a lot: RStudio’s Packrat project, Revolution Analytic’s “snapshot” feaure, and Hadley Wickham’s testthat package to name a few. Another interesting package under development is Edwin de Jonge’s “validate” package.

Reports & Tables

There are several packages that can be used to make very nice packages:

printr xtable stargazer tables pander