Use of this document

This is a study note for using \(devtools\) package for building libraries. For more details on the study material see:

1. Development Tools

Once you have a package, the devtools package provides functionality to support your development. Functions in devtools pacage operate on the active project (the folder of your project):

2 File structure

Once we create a repository for the new library following by [Creat Project >> New Directory >> R Package >> define Package name & directory], there will be five objects Within folder that has the same name as the new R package:

2.1 DESCRIPTION

Description file reference

Package: happyR
Type: Package
Title: What the Package Does (Title Case)
Version: 0.1.0
Author: Who wrote it
Maintainer: The package maintainer <yourself@somewhere.net>
Description: More about what it does (maybe more than one line)
    Use four spaces when indenting paragraphs within the Description.
License: What license is it under?
Encoding: UTF-8
LazyData: true
Imports: package1,package2
Suggests: package3
Depends: package4,package5

2.1.1 Licenses

Most often used:

  • GPL-2/3: “run, copy, distribute, study, change and improve the software” now and in the future, i.e. preserve open-source nature
  • GPL-3 clears up some inconsistencies and ambiguities in GPL-2 and should be used in new projects
  • MIT releases software completely and also allows use in commercial projects, removes liability of provider.

licenses available:

Flowchart for picking a (type of) license

2.2 man

.Rd is the extension used for R documentation. Rd files are clearly structured, yet, we DO NOT want to write these ourselves (way too many places to mess up)

The documentation workflow are the following:

  1. Add tags to your .R files in form of roxygen comments. i.g. #' some comment.
  2. Run devtools::document() (or press Ctrl/Cmd + Shift + D in RStudio) to convert roxygen comments to .Rd files.
  3. Preview documentation with ?.
  4. Rinse and repeat until the documentation looks the way you want.

2.3 NAMESPACE

The following three Roxygen tags in the .R file do not write anything into the .Rd help file instead, they write to NAMESPACE.

  • @export makes the function visible to package users
  • @import makes the imported package visible to the function code so you don’t have to use package::function() notation
  • @importFrom makes the imported function from the specified package visible (a more limited version of @import)

Note that Import in the DESCRIPTION file and @import for NAMESPACE aren’t related - same word, two totally separate concepts

3. Workflow

The workflow includeS three essential parts:

  1. Documentation
  2. Function Develpment and Testing
  3. Package Wrapping

Importantly, usethis is a workflow package: it automates repetitive tasks that arise during project setup and development, both for R packages and non-package projects. Below is a quick look at how usethis can help to set up a package. But remember, many usethis functions are also applicable to analytical projects that are not packages. link to usethis

library(usethis)

# Create a new package -------------------------------------------------
tmp <- file.path("C:\Project\testPkg", "testPkg")
create_package(tmp)
#> ??? Setting active project to 'C:\Project\testPkg'
#> ??? Creating 'R/'
#> ??? Creating 'man/'
#> ??? Writing 'DESCRIPTION'
#> ??? Writing 'NAMESPACE'

# Modify the description ----------------------------------------------
use_mit_license("My Name")
#> ??? Setting License field in DESCRIPTION to 'MIT + file LICENSE'
#> ??? Writing 'LICENSE.md'
#> ??? Adding '^LICENSE\\.md$' to '.Rbuildignore'
#> ??? Writing 'LICENSE'

use_package("MASS", "Suggests")
#> ??? Adding 'MASS' to Suggests field in DESCRIPTION
#> ??? Use `requireNamespace("MASS", quietly = TRUE)` to test if package is installed
#> ??? Then use `MASS::fun()` to refer to functions.

# Set up other files -------------------------------------------------

use_package_doc()
#> ??? Writing 'R/mypkg-package.R'

use_readme_rmd()
#> ??? Writing 'README.Rmd'

use_readme_md()
#> ??? Writing 'README.md'

use_news_md()
#> ??? Writing 'NEWS.md'

# Use git ------------------------------------------------------------
use_git()
#> ??? Initialising Git repo
#> ??? Adding '.Rhistory', '.RData', '.Rproj.user' to '.gitignore'

# create github repo ------------------------------------------------------------
use_github(protocol = "https")
Creating GitHub repository
#> ??? Adding GitHub remote
#> ??? Adding GitHub links to DESCRIPTION
#> ??? Setting URL field in DESCRIPTION to 'https://github.com/WeiquanLuo/testPkg'
#> ??? Setting BugReports field in DESCRIPTION to 'https://github.com/WeiquanLuo/testPkg/issues'
#> ??? Pushing to GitHub and setting remote tracking branch
#> ??? Opening URL https://github.com/WeiquanLuo/testPkg

# Set up various packages ---------------------------------------------
use_roxygen_md()
#> ??? Setting Roxygen field in DESCRIPTION to 'list(markdown = TRUE)'
#> ??? Setting RoxygenNote field in DESCRIPTION to '6.1.1'
#> ??? Run `devtools::document()`

# Add pkg tag ---------------------------------------------
use_travis()

use_coverage(pkg = ".", type = c("codecov"))


use_rcpp()
#> ??? Adding 'Rcpp' to LinkingTo field in DESCRIPTION
#> ??? Adding 'Rcpp' to Imports field in DESCRIPTION
#> ??? Creating 'src/'
#> ??? Adding '*.o', '*.so', '*.dll' to 'src/.gitignore'
#> ??? Include the following roxygen tags somewhere in your package
#>   #' @useDynLib mypkg, .registration = TRUE
#>   #' @importFrom Rcpp sourceCpp
#>   NULL
#> ??? Run `devtools::document()`

use_revdep()
#> ??? Creating 'revdep/'
#> ??? Adding '^revdep$' to '.Rbuildignore'
#> ??? Adding 'checks', 'library', 'checks.noindex', 'library.noindex', 'data.sqlite', '*.html' to 'revdep/.gitignore'
#> ??? Writing 'revdep/email.yml'
#> ??? Run checks with `revdepcheck::revdep_check(num_workers = 4)`

# Test ---------------------------------------------
use_test("my-test")
#> ??? Adding 'testthat' to Suggests field in DESCRIPTION
#> ??? Creating 'tests/testthat/'
#> ??? Writing 'tests/testthat.R'
#> ??? Writing 'tests/testthat/test-my-test.R'


# use internal data ---------------------------------------------
x <- 1
y <- 2
use_data(x, y)
#> ??? Creating 'data/'
#> ??? Saving 'x', 'y' to 'data/x.rda', 'data/y.rda'

3.1 Documentation

Both the whole R package and functions are needed to be well documented

3.1.1 Dependency Documentation

We need to document for what dependency packages are used in the developing package into the DESCRIPTION file. Importantly:

  • Imports: packages in this list must be present for your package to work
    • Imports are NOT attached when your package is loaded
    • You must refer to functions in these packages using package::function() or import them using roxygen tags (@import, @importFrom)
  • Suggests: packages in this list may add functionality but are not necessary
    • e.g. for example data, unit tests, build vignettes
  • Depends: packages in this list are attached when your package is loaded

Hint: It is better practice to use Imports rather than Depends. Depends might overwrite a previously loaded function of the same name (cause of some of the plyr - dplyr animosity)

Use usethis::use_package(package, type = "Imports") to adds a CRAN package dependency to DESCRIPTION.

3.1.2 Function Documentation

We need to document for the each function in .R file. In each of .R function, use roxygen comments make writing helper function to .Rd easier.

Roxygen tags
Tag Purpose
@export Make the function visible to users of the package
@param Describe inputs
@return Describe outputs
@examples Show how the function works
@author “Who wrote the function (if different from package)”
@seealso Pointers to related functions
@aliases Make it easier for users to find
@rdname Useful for functions that are invalid filenames and for combining docs
@import Call all functions from another package natively (without package::function)
@importFrom Call a single function from another package natively
Commands in Roxygen
Tag Purpose
\code{} Discuss R code
\link{} Make link to another function. Usually wrapped in \code{}
\eqn{} Inline equation (standard latex)
\emph{} Italic text
\strong{} Bold text

Example of Roxygen comment for a .R function

#'@name team_11
#'@title team_11's function for 2-level list extraction
#'@description This function extracts data from a shapefile with 2 levels in file$geometry
#'@usage team_11(tile, tolerance)
#'@param file path of a geometry file, extension should be .shp.
#'@param tolerance Tolerance level for thinning shape file. A percentage between 0 and 1.
#'@details Converts the geometry section of a shape file to latitude-longitude format
#' \itemize{
#' \item name = subregion name depicted by the data
#' \item region = coded subregion
#' \item group = indicates which polygon a set of points corresponds to
#' \item long = longitude of the point
#' \item lat = latitude of the point
#' }
#'@return a small shape file in latitude-longitude format
#'@author Lab 2 team 11 from STAT 585 Spring 2019
#'@seealso team_10, team_5
#'@examples
#'dsn="data/gadm36_AUS_shp/gadm36_AUS_1.shp"
#'tmp=team_11(file = dsn)
#'@export
#'@import tidyverse
#'@import bnlearn
#'@importFrom sf read_sf st_as_sf
#'@importFrom maptools thinnedSpatialPoly
#'@importFrom purrr map_depth flatten map_dfr

3.1.3 Documentation Wraping

  • Use devtools::document() to write the documentation from .R function and DESCRIPTION of the library to the.Rd function helper files and NAMESPACE
  • Render the helper file with calling ?myfun

The example resulting DESCRIPTION will be the following:

Package: lab3team12
Type: Package
Title: Converts shape geometry to lat-long format
Version: 0.1.0
Author: lab3team12 from STAT 585 Spring 2019
Package: lab3team12
Maintainer: Weiquan Luo <weiquanl@iastate.edu>
Description: Several functions that take a shape file (extension .shp) and convert multipolygon into
    a data frame containing latitude and longitude.
License: MIT + file LICENSE
URL: https://github.com/WeiquanLuo/lab3Group9
BugReports: https://github.com/WeiquanLuo/lab3Group9/issues
Imports: 
    purrr,
    maps,
    maptools,
    rgeos,
    tidyverse,
    tidyr,
    sf,
    magrittr,
    assertthat,
    checkmate,
    dplyr,
    tibble
Depends: R (>= 2.10)
Encoding: UTF-8
LazyData: true
RoxygenNote: 6.1.1
Suggests: 
    ggplot2,
    testthat

3.2 Function Develpment and Testing

Test Driven Development is an approach to programming where the unit tests are written before the actual code

  • Requirements are decided in advance
  • Code must meet the requirements (and only the requirements)

Advantage:

  • Fewer bugs
    • Essential functions are tested and issues are caught quickly
    • Visual confirmation that essential features are working
    • When debugging, write a test to pinpoint the bug (and stop it from reoccurring)
  • Better code structure
    • Modular code: do only one thing per function
    • Document functions as you write them to satisfy tests
  • Robust code
    • make big changes quickly and without downstream problems.
    • Any changes that break things should be caught with tests
  • (in theory) Easier to start after a break: pick up at the last failed test and write code to satisfy that test

We will use:

  • \(usethis\): library with helper functions for R package development
  • \(testthat\): library for unit testing for R packages
    • Unit tests test specific functions you’ve written,

3.2.1 Setup Automated Testing

Under the library project directory, there will

  • usethis::use_testthat(): sets up testing infrastructure, creating \tests\testthat.R and \tests\testthat\, and adding testthat to the suggested packages.
  • use_test(): creates \tests\testthat\test-<name>.R and opens it for editing.
usethis::use_testthat()
use_test(name = "myfun")

3.2.2 Writing test function

Test_that: evaluate one feature of a function. They are run automatically after you make changes to the code. A test file consists of :

  • context - a description of the test blocks in the file
  • one or more test blocks: test_that(description, {test statements}).
a series of expectation functions in testthat package. See more with help(package = "testthat")
function in test statements description
expect_equal(obj, value) Is the object equal to a value?
expect_error(expr) Does the expression produce an error?
expect_gt(obj, value) Is the object greater than the value?
expect_length(obj, value) Does the object have length value?

These functions are silent if the expectation is met, and throw an error otherwise. Expectations are used to construct tests.

For example, test_fun_myfun.R is:

context("test-basics")

# Test block
test_that(
  # description
  "multiplication works",
  { # test statements inside this block
    expect_equal(2 * 2, 4)
  }
)

3.2.3 Test the function

Save your test file, and then you can test your package with devtools::test() or Ctrl/CMD + Shift + T.

3.3. Package Wrapping

3.3.1 Build

There two way to build the library:

  • devtools::build(): Building converts a package source directory into a single bundled file for delibery package. a tar.gz file will be create outside of the library folder.
  • Ctrl/Cmd + Shift + b: build package in cache (preferable when development)
Build in cache
Description Windows & Linux Mac
Build and Reload Ctrl+Shift+B Cmd+Shift+B
Load All (devtools) Ctrl+Shift+L Cmd+Shift+L
Test Package (Desktop) Ctrl+Shift+T Cmd+Shift+T
Test Package (Web) Ctrl+Alt+F7 Cmd+Alt+F7
Check Package Ctrl+Shift+E Cmd+Shift+E
Document Package Ctrl+Shift+D Cmd+Shift+D

3.3.2 Load

There two way to load the library:

  • devtools::install_git("https://github.com/WeiquanLuo/lab3team12"): After push the repository of the library to github, install the library with this code.
  • devtools::load_all() or Ctrl/Cmd + Shift + L: simulates what happens when a package is installed and loaded with library().

3.3.2 Check

Package checks test the whole package, including:

  • the package has the correct file structure
  • the package is installable
  • there is a recognized license
  • there is an author/maintaner (with contact information)
  • documentation exists and is correctly structured
  • imports and dependencies are correctly specified
  • examples can be run
  • all of your tests work correctly
  • and more…

To submit to CRAN, you must not have any warnings or errors. Check your local package with devtools::check() or Ctrl/Cmd + Shift + E. Three levels of faults are the following:

  • ERROR: severe problem that has to be fixed beofr esubmitting to CRAN - you should fix it anyway
  • WARNING: also has to be fixed before going onto CRAN, but not as severe.
  • NOTE: mild problem. You should try to get rid of all notes before submitting to CRAN, but sometimes there is a specific reason that does not allow you to fix all notes. In that case, write a careful description why this note happens in the submission comments.

3.4 Coverage

Test coverage for a package is the coverage that functions in package are used in the test. To check the testcoverage, run the following code in the package directory to result a grapical presentation of the overall and detail coverage.

devtools::test_coverage(pkg ="C:/Users/Weiquan Luo/Dropbox/ISU Grad Study/Spring 2019/STAT 585/Project/ggfun")

3.5 Data

3.5.1 Internal data

If the DESCRIPTION contains LazyData: true, then datasets will be lazily loaded. Lazy loading datasets will be lazily loaded, which they won't occupy any memory until you use them.

3.5.1.1 Save Internal data

To save multiple data object into path:/data/ as seperate .rda file, use the following code. Each data will be saved in the data object name with suffix .rda. Each .rda object contain only one data.

x <- rnorm(10, mean = 0, sd = 1)
devtools::use_data(x, mtcars, internal = TRUE)

To write Documenting for datasets, for reach dataset, create .r file in folder R with the same name as the .rda file, where the the data is stored, then use roxygen2 to write documentation.

3.5.1.2 Refer Internal data

To refer the data, call by the function data().

data(x)
data(mtcars)

3.5.2 External data

3.5.2.1 Save External data

Non-R data should be placed in the folder inst, so we create a folder in the package folder “inst/shape/” for shapefile, or “inst/extdata/” for other type such as .csv.

3.5.2.2 Refer External data

To refer the name of the data, call the following after load the package:

# open the sub folder in `inst` folder for specific data file. 
system.file("extdata", "mydata.csv", package = "mypackage")
system.file("shape", "mydata.shp", package = "mypackage")
system.file("extdata", "2010.csv", package = "testdat")

For info about data, check [http://r-pkgs.had.co.nz/data.html]

4. Package development

4.1 Continuous Integration (CI)

4.1.1 Travis CI: building, testing, deploying

When you run a build, Travis CI clones your GitHub repository into a brand new virtual environment, and carries out a series of tasks to build and test your code. If one or more of those tasks fails, the build is considered broken. If none of the tasks fail, the build is considered passed, and Travis CI can deploy your code to a web server, or application host.

To use Travis.ci to check your package, you need to do the following two thing:

  • have the .rmd file with output formate as rmarkdown::github_document to generate .md file. The configuration of the .Rmd file is show at the following:
---
title: "pkg title"
author: "Weiquan Luo"
date: "2021-01-21"
output: rmarkdown::github_document
always_allow_html: yes
---
  • sign up for [https://travis-ci.org/], link to the github personal account, and activate the repo on [travis-ci.org].

Next, by running the following codeon rstudio console, we generate a line of Travis code in clipboard. The use_travis() will try to add this Travis code to the .md file, and it is also copyed to the clipboard. Or, we can manually add the link of code into the .Rmd file. By kniting the .Rmd file, the same line of code will be represent in the .md file as well.

devtools::use_travis()
usethis::use_travis() # alternative

Example Travis code: Travis build status

The presenting of the Travis code will turn on travis for your repo. Once the this .md is push to github. the Travis CI will build and evalue the package in the repo automatively.

Travs CI Core Concepts

4.1.2 Codecov: run tests, generate coverage

After writing some test for .R functions, by running the following code, you will get a code that can be added to your README file to display a codecov badge. i.g.: Coverage status

use_coverage(pkg = ".", type = c("codecov"))

Now log in to codecov.io using the GitHub account. Give codecov access to the project where you want to cover the code. This should create a screen where you can see a token which needs to be copied. Once this is completed, go back to R and run the following commands to use \(covr\):

install.packages("covr")
library(covr)
codecov(token = "YOUR_TOKEN_GOES_HERE")

4.2 pkgdown: make package websites

To build a website for a quick introduction for the pacakge, we can use the function fuild_site in \(pkgdown\) package. To successful create the website, it will require complete .Rmd, Rd for all functions and data, NAMESPACE, vignettes/pkgName-vignettes.Rmd for the package. All file for generating the website is saved in the /docs/ folder.

pkgdown::build_site()

4.3 Vignette

usethis::use_vignette("pkgName")