4  Workflow

Before we start with Quarto, we need to make sure that you understand file storage hygiene.

We can prevent unexpected problems if we can maintain an order to your files, paths, and directories. A common problem that arises is R not knowing where a certain file is. For example, we get the error:

read.csv("my-very-important-data-file-somewhere.csv")
Warning in file(file, "rt"): cannot open file
'my-very-important-data-file-somewhere.csv': No such file or directory
Error in `file()`:
! cannot open the connection

Because R doesn’t know where "my-very-important-data-file-somewhere.csv" is.

Practicing good file storage hygiene will help maintain an order to files, paths, and directories. This will make you more productive in the future, because you’ll spend less time fighting against file paths.

Not sure what a file path is? We explain that as well.

4.1 Overview

  • Teaching 10 minutes
  • Exercises 10 minutes

4.2 Questions

  • Where should I put all my files?
  • What is an RStudio project, anyway?
  • What is a file path?

4.3 Objectives

  • Understand what a file path is
  • Set up an RStudio Project to organise your work
  • Put some data in your project to set up the next tasks
NoteYour Turn

In groups of 2-4 discuss:

  1. What your normal “workflow” is for starting a new project
  2. Possible challenges that might arise when maintaining your project

4.4 When you start a new project: Open a new RStudio project

This section is heavily influenced by Jenny Bryan’s great blog post on project based workflows.

Sometimes this is the first line of an R Script or R markdown file.

setwd("c:/really/long/file/path/to/this/directory")
TipQuestion

What do you think the setwd code does?

4.4.1 So what does this do?

This says, “set my working directory to this specific working directory”.

It means that you can read in data and other things like this:

data <- read_csv("data/mydata.csv")

Instead of

data <- read_csv("c:/really/long/file/path/to/this/directory/data/mydata.csv")

So while this has the effect of making the file paths work in your file, it is a problem. It is a problem because, among other things, using setwd() like this:

  • Has 0% chance of working on someone else’s machine (this could include you in 6 months!)
  • Your file is not self-contained and portable. (Think: “What if this folder moved to /Downloads, or onto another machine?”)

So, to get this to work, you need to hand edit the file path to your machine.

This is painful.

When you do this all the time, it gets old, fast.

4.5 What is a file path?

This might all be a bit confusing if you don’t know what a file path is. A file path is the machine-readable directions to where files on your computer live. So, the file path:

/Users/njtierney/Desktop/qmd4sci-exercises/demo.R

Describes the location of the file “demo.R”. This could be visualised as:

users
  └── njtierney
      └── Desktop
          └──  qmd4sci-exercises
               └── demo.R << THIS IS THE FILE HERE
               └── exercises
               └── exploratory-data-analysis
                   └── eda-document.qmd
                   └── eda-script.R
               └──  data
                    └──  gapminder.csv

So, if you want to read in the gapminder.csv file, you might need to write code like this:

gapminder <- read_csv("/Users/njtierney/Desktop/qmd4sci-exercises/data/gapminder.csv")

As we now know, this is a problem, because this is not portable code. It is unlikely someone else will have the gapminder.csv data stored under the folders, "Users/njtierney/Desktop".

If you have an RStudio project file inside the qmd4sci-exercises folder, you can instead write the following:

gapminder <- read_csv("data/gapminder.csv")
NoteYour Turn
  • (1-2 minutes) Imagine you see the following directory path: "/Users/miles/etc1010/week1/data/health.csv" what are the folders above the file, health.csv?

4.6 Is there an answer to the madness?

This file path situation is a real pain. Is there an answer to the madness?

The answer is yes!

I highly recommend when you start on a new idea, new research project, paper. Anything that is new. It should start its life as an rstudio project.

An rstudio project helps keep related work together in the same place. Amongst other things, they:

  • Keep all your files together.
  • Set the working directory to the project directory.
  • Starts a new session of R.
  • Restore previously edited files into the editor tabs.
  • Restore other rstudio settings.
  • Allow for multiple R projects open at the same time.

This helps keep you sane, because:

  • Your projects are each independent.
  • You can work on different projects at the same time.
  • Objects and functions you create and run from project idea won’t impact one another.
  • You can refer to your data and other projects in a consistent way.

And finally, the big one:

RStudio projects help resolve file path problems, because they automatically set the working directory to the location of the rstudio project.

Let’s open one together.

NoteYour Turn: Use your own rstudio project

As mentioned in Section 1.7, our course exercises exist at https://github.com/njtierney/qmd4sci-exercises.

You can download them and then have RStudio automagically open by following the following steps:

  1. In RStudio, and run the following code to start a new rstudio project called “qmd4sci-exercises”.
usethis::use_course("njtierney/qmd4sci-exercises")

  1. Follow the prompts to download this to your desktop and then run the rstudio project. (You can move it later if you like!)

  2. You are now in an rstudio project!

NoteYour Turn: open the demo.R file
  1. Run the code inside the demo.R file
  2. Why does the read_csv code work?
  3. Run the code inside the exploratory-data-analysis folder - eda-script.R.
  4. Does the read_csv code work?
  5. Run the code inside the exploratory-data-analysis folder - eda-document.qmd, by clicking the “render” button (we’ll go into this in more detail soon!)
  6. Does it work?

4.7 The “here” package

Although RStudio projects help resolve file path problems, in some cases you might have many folders in your r project. To help navigate them appropriately, you can use the here package to provide the full path directory, in a compact way.

here::here("data")

returns

[1] "/Users/nick/github/njtierney/qmd4sci-exercises/data"

And

here::here("data", "gapminder.csv")

returns

[1] "/Users/nick/github/njtierney/qmd4sci-exercises/data/gapminder.csv"

(Note that these absolute file paths will indeed be different on my computer compared to yours - super neat!)

You can read the above here code as:

In the folder data, there is a file called gapminder.csv, can you please give me the full path to that file?

This is really handy for a few reasons:

  1. It makes things completely portable
  2. Quarto documents have a special way of looking for files, this helps eliminate file path pain.
  3. If you decide to not use RStudio projects, you have code that will work on any machine
NoteYour Turn: Fix the quarto doc

You got an error earlier when rendering exploratory-data-analysis/eda-document.qmd.

Your quarto document assumes it’s directory where it lives, is the home - so it cannot see “data”. Given when we know now about here::here(), how would you solve this?

4.8 Remember

If the first line of your R script is

setwd("C:\Users\jenny\path\that\only\I\have")

I will come into your office and SET YOUR COMPUTER ON FIRE 🔥.

– Jenny Bryan

You can create an Rstudio project by going to:

file > new project > new directory > new project > name your project > create project.

You can also click on the create project button in the top left corner

Then go to new directory, if it is a new folder - otherwise if you have an existing folder you have - click on existing directory.

Then go to new project

Then write the name of your project. I think it is usually worthwhile spending a bit of time thinking of a name for your project. Even if it is only a few minutes, it can make a difference. You want to think about:

  • Keeping it short.
  • No spaces.
  • Combining words.

For example, I had a project looking at bat calls, so I called it screech, because bats make a screech-y noise. But maybe you’re doing some global health analysis so you call it “world-health”.

And click “create project”.

If you want to use these nice features of the “.Rproj” file but are working in VS Code, or elsewhere (but still using R). You can still enjoy the benefits of the here::here() package, if you have a .Rproj file. You can create one in R by typing

library(usethis)
use_rstudio()

Which will create a .Rproj file, named after the directory you are in.

For more details on this, read: https://positron.posit.co/migrate-rstudio-rproj.html

4.9 Summary

In this lesson we’ve:

  • Learnt what file paths are
  • How to setup an rstudio project
  • How to construct full file paths with the here package