Welcome

These slides available at: https://arcus.github.io/demystifying_r_rstudio_skills_series/session_2.html

• Use keyboard arrow keys to
  • advance ( → ) and
  • go back ( ← )
• Type “s” to see speaker notes
• Type “?” to see other keyboard shortcuts

As we prepare to get started, I’d like to encourage you to open these slides on your own browser, so you will have them available to refer to later. There are also a number of links you may want to click on. I’ll copy paste that URL, where the slide deck is located, into chat, so everyone has it.

This first slide is just to help you navigate this slide deck if you look at these slides on your own! And yes, there are ample speaker notes, so if you missed something I said, chances are it’s going to be in the speaker notes of the slide.

About Arcus / Your Presenter

Arcus is an initiative by the Research Institute aimed at promoting data discovery and reuse and increasing research reproducibility.

• Arcus app: https://arcus.chop.edu
• Arcus Sharepoint site: https://chop365.sharepoint.com/sites/Arcus

Among the many teams in Arcus, I represent Arcus Education!

Thanks again for joining us, and welcome to our session today! I’d like to begin by briefly introducing myself and introducing Arcus, for those of you who may not be familiar with us. My name is ______ [add pronouns here if you want], and I’ll be leading today’s session.

I’ve already talked a bit about Arcus in the first session of this Skills Series, so I’ll just skip most of that this time and simply say that Arcus is an initiative by the Research Institute aimed at promoting data discovery and reuse and increasing research reproducibility.

You can find out more by checking out the Arcus application itself or the Arcus Sharepoint site, which has a calendar of events and lots more information about Arcus.

And finally, a little more about me: I work in Arcus Education. Our role is to support CHOP scientists by helping researchers acquire data science skills. That’s why we have Skills Series like this one!

Arcus Education

Arcus education provides data science training to researchers …

(and often this is useful to non-researchers too!).

https://arcus.chop.edu/i-want-to/arcus-education

Email us! arcus-education@chop.edu

The Arcus Education team does lots of different kinds of education. We work one on one with researchers in office hours, we write how-to guides to help people understand how to use Arcus tools, we create asynchronous modules that people can use to acquire skills in their own time, and we do synchronous events like this one.

Please check out that education page and find out more about some of our services. You can see I’ve added a screenshot of that page to the slide here on the left side. You can also email us at arcus dash education at chop dot edu.

Demystifying R and RStudio

Arcus Education provides “Skills Series” for the entire CHOP community.

This Skills Series is a short, 2-session series aimed at Demystifying R and RStudio!

• Session 1: Introduction to R/RStudio
• Session 2: Introduction to Literate Statistical Programming

One of our methods in Education is the Skills Series, and that’s what you’re in today.

In this series, we’re going to take just two sessions to help demystify R and RStudio. If you have heard people talk about these concepts and you have only a vague idea what they are, and want to know more, without having to actually download anything or write code, at least not yet, this is the right series for you.

After you finish these two sessions, you’ll understand what R is, what RStudio is, why people use these tools, and next steps for getting access to them and using them, if you want.

Session 2 Itinerary

Literate Statistical Programmi

• Review of R and RStudio
• Literate programming is a programming paradigm
• Research reproducibility reminders
• Quarto documents
• Next steps

Goals:

• Describe what makes programming “literate”
• Explain the real-life consequence of irreproducible research
• Name one way Quarto documents can be helpful in data analysis

So let’s get started with this second session. In this session we’re going to cover the concept of literate statistical programming and why it’s a great practice for data analysis. Literate statistical programming can be done in various languages, but we’ll talk about how to do it in R, using RStudio, in this webinar.

Let’s start with a quick review of some of what we talked about in the first session of this two-part series.

R Vs RStudio

R Programming language for data analysis

RStudio Integrated development environment (IDE)

Last time we made an important distinction between R and RStudio.

The first logo you see here is for R. R is a statistical programming language that’s great for doing data analysis. We talked in the first session about a few things that make R useful:

• R was built specifically for statistical data analysis, so it’s got a friendly and focused user community and lots of great online resources for learning.
• R is open source, which means that it is “free” in two ways – in the meaning of having no cost and also in that can be used widely without intellectual property concerns or licensing restrictions.
• R makes it possible to work with data and build scripts that do lots of things, from ingesting data to building data visualizations.

So that’s R, it’s a language with a lot of benefits.

Then, there’s RStudio. RStudio was the original name of a company as well as the name of a piece of software the company makes. A couple of years ago, the company changed its name from RStudio to Posit, but they kept the old name on at least some of their software.

You can think of RStudio (the software) as a fancy text editor for writing R code. The technical term for a fancy text editor for writing code is “Integrated Development Environment”, or I-D-E.

You don’t have to use RStudio to write R, but it makes it a lot easier. Last time we gave the example of typing a paper in Microsoft Word. If you didn’t have Microsoft Word, you could write down your thoughts in the Notes app on your phone. But Microsoft Word has lots of add-ons, like spell-check and word count and formatting that make it much more pleasant to write a paper there, rather than in the notes app on your phone!

One of the things you can do in RStudio really well is write code in a particular paradigm called literate statistical programming, and that’s what we’re going to talk about next.

Literate Statistical Programming

Literate programming – Donald Knuth’s term for programming that is effective not just for computers but for people.

Statistical programming – when you analyze data statistically, using a programming language.

Literate statistical programming – when you create scripts (for example in R) that describe for the computer and for human readers the analysis you’re doing and why and how you’re doing each step.

Literate statistical programming is a great way to make sure that your data analysis steps can be redone in the future with ease. Let’s explain this term a little bit.

“Literate programming” was coined decades ago by an important figure in computer science, Donald Knuth [Kuh-nuth]. He realized that while computer code was obviously being consumed by computers, it was also being consumed by humans, and it needed to be readable. And code needed to be better than barely readable, but instead actually be fluid and easy for humans to read. Literate programming relies not just on a short, cryptic comment here or there, but on real documentation that is interlaced with code.

When we combine that concept with the idea of statistical programming, where we’re analyzing data, we get literate statistical programming.

Literate Statistical Prgramming is what you do when you create a script, say for instance in R. In that script, you go through the steps of writing code that handles ingesting data, cleaning and analyzing it, and displaying your data and findings, and you also think about how best to describe what you’re doing in human language, with full paragraphs, and headers, and figure captions, and bulleted lists, and maybe html links out to helpful documents, and other notes that help you remember and help others learn what you’re doing and why.

Let’s show you an example!

A Simple Example

We describe what we do and use headers, bullet points, and other formatting to make it easier for humans to make sense of the code.

Last time, we showed you some of our analysis code, code we use to analyze our effectiveness in a webinar we give called “Arcus 101.”

Here is a snippet of a script we wrote, and you can see that it has some description of what the code is doing, to help humans. We put in a header, to separate a section that has to do more with bringing the data in and cleaning it. Then we write a paragraph describing what we’re going to do, followed by a chunk of code.

After that code executes, we’ll have some data. So we have some additional instructions, this time in the form of a bulleted list. I’m not showing you all of the code that makes that bulleted list happen, because it’s kind of long, but the code chunk with the grey background below that bulleted list does all the things in that bulleted list.

Hopefully, what this does is make it a lot easier for anyone on my team to pick up this code and work with it, even if they’re not the person who wrote the code. This is what literate statistical programming looks like, and working in R and RStudio is a great way to get started working this way.

This can help you create a kind of lab notebook that records each step you do with your data, in human language and in computer language. This is part of doing reproducible data analyses, which will reduce risk and make “future you” much happier.

And I do want to talk briefly about the very real risks that can happen when people don’t do reproducible data analysis.

The Duke Cancer Scandal

• Chemo sensitivity from microarrays
• Serious errors in data analysis
• Clinical trials based on flawed models
• Papers retracted, lawsuits settled

Consider the following case study, which highlights potential consequences of irreproducible research, or research that’s really hard or impossible to re-do in the same way.

A few years ago, researchers at Duke University tried to use gene expression data of tumor cells to predict sensitivity to chemotherapeutic agents. This approach generated a lot of excitement at the time, and the resulting work was published in high-profile journals.

Unfortunately, there were a number of serious errors in the data analysis.

Even more unfortunately, patients were enrolled in clinical trials and allocated based on flawed models. It’s likely that some patients were actually treated with the chemo they are least likely to respond to rather than the chemo that’s most likely to work.

In the end, 18 papers were retracted, and Duke settled more than 10 lawsuits for an undisclosed amount of money.

An Easy Mistake

Duke

"1881_at"

"31321_at"

"31725_s_at"

"32307_r_at"

MD Anderson

"1882_g_at"

"31322_at"

"31726_at"

"32308_r_at"

Do you see the off-by-one indexing error?

Two biostatisticians at MD Anderson uncovered these mistakes in painstaking work. Let’s look at one of the errors they found.

What you see on the left are the names of are a few of the hundreds of microarray probe sets – each roughly corresponds to a gene – that the Duke investigators reported to predict sensitivity to a particular chemotherapeutic agent..

And on the right are the probe sets that the MD Anderson team got. You can see that they’re not the same.

Can you see what went wrong?

If you look at the values long enough, you might notice a pattern: the number of the probe set that Duke reported is always one less than the number of the probe that MD Anderson found when they re-did the analysis.

This is what’s called an off-by-one indexing error. This happens when you use a tool like Excel and accidentally miss one row; or you have one dataset that has a header and another one that doesn’t. The result is that all the values in the affected column are shifted by one. This is a simple error to make, but it completely invalidates all downstream results.

Easy to make mistakes…

Off-by-one indexing error

Sensitive / resistant label reversal

Confounding in experimental design

Inclusion of data from non-reported sources

Wrong figure shown

… add up to huge patient consequences!

The off-by-one indexing error was just one of many simple errors the MD Anderson team discovered.

These errors are so easy to make, and anyone can slip up. The important thing is having safeguards to catch mistakes! Without good documentation of how data was analyzed or a reproducible workflow based in automated scripts, it’s very hard to catch these mistakes.

Now, not all of these problems could have been fixed just by using literate statistical programming, but some of the really important ones could have been fixed or at least detected a lot more quickly, possibly before patients were harmed and before papers went to press.

A key issue in this case study is that the Duke investigators used “point and click” tools like Excel, without automated scripts, without using a literate statistical programming approach. There was a lot of manual human intervention here, things like the use of spreadsheets… work that was really labor intensive and had lots of steps, many of which were probably never written down but just kept in people’s heads.

This prevented peers and independent investigators from catching errors in the analysis, until it was too late.

Unfortunately, the Duke case study is only one example where the barrier to reproducibility was that people used “point-and-click” type tools for analyzing large and complicated sets of data. There are lots of other examples of where point and click analysis has resulted in analyses being wrong. Now, with the right safeguards, Excel CAN be the right fit for some simple tasks, but for high-stakes things like research results, it can hide your mistakes and compound them. No one wants that!

Are You a “Team of One” ?

• Can I redo the analysis with this month’s data?
• Why do the data in Table 1 not seem to agree with Figure 2?
• Why did I decide to omit these six samples from my analysis?

Your closest collaborator is you from 6 months ago…

Using scripts to document your work in a literate statistical programming approach doesn’t only help people outside a study understand how things were done. They also help your local team, even if you’re just a team of one.

Consider the following 3 statements and ask yourself if they sound familiar.

• Can I redo the analysis with this month’s data? Can I do it in a hurry?
• Why do the data in Table 1 not seem to agree with Figure 2?
• Why did I decide to omit these six samples from my analysis?

Keep in mind, your closest collaborator is you from 6 months ago, but past you can’t answer your emails!

If you use literate data analysis scripts in R, scripts that record each step you do in code and in human language, it can be much easier to retrace your steps or rerun your entire analysis with just a couple of clicks to re-run the script. You could, for example, have a script that goes and gets the latest data directly from REDCap, removes any outliers according to pre-established rules, does some statistical tests on two groups, creates a table and data visualization, and saves all of that in a .pdf that is already formatted with the special rules of the journal you want to submit to. You could keep collecting data via REDCap and simply re-run that script with a single click and get an updated report. This will make your life easier in the long run, even though it means an up-front investment of writing code.

Introducing Quarto

I hope I have convinced you that literate statistical programming can help make your data analysis more reproducible, and that this can have really important consequences.

One way to start doing literate statistical programming is to use Quarto, and another is a very similar document format that preceded Quarto and is very much like it, called R Markdown. My team’s code that analyzes our survey data, the code you’ve seen already, is a Quarto document.

Quarto and R Markdown allow you to write computer code mixed in with human language that documents the purpose of the code and details about the decisions you made in your analysis.

Quarto documents are composed of 3 basic types of building blocks.

The first is the header which includes information about the document, such as its title, author, and the desired output format when the document is rendered. A lot of this information is optional so a header can be pretty short, like the one we see here, or much longer.

The second type of building block is text. Text can include special kinds of marks (Markdown) that add styling. This includes hash marks that turn a line of text into a header, asterisks that can create italics, lines of dashes that turn into horizontal lines, and more. Markdown is used everywhere in tech, not just in R Markdown or Quarto. Markdown is useful to learn if you’re going to use GitHub, or Python, or LaTex [pronounced “lah-tek”], or make posts on Stack Overflow, or use Reddit, or just about anything in the tech world where you’re expected to write some sort of documentation.

The third is code chunks. Code chunks contain R code that can be executed to output results.

Output Formats

Quarto allows you create documents that interlace:

• Your reasoning about your code
• The code you write
• The output of the code you write

Which helps future-you AND your colleagues!

You can use a Quarto or R Markdown document interactively, and run code from within it. You could run a single code chunk to ingest data and just stop there, for example. But you can also use this kind of script to make a final output document, like a short html file, a slide deck, a manuscript you want to have a in .pdf format, or even a whole website. In fact, the online slide deck I’m presenting from was built in Quarto. These slides, which use html and javascript, were written in RStudio, using Markdown, and if I wanted to, I could run R code inside these slides.

To make an output document, you click a button in RStudio. The button might say “Render”, “Preview”, or “Knit”, depending on the kind of document you’re creating, and whether you’re working in Quarto or R Markdown. The words all mean kind of the same thing.

Ready to Try?

• Get access to R and RStudio!
  • Posit.cloud (just for learning, NOT for working in CHOP data)
  • Ask to get R and RStudio installed (and consider Git and GitHub, too!)
• Learn a little basic R
  • We suggest learning “tidyverse” and working with data frames first
  • We very strongly suggest learning within the context of RStudio
  • Arcus has some resources (live/synchronous and asynchronous)
  • So do lots of other people! Consider how you like to learn.
• Join CHOP’s R User Group
• Pick one task you do manually and try to re-do it in R
• Don’t be afraid to fail (I do!)

If what you’ve learned today, and possibly in the first session, has inspired you to want to learn to write R code and to use RStudio to do that, there are lots of ways to do that.

First, do get access to R and RStudio so you can use these tools.

Posit.cloud is an online service that has a generous free tier and will allow you to practice your R and RStudio skills without downloading anything – but you should never use this for real work data.

Or, you can get software installed on your CHOP device. We’ll share some slides about how to do that in just a few moments.

Second, learn to use R!

We strongly suggest starting by learning about working in tabular data using the tidyverse. Even if your eventual goal is to do genomics or computer vision or machine learning in R, starting with tabular data analysis using the tidyverse set of packages will help establish good habits and give you a good foundation.

Also, we want to re-assert that while RStudio is optional, it is highly, highly recommended. You should learn within the context of using RStudio software, because it has a lot of helpful scaffolding that will make your experience less frustrating.

Now, where should you go to learn R skills?

My team has some offerings here at CHOP, but we’re certainly not the only game in town. How do you like to learn? Are you motivated by grades, and you really thrive in a class setting? Many universities offer courses in R, often related to things like biostatistics or public health. Do you like videos and short-form courses that you get a certificate for? Coursera and other massive open online course providers have great R courses, which again can sometimes be related to a specific interest, like Bayesian statistics or data visualization. Do you prefer books? There are some great books out there, especially ones by people like Hadley Wickham and Garrett Grolemund. In the next few slide we’ll have some links for CHOP-based resources, but please know that there’s a wealth of other stuff out there that you may also really enjoy.

CHOP has also had an R user group for a number of years, and we encourage you to join it! There’s a mailing list as well as a Slack workspace that people use to communicate, share successes, ask questions, and get advice.

Learning to code is like learning a sport – you have to do it and build muscle memory. So take something you do regularly, whether that’s analyzing your department’s budget, or reporting on recruitment for your study, or looking at infection prevalence on your floor, or even looking at the data from your favorite sports team or political party… take one thing that you do using a point and click, manual approach, and try to move it to analysis with R. At first you’ll feel like it’s awkward and hard, like you’re writing with your non-dominant hand. But after a while, you’ll gain skills and figure out how to apply what you’ve learned to other tasks.

And finally, writing code is hard! But your whole job is hard, or at least it was hard at first, before you learned the skills, right? So when it feels like you just keep messing up, I want to encourage you by saying that I fail too. I not only write R to analyze data, but I teach this stuff, and I still mess up, have to Google things, need to ask for help, and learn new things from my colleagues. So please be kind to yourselves as you learn.

Getting Access to R / RStudio

• Slide titled Working with R at CHOP
• Slide titled What to Get Installed
• Slide titled Researchers ONLY at CHOP
• Slide titled Posit.Cloud (be careful!)

There are some detailed slides we went over in the first session of this series, and I’m just going to give those links here. Please check these out, these have helpful links! The first two slides are general purpose and describe how to get R, RStudio, Git, and GitHub installed. The third slide is of particular use if you work in Research, and the fourth talks about Posit.cloud.

First Steps in R and RStudio

Five Mondays at 1 pm, April and First Monday in May:

Session 1: Review and Setup

Session 2: Projects and File Ingestion

Session 3: Exploring Data Visually Using ggplot2

Session 4: Selecting Data Using dplyr

Session 5: Putting it All Together: Communicating

If you’ve enjoyed this two-session series demystifying R and RStudio, you might benefit from a five-session series that will walk you through your first steps in using R and RStudio. We’re offering that in April and May!

Signups for this webinar series are on your screen.

A little bit about the scope of this Skills Series:

This series is intended to be a gentle introduction to using R and RStudio for people who interact with data and want to work in the R statistical programming language. This course is geared towards beginners who are comfortable doing basic tasks with data that comes in rows and columns (for example, organizing data in Excel) but have no programming background.

The workshop will cover how to get started using the R statistical programming language in your work. We’ll talk about how to import data, transform data, and create data visualizations in R. To keep this workshop series short, our scope is limited, and we won’t go into details that are specific to the conduct of research, like modeling and statistical tests. For that, we are planning a future Skills Series we’re going to call Next Steps in R for Research.

More R at CHOP

• Join the CHOP R User Group
• Check out some asynchronous modules
• We offer periodic sessions: Join the CHOP R User Group or keep your eyes on the Arcus Sharepoint page to be the first to find out the dates!

And as promised, a few links here. You can join the CHOP R User Group. You may also want to check out some asynchronous modules my team built, which you can use to learn R in small bits of time, like on your lunch break. Additionally, we plan to have additional Skills Series like this one. I’ve already talked about “First Steps in R and RStudio”, but keep your eyes open for other announcements too, either in the R User Group or on the [rcus Sharepoint page!

Q&A / Was This Effective?

As you can tell from our data analysis, we like to measure our effectiveness.

Goals:

• Describe what makes programming “literate”
• Explain the real-life consequence of irreproducible research
• Name one way Quarto documents can be helpful in data analysis

Now, this is not the final slide, but we’re a group that likes to measure our effectiveness and make changes based on what we learn. So before we do questions and answers, we like to make sure to give our learners a poll to see if we made a good use of your time. We’re going to ask if we were effective in reaching our goals today, and those goals are on the screen in order to help you answer that question. Once you answer that poll, feel free to type a question into chat or come off mute and ask to clarify anything I can explain a bit better. I’ll stop our session of Q&A with about 5 minutes to spare because I do have just a couple of finishing slides I want to show.

Homework

If you want, totally optional additional learning:

Module	Description	Duration
Learning to Learn Data Science	Discover how learning data science is different than learning other subjects.	20 mins
Reproducibility, Generalizability, and Reuse	This module provides learners with an approachable introduction to the concepts and impact of research reproducibility….	60 min
R Basics	Are you brand new to R, and ready to get started? This module teaches concepts and vocabulary related to R, RStudio, and R Markdown….	60 min

• An article on Excel in research, with some warnings and some next steps for risk mitigation

OK, as we get close to the end, I know that for some people, this topic ignited a bit of excitement about the ideas around learning how to code or learning more about reproducibility. I’ve also added R Basics. These are all examples of some of our asynchronous modules which I mentioned earlier, and there are over sixty of these modules, so try checking these out.

And, if you’re an Excel user, and you want to understand the risks of using Excel and how to mitigate them, there’s an article here that might be interesting to read.

Acknowledgements

• R User Group leadership, especially Stephan Kadauke
• Former learners at CHOP, Penn, Drexel, University of Botswana
• DART study participants and pilots around the world
• Ed Himelblau
• You!

We want to thank the people that contributed to this work. R User group leaders, especially Stephen [shte-fan] Kadauke [cud-awk-ey] created some of the materials that we started with years ago that have changed over time to take on this current format. We’re grateful to all the students at CHOP and beyond who have taken R workshops and given us valuable feedback that has improved our approach, especially our DART study participants. We really enjoy the humor and science of Ed Himelblau and are grateful for his generosity in supporting our work with his art, which was featured in our first session. And, of course, thank you, our participants today!

With that, I want to thank you for your time and enthusiasm today. Your time is an incredibly valuable resource and I hope that this webinar series was a good use of it. If there’s any feedback, positive or negative, that you’d be willing to offer, please feel free to drop me a line or email arcus dash education at chop dot edu.