Many of you have asked about how to make prettier tables with R and Quarto. There’s a whole world of packages for making beautiful tables with R! Four of the most common ones are {gt}, the brand new and invented-to-improve-on-{kableExtra} {tinytable}, {kableExtra}, and {flextable}:

library(tidyverse)

cars_summary <- mpg |> 
  group_by(year, drv) |>
  summarize(
    n = n(),
    avg_mpg = mean(hwy),
    median_mpg = median(hwy),
    min_mpg = min(hwy),
    max_mpg = max(hwy)
  ) |> 
  ungroup()

library(gt)

cars_summary |> 
  group_by(year) |> 
  gt() |> 
  cols_label(
    drv = "Drive",
    n = "N",
    avg_mpg = "Average",
    median_mpg = "Median",
    min_mpg = "Minimum",
    max_mpg = "Maximum"
  ) |> 
  tab_spanner(
    label = "Highway MPG",
    columns = c(avg_mpg, median_mpg, min_mpg, max_mpg)
  ) |> 
  fmt_number(
    columns = avg_mpg,
    decimals = 2
  ) |> 
  tab_options(
    row_group.as_column = TRUE
  )

library(tinytable)

cars_summary |> 
  select(-year) |> 
  rename(
    "Drive" = drv,
    "N" = n,
    "Average" = avg_mpg,
    "Median" = median_mpg,
    "Minimum" = min_mpg,
    "Maximum" = max_mpg
  ) |> 
  tt() |> 
  group_tt(
    i = list("1999" = 1, "2008" = 4),
    j = list("Highway MPG" = 3:6)
  ) |>
  style_tt(
    i = c(1, 5),
    bold = TRUE,
    line = "b",
    line_color = "#cccccc"
  ) |> 
  style_tt(j = 2:6, align = "c") |> 
  format_tt(j = 3, digits = 2, num_fmt = "decimal")

library(kableExtra)

cars_summary |> 
  select(-year) |> 
  kbl(
    col.names = c("Drive", "N", "Average", "Median", "Minimum", "Maximum"),
    digits = 2
  ) |> 
  kable_styling() |> 
  pack_rows("1999", 1, 3) |> 
  pack_rows("2008", 4, 6) |> 
  add_header_above(c(" " = 2, "Highway MPG" = 4))

library(flextable)

cars_summary |> 
  rename(
    "Year" = year,
    "Drive" = drv,
    "N" = n,
    "Average" = avg_mpg,
    "Median" = median_mpg,
    "Minimum" = min_mpg,
    "Maximum" = max_mpg
    ) |> 
  mutate(Year = as.character(Year)) |> 
  flextable() |> 
  colformat_double(j = "Average", digits = 2) |>
  add_header_row(values = c(" ", "Highway MPG"), colwidths = c(3, 4)) |> 
  align(i = 1, part = "header", align = "center") |> 
  merge_v(j = ~ Year) |> 
  valign(j = 1, valign = "top")

library(modelsummary)

model1 <- lm(hwy ~ displ, data = mpg)
model2 <- lm(hwy ~ displ + drv, data = mpg)

modelsummary(
  list(model1, model2),
  stars = TRUE,
  # Rename the coefficients
  coef_rename = c(
    "(Intercept)" = "Intercept",
    "displ" = "Displacement",
    "drvf" = "Drive (front)",
    "drvr" = "Drive (rear)"),
  # Get rid of some of the extra goodness-of-fit statistics
  gof_omit = "IC|RMSE|F|Log",
  # Use {tinytable} (this is optional since it's the default)
  output = "tinytable"
) |> 
  style_tt(i = 3:4, j = 2, background = "yellow") |> 
  style_tt(i = 3:4, j = 3, background = "black", color = "white")

Table captions and numbering

One really nice Quarto feature is the ability to automatically generate cross references to figures and tables. To do this, you’ll need to do three things:

1. Add a chunk label using the label option in the chunk options. For tables, you need to use a tbl- prefix; for figures you need to use a fig- prefix.
2. Add a caption using either the tbl-cap or fig-cap option in the chunk options.
3. Reference the table or figure in your text by typing @tbl-whatever-your-label-is or @fig-whatever-your-label-is

Here’s how it all works—type this…

Blah blah blah I'm typing about causal inference; see @fig-example.

```{r}
#| label: fig-example
#| fig-cap: "This is a histogram"
#| fig-width: 5
#| fig-height: 3

ggplot(mpg, aes(x = hwy)) + 
  geom_histogram(binwidth = 2, color = "white", fill = "darkorange") +
  theme_bw()
```

And more blah stuff here, and @tbl-other-example has some results in it.

```{r}
#| label: tbl-other-example
#| tbl-cap: "A table with stuff in it"

mpg |> 
  select(year, cyl, trans, drv) |> 
  slice(1:4) |> 
  tt()
```

… to get this:

Blah blah blah I’m typing about causal inference; see Figure 1.

And more blah stuff here, and Table 1 has some results in it.

year	cyl	trans	drv
1999	4	auto(l5)	f
1999	4	manual(m5)	f
2008	4	manual(m6)	f
2008	4	auto(av)	f

Hi again everyone!

I recently posted a couple blog posts with some extra details and resources about core concepts in the class. You should check them out!

• Visualizing {dplyr}’s mutate(), summarize(), group_by() and ungroup() with animations - I made some little animations showing what happens when you group, summarize, and mutate
• Demystifying causal inference estimands: ATE, ATT, and ATU - This is a helpful little exploration of the ATE, ATT, and ATU—those things that we talked about weeks ago at the beginning of the semester but haven’t really done a lot with since.

Hi everyone!

Exam 2 opens on Thursday morning and will be available until 11:59 PM on April 19. It’s like Exam 1—open book, open note, open internet, just not open other people. There’s a study guide here.

You should have access to all the answer keys after you submit each assignment. If you submitted an assignment to me via e-mail, let me know and I can make the answer key available to you.

Hi everyone!

A few quick important announcements for these last few weeks of the semester:

• I (finally!) just updated your grades for everything turned in to iCollege so far. Please check your grades to make sure I’m not missing anything!
• You can still turn stuff in late! I’ve been, like, super lenient about late penalties (despite what I said in the syllabus), so you can still make up missing work
• You can turn work in up until Friday, April 26 at 11:59 PM—the Friday before the final project is due. That’s the final final deadline for all assignments that aren’t the final project.

And remember, please reach out for help if you’re struggling! Don’t suffer in silence—I’m here to help you!

Hi everyone!

I hope you’re all enjoying spring break and this beautiful Atlanta weather!

I just wanted to point out some really helpful resources and extra guides for all the causal inference methods we’ve been talking about this semester. There are a couple overarching organizations that focus on evidence-based policy that provide a ton of excellent free resources:

Hi everyone!

I’m like 90% better, so we’re back to our normal schedule tomorrow. We will have an in-person lab tomorrow evening. Be prepared to talk all about DAGs and potential outcomes and causal inference!

Hi everyone!

I was hoping I’d be able to run the in-class lab this evening over Webex, but lol nope. I’m wiped out and can barely talk.

This sucks because we were going to talk about important DAG stuff and answer all your questions about causal models and do an activity where we created both a logic model and a DAG for a program (since they’re very different things with different purposes). We’ll do that all next week instead.

Accordingly, I’ve changed the deadline for the causal model assignment that would be due next week. In order to give you time to get your questions answered about DAGs, the causal model assignment is due on February 22 now. I’ve changed the deadline on the class website and on iCollege.

Hi everyone!

Despite our best efforts, we’ve had a bit of a COVID outbreak at home, and I unfortunately just tested positive this morning.

So sadly, like last week, there will be no in-person session tomorrow. Depending on how I’m feeling, I’ll run the in-person lab on Webex again, like last week. I’ll make that call tomorrow afternoon—I’ll let you all know by 2 PM.

So no in-person stuff tomorrow evening, but maybe synchronous online stuff tomorrow evening.

Hi everyone!

Despite our constant masking and careful social interactions, one of my daughters just tested positive for COVID this evening. Out of an abundance of caution (and because kids are little germ factories and it’s inevitable that the others will start testing positive soon), I’m not going to hold the in-person lab tomorrow—I don’t want to inadvertently get any of you sick in case I asymptomatically have it too.

I’m not leaving you content-less though! I’ll run the lab on Webex from my home tomorrow at the usual class time using the same Webex link that we use for streaming from my middle-of-the-classroom laptop. The link is accessible on iCollege in the top navigation bar under “Webex”. We’ll talk about your questions, we’ll work a bit more on regression stuff, and we’ll work through another example of a logic model and compare and contrast it to a DAG.

So no in-person stuff tomorrow evening, but yes synchronous online stuff tomorrow evening.

Hi everyone!

One quick note about your grades on iCollege. A few of you have noticed that you now have 0s for assignments that you haven’t turned in yet. Don’t worry! I just had iCollege update all the missing grades to 0 for the sake of more accurate grade calculation. Before today, if you checked your grades on iCollege, you’d be failing even if you turned in everything and got 100% on everything. That’s because the denominator was 930 points—the total number of points you can get this semester. But that’s inaccurate—we’ve only a handful of things so far (weekly check-ins 1, 2, 3, and problem set 1), so if you got the maximum score on those, you’d have a 60/930, or a 6.5% F. That’s not useful information! There’s no use including those in the denominator for now.

So now you should be able to see what your current grade is with the correct denominator (i.e. the points for everything from the beginning of the semester up to today, which is just 60 points), meaning the percentages you see are more accurate.

If you have a 0, don’t worry! Once you turn in past exercises the 0 will get replaced with the actual score. You can still turn in late work! The 0 is just a mathematical quirk to get iCollege to cooperate.

When working through the examples or problem sets, you will get stuck. It’s inevitable.

You absolutely can (and should) reach out when stuck. I’m super responsive on Slack and e-mail. If you ask a question on the #help channel in Slack, others can help too. No question is too tricky or embarrassing, I promise. Computers are extraordinarily literal and tiny typos will often mess you up for a long time—I’ve lost countless time because of missing commas and misspelled words (curse you lenght()). Please feel comfortable reaching out.

I’m a big believer in the 30-minute rule:

There are a few easy guidelines to remember when asking for help:

1. Be kind.

2. Ask questions with as complete information as possible. Do not just post a photo from your phone and say something like “help?” or “my code isn’t working” and that’s it—provide more background (it’s hard to read your computer’s mind).

   Some people call this low resolution writing vs. high resolution writing. With low resolution writing on Slack, you’d type “My code isn’t working,” and then you’ll have to wait for someone to respond with “What isn’t working?,” and then you’ll respond with “My code for question 3,” and then you’ll have to wait for someone to respond with “What part of question 3?” and so on and it can take forever. Avoid this where possible. Embrace high resolution writing instead. Have your initial message be something like “In question 3 of the problem set, I’m trying to run a regression model that includes income as an independent variable. I’m running this code (CODE HERE), but I’m getting an error that says (ERROR HERE).” You’ll get help way faster that way.

   Explain specifically what you’re trying to do and provide code when possible. You can actually format R code on Slack if you click on the little lightning icon in the bottom left of the typing area and search for “text snippet”—that’ll open a dialog that will let you paste in text and add R syntax highlighting. You can also paste your code between triple backticks on Slack and it’ll format it in monospaced font (though not with the neat syntax highlighting that you get when using Slack’s text snippet thing):

   ```
   ggplot(blah) + geom_point()
   ```

   You can also take screenshots to supplement your question (use ⌘+shift+4 on macOS to save a screenshot to your Desktop, or ⌘+⌥+shift+4 to save a screenshot directly to the clipboard; use Windows+shift+S on Windows to save a screenshot directly to the clipboard). Do not send a screenshot as your entire question though (it’s very tempting to just post a picture of an error and hope that someone can fix it)—provide context. Screenshots are helpful, but code/text is better—it’s harder for people to get the code out of a screenshot in Slack and into RStudio on their computer to troubleshoot, since you can’t copy/paste from an image.

3. Try making your question a reproducible example (or reprex). Reprexes are the best way to (1) get help online and (2) fix issues on your own.

   Making a good reprex is tricky, but it’s a very valuable skill to know (regardless of programming language!). Here are some helpful resources for making them:

   • What’s a reproducible example (reprex) and how do I do one?
   • So you’ve been asked to make a reprex
   • The reprex package

4. Ask in public. It is very tempting to send me private messages on Slack for assistance, and that’s great and I’m happy to respond like that. However, one way to help build a stronger community for this class is to ask questions in public in the different #help channels instead. There are a couple reasons for this:

   • It reduces duplication: Many of you will have almost identical questions and I often end up copying and pasting my answers between different private conversations asking for help. Having those questions and answers in the public #help channels instead will let you get answers to common questions faster
   • It allows you to help: Some of you have R experience already, and even if you don’t, as the semester goes on, you’ll get more comfortable with it and will start being able to answer your classmates’ questions. You might have just fixed a similar issue in a past exercise, or you might be able to spot a typo in their code, or you might otherwise know how to help. Step in and help! Slack is for building a community, not just for getting assistance from me.

And that’s it. Ask questions in ways that will help answerers answer them and be nice about it. When answering questions, be nice about it. Ask lots of questions. Answer lots of questions. Do it all in public.

Once again, do not suffer in silence. I’ve had past students tell me that’s like the one thing they’ll remember from my classes—do not suffer in silence. I mean it, and I’ll keep saying it throughout the semester (because often in your past courses and degrees, you’ve been discouraged from reaching out or from building communities or whatever—that is not the case here).

Remember that you can always sign up for a time to meet with me at my Calendly page! I’m also on Slack and accessible via e-mail!

The most common question from the pre-class survey was some variation of this:

I have no background in programming or in R—will I be able to learn things and succeed in this class?

Absolutely.

I’ve designed this class to be accessible by people with any level of R experience. If you’ve never used R—awesome! If you’ve been using R for years—awesome! You’ll all learn new skills.

To get the most out of this class and minimize the inevitable frustration that comes with learning a programming language, do not try to do everything on your own. Community is essential.

Learning programming is a little different from learning other skills or materials. In other classes you’ve taken, you read articles or textbooks, take notes on them, talk about the readings in class, maybe complete problem sets or exercises about the materials, and then you take a test.

With programming, though, the approach is typically different. You read things, you copy code, you paste that code into a file on your computer, and you tinker with the code to see how it works and what changes as you change things.

The R community in particular has embraced this approach to learning how to code. Posit, the company that develops RStudio, has a whole team of professional educators dedicated to improving R pedagogy, and they use this approach in all their teaching materials. The Software Carpentry project (a group dedicated to teaching data and programming skills) does too. I’m a fan of it as well. As one of the textbooks I teach from in another class states:

Take the “copy, paste, and tweak” approach: Especially when you learn your first programming language or you need to understand particularly complicated code, it is often much easier to take existing code that you know works and modify it to suit your ends. This is as opposed to trying to type out the code from scratch. We call this the “copy, paste, and tweak” approach. So early on, we suggest not trying to write code from memory, but rather take existing examples we have provided you, then copy, paste, and tweak them to suit your goals. After you start feeling more confident, you can slowly move away from this approach and write code from scratch. Think of the “copy, paste, and tweak” approach as training wheels for a child learning to ride a bike. After getting comfortable, they won’t need them anymore.

To make your copying/pasting life easier, each topic has an extensive example page and an accompanying video showing how to work through the different analyses. Copy/paste/tweak liberally from the example pages!

Every chunk of code on the course website has a neat little clipboard icon that you can use to copy the whole chunk:

# Click on the clipboard icon in the top right corner → to copy this code
x <- 1:5
x + 5
## [1]  6  7  8  9 10

Use that button!

Inevitably in the classes where I teach R, I have students say things like “I’m trying to do this without looking at any documentation!” or “I can’t do this without googling—I’m a failure!”. While the effort to be fully independent and perfect with code is noble, it’s totally unnecessary. Everyone looks stuff up all the time—being able to do something without looking stuff up shouldn’t be your end goal.

Eventually you’ll be able to whip out basic ggplot(..., aes(...)) + geom_point() + geom_smooth() + labs() kinds of things without copying and pasting—that comes naturally over time, and you see me do that in the videos. But as soon as I have to start changing axis breaks or do anything beyond the standard stuff, I immediately start googling and looking at the documentation. Everyone does it. The authors of these packages do it. That’s why documentation exists. So don’t feel bad if you do it too. It’s the dirty little secret of all programming everywhere—it’s impossible to write code without referring to documentation or other people’s code (or your past code).

This is why I have you do stuff in Quarto Markdown. Everything in your problem sets is completely reproducible. The document tells you exactly what you ran to get the data and results and figures that you made. Copy and paste from your past assignments liberally. Copy and paste from the examples liberally.

Do not be super noble and force yourself to write everything from scratch!

Finally, if you ever get frustrated when running code or making a plot because it goes wild and vomits geoms all over the page, don’t worry! It happens to everyone! There’s actually a whole hashtag on Twitter for #accidentalaRt (and a corresponding @accidental__aRt Twitter account that collects these plots) where people post their plotting disasters. Scroll through the feed and see the marvelous accidental art people make. It’s fantastic.

Hi everyone!

Tomorrow’s the first day of class! Exciting!

Since we’re still in a pandemic, I’ve structured the class in way that limits our exposure to each other as much as possible. As noted in the syllabus, we’re using a flipped classroom this semester. What this means in practice is that you’ll do all the readings and watch asynchronous lecture videos before class meets on Thursdays.

Our in-person time will not involve lectures. Instead, it’ll be a more hands-on lab for working with R and answering the questions that come up from doing the readings and watching the lectures. That’s the point of the weekly check-in assignment—the only way this in-person time will be effective is if you do the readings + videos and ask questions in the weekly check-in.

The videos for the first day of class are live and ready to watch (along with the readings). If possible, please watch them (and do the readings) before class tomorrow so that we can start with the lab-like in-person time and get a good solid introduction to R and RStudio + answer the questions you have from the first session’s materials.

Ideally, please try to submit the first weekly check-in on iCollege by tomorrow too so that we can have some good Q&A time during our in-person time. BUT no worries if not! The first weekly check-in isn’t technically due until next week.

If you don’t have time to get through the materials before the first day, that’s totally fine—you can do it later and there’s no late penalty or anything for the first week. But if you can, let’s get started!

Haha one last thing for real!

Over the course of the semester, I’ll be sending you lots of updates and announcements and FAQs.

My philosophy for this class is to let you maintain access to as much content as possible, which is why so little of the the class material is on iCollege.

I will post all these announcements and e-mails at the “News” section of the course website. This means that even after you finish the class and lose access to iCollege, and even after you graduate from GSU and lose access to your gsu.edu e-mail address, you’ll still have a searchable archive of messages and FAQs from this class.

I’ll let you know when there’s a new item in the news section in a few different ways:

1. I’ll e-mail you a link to the newest message. I’ll use the e-mail address you have listed in PAWS—if you want me to use a different address, let me know and I’ll update my list
2. New messages should post automatically to the #general channel in Slack
3. If you use an RSS feed reader (RIP Google Reader), you can subscribe to an RSS feed of the news section. There’s a link at the top of the news page.

Phew. That should be it from me for a while. Remember to read the syllabus and the rest of the course website, answer the welcome survey, sign up for Slack, and join the Posit.cloud workspace. Also, remember to take care of yourselves! I mean every word in the “Learning during a pandemic” section of the syllabus—this all sucks so much, and I will try my hardest to accommodate your needs throughout the semester.

Thanks!

From reading the syllabus you’ll notice that we’re using R in this class (though you can certainly use Stata too if you really want). R is free (!!), and highly sought after by employers, but it has a little bit of an initial learning curve. You’ll get the hang of it, though, I promise! I’m a certified RStudio instructor (see https://education.rstudio.com/trainers/), so I’ve been specially trained to teach statistical programming. Again, I promise that you’ll learn a ton.

While R and RStudio are free and open source, they can sometimes be tricky to install. To make life simpler, you can run RStudio in your browser without having to install anything (!!!) using a service called Posit.cloud. Over the course of the semester you’ll want to eventually install everything on your actual computer, since it’ll run faster there, but you won’t need to install anything to get started. I’ll e-mail you a link to join the class workspace.

In the meantime, you can also install everything on your actual computer if you follow these instructions (eventually you should do this—it’s best to run R on your own computer, but it’s totally fine to run everything at Posit.cloud for the first few weeks)

Once again, I promise that you can do this and have a blast in this class!

Hello everyone!

I’m Andrew Heiss, your professor for PMAP 8521 (Evaluation Research) this spring, and I’m so excited for the class! In this class you’ll learn all about causal inference, or how to legally claim causation with statistics. In your past stats courses you were always taught “correlation isn’t causation,” which is mostly true, except when it’s not. In this class you’ll get to legitimately make causal claims.

We’ll cover fun tools like directed acyclic graphs (DAGs), randomized controlled trials, difference-in-differences analysis, regression discontinuity analysis, and instrumental variables. You’ll also learn the statistical language R, which is free (which means you can keep using it after you graduate and not have to pay for really really expensive SPSS or Stata licenses). I’ve had many former students get jobs because of the R part of this class—tons of public and nonprofit organizations are looking for R skills.

I have a few important announcements before class:

• We are meeting in person, but as you’ll see in the syllabus (see the “Masking and vaccines” section there, I’m trying to take as many precautions as possible since there’s still a global pandemic. For instance…

• …I’ve made the course a flipped course, which means you’ll do all the readings and watch the lectures on YouTube before class on Thursdays (see here for more details). We’ll spend our class time answering questions, doing activities, and learning a lot of R together. This means that the bulk of the course material is fully online and asynchronous.

• Before class, I’d love to get to know each you a little first, so I’ve created a quick survey to fill out. I’ve sent you a link to it via e-mail. Please take it at your earliest convenience.

• The entire course is available at a special class website at https://evalsp24.classes.andrewheiss.com/ (surprise—you’re at it right now). Bookmark this site—it’ll be your best friend for the next semester. I only use iCollege for collecting your assignments, posting answer keys, posting a couple scanned book chapters, and offering the two exams (since it’s password protected). This website is the official source of dates and all other class information. Because it’s not part of iCollege, you’ll be able to reference it even after you graduate and lose access to GSU resources. You can even share it with others—it’s just a website!

• Please read the main explanatory pages at the course website at your earliest convenience. The instructions and expectations for the class are divided across different pages, all accessible from the menu bar at the top of the site. Please read the main pages for the syllabus, schedule, content, assignments, and examples.

• We’re using zero physical textbooks in this class. Every reading and book and piece of software in this class is 100% free (see the syllabus for more information about that).

Quick background about me: I’m an assistant professor here at the Andrew Young School, where I’ve been teaching MPA/MPP microeconomics, program evaluation, and data visualization. I moved here from Utah in Fall 2019, where I was a visiting professor of public management at the Marriott School of Business at Brigham Young University (BYU). While at BYU, I taught microeconomics, statistics, and data visualization (basically the same stuff I’m teaching now). I finished my PhD in public policy and political science from Duke in 2017, and before then I finished my MPA in nonprofit management from BYU in 2012. When I’m not teaching, I research international nonprofits and human rights.

When I’m not teaching stats and evaluation, I research international nonprofits and political science. And when I’m not teaching or researching, I’m normally chilling at home with my 6 (!!) kids (see https://www.heissatopia.com/ for photos and hilarious stories).

Again, I’m really excited to get started next week. This semester should be a blast!