Hi, I’m James

I’m D&DSL at 360info @ Monash Uni

Really excited to talk about Quarto

Some people today have used R Markdown and are wondering what this new thing is

Others might be notebook fans or have never used any tools in this space

That’s okay - I’m going to cover all the basics and then talk a bit about why I think Quarto has a lot of potential

Quarto

Create beautiful documents with R, Python, Julia and Observable

JAMES GOLDIE

Data and Digital Storytelling Lead, 360info

Today

I want to get a feel for everyone’s experience today, so I’m going to start out qith a quick poll

before i get into Quarto.

For people who’ve used these kinds of things before, Quarto has two really cool differences.

One is that it’s language-independent, so anyone can use it

The other is that it adds a language called Observable JavaScript that makes learning JavaScript easier.

These two things give Quarto a really strong basis for helping data scientists learn to build compelling visualisations for the web.

1. Hello!
2. What are reproducible reports?
3. What’s Quarto? Why can anyone use it?
4. Web content

Slides

https://runapp2022.talks.jamesgoldie.dev

(Code for these slides is at https://github.com/jimjam-slam/talk-runapp-quarto-2022)

Hi!

(click)

I’ve been working for 360info for about 9 months

We’re a research-driven newswire, and we give everything away for free under Creative Commons so that all journalists can benefit from it

We’re founded by Andrew Jaspan, the man who also founded The Conversation

As our team grows, I’m doing some of everything:

• analysing data
• build charts and maps
• working with our infrastructure partners

I used to be a climate + health researcher

•

Worked with CLEX and MCCCRH on climate change communication

•

Now I’m a data journalist with

Poll: what tools do you use?

Would love to find out more about all of you and what you’re learning

No personal info in this poll, just 2 Qs

1. What have you used before?
2. What do you want to learn?

Take a moment to fill it in on your phone

Normally if you wanted to do a poll in a presentation, you’d use a third-party website like Slido and embed it

For today’s poll I decided to take a risk and build it entirely in Quarto.

Take the survey at https://forms.gle/EdADv8GvDxUrrRmy8

md`## Poll results (Respondents: ${respondentCount})`

import { liveGoogleSheet } from "@jimjamslam/live-google-sheet";
import { aq, op } from "@uwdata/arquero";

surveyResults = liveGoogleSheet(
  "https://docs.google.com/spreadsheets/d/e/" +
    "2PACX-1vSVQzZlkSWWR38FRKyFbO2WGYo04ehrJt4TvWkRVOJ7WPRmfQzPAf7AjtyV2EjIY-2DwKW-SfzLoEQw/" +
    "pub?gid=1368316628&single=true&output=csv",
  15000, 1, 2);

respondentCount = surveyResults.length;

// get the counts of people using and interested in tools
countsUsed = aq.from(surveyResults)
  .derive({ used: d => op.split(d.responseUsed, ", ") })
  .select("used")
  .unroll("used")
  .groupby("used")
  .count()
  .derive({ measure: d => "Have used" })
  .rename({ used: "tool" })

countsWant = aq.from(surveyResults)
  .derive({ want: d => op.split(d.responseWant, ", ") })
  .select("want")
  .unroll("want")
  .groupby("want")
  .count()
  .derive({ measure: d => "Want to learn" })
  .rename({ want: "tool" })

countsAll = [...countsUsed.objects(), ...countsWant.objects()];

plotUsed = Plot.plot({
  marks: [
    Plot.barX(countsAll, { y: "tool", x: "count", fill: "tool" }),
    Plot.ruleX([respondentCount], { stroke: "#ffffff99" })
  ],
  x : { label: "" },
  y : { label: "", tickSize: 0 },
  facet: { data: countsAll, x: "measure", label: "" },
  marginLeft: 140,
  style: {
    width: 1350,
    height: 500,
    fontSize: 12,
  }
});

What are
reproducible reports?

If you work in Python or Julia, you may have heard of notebooks

R users might have heard of reproducible reports

All examples of literate programming

Literate programming is about explaining your code by combining code with written prose and graphics

Very nice to read!

Readable even if you didn’t write it

File format is opaque and hard to version control

Files shared or hosted on special notebook services

Notebooks

• Contain:
  • Writing
  • Code
  • Results in the file too

Reproducible reports

Source documents are rendered to produce results

R Markdown is more common among R users

Instead of code and results together, we render a source document

Contains writing and code, but not results!

An R Markdown document is rendered. The final report is a separate file!

Plain text source file

Easy to see changes over time in version control

Docs, books, websites, presentations…

Need R to run it

The RMarkdown ecosystem

Because R Markdown gets rendered, it can make lots of different kinds of documents!

R packages for just about every kind of document:

xaringan • distill • blogdown • hugodown • bookdown • thesisdown • oxforddown • unswthesisdown • pagedown • flexdashboard • rmdformats • rticles • prettydoc • markdowntemplates • and more…

Quarto:
Literate programming foreveryone

So we have two very different approaches to explaining your data.

Notebooks are a really nice writing experience, and it’s good to have code and results in one place.

But they’re difficult to share or collaborate on without dedicated services.

RMarkdown docs have a massive ecosystem for lots of different outputs, and they’re very clear and easy to share. But not everyone uses R!

Quarto tries to have its cake and eat it too.

Notebooks or documents

You don’t have to choose between writing in notebooks or documents - they both work!

Here we have a Quarto report, written two different ways: a notebook and a document.

Either of these can be rendered - into a webpage, a blog post, a slideshow, a Word document, a PDF…

If you’re used to documents, Quarto ones look almost the same!

We still have chunk options (cell options for notebook people). They sit as special comments at the start of the cell, and they’re written in YAML. You can add them to notebook cells too!

(The old RMarkdown way still works.)

Frontmatter still works largely the same as RMarkdown, but you can add it to notebooks too!

R, Python, Julia

The other big change here is that you can use your favourite language in either format.

You can write Python chunks in a document like the one on the right, and you can render it without needing to have R installed.

On the flip side, you can write R cells in a notebook. Or Julia. Or all three!

Quarto is a separate tool that runs in the shell. It comes with everything you need, except the programming languages you want to use.

Editors

The examples I’ve shown are in Visual Studio Code, which is a code editor that I love.

Quarto works really nicely there. It even has a help panel that switches languages as you type!

Quarto also works fantastically with the JupyterLab editor and with RStudio.

RStudio visual editor

If you use RStudio and you’re used to writing documents but like the way notebooks look, you don’t even have to choose!

RStudio has a notebook-style editor for documents.

You can switch back and forth as you like, and it has a command palette to help you insert things.

R Markdown has been able to include code from other languages for a while, but I think this is a really big step to facilitating cross-language collaboration.

For someone who only works in Python or Julia, it’s a really tough sell to ask them to install R and several packages in order to write documents.

With Quarto, you can have a website with people contributing posts as a mix of notebooks and documents in any (or all languages).

Spicing up Quarto
with reactivity and Observable JS

Quarto has one other big feature that I think we need to talk about

It’s called Observable JS, and it can make your reports, your books and your slides reactive.

So when we render our document, code runs and it changes how the report looks.

Maybe it creates a plot from some data, or prints some statistics.

But sometimes we want our document to change as it’s being read.

We want to let users to make decisions and for the document to react to things changing.

Reactivity?

HUH?

Example: footy scores

So let’s say we had data on footy scores for different teams over a number of years.

We could try to make one graphic that shows all the data, but it’s a lot!

And maybe different readers care about different teams.

We can let our chart react to the decisions our users make about footy team and year.

menu + slider => chart of footy scores

viewof footyTeam2 = Inputs.select(
  ["Pies", "Blues", "Bombers", "Cats"],
  { value: "Blues", label: "Footy team" })

viewof year2 = Inputs.range([1990, 2005],
  { value: 1990, step: 1, label: "Year", width: 300 })

scores2 = FileAttachment("data/footy-scores.csv").csv({ typed: true })
filteredScores2 = scores.filter(d => d.team == footyTeam2 && d.year == year2)

Plot.plot({
  marks: [
    Plot.barY(filteredScores2, { x: "game", y: "score", fill: "game" })
  ],
  x: { label: "Game number", labelOffset: 70 },
  y: { label: "Score" },
  width: 800,
  height: 500,
  marginLeft: 60,
  marginBottom: 90,
  marginTop: 60,
  style: {
    fontSize: 24
  }
})

Code is usually about doing things

Programs and websites do this all the time! There’s a user interface with buttons and dials and sliders, and when we play with them, stuff happens.

But making that happen can be really tiring.

Web and app developers deal with this all the time:

• write code to create the button
• then write code to check its value all the time
• then write code to do things every time it changes

That’s powerful, but it gets old really quickly!

• Write code to create a control
• Write code to check its value all the time
• Write code to do things every time it changes
• Write code to pass updates from one thing to another

This gets old quickly!

😩

Reactive code just updates itself

Describe a control or input

Describe an output that reacts to the control’s changes

… there is no step 3

(the output takes care of itself!)

This is how Shiny and Dash work!

menu + slider => chart of footy scores

# describe some controls - a dropdown menu
# and a slider - and a plot

ui <- fluidPage(

  selectInput("footyTeam",
    label = "Footy team",
    choices = c("Pies", "Blues",
      "Bombers", "Cats"),
    selected = "Blues"),
  sliderInput("year", label = "Year",
    1990, 2005, value = 1991),
  plotOutput("footyScores")
  
)

# for the plot, filter our data and
# draw a line chart

server <- function(input, out) {
  output$footyScores <- renderPlot({

    scores %>%
    filter(
      year == input$year,
      house == input$footyTeam) %>%
    {
      ggplot(.) +
        aes(x = game, y = score) +
        geom_line()
    }
    
  })}

So why not use Shiny?

(ref. slides)

Services like shinyapps.io serve websites to people, but they leave a copy of R running to allow things to react like this

Shiny already works with R Markdown, and it works with Quarto too

But you need R running to react to things as people read the document, not just when you render it

This requires a special web server! (eg. shinyapps.io)

So why not use Shiny?

(ref. slides)

My use is different to many of yours!

If you want to make something to be used internally, or by special stakeholders, Shiny is great!

In media, charts could be viewed 100k+ times in days

(If I’m lucky…)

I can’t afford a Shiny server for that kind of traffic!

Enter Observable JS

Quarto gives us a new kind of code chunk called OJS

It lets us write JavaScript that is naturally reactive

Observable JS: no server required

We make a slider called x…

viewof x = Inputs.range(
  [0, 100], { step: 1 })

… and then reference it:

md`The square of ${x} is ${x**2}!
  How about that!`

The text reacts to the changing value of x. Nice!

Footy example with Quarto + OJS

• Plot
• Code

viewof footyTeam = Inputs.select(
  ["Pies", "Blues", "Bombers", "Cats"],
  { value: "Blues", label: "Footy team" })

viewof year = Inputs.range([1990, 2005],
  { value: 1990, step: 1, label: "Year", width: 300 })

viewof footyTeam = Inputs.select(
  ["Pies", "Blues", "Bombers", "Cats"],
  { value: "Blues", label: "Footy team" })

viewof year = Inputs.range(
  [1990, 2005],
  {
    value: 1990,
    step: 1, label: "Year",
    width: 300
  })

• Plot
• Code

scores = FileAttachment("data/footy-scores.csv").csv({ typed: true })
filteredScores = scores.filter(d => d.team == footyTeam && d.year == year)

Plot.plot({
  marks: [
    Plot.barY(filteredScores, { x: "game", y: "score", fill: "game" })
  ],
  x: { label: "Game number", labelOffset: 70 },
  y: { label: "Score" },
  width: 800,
  height: 500,
  marginLeft: 60,
  marginBottom: 90,
  marginTop: 60,
  style: {
    fontSize: 24
  }
})

scores = FileAttachment("data/footy-scores.csv")
  .csv({ typed: true })

// filter it using the controls
filteredScores = scores.filter(
  d => d.team == footyTeam &&
  d.year == year)

// make the chart
Plot.plot({
  marks: [
    Plot.barY(filteredScores, {
      x: "game",
      y: "score",
      fill: "game"
    })
  ],
  x: {
    label: "Game number",
    labelOffset: 70
  },
  y: { label: "Score" },
  width: 800,
  height: 500,
  marginLeft: 60,
  marginBottom: 90,
  marginTop: 60,
  style: { fontSize: 24 }
})

This is really powerful. If we don’t need a special server to make charts that react to users, they can be as popular as we like!

I can put Quarto documents on GitHub Pages, Netlify, or even Cloudflare Pages, which has unlimited bandwidth.

Quarto +

OJS means

• You can do your data analysis in R, Python or Julia
• You can make visuals that react to users
• You don’t need to worry about the cost if it gets popular
• Learning JavaScript is as easy as possible

R/Python Widgets

“Web content” in R Markdown usually involves a special R or Python package (eg. htmlwidgets)

👍 Convenient

👎 Customising means learning JavaScript and learning how an R package works

Quarto + OJS

It’s just JavaScript!

You get great libraries bundled in to try out JS:

• Observable Plot for making charts
• Observable Inputs for making controls
• Arquero for data analysis (it’s just like dplyr)

Seriously, just like dplyr, I’m not even kidding

Remember our survey before? That was doing data analysis on-the-fly with Arquero.

(click)

These are basically dplyr or pandas verbs!

Remember the survey before?

I did that with Observable Plot and Arquero.

Here’s the code I used to tally up survey responses:

countsUsed = aq.from(surveyResults)
  .derive({
    used: d => op.split(d.responseUsed, ", ")
  })
  .select("used")
  .unroll("used")
  .groupby("used")
  .count()
  .derive({ measure: d => "Have used" })
  .rename({ used: "tool" })

Use the whole web

And when you’re ready, you can use any other JavaScript library

(click to see)

This was a chart we built recently showing that many commercial satellites are used by governments and the military too.

(click to Italy)

This uses the d3-euler library, as well as a popup library called micromodal.

There’s a lot we could do to improve this, but the actual JavaScript being used here would look very familar to an R or Python user!

This isn’t much more complex than the dropdown menu and footy score chart

Conclusions

So we’ve seen these two major features of Quarto and how they make it stand out from R Markdown

Quarto helps you write documents…

… whether you use R, Python or Julia

… whether you like notebooks or documents

… whether you write in a code editor or a notebook editor

Quarto help you make documents that react to users

… and you might learn JavaScript by accident!

Between these two features, I think Quarto has a really powerful funnel effect

Learning a new language or framework is a big commitment.

Do you try to learn something on the weekend, hope you stay motivated and that you’ll have the time on top of the rest of your life?

Or do you take a risk and try to build something for work using a language you don’t know?

It’s taken me years to start using d3.js for exactly this reason

Quarto makes JavaScript and web visualisation accessible by inches, no matter which data science language you’re coming from

A checklist for trying Quarto

🔴 Download Quarto from quarto.org and have a look at the Guide

🟠 Try to render an existing R Markdown document or Jupyter notebook using Quarto and your favourite editor

🟢 Switch out one of your charts for an Observable Plot chart

🔵 Try switching some dplyr or pandas verbs for some Arquero verbs

🟣 Try adding something to your document by importing an Observable notebook or a JavaScript library

Thanks for listening!

Questions?

@jimjam-slam

360info.org

github.com/360-info