This is an R Markdown document. Follow the link to learn more about R Markdown and the notebook format used during the workshop.

The cheatsheet is very useful when working with ggplot2: https://www.rstudio.org/links/data_visualization_cheat_sheet

Setup

library(dplyr)
library(ggplot2)
library(readr)

Data

We’re using two data sets, one from a package, one built into R.

You only need to do this once: install the remotes package (or you can substitute devtools if you have that), and then use that package to install the palmerpenguins package from GitHub.

install.packages("palmerpenguins")

If you have it installed, load it and look at the penguins data set:

library(palmerpenguins)
penguins

We’re also going to use average yearly temperatures in New Hampshire, a data set built into R (nhtemp). But we need to put it in a data frame first to make it easier to use (it’s in a special time series object):

nh <- data.frame(temp=c(nhtemp), year=1912:1971)
nh

Introductions

Why ggplot? For me, it makes exploring my data, especially groups in my data, much easier. I can easily make some plots with ggplot that are difficult enough to make with base R graphics that I just won’t do them routinely when exploring my data. This is bad, because we should visualize our data! I don’t want the code to get in the way of that. ggplot is a package where I recommend first learning the basic syntax, and then go back and learn some more about the theory behind the package – what the logic is to how it works. It took me a few iterations of working with ggplot to get comfortable with it, and to understand how to think in a way that was compatible with how it approaches visualization. But once I did, it made data visualization easier and more fun.

The syntax of ggplot2 is a little bit different than other tidyverse packages, in part because it predates them. But it does share a few things with other tidyverse packages, such as:

The package is called ggplot2 (there is no first version), the main function in the package is called ggplot, and “ggplot” generally refers to anything with the package (I and others often don’t bother to say the 2 in the package name).

ggplot Basics

Each plot with ggplot has the same template for the code:

## call to ggplot and the name of the data frame
ggplot(dataframename, 
       ## which variables to use for which plot components 
       aes(x = column_name, y = column_name)) +   
  # what type of plot
  geom_****()  

The first line says what data and variables to use, and the second line says what type of plot to make. We add the components of the plot together, which is a somewhat unique syntax.

For example:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm)) + 
  geom_point()
## Warning: Removed 2 rows containing missing values (geom_point).

The warning message is telling you that two observations (rows) in your data frame are not plotted because there were missing values. You can get this warning from explicit missing values or in cases where you set the limits of the axes in a way that excludes some data points.

The first line alone (without the + at the end) makes a plot set up without adding the data:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm))

If we change the geom function we use, the plot type changes:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm)) + 
  geom_bin2d()
## Warning: Removed 2 rows containing non-finite values (stat_bin2d).

This is a density plot, where the color of each rectangle indicates how many observations (points on the above plot) fall in that area.

EXERCISE

Fill in the ___ to make a line plot using geom_line() with the nh data frame, with year as the x variable and temp as the y:

ggplot(___, aes(x = ___, y = ___)) + 
  ___()

Different Plot Types

The above plots used two continuous variables. The cheat sheet has the geoms organized by how many and what type of variables you want to plot. For example, we can make a histogram, which is of a single variable:

ggplot(penguins, aes(x = bill_depth_mm)) + 
  geom_histogram()
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## Warning: Removed 2 rows containing non-finite values (stat_bin).

The warning message is how many bins (bars) there are in the histogram. It defaults to 30, but it wants you to explicitly pick a number instead:

ggplot(penguins, aes(x = bill_depth_mm)) + 
  geom_histogram(bins = 25) 
## Warning: Removed 2 rows containing non-finite values (stat_bin).

We can also have multiple geoms on the same plot:

ggplot(nh, aes(x=year, y=temp)) + 
  geom_line() + 
  geom_point()

ggplot(nh, aes(x=year, y=temp)) + 
  geom_point() + 
  geom_line() + 
  geom_smooth()
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

EXERCISE

Make a histogram (geom_histogram()) of the temp variable in the nh dataset

Other Aesthetics and Grouping

We can set other aesthetics of the plot (color, fill, linetype, marker, etc.) according to variables in our data.

Note that if you want a line or color for each group, those groups need to be defined by a categorical variable. This is the opposite of how data usually needs to be structured when using base R plotting functions. If the values for the two groups are in separate columns in your dataset, you’ll need to reshape your data to get them in a single column instead – something covered in the tidyr session of these workshops.

Each aesthetic – each thing you define in aes() – needs to be a single column in your data set.

With categorical variables, this helps us see the groups in our data:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm,
                     color = species)) +  
  geom_point()
## Warning: Removed 2 rows containing missing values (geom_point).

Note that a legend was added automatically.

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm,
                     color = species)) +  
  geom_point() + 
  geom_smooth(method="lm", se=FALSE)
## `geom_smooth()` using formula 'y ~ x'
## Warning: Removed 2 rows containing non-finite values (stat_smooth).
## Warning: Removed 2 rows containing missing values (geom_point).

method=lm tells it how to fit a line to the data; “lm” means fit a linear model with the lm function. se=FALSE means to not display the standard errors on the plot.

Note that the smoothing was done by group. Specifying color effectively grouped our data, much like what happens with group_by in dplyr.

EXERCISE

Repeat the same plot as above, bill_length_mm vs. bill_depth_mm, but color by body_mass_g instead of species.

Labels/Titles

A convenient way to label axes and legends and set the title is with the labs() function, added as a component of the plot:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm,
                     color = species)) +  
  geom_point() + 
  labs(title="Penguin Bill Dimensions",
       y="Depth (mm)", 
       x="Length (mm)",
       color="Body Mass",
       subtitle="Three Species",
       caption="Source: palmerpenguins")
## Warning: Removed 2 rows containing missing values (geom_point).

EXERCISE

Plot flipper_length_mm vs. body_mass_g and color by sex. Label the axes and title your plot.

Axis Limits

To change the axes min and max values, like we would with xlim or ylim in base R graphics, we use a scale function and set the limits argument:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm)) +  
  geom_point() + 
  scale_x_continuous(limits=c(30, 50))
## Warning: Removed 54 rows containing missing values (geom_point).

Facets

In addition to using color, fill, linetype, etc. to denote groups, we can also make a separate plot for each group, where the plots are aligned and share axes. These are called facets:

ggplot(penguins, aes(flipper_length_mm)) + 
  geom_histogram() + 
  facet_grid(sex ~ .)
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## Warning: Removed 2 rows containing non-finite values (stat_bin).

The facet function syntax is using the formula syntax used elsewhere in R, but it can be tough to remember. It’s on the cheat sheet though so you can look it up. There are other options for laying out plots horizontally or in a grid.

Note that the plots share an x-axis, and the y-axis has the same range in each plot.

If you want to put multiple plots in a single image together, but they are not facets of a single plot, see the patchwork package.

EXERCISE

Make a histogram (set a value for bins) of body_mass_g for each species:

ggplot(penguins, aes(___)) + 
  ___(___) + 
  facet_grid(___ ~ .)

Styling

Themes control the look of the plot not related to the data: fonts, backgrounds, grid lines, axis lines, etc.

There are some built-in themes (theme_minimal(), theme_bw(), theme_classic(), etc.), but you can also control the styling further with the theme() function:

ggplot(penguins, aes(body_mass_g)) + 
  geom_histogram(bins = 30) + 
  theme_minimal()
## Warning: Removed 2 rows containing non-finite values (stat_bin).

For further control, use theme instead or in addition:

ggplot(penguins, aes(body_mass_g)) + 
  geom_histogram(bins = 30) + 
  theme_minimal() + 
  theme(panel.grid.major.x = element_blank(),  # get rid of the vertical grid lines
        panel.grid.minor.x = element_blank())
## Warning: Removed 2 rows containing non-finite values (stat_bin).

Many theme elements are controlled by setting options in additional element functions:

  • element_blank() - omit the element
  • element_text()
  • element_line()
  • element_rect()

These are options that you will generally google for how to do: “ggplot remove gridlines”, “ggplot change font size”, etc.

In addition to themes, you can control some aspects of the data-driven elements of the plot directly, without using a variable to define them. For example with color or size, if we specify them outside of a call to aes() within the geom function:

ggplot(penguins, aes(x = bill_length_mm, 
                     y = bill_depth_mm)) +  
  geom_point(color="red", size=3) 
## Warning: Removed 2 rows containing missing values (geom_point).

it colors all of the points the same color.

EXERCISE

Change the fill color for the bars in our histogram above to be “darkgreen”. The code from above is copied below to get you started.

ggplot(penguins, aes(body_mass_g)) + 
  geom_histogram(bins = 30) + 
  theme_minimal() + 
  theme(panel.grid.major.x = element_blank(),  
        panel.grid.minor.x = element_blank())

Pipes

We use +, not %>%, to join together components of our ggplot. But we can still pipe the results of other commands into ggplot - I just didn’t above because we were just plotting the data frame as it is. For example, to plot only the Adelie penguins:

penguins %>%
  filter(species == "Adelie") %>%
  ggplot(aes(x=bill_length_mm, y=bill_depth_mm)) + 
  geom_point()
## Warning: Removed 1 rows containing missing values (geom_point).

Learn More

The first chapter of R for Data Science https://r4ds.had.co.nz/ covers ggplot well.

For more on ggplot2, see our guide with free (for Northwestern folks) resources.

We’re happy to help you use ggplot on your own data. Request a free consultation: https://www.it.northwestern.edu/research/consultation/data-services.html

LS0tCnRpdGxlOiAnZ2dwbG90OiBRdWljayBJbnRyb2R1Y3Rpb24nCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICBjb2RlX2Rvd25sb2FkOiBUUlVFCiAgICB0b2M6IHRydWUKICAgIHRvY19kZXB0aDogMgplZGl0b3Jfb3B0aW9uczoKICBjaHVua19vdXRwdXRfdHlwZTogY29uc29sZQotLS0KCmBgYHtyLCBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGV2YWw9VFJVRSwgZmlnLndpZHRoPTUsIGZpZy5oZWlnaHQ9My41KQpgYGAKClRoaXMgaXMgYW4gW1IgTWFya2Rvd25dKGh0dHBzOi8vcm1hcmtkb3duLnJzdHVkaW8uY29tLykgZG9jdW1lbnQuICBGb2xsb3cgdGhlIGxpbmsgdG8gbGVhcm4gbW9yZSBhYm91dCBSIE1hcmtkb3duIGFuZCB0aGUgbm90ZWJvb2sgZm9ybWF0IHVzZWQgZHVyaW5nIHRoZSB3b3Jrc2hvcC4KClRoZSBjaGVhdHNoZWV0IGlzIHZlcnkgdXNlZnVsIHdoZW4gd29ya2luZyB3aXRoIGdncGxvdDI6IGh0dHBzOi8vd3d3LnJzdHVkaW8ub3JnL2xpbmtzL2RhdGFfdmlzdWFsaXphdGlvbl9jaGVhdF9zaGVldAoKCiMgU2V0dXAKCmBgYHtyLCB3YXJuaW5nPUZBTFNFLCBlcnJvcj1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KbGlicmFyeShkcGx5cikKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KHJlYWRyKQpgYGAKCgojIERhdGEKCldlJ3JlIHVzaW5nIHR3byBkYXRhIHNldHMsIG9uZSBmcm9tIGEgcGFja2FnZSwgb25lIGJ1aWx0IGludG8gUi4KCllvdSBvbmx5IG5lZWQgdG8gZG8gdGhpcyBvbmNlOiBpbnN0YWxsIHRoZSByZW1vdGVzIHBhY2thZ2UgKG9yIHlvdSBjYW4gc3Vic3RpdHV0ZSBkZXZ0b29scyBpZiB5b3UgaGF2ZSB0aGF0KSwgYW5kIHRoZW4gdXNlIHRoYXQgcGFja2FnZSB0byBpbnN0YWxsIHRoZSBwYWxtZXJwZW5ndWlucyBwYWNrYWdlIGZyb20gR2l0SHViLgoKYGBge3IsIGluc3RhbGxwYWNrYWdlcywgZXZhbD1GQUxTRX0KaW5zdGFsbC5wYWNrYWdlcygicGFsbWVycGVuZ3VpbnMiKQpgYGAKCklmIHlvdSBoYXZlIGl0IGluc3RhbGxlZCwgbG9hZCBpdCBhbmQgbG9vayBhdCB0aGUgcGVuZ3VpbnMgZGF0YSBzZXQ6CgpgYGB7cn0KbGlicmFyeShwYWxtZXJwZW5ndWlucykKcGVuZ3VpbnMKYGBgCgpXZSdyZSBhbHNvIGdvaW5nIHRvIHVzZSBhdmVyYWdlIHllYXJseSB0ZW1wZXJhdHVyZXMgaW4gTmV3IEhhbXBzaGlyZSwgYSBkYXRhIHNldCBidWlsdCBpbnRvIFIgKGBuaHRlbXBgKS4gIEJ1dCB3ZSBuZWVkIHRvIHB1dCBpdCBpbiBhIGRhdGEgZnJhbWUgZmlyc3QgdG8gbWFrZSBpdCBlYXNpZXIgdG8gdXNlIChpdCdzIGluIGEgc3BlY2lhbCB0aW1lIHNlcmllcyBvYmplY3QpOgoKYGBge3J9Cm5oIDwtIGRhdGEuZnJhbWUodGVtcD1jKG5odGVtcCksIHllYXI9MTkxMjoxOTcxKQpuaApgYGAKCiMgSW50cm9kdWN0aW9ucwoKV2h5IGdncGxvdD8gIEZvciBtZSwgaXQgbWFrZXMgZXhwbG9yaW5nIG15IGRhdGEsIGVzcGVjaWFsbHkgZ3JvdXBzIGluIG15IGRhdGEsIG11Y2ggZWFzaWVyLiAgSSBjYW4gZWFzaWx5IG1ha2Ugc29tZSBwbG90cyB3aXRoIGdncGxvdCB0aGF0IGFyZSBkaWZmaWN1bHQgZW5vdWdoIHRvIG1ha2Ugd2l0aCBiYXNlIFIgZ3JhcGhpY3MgdGhhdCBJIGp1c3Qgd29uJ3QgZG8gdGhlbSByb3V0aW5lbHkgd2hlbiBleHBsb3JpbmcgbXkgZGF0YS4gIFRoaXMgaXMgYmFkLCBiZWNhdXNlIHdlIHNob3VsZCB2aXN1YWxpemUgb3VyIGRhdGEhICBJIGRvbid0IHdhbnQgdGhlIGNvZGUgdG8gZ2V0IGluIHRoZSB3YXkgb2YgdGhhdC4gIGdncGxvdCBpcyBhIHBhY2thZ2Ugd2hlcmUgSSByZWNvbW1lbmQgZmlyc3QgbGVhcm5pbmcgdGhlIGJhc2ljIHN5bnRheCwgYW5kIHRoZW4gZ28gYmFjayBhbmQgbGVhcm4gc29tZSBtb3JlIGFib3V0IHRoZSB0aGVvcnkgYmVoaW5kIHRoZSBwYWNrYWdlIC0tIHdoYXQgdGhlIGxvZ2ljIGlzIHRvIGhvdyBpdCB3b3Jrcy4gSXQgdG9vayBtZSBhIGZldyBpdGVyYXRpb25zIG9mIHdvcmtpbmcgd2l0aCBnZ3Bsb3QgdG8gZ2V0IGNvbWZvcnRhYmxlIHdpdGggaXQsIGFuZCB0byB1bmRlcnN0YW5kIGhvdyB0byB0aGluayBpbiBhIHdheSB0aGF0IHdhcyBjb21wYXRpYmxlIHdpdGggaG93IGl0IGFwcHJvYWNoZXMgdmlzdWFsaXphdGlvbi4gIEJ1dCBvbmNlIEkgZGlkLCBpdCBtYWRlIGRhdGEgdmlzdWFsaXphdGlvbiBlYXNpZXIgYW5kIG1vcmUgZnVuLiAKClRoZSBzeW50YXggb2YgZ2dwbG90MiBpcyBhIGxpdHRsZSBiaXQgZGlmZmVyZW50IHRoYW4gb3RoZXIgdGlkeXZlcnNlIHBhY2thZ2VzLCBpbiBwYXJ0IGJlY2F1c2UgaXQgcHJlZGF0ZXMgdGhlbS4gIEJ1dCBpdCBkb2VzIHNoYXJlIGEgZmV3IHRoaW5ncyB3aXRoIG90aGVyIHRpZHl2ZXJzZSBwYWNrYWdlcywgc3VjaCBhczoKCiogSXQgZXhwZWN0cyB0byB3b3JrIHdpdGggYSBkYXRhIGZyYW1lLCBhbmQgdGhlIGRhdGEgZnJhbWUgaXMgdGhlIGZpcnN0IGlucHV0CiogWW91IGNhbiByZWZlcmVuY2UgdGhlIG5hbWVzIG9mIGNvbHVtbnMgd2l0aG91dCBxdW90ZXMgb3IgJCBzeW50YXgKKiBJdCdzIGVhc3kgdG8gd29yayB3aXRoIGdyb3VwcyBpbiB5b3VyIGRhdGEKKiBJbnN0ZWFkIG9mIGZ1bmN0aW9ucyB3aXRoIGxvdHMgb2YgZGlmZmVyZW50IGFyZ3VtZW50cyAoYWx0aG91Z2ggc29tZSBvZiB0aGVzZSBmdW5jdGlvbnMgZG8gaGF2ZSBhIGxvdCksIHlvdSBjb21iaW5lIG11bHRpcGxlIGZ1bmN0aW9uIGNhbGxzIHRvZ2V0aGVyIHRvIGFjaGlldmUgd2hhdCB5b3Ugd2FudCB0byBkbwoKVGhlIHBhY2thZ2UgaXMgY2FsbGVkIGdncGxvdDIgKHRoZXJlIGlzIG5vIGZpcnN0IHZlcnNpb24pLCB0aGUgbWFpbiBmdW5jdGlvbiBpbiB0aGUgcGFja2FnZSBpcyBjYWxsZWQgZ2dwbG90LCBhbmQgImdncGxvdCIgZ2VuZXJhbGx5IHJlZmVycyB0byBhbnl0aGluZyB3aXRoIHRoZSBwYWNrYWdlIChJIGFuZCBvdGhlcnMgb2Z0ZW4gZG9uJ3QgYm90aGVyIHRvIHNheSB0aGUgMiBpbiB0aGUgcGFja2FnZSBuYW1lKS4gIAoKIyBnZ3Bsb3QgQmFzaWNzCgpFYWNoIHBsb3Qgd2l0aCBnZ3Bsb3QgaGFzIHRoZSBzYW1lIHRlbXBsYXRlIGZvciB0aGUgY29kZToKCmBgYHtyLCBldmFsPUZBTFNFfQojIyBjYWxsIHRvIGdncGxvdCBhbmQgdGhlIG5hbWUgb2YgdGhlIGRhdGEgZnJhbWUKZ2dwbG90KGRhdGFmcmFtZW5hbWUsIAogICAgICAgIyMgd2hpY2ggdmFyaWFibGVzIHRvIHVzZSBmb3Igd2hpY2ggcGxvdCBjb21wb25lbnRzIAogICAgICAgYWVzKHggPSBjb2x1bW5fbmFtZSwgeSA9IGNvbHVtbl9uYW1lKSkgKyAgIAogICMgd2hhdCB0eXBlIG9mIHBsb3QKICBnZW9tXyoqKiooKSAgCmBgYAoKVGhlIGZpcnN0IGxpbmUgc2F5cyB3aGF0IGRhdGEgYW5kIHZhcmlhYmxlcyB0byB1c2UsIGFuZCB0aGUgc2Vjb25kIGxpbmUgc2F5cyB3aGF0IHR5cGUgb2YgcGxvdCB0byBtYWtlLiAgV2UgYWRkIHRoZSBjb21wb25lbnRzIG9mIHRoZSBwbG90IHRvZ2V0aGVyLCB3aGljaCBpcyBhIHNvbWV3aGF0IHVuaXF1ZSBzeW50YXguICAKCkZvciBleGFtcGxlOgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKHggPSBiaWxsX2xlbmd0aF9tbSwgCiAgICAgICAgICAgICAgICAgICAgIHkgPSBiaWxsX2RlcHRoX21tKSkgKyAKICBnZW9tX3BvaW50KCkKYGBgCgpUaGUgd2FybmluZyBtZXNzYWdlIGlzIHRlbGxpbmcgeW91IHRoYXQgdHdvIG9ic2VydmF0aW9ucyAocm93cykgaW4geW91ciBkYXRhIGZyYW1lIGFyZSBub3QgcGxvdHRlZCBiZWNhdXNlIHRoZXJlIHdlcmUgbWlzc2luZyB2YWx1ZXMuICBZb3UgY2FuIGdldCB0aGlzIHdhcm5pbmcgZnJvbSBleHBsaWNpdCBtaXNzaW5nIHZhbHVlcyBvciBpbiBjYXNlcyB3aGVyZSB5b3Ugc2V0IHRoZSBsaW1pdHMgb2YgdGhlIGF4ZXMgaW4gYSB3YXkgdGhhdCBleGNsdWRlcyBzb21lIGRhdGEgcG9pbnRzLgoKVGhlIGZpcnN0IGxpbmUgYWxvbmUgKHdpdGhvdXQgdGhlICsgYXQgdGhlIGVuZCkgbWFrZXMgYSBwbG90IHNldCB1cCB3aXRob3V0IGFkZGluZyB0aGUgZGF0YToKCmBgYHtyfQpnZ3Bsb3QocGVuZ3VpbnMsIGFlcyh4ID0gYmlsbF9sZW5ndGhfbW0sIAogICAgICAgICAgICAgICAgICAgICB5ID0gYmlsbF9kZXB0aF9tbSkpCmBgYAoKCklmIHdlIGNoYW5nZSB0aGUgZ2VvbSBmdW5jdGlvbiB3ZSB1c2UsIHRoZSBwbG90IHR5cGUgY2hhbmdlczoKCmBgYHtyfQpnZ3Bsb3QocGVuZ3VpbnMsIGFlcyh4ID0gYmlsbF9sZW5ndGhfbW0sIAogICAgICAgICAgICAgICAgICAgICB5ID0gYmlsbF9kZXB0aF9tbSkpICsgCiAgZ2VvbV9iaW4yZCgpCmBgYAoKVGhpcyBpcyBhIGRlbnNpdHkgcGxvdCwgd2hlcmUgdGhlIGNvbG9yIG9mIGVhY2ggcmVjdGFuZ2xlIGluZGljYXRlcyBob3cgbWFueSBvYnNlcnZhdGlvbnMgKHBvaW50cyBvbiB0aGUgYWJvdmUgcGxvdCkgZmFsbCBpbiB0aGF0IGFyZWEuCgoKIyMjIEVYRVJDSVNFCgpGaWxsIGluIHRoZSBgX19fYCB0byBtYWtlIGEgbGluZSBwbG90IHVzaW5nIGBnZW9tX2xpbmUoKWAgd2l0aCB0aGUgYG5oYCBkYXRhIGZyYW1lLCB3aXRoIGB5ZWFyYCBhcyB0aGUgeCB2YXJpYWJsZSBhbmQgYHRlbXBgIGFzIHRoZSB5OgoKYGBge3IsIGV2YWw9RkFMU0V9CmdncGxvdChfX18sIGFlcyh4ID0gX19fLCB5ID0gX19fKSkgKyAKICBfX18oKQpgYGAKCiMjIERpZmZlcmVudCBQbG90IFR5cGVzCgpUaGUgYWJvdmUgcGxvdHMgdXNlZCB0d28gY29udGludW91cyB2YXJpYWJsZXMuICBUaGUgW2NoZWF0IHNoZWV0XShodHRwczovL3d3dy5yc3R1ZGlvLm9yZy9saW5rcy9kYXRhX3Zpc3VhbGl6YXRpb25fY2hlYXRfc2hlZXQpIGhhcyB0aGUgZ2VvbXMgb3JnYW5pemVkIGJ5IGhvdyBtYW55IGFuZCB3aGF0IHR5cGUgb2YgdmFyaWFibGVzIHlvdSB3YW50IHRvIHBsb3QuICBGb3IgZXhhbXBsZSwgd2UgY2FuIG1ha2UgYSBoaXN0b2dyYW0sIHdoaWNoIGlzIG9mIGEgc2luZ2xlIHZhcmlhYmxlOgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKHggPSBiaWxsX2RlcHRoX21tKSkgKyAKICBnZW9tX2hpc3RvZ3JhbSgpCmBgYAoKVGhlIHdhcm5pbmcgbWVzc2FnZSBpcyBob3cgbWFueSBiaW5zIChiYXJzKSB0aGVyZSBhcmUgaW4gdGhlIGhpc3RvZ3JhbS4gIEl0IGRlZmF1bHRzIHRvIDMwLCBidXQgaXQgd2FudHMgeW91IHRvIGV4cGxpY2l0bHkgcGljayBhIG51bWJlciBpbnN0ZWFkOgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKHggPSBiaWxsX2RlcHRoX21tKSkgKyAKICBnZW9tX2hpc3RvZ3JhbShiaW5zID0gMjUpIApgYGAKCldlIGNhbiBhbHNvIGhhdmUgbXVsdGlwbGUgZ2VvbXMgb24gdGhlIHNhbWUgcGxvdDoKCmBgYHtyfQpnZ3Bsb3QobmgsIGFlcyh4PXllYXIsIHk9dGVtcCkpICsgCiAgZ2VvbV9saW5lKCkgKyAKICBnZW9tX3BvaW50KCkKYGBgCgpgYGB7cn0KZ2dwbG90KG5oLCBhZXMoeD15ZWFyLCB5PXRlbXApKSArIAogIGdlb21fcG9pbnQoKSArIAogIGdlb21fbGluZSgpICsgCiAgZ2VvbV9zbW9vdGgoKQpgYGAKCgojIyMgRVhFUkNJU0UKCk1ha2UgYSBoaXN0b2dyYW0gKGBnZW9tX2hpc3RvZ3JhbSgpYCkgb2YgdGhlIGB0ZW1wYCB2YXJpYWJsZSBpbiB0aGUgYG5oYCBkYXRhc2V0CgpgYGB7ciwgZXZhbD1GQUxTRX0KCmBgYAoKCiMjIE90aGVyIEFlc3RoZXRpY3MgYW5kIEdyb3VwaW5nCgpXZSBjYW4gc2V0IG90aGVyIGFlc3RoZXRpY3Mgb2YgdGhlIHBsb3QgKGNvbG9yLCBmaWxsLCBsaW5ldHlwZSwgbWFya2VyLCBldGMuKSBhY2NvcmRpbmcgdG8gdmFyaWFibGVzIGluIG91ciBkYXRhLiAgCgpOb3RlIHRoYXQgaWYgeW91IHdhbnQgYSBsaW5lIG9yIGNvbG9yIGZvciBlYWNoIGdyb3VwLCB0aG9zZSBncm91cHMgbmVlZCB0byBiZSBkZWZpbmVkIGJ5IGEgY2F0ZWdvcmljYWwgdmFyaWFibGUuICBUaGlzIGlzIHRoZSBvcHBvc2l0ZSBvZiBob3cgZGF0YSB1c3VhbGx5IG5lZWRzIHRvIGJlIHN0cnVjdHVyZWQgd2hlbiB1c2luZyBiYXNlIFIgcGxvdHRpbmcgZnVuY3Rpb25zLiBJZiB0aGUgdmFsdWVzIGZvciB0aGUgdHdvIGdyb3VwcyBhcmUgaW4gc2VwYXJhdGUgY29sdW1ucyBpbiB5b3VyIGRhdGFzZXQsIHlvdSdsbCBuZWVkIHRvIHJlc2hhcGUgeW91ciBkYXRhIHRvIGdldCB0aGVtIGluIGEgc2luZ2xlIGNvbHVtbiBpbnN0ZWFkIC0tIHNvbWV0aGluZyBjb3ZlcmVkIGluIHRoZSB0aWR5ciBzZXNzaW9uIG9mIHRoZXNlIHdvcmtzaG9wcy4KCkVhY2ggYWVzdGhldGljIC0tIGVhY2ggdGhpbmcgeW91IGRlZmluZSBpbiBhZXMoKSAtLSBuZWVkcyB0byBiZSBhIHNpbmdsZSBjb2x1bW4gaW4geW91ciBkYXRhIHNldC4KCldpdGggY2F0ZWdvcmljYWwgdmFyaWFibGVzLCB0aGlzIGhlbHBzIHVzIHNlZSB0aGUgZ3JvdXBzIGluIG91ciBkYXRhOgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKHggPSBiaWxsX2xlbmd0aF9tbSwgCiAgICAgICAgICAgICAgICAgICAgIHkgPSBiaWxsX2RlcHRoX21tLAogICAgICAgICAgICAgICAgICAgICBjb2xvciA9IHNwZWNpZXMpKSArICAKICBnZW9tX3BvaW50KCkKYGBgCgpOb3RlIHRoYXQgYSBsZWdlbmQgd2FzIGFkZGVkIGF1dG9tYXRpY2FsbHkuCgpgYGB7cn0KZ2dwbG90KHBlbmd1aW5zLCBhZXMoeCA9IGJpbGxfbGVuZ3RoX21tLCAKICAgICAgICAgICAgICAgICAgICAgeSA9IGJpbGxfZGVwdGhfbW0sCiAgICAgICAgICAgICAgICAgICAgIGNvbG9yID0gc3BlY2llcykpICsgIAogIGdlb21fcG9pbnQoKSArIAogIGdlb21fc21vb3RoKG1ldGhvZD0ibG0iLCBzZT1GQUxTRSkKYGBgCgpgbWV0aG9kPWxtYCB0ZWxscyBpdCBob3cgdG8gZml0IGEgbGluZSB0byB0aGUgZGF0YTsgImxtIiBtZWFucyBmaXQgYSBsaW5lYXIgbW9kZWwgd2l0aCB0aGUgYGxtYCBmdW5jdGlvbi4gIGBzZT1GQUxTRWAgbWVhbnMgdG8gbm90IGRpc3BsYXkgdGhlIHN0YW5kYXJkIGVycm9ycyBvbiB0aGUgcGxvdC4KCk5vdGUgdGhhdCB0aGUgc21vb3RoaW5nIHdhcyBkb25lIGJ5IGdyb3VwLiAgU3BlY2lmeWluZyBgY29sb3JgIGVmZmVjdGl2ZWx5IGdyb3VwZWQgb3VyIGRhdGEsIG11Y2ggbGlrZSB3aGF0IGhhcHBlbnMgd2l0aCBgZ3JvdXBfYnlgIGluIGRwbHlyLgoKCiMjIyBFWEVSQ0lTRQoKUmVwZWF0IHRoZSBzYW1lIHBsb3QgYXMgYWJvdmUsIGBiaWxsX2xlbmd0aF9tbWAgdnMuIGBiaWxsX2RlcHRoX21tYCwgYnV0IGBjb2xvcmAgYnkgYGJvZHlfbWFzc19nYCBpbnN0ZWFkIG9mIGBzcGVjaWVzYC4KCmBgYHtyfQoKYGBgCgoKCiMjIExhYmVscy9UaXRsZXMKCkEgY29udmVuaWVudCB3YXkgdG8gbGFiZWwgYXhlcyBhbmQgbGVnZW5kcyBhbmQgc2V0IHRoZSB0aXRsZSBpcyB3aXRoIHRoZSBsYWJzKCkgZnVuY3Rpb24sIGFkZGVkIGFzIGEgY29tcG9uZW50IG9mIHRoZSBwbG90OgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKHggPSBiaWxsX2xlbmd0aF9tbSwgCiAgICAgICAgICAgICAgICAgICAgIHkgPSBiaWxsX2RlcHRoX21tLAogICAgICAgICAgICAgICAgICAgICBjb2xvciA9IHNwZWNpZXMpKSArICAKICBnZW9tX3BvaW50KCkgKyAKICBsYWJzKHRpdGxlPSJQZW5ndWluIEJpbGwgRGltZW5zaW9ucyIsCiAgICAgICB5PSJEZXB0aCAobW0pIiwgCiAgICAgICB4PSJMZW5ndGggKG1tKSIsCiAgICAgICBjb2xvcj0iQm9keSBNYXNzIiwKICAgICAgIHN1YnRpdGxlPSJUaHJlZSBTcGVjaWVzIiwKICAgICAgIGNhcHRpb249IlNvdXJjZTogcGFsbWVycGVuZ3VpbnMiKQpgYGAKCiMjIyBFWEVSQ0lTRQoKUGxvdCBgZmxpcHBlcl9sZW5ndGhfbW1gIHZzLiBgYm9keV9tYXNzX2dgIGFuZCBjb2xvciBieSBgc2V4YC4gIExhYmVsIHRoZSBheGVzIGFuZCB0aXRsZSB5b3VyIHBsb3QuCgpgYGB7cn0KCmBgYAoKCgojIyBBeGlzIExpbWl0cwoKVG8gY2hhbmdlIHRoZSBheGVzIG1pbiBhbmQgbWF4IHZhbHVlcywgbGlrZSB3ZSB3b3VsZCB3aXRoIGB4bGltYCBvciBgeWxpbWAgaW4gYmFzZSBSIGdyYXBoaWNzLCB3ZSB1c2UgYSBzY2FsZSBmdW5jdGlvbiBhbmQgc2V0IHRoZSBgbGltaXRzYCBhcmd1bWVudDoKCmBgYHtyfQpnZ3Bsb3QocGVuZ3VpbnMsIGFlcyh4ID0gYmlsbF9sZW5ndGhfbW0sIAogICAgICAgICAgICAgICAgICAgICB5ID0gYmlsbF9kZXB0aF9tbSkpICsgIAogIGdlb21fcG9pbnQoKSArIAogIHNjYWxlX3hfY29udGludW91cyhsaW1pdHM9YygzMCwgNTApKQpgYGAKCgoKIyMgRmFjZXRzCgpJbiBhZGRpdGlvbiB0byB1c2luZyBjb2xvciwgZmlsbCwgbGluZXR5cGUsIGV0Yy4gdG8gZGVub3RlIGdyb3Vwcywgd2UgY2FuIGFsc28gbWFrZSBhIHNlcGFyYXRlIHBsb3QgZm9yIGVhY2ggZ3JvdXAsIHdoZXJlIHRoZSBwbG90cyBhcmUgYWxpZ25lZCBhbmQgc2hhcmUgYXhlcy4gIFRoZXNlIGFyZSBjYWxsZWQgZmFjZXRzOgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKGZsaXBwZXJfbGVuZ3RoX21tKSkgKyAKICBnZW9tX2hpc3RvZ3JhbSgpICsgCiAgZmFjZXRfZ3JpZChzZXggfiAuKQpgYGAKClRoZSBmYWNldCBmdW5jdGlvbiBzeW50YXggaXMgdXNpbmcgdGhlIGZvcm11bGEgc3ludGF4IHVzZWQgZWxzZXdoZXJlIGluIFIsIGJ1dCBpdCBjYW4gYmUgdG91Z2ggdG8gcmVtZW1iZXIuICBJdCdzIG9uIHRoZSBjaGVhdCBzaGVldCB0aG91Z2ggc28geW91IGNhbiBsb29rIGl0IHVwLiAgVGhlcmUgYXJlIG90aGVyIG9wdGlvbnMgZm9yIGxheWluZyBvdXQgcGxvdHMgaG9yaXpvbnRhbGx5IG9yIGluIGEgZ3JpZC4gIAoKTm90ZSB0aGF0IHRoZSBwbG90cyBzaGFyZSBhbiB4LWF4aXMsIGFuZCB0aGUgeS1heGlzIGhhcyB0aGUgc2FtZSByYW5nZSBpbiBlYWNoIHBsb3QuCgpJZiB5b3Ugd2FudCB0byBwdXQgbXVsdGlwbGUgcGxvdHMgaW4gYSBzaW5nbGUgaW1hZ2UgdG9nZXRoZXIsIGJ1dCB0aGV5IGFyZSBub3QgZmFjZXRzIG9mIGEgc2luZ2xlIHBsb3QsIHNlZSB0aGUgcGF0Y2h3b3JrIHBhY2thZ2UuCgojIyMgRVhFUkNJU0UKCk1ha2UgYSBoaXN0b2dyYW0gKHNldCBhIHZhbHVlIGZvciBiaW5zKSBvZiBgYm9keV9tYXNzX2dgIGZvciBlYWNoIGBzcGVjaWVzYDoKCmBgYHtyLCBldmFsPUZBTFNFfQpnZ3Bsb3QocGVuZ3VpbnMsIGFlcyhfX18pKSArIAogIF9fXyhfX18pICsgCiAgZmFjZXRfZ3JpZChfX18gfiAuKQpgYGAKCgoKIyMgU3R5bGluZyAKClRoZW1lcyBjb250cm9sIHRoZSBsb29rIG9mIHRoZSBwbG90IG5vdCByZWxhdGVkIHRvIHRoZSBkYXRhOiBmb250cywgYmFja2dyb3VuZHMsIGdyaWQgbGluZXMsIGF4aXMgbGluZXMsIGV0Yy4KClRoZXJlIGFyZSBzb21lIGJ1aWx0LWluIHRoZW1lcyAoYHRoZW1lX21pbmltYWwoKWAsIGB0aGVtZV9idygpYCwgYHRoZW1lX2NsYXNzaWMoKWAsIGV0Yy4pLCBidXQgeW91IGNhbiBhbHNvIGNvbnRyb2wgdGhlIHN0eWxpbmcgZnVydGhlciB3aXRoIHRoZSBgdGhlbWUoKWAgZnVuY3Rpb246CgpgYGB7cn0KZ2dwbG90KHBlbmd1aW5zLCBhZXMoYm9keV9tYXNzX2cpKSArIAogIGdlb21faGlzdG9ncmFtKGJpbnMgPSAzMCkgKyAKICB0aGVtZV9taW5pbWFsKCkKYGBgCgpGb3IgZnVydGhlciBjb250cm9sLCB1c2UgYHRoZW1lYCBpbnN0ZWFkIG9yIGluIGFkZGl0aW9uOgoKYGBge3J9CmdncGxvdChwZW5ndWlucywgYWVzKGJvZHlfbWFzc19nKSkgKyAKICBnZW9tX2hpc3RvZ3JhbShiaW5zID0gMzApICsgCiAgdGhlbWVfbWluaW1hbCgpICsgCiAgdGhlbWUocGFuZWwuZ3JpZC5tYWpvci54ID0gZWxlbWVudF9ibGFuaygpLCAgIyBnZXQgcmlkIG9mIHRoZSB2ZXJ0aWNhbCBncmlkIGxpbmVzCiAgICAgICAgcGFuZWwuZ3JpZC5taW5vci54ID0gZWxlbWVudF9ibGFuaygpKQpgYGAKCk1hbnkgdGhlbWUgZWxlbWVudHMgYXJlIGNvbnRyb2xsZWQgYnkgc2V0dGluZyBvcHRpb25zIGluIGFkZGl0aW9uYWwgYGVsZW1lbnRgIGZ1bmN0aW9uczoKCiogYGVsZW1lbnRfYmxhbmsoKWAgLSBvbWl0IHRoZSBlbGVtZW50CiogYGVsZW1lbnRfdGV4dCgpYCAKKiBgZWxlbWVudF9saW5lKClgCiogYGVsZW1lbnRfcmVjdCgpYAoKVGhlc2UgYXJlIG9wdGlvbnMgdGhhdCB5b3Ugd2lsbCBnZW5lcmFsbHkgZ29vZ2xlIGZvciBob3cgdG8gZG86ICJnZ3Bsb3QgcmVtb3ZlIGdyaWRsaW5lcyIsICJnZ3Bsb3QgY2hhbmdlIGZvbnQgc2l6ZSIsIGV0Yy4KCgpJbiBhZGRpdGlvbiB0byB0aGVtZXMsIHlvdSBjYW4gY29udHJvbCBzb21lIGFzcGVjdHMgb2YgdGhlIGRhdGEtZHJpdmVuIGVsZW1lbnRzIG9mIHRoZSBwbG90IGRpcmVjdGx5LCB3aXRob3V0IHVzaW5nIGEgdmFyaWFibGUgdG8gZGVmaW5lIHRoZW0uICBGb3IgZXhhbXBsZSB3aXRoIGNvbG9yIG9yIHNpemUsIGlmIHdlIHNwZWNpZnkgdGhlbSBvdXRzaWRlIG9mIGEgY2FsbCB0byBgYWVzKClgIHdpdGhpbiB0aGUgZ2VvbSBmdW5jdGlvbjoKCmBgYHtyfQpnZ3Bsb3QocGVuZ3VpbnMsIGFlcyh4ID0gYmlsbF9sZW5ndGhfbW0sIAogICAgICAgICAgICAgICAgICAgICB5ID0gYmlsbF9kZXB0aF9tbSkpICsgIAogIGdlb21fcG9pbnQoY29sb3I9InJlZCIsIHNpemU9MykgCmBgYAoKaXQgY29sb3JzIGFsbCBvZiB0aGUgcG9pbnRzIHRoZSBzYW1lIGNvbG9yLiAgCgojIyMgRVhFUkNJU0UKCkNoYW5nZSB0aGUgYGZpbGxgIGNvbG9yIGZvciB0aGUgYmFycyBpbiBvdXIgaGlzdG9ncmFtIGFib3ZlIHRvIGJlICJkYXJrZ3JlZW4iLiAgVGhlIGNvZGUgZnJvbSBhYm92ZSBpcyBjb3BpZWQgYmVsb3cgdG8gZ2V0IHlvdSBzdGFydGVkLgoKYGBge3IsIGV2YWw9RkFMU0V9CmdncGxvdChwZW5ndWlucywgYWVzKGJvZHlfbWFzc19nKSkgKyAKICBnZW9tX2hpc3RvZ3JhbShiaW5zID0gMzApICsgCiAgdGhlbWVfbWluaW1hbCgpICsgCiAgdGhlbWUocGFuZWwuZ3JpZC5tYWpvci54ID0gZWxlbWVudF9ibGFuaygpLCAgCiAgICAgICAgcGFuZWwuZ3JpZC5taW5vci54ID0gZWxlbWVudF9ibGFuaygpKQpgYGAKCgoKIyMgUGlwZXMKCldlIHVzZSBgK2AsIG5vdCBgJT4lYCwgdG8gam9pbiB0b2dldGhlciBjb21wb25lbnRzIG9mIG91ciBnZ3Bsb3QuICBCdXQgd2UgY2FuIHN0aWxsIHBpcGUgdGhlIHJlc3VsdHMgb2Ygb3RoZXIgY29tbWFuZHMgaW50byBnZ3Bsb3QgLSBJIGp1c3QgZGlkbid0IGFib3ZlIGJlY2F1c2Ugd2Ugd2VyZSBqdXN0IHBsb3R0aW5nIHRoZSBkYXRhIGZyYW1lIGFzIGl0IGlzLiAgRm9yIGV4YW1wbGUsIHRvIHBsb3Qgb25seSB0aGUgQWRlbGllIHBlbmd1aW5zOgoKYGBge3J9CnBlbmd1aW5zICU+JQogIGZpbHRlcihzcGVjaWVzID09ICJBZGVsaWUiKSAlPiUKICBnZ3Bsb3QoYWVzKHg9YmlsbF9sZW5ndGhfbW0sIHk9YmlsbF9kZXB0aF9tbSkpICsgCiAgZ2VvbV9wb2ludCgpCmBgYAoKCiMgTGVhcm4gTW9yZQoKVGhlIGZpcnN0IGNoYXB0ZXIgb2YgUiBmb3IgRGF0YSBTY2llbmNlIGh0dHBzOi8vcjRkcy5oYWQuY28ubnovIGNvdmVycyBnZ3Bsb3Qgd2VsbC4gIAoKRm9yIG1vcmUgb24gZ2dwbG90Miwgc2VlIFtvdXIgZ3VpZGVdKGh0dHBzOi8vc2l0ZXMubm9ydGh3ZXN0ZXJuLmVkdS9yZXNlYXJjaGNvbXB1dGluZy8yMDIwLzA0LzEzL29ubGluZS1sZWFybmluZy1yZXNvdXJjZXMtci1nZ3Bsb3QyLykgd2l0aCBmcmVlIChmb3IgTm9ydGh3ZXN0ZXJuIGZvbGtzKSByZXNvdXJjZXMuCgpXZSdyZSBoYXBweSB0byBoZWxwIHlvdSB1c2UgZ2dwbG90IG9uIHlvdXIgb3duIGRhdGEuICBSZXF1ZXN0IGEgZnJlZSBjb25zdWx0YXRpb246IGh0dHBzOi8vd3d3Lml0Lm5vcnRod2VzdGVybi5lZHUvcmVzZWFyY2gvY29uc3VsdGF0aW9uL2RhdGEtc2VydmljZXMuaHRtbAo=