The output of most of the R chunks isn’t included in the HTML version of the file to keep it to a more reasonable file size. You can run the code in R to see the output.
This is an R Markdown document. Follow the link to learn more about R Markdown and the notebook format used during the workshop.
Setup
library(tidyverse)
## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──
## ✓ ggplot2 3.3.5 ✓ purrr 0.3.4.9000
## ✓ tibble 3.1.6 ✓ dplyr 1.0.7
## ✓ tidyr 1.1.4 ✓ stringr 1.4.0
## ✓ readr 2.0.2 ✓ forcats 0.5.1
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
The data is from the Stanford Open Policing Project and includes vehicle stops by the Evanston police in 2017. We’re reading the data in from a URL directly.
police <- read_csv("https://raw.githubusercontent.com/nuitrcs/r-tidyverse/main/data/ev_police.csv",
col_types=c( "location"="c"))
dplyr
dplyr is at the core of the tidyverse. It is for working with data frames. It contains six main functions, each a verb, of actions you frequently take with a data frame. We’re covering 3 of those functions today (select, filter, mutate), and 3 more next session (group_by, summarize, arrange).
Each of these functions takes a data frame as the first input. Within the function call, we can refer to the column names without quotes and without $ notation.
Select: Choose Columns
The previous session covered the basics of select
but there are many more options for how we can specify which columns to choose.
First, let’s remember what the column names are:
names(police)
Recall, the select
function takes as the first input a data frame, and then we can list one or more columns, names unquoted, that we want to select. The columns will be ordered in the order we specify them.
select(police, outcome, date)
Ranges
There are a number of select helper functions and special syntax options that allow us to choose multiple columns.
First, we can use : for range, but with names in addition to numbers:
select(police, raw_DriverRace:raw_ResultOfStop)
We can select the rightmost columns with last_col()
:
select(police, last_col())
Last 4 columns (the input to the function is the offset # of columns from the right edge):
select(police, last_col(0:3))
Excluding columns
We can also say which columns we don’t want by putting a -
in front of the name:
select(police, -raw_row_number, -subject_age)
When using negated -
column names, if we start with negations, it will include all other columns, even if we try to specify some:
select(police, -raw_row_number, -subject_age, time:outcome)
To both specify the columns wanted and exclude some that would otherwise be selected, the exclusions need to come at the end:
select(police, location:type, -department_id)
Reordering and renaming
We’ve already seen that columns will appear in the result in the order that we list the names.
The everything()
helper function can be useful if you want to pull a few columns over to the left so that they are the ones that show first when you look at the data:
select(police, outcome, everything())
Each column only gets included once, in the position that it first appears. So “outcome” becomes the leftmost column above and no longer appears in it’s original spot.
We can also rename columns while using select()
. The syntax is new_name = old_name
.
select(police, raw_id=raw_row_number, date, time)
or we can use rename()
to only rename, without affecting which columns are included or their order (all of the columns are kept in the same order):
rename(police, raw_id=raw_row_number)
Remember, this doesn’t change police because we didn’t save the result. So far, we’ve just been printing the copy of the data frame that is returned by the function. If we want to change our data frame, we’d need to save the result back to the police
object.
police <- rename(police, raw_id=raw_row_number)
EXERCISE
Remember: run the cells above to load tidyverse and import the data.
Using select
and/or rename
as needed:
- Rename subject_age to age, subject_race to race, and subject_sex to sex, but keep the columns in their original order
- Exclude the department_id and department_name columns
Hint: remember that you can chain dplyr commands together with %>%
.
Matching names
We can also select by matching patterns in the names of the columns. The patterns to match are in quotes because they aren’t column names – just character data.
select(police, starts_with("contraband"))
select(police, ends_with("issued"))
select(police, contains("vehicle"))
We can also put a -
in front of these helper functions to exclude columns:
select(police, -contains("subject"))
And there are even more select helper functions.
EXERCISE
Use select()
to get a copy of police
without the columns that start with “raw”.
Hint: If you mess up your police
dataset, re-run the cell near the top of the file under the Data header and read the data in again fresh.
Selecting with Vectors or Functions
What if we have the names of the columns we want to select in a vector already? For example:
analysis_vars <- c("search_basis", "reason_for_stop")
Perhaps we built this vector programatically (we wrote code to determine the values, instead of typing them ourselves), so we can’t just rewrite it to:
select(police, search_basis, reason_for_stop)
If we just give the vector to select
, it looks like we expect “analysis_vars” to be a column name in police. We get a warning:
select(police, analysis_vars)
This warning tells us what we should do instead, which is use all_of
:
select(police, all_of(analysis_vars))
This makes it clearer that “analysis_vars” isn’t the name of a column in police.
What if we want to select columns of a certain type – for example, only the numeric columns?
select(police, where(is.numeric))
is.numeric
is the name of a function. We just use the name without ()
. This function is applied to each column, and if it returns TRUE, then the column is selected. Like above with using a vector, we wrap the function we want to use in where
to make it clear that we’re using a function, not looking for a column named “is.numeric”).
where
can be used with any function that returns a single TRUE or FALSE value for each column.
Filter: Choose Rows
The filter()
function lets us choose which rows of data to keep by writing expressions that return TRUE or FALSE for every row in the data frame. Recall from last session:
filter(police, date == "2017-01-02")
We can do complex conditions as we could do with []
filter(police, subject_race == "hispanic" & subject_sex == "female")
If we include multiple comma-separated conditions, they are joined with &
and. So this following is equivalent to the above.
filter(police, subject_race == "hispanic", subject_sex == "female")
EXERCISE
- Filter
police
to choose the rows where location is 60201 or 60202
- Filter
police
to choose the rows where location is 60201 or 60202 and subject_sex is “male”
Hints:
- The “or” operator is
|
; the “and” operator is &
Bonus: slice variants
Last session, we saw slice()
briefly as a way to choose which rows we want by their integer index value. But, there are some useful variants on the slice
function that help us select rows that have the maximum or minimum value of a particular variable:
slice_max(mtcars, hp)
By default it just gives us one row, but we can ask for more than one by setting the n
argument:
slice_max(mtcars, hp, n=3)
We got 4 rows above because there was a tie at position 3. There’s an option with_ties
that can change how ties are handled.
There’s also a minimum version:
slice_min(mtcars, disp)
Mutate: Change or Create Columns
mutate()
is used to both change the values of an existing column and make a new column.
We name the column we’re mutating and set the value. If the name already exists, it will update the column. If the name doesn’t exist, it will create a new variable (column is appended at the end of existing columns).
mutate(police, vehicle_age = 2017 - vehicle_year) %>%
select(starts_with("vehicle")) # just to pick a few columns to look at
We can put multiple mutations in the same call to mutate, with the expressions separated by commas:
mutate(police,
vehicle_age = 2017 - vehicle_year,
old_car = vehicle_year < 2000)
Within a call to mutate, we can refer to variables we made or changed earlier in the same call as well. Here, we create vehicle_age, and then use it to create vehicle_age_norm:
police %>%
mutate(vehicle_age = 2017 - vehicle_year,
vehicle_age_norm = ifelse(vehicle_age < 0, # ifelse test condition
0, # value if true
vehicle_age) # value if false
) %>%
# below is just making it easier for us to see what we changed
select(starts_with("vehicle")) %>%
filter(vehicle_age < 0)
Side note: there is a tidyverse version of ifelse()
called if_else()
that works generally the same except it is stricter about checking data types.
mutate()
can also change an existing column. The location column in the data contains zip codes, that were read in as numeric values. This means the leading zero on some zip codes has been lost. Convert location to character data, and add back in the leading 0 if it should be there.
Here I’ll change the location column twice in the same call with two different transformations:
police %>%
mutate(location = as.character(location), # first convert to character, then recode below
location = ifelse(nchar(location) == 4, # ifelse test (vector of TRUE and FALSE)
paste0("0", location), # value if TRUE
location)) %>% # value if FALSE
select(location) %>% # selecting just the column we mutated to look at
filter(startsWith(location, "0")) # selecting a few rows to look at the change
Remember that when using mutate()
, you’re operating on the entire column at once, so you can’t select just a subset of the vector as you would with []
. This means more frequently using functions like ifelse()
or helper functions such as na_if()
, replace_na()
, or recode()
.
na_if
replaces an existing value with NA
. replace_na
does roughly the opposite: replaces NA
with a new value.
mutate(police, vehicle_make = na_if(vehicle_make, "UNK"))
na_if()
can only can check and replace one value at a time; it also can’t be used with any expressions (x <= 1
) – only single values.
EXERCISE
If beat is “/” or “CHICAGO”, set it to NA
instead using mutate()
.
Hint: it’s ok if you take two steps to do this.
Recap
You now can use select
and filter
to subset your data in a wide variety of ways, and mutate
to update variables or create new ones.
Next session: the three other common dplyr “verb” functions for working with data frames: group_by
, summarize
, and arrange
.
LS0tCnRpdGxlOiAnZHBseXI6IHNlbGVjdCwgZmlsdGVyLCBtdXRhdGUnCm91dHB1dDoKICBodG1sX2RvY3VtZW50OgogICAgZGZfcHJpbnQ6IHBhZ2VkCiAgICBjb2RlX2Rvd25sb2FkOiBUUlVFCiAgICB0b2M6IHRydWUKICAgIHRvY19kZXB0aDogMgplZGl0b3Jfb3B0aW9uczoKICBjaHVua19vdXRwdXRfdHlwZTogaW5saW5lCi0tLQoKYGBge3IsIHNldHVwLCBpbmNsdWRlPUZBTFNFfQojIHlvdSBkb24ndCBuZWVkIHRvIHJ1biB0aGlzIHdoZW4gd29ya2luZyBpbiBSU3R1ZGlvCmtuaXRyOjpvcHRzX2NodW5rJHNldChldmFsPUZBTFNFKSAgIyB3aGVuIG1ha2luZyB0aGUgaHRtbCB2ZXJzaW9uIG9mIHRoaXMgZmlsZSwgZG9uJ3QgZXhlY3V0ZSB0aGUgY29kZQpgYGAKCipUaGUgb3V0cHV0IG9mIG1vc3Qgb2YgdGhlIFIgY2h1bmtzIGlzbid0IGluY2x1ZGVkIGluIHRoZSBIVE1MIHZlcnNpb24gb2YgdGhlIGZpbGUgdG8ga2VlcCBpdCB0byBhIG1vcmUgcmVhc29uYWJsZSBmaWxlIHNpemUuICBZb3UgY2FuIHJ1biB0aGUgY29kZSBpbiBSIHRvIHNlZSB0aGUgb3V0cHV0LioKClRoaXMgaXMgYW4gW1IgTWFya2Rvd25dKGh0dHBzOi8vcm1hcmtkb3duLnJzdHVkaW8uY29tLykgZG9jdW1lbnQuICBGb2xsb3cgdGhlIGxpbmsgdG8gbGVhcm4gbW9yZSBhYm91dCBSIE1hcmtkb3duIGFuZCB0aGUgbm90ZWJvb2sgZm9ybWF0IHVzZWQgZHVyaW5nIHRoZSB3b3Jrc2hvcC4KCiMgU2V0dXAKCmBgYHtyLCBldmFsPVRSVUV9CmxpYnJhcnkodGlkeXZlcnNlKQpgYGAKCgpUaGUgZGF0YSBpcyBmcm9tIHRoZSBbU3RhbmZvcmQgT3BlbiBQb2xpY2luZyBQcm9qZWN0XShodHRwczovL29wZW5wb2xpY2luZy5zdGFuZm9yZC5lZHUvZGF0YS8pIGFuZCBpbmNsdWRlcyB2ZWhpY2xlIHN0b3BzIGJ5IHRoZSBFdmFuc3RvbiBwb2xpY2UgaW4gMjAxNy4gIFdlJ3JlIHJlYWRpbmcgdGhlIGRhdGEgaW4gZnJvbSBhIFVSTCBkaXJlY3RseS4gIAoKYGBge3IsIGV2YWw9VFJVRX0KcG9saWNlIDwtIHJlYWRfY3N2KCJodHRwczovL3Jhdy5naXRodWJ1c2VyY29udGVudC5jb20vbnVpdHJjcy9yLXRpZHl2ZXJzZS9tYWluL2RhdGEvZXZfcG9saWNlLmNzdiIsCiAgICAgICAgICAgICAgICAgICBjb2xfdHlwZXM9YyggImxvY2F0aW9uIj0iYyIpKQpgYGAKCgojIGRwbHlyCgpkcGx5ciBpcyBhdCB0aGUgY29yZSBvZiB0aGUgdGlkeXZlcnNlLiAgSXQgaXMgZm9yIHdvcmtpbmcgd2l0aCBkYXRhIGZyYW1lcy4gIEl0IGNvbnRhaW5zIHNpeCBtYWluIGZ1bmN0aW9ucywgZWFjaCBhIHZlcmIsIG9mIGFjdGlvbnMgeW91IGZyZXF1ZW50bHkgdGFrZSB3aXRoIGEgZGF0YSBmcmFtZS4gIFdlJ3JlIGNvdmVyaW5nIDMgb2YgdGhvc2UgZnVuY3Rpb25zIHRvZGF5IChzZWxlY3QsIGZpbHRlciwgbXV0YXRlKSwgYW5kIDMgbW9yZSBuZXh0IHNlc3Npb24gKGdyb3VwX2J5LCBzdW1tYXJpemUsIGFycmFuZ2UpLgoKRWFjaCBvZiB0aGVzZSBmdW5jdGlvbnMgdGFrZXMgYSBkYXRhIGZyYW1lIGFzIHRoZSBmaXJzdCBpbnB1dC4gIFdpdGhpbiB0aGUgZnVuY3Rpb24gY2FsbCwgd2UgY2FuIHJlZmVyIHRvIHRoZSBjb2x1bW4gbmFtZXMgd2l0aG91dCBxdW90ZXMgYW5kIHdpdGhvdXQgJCBub3RhdGlvbi4KCiMgU2VsZWN0OiBDaG9vc2UgQ29sdW1ucwoKVGhlIHByZXZpb3VzIHNlc3Npb24gY292ZXJlZCB0aGUgYmFzaWNzIG9mIGBzZWxlY3RgIGJ1dCB0aGVyZSBhcmUgbWFueSBtb3JlIG9wdGlvbnMgZm9yIGhvdyB3ZSBjYW4gc3BlY2lmeSB3aGljaCBjb2x1bW5zIHRvIGNob29zZS4KCkZpcnN0LCBsZXQncyByZW1lbWJlciB3aGF0IHRoZSBjb2x1bW4gbmFtZXMgYXJlOgoKYGBge3J9Cm5hbWVzKHBvbGljZSkKYGBgCgpSZWNhbGwsIHRoZSBgc2VsZWN0YCBmdW5jdGlvbiB0YWtlcyBhcyB0aGUgZmlyc3QgaW5wdXQgYSBkYXRhIGZyYW1lLCBhbmQgdGhlbiB3ZSBjYW4gbGlzdCBvbmUgb3IgbW9yZSBjb2x1bW5zLCBuYW1lcyB1bnF1b3RlZCwgdGhhdCB3ZSB3YW50IHRvIHNlbGVjdC4gIFRoZSBjb2x1bW5zIHdpbGwgYmUgb3JkZXJlZCBpbiB0aGUgb3JkZXIgd2Ugc3BlY2lmeSB0aGVtLgoKYGBge3J9CnNlbGVjdChwb2xpY2UsIG91dGNvbWUsIGRhdGUpCmBgYAoKCiMjIFJhbmdlcwoKVGhlcmUgYXJlIGEgbnVtYmVyIG9mIHNlbGVjdCBoZWxwZXIgZnVuY3Rpb25zIGFuZCBzcGVjaWFsIHN5bnRheCBvcHRpb25zIHRoYXQgYWxsb3cgdXMgdG8gY2hvb3NlIG11bHRpcGxlIGNvbHVtbnMuCgpGaXJzdCwgd2UgY2FuIHVzZSA6IGZvciByYW5nZSwgYnV0IHdpdGggbmFtZXMgaW4gYWRkaXRpb24gdG8gbnVtYmVyczoKCmBgYHtyfQpzZWxlY3QocG9saWNlLCByYXdfRHJpdmVyUmFjZTpyYXdfUmVzdWx0T2ZTdG9wKQpgYGAKCldlIGNhbiBzZWxlY3QgdGhlIHJpZ2h0bW9zdCBjb2x1bW5zIHdpdGggYGxhc3RfY29sKClgOgoKYGBge3J9CnNlbGVjdChwb2xpY2UsIGxhc3RfY29sKCkpCmBgYAoKTGFzdCA0IGNvbHVtbnMgKHRoZSBpbnB1dCB0byB0aGUgZnVuY3Rpb24gaXMgdGhlIG9mZnNldCAjIG9mIGNvbHVtbnMgZnJvbSB0aGUgcmlnaHQgZWRnZSk6CgpgYGB7cn0Kc2VsZWN0KHBvbGljZSwgbGFzdF9jb2woMDozKSkKYGBgCgoKIyMgRXhjbHVkaW5nIGNvbHVtbnMKCldlIGNhbiBhbHNvIHNheSB3aGljaCBjb2x1bW5zIHdlIGRvbid0IHdhbnQgYnkgcHV0dGluZyBhIGAtYCBpbiBmcm9udCBvZiB0aGUgbmFtZToKCmBgYHtyfQpzZWxlY3QocG9saWNlLCAtcmF3X3Jvd19udW1iZXIsIC1zdWJqZWN0X2FnZSkKYGBgCgpXaGVuIHVzaW5nIG5lZ2F0ZWQgYC1gIGNvbHVtbiBuYW1lcywgaWYgd2Ugc3RhcnQgd2l0aCBuZWdhdGlvbnMsIGl0IHdpbGwgaW5jbHVkZSBhbGwgb3RoZXIgY29sdW1ucywgZXZlbiBpZiB3ZSB0cnkgdG8gc3BlY2lmeSBzb21lOgoKYGBge3J9CnNlbGVjdChwb2xpY2UsIC1yYXdfcm93X251bWJlciwgLXN1YmplY3RfYWdlLCB0aW1lOm91dGNvbWUpCmBgYAoKVG8gYm90aCBzcGVjaWZ5IHRoZSBjb2x1bW5zIHdhbnRlZCBhbmQgZXhjbHVkZSBzb21lIHRoYXQgd291bGQgb3RoZXJ3aXNlIGJlIHNlbGVjdGVkLCB0aGUgZXhjbHVzaW9ucyBuZWVkIHRvIGNvbWUgYXQgdGhlIGVuZDoKCmBgYHtyfQpzZWxlY3QocG9saWNlLCBsb2NhdGlvbjp0eXBlLCAtZGVwYXJ0bWVudF9pZCkKYGBgCgoKIyMgUmVvcmRlcmluZyBhbmQgcmVuYW1pbmcKCldlJ3ZlIGFscmVhZHkgc2VlbiB0aGF0IGNvbHVtbnMgd2lsbCBhcHBlYXIgaW4gdGhlIHJlc3VsdCBpbiB0aGUgb3JkZXIgdGhhdCB3ZSBsaXN0IHRoZSBuYW1lcy4gIAoKVGhlIGBldmVyeXRoaW5nKClgIGhlbHBlciBmdW5jdGlvbiBjYW4gYmUgdXNlZnVsIGlmIHlvdSB3YW50IHRvIHB1bGwgYSBmZXcgY29sdW1ucyBvdmVyIHRvIHRoZSBsZWZ0IHNvIHRoYXQgdGhleSBhcmUgdGhlIG9uZXMgdGhhdCBzaG93IGZpcnN0IHdoZW4geW91IGxvb2sgYXQgdGhlIGRhdGE6CgpgYGB7cn0Kc2VsZWN0KHBvbGljZSwgb3V0Y29tZSwgZXZlcnl0aGluZygpKQpgYGAKCkVhY2ggY29sdW1uIG9ubHkgZ2V0cyBpbmNsdWRlZCBvbmNlLCBpbiB0aGUgcG9zaXRpb24gdGhhdCBpdCBmaXJzdCBhcHBlYXJzLiAgU28gIm91dGNvbWUiIGJlY29tZXMgdGhlIGxlZnRtb3N0IGNvbHVtbiBhYm92ZSBhbmQgbm8gbG9uZ2VyIGFwcGVhcnMgaW4gaXQncyBvcmlnaW5hbCBzcG90LiAgCgpXZSBjYW4gYWxzbyByZW5hbWUgY29sdW1ucyB3aGlsZSB1c2luZyBgc2VsZWN0KClgLiAgVGhlIHN5bnRheCBpcyBgbmV3X25hbWUgPSBvbGRfbmFtZWAuCgpgYGB7cn0Kc2VsZWN0KHBvbGljZSwgcmF3X2lkPXJhd19yb3dfbnVtYmVyLCBkYXRlLCB0aW1lKQpgYGAKCgpvciB3ZSBjYW4gdXNlIGByZW5hbWUoKWAgdG8gb25seSByZW5hbWUsIHdpdGhvdXQgYWZmZWN0aW5nIHdoaWNoIGNvbHVtbnMgYXJlIGluY2x1ZGVkIG9yIHRoZWlyIG9yZGVyIChhbGwgb2YgdGhlIGNvbHVtbnMgYXJlIGtlcHQgaW4gdGhlIHNhbWUgb3JkZXIpOgoKYGBge3J9CnJlbmFtZShwb2xpY2UsIHJhd19pZD1yYXdfcm93X251bWJlcikKYGBgCgpSZW1lbWJlciwgdGhpcyBkb2Vzbid0IGNoYW5nZSBwb2xpY2UgYmVjYXVzZSB3ZSBkaWRuJ3Qgc2F2ZSB0aGUgcmVzdWx0LiAgU28gZmFyLCB3ZSd2ZSBqdXN0IGJlZW4gcHJpbnRpbmcgdGhlIGNvcHkgb2YgdGhlIGRhdGEgZnJhbWUgdGhhdCBpcyByZXR1cm5lZCBieSB0aGUgZnVuY3Rpb24uICBJZiB3ZSB3YW50IHRvIGNoYW5nZSBvdXIgZGF0YSBmcmFtZSwgd2UnZCBuZWVkIHRvIHNhdmUgdGhlIHJlc3VsdCBiYWNrIHRvIHRoZSBgcG9saWNlYCBvYmplY3QuCgpgYGB7cn0KcG9saWNlIDwtIHJlbmFtZShwb2xpY2UsIHJhd19pZD1yYXdfcm93X251bWJlcikKYGBgCgoKCiMjIyBFWEVSQ0lTRQoKUmVtZW1iZXI6IHJ1biB0aGUgY2VsbHMgYWJvdmUgdG8gbG9hZCB0aWR5dmVyc2UgYW5kIGltcG9ydCB0aGUgZGF0YS4KClVzaW5nIGBzZWxlY3RgIGFuZC9vciBgcmVuYW1lYCBhcyBuZWVkZWQ6CgotIFJlbmFtZSBzdWJqZWN0X2FnZSB0byBhZ2UsIHN1YmplY3RfcmFjZSB0byByYWNlLCBhbmQgc3ViamVjdF9zZXggdG8gc2V4LCBidXQga2VlcCB0aGUgY29sdW1ucyBpbiB0aGVpciBvcmlnaW5hbCBvcmRlcgotIEV4Y2x1ZGUgdGhlIGRlcGFydG1lbnRfaWQgYW5kIGRlcGFydG1lbnRfbmFtZSBjb2x1bW5zCgpIaW50OiByZW1lbWJlciB0aGF0IHlvdSBjYW4gY2hhaW4gZHBseXIgY29tbWFuZHMgdG9nZXRoZXIgd2l0aCBgJT4lYC4KCgpgYGB7cn0KCmBgYAoKCgojIyBNYXRjaGluZyBuYW1lcwoKCldlIGNhbiBhbHNvIHNlbGVjdCBieSBtYXRjaGluZyBwYXR0ZXJucyBpbiB0aGUgbmFtZXMgb2YgdGhlIGNvbHVtbnMuICBUaGUgcGF0dGVybnMgdG8gbWF0Y2ggYXJlIGluIHF1b3RlcyBiZWNhdXNlIHRoZXkgYXJlbid0IGNvbHVtbiBuYW1lcyAtLSBqdXN0IGNoYXJhY3RlciBkYXRhLgoKYGBge3J9CnNlbGVjdChwb2xpY2UsIHN0YXJ0c193aXRoKCJjb250cmFiYW5kIikpCmBgYAoKYGBge3J9CnNlbGVjdChwb2xpY2UsIGVuZHNfd2l0aCgiaXNzdWVkIikpCmBgYAoKYGBge3J9CnNlbGVjdChwb2xpY2UsIGNvbnRhaW5zKCJ2ZWhpY2xlIikpCmBgYAoKV2UgY2FuIGFsc28gcHV0IGEgYC1gIGluIGZyb250IG9mIHRoZXNlIGhlbHBlciBmdW5jdGlvbnMgdG8gZXhjbHVkZSBjb2x1bW5zOgoKYGBge3J9CnNlbGVjdChwb2xpY2UsIC1jb250YWlucygic3ViamVjdCIpKQpgYGAKCgpBbmQgdGhlcmUgYXJlIGV2ZW4gbW9yZSBbc2VsZWN0IGhlbHBlciBmdW5jdGlvbnNdKGh0dHBzOi8vdGlkeXNlbGVjdC5yLWxpYi5vcmcvcmVmZXJlbmNlL3NlbGVjdF9oZWxwZXJzLmh0bWwpLiAgCgojIyMgRVhFUkNJU0UKClVzZSBgc2VsZWN0KClgIHRvIGdldCBhIGNvcHkgb2YgYHBvbGljZWAgd2l0aG91dCB0aGUgY29sdW1ucyB0aGF0IHN0YXJ0IHdpdGggInJhdyIuCgpgYGB7cn0KCmBgYAoKSGludDogSWYgeW91IG1lc3MgdXAgeW91ciBgcG9saWNlYCBkYXRhc2V0LCByZS1ydW4gdGhlIGNlbGwgbmVhciB0aGUgdG9wIG9mIHRoZSBmaWxlIHVuZGVyIHRoZSBEYXRhIGhlYWRlciBhbmQgcmVhZCB0aGUgZGF0YSBpbiBhZ2FpbiBmcmVzaC4KCgojIyBTZWxlY3Rpbmcgd2l0aCBWZWN0b3JzIG9yIEZ1bmN0aW9ucwoKV2hhdCBpZiB3ZSBoYXZlIHRoZSBuYW1lcyBvZiB0aGUgY29sdW1ucyB3ZSB3YW50IHRvIHNlbGVjdCBpbiBhIHZlY3RvciBhbHJlYWR5PyAgRm9yIGV4YW1wbGU6CgpgYGB7cn0KYW5hbHlzaXNfdmFycyA8LSBjKCJzZWFyY2hfYmFzaXMiLCAicmVhc29uX2Zvcl9zdG9wIikKYGBgCgpQZXJoYXBzIHdlIGJ1aWx0IHRoaXMgdmVjdG9yIHByb2dyYW1hdGljYWxseSAod2Ugd3JvdGUgY29kZSB0byBkZXRlcm1pbmUgdGhlIHZhbHVlcywgaW5zdGVhZCBvZiB0eXBpbmcgdGhlbSBvdXJzZWx2ZXMpLCBzbyB3ZSBjYW4ndCBqdXN0IHJld3JpdGUgaXQgdG86IAoKYGBge3J9CnNlbGVjdChwb2xpY2UsIHNlYXJjaF9iYXNpcywgcmVhc29uX2Zvcl9zdG9wKQpgYGAKCklmIHdlIGp1c3QgZ2l2ZSB0aGUgdmVjdG9yIHRvIGBzZWxlY3RgLCBpdCBsb29rcyBsaWtlIHdlIGV4cGVjdCAiYW5hbHlzaXNfdmFycyIgdG8gYmUgYSBjb2x1bW4gbmFtZSBpbiBwb2xpY2UuICBXZSBnZXQgYSB3YXJuaW5nOgoKYGBge3J9CnNlbGVjdChwb2xpY2UsIGFuYWx5c2lzX3ZhcnMpCmBgYAoKVGhpcyB3YXJuaW5nIHRlbGxzIHVzIHdoYXQgd2Ugc2hvdWxkIGRvIGluc3RlYWQsIHdoaWNoIGlzIHVzZSBgYWxsX29mYDoKCmBgYHtyfQpzZWxlY3QocG9saWNlLCBhbGxfb2YoYW5hbHlzaXNfdmFycykpCmBgYAoKVGhpcyBtYWtlcyBpdCBjbGVhcmVyIHRoYXQgImFuYWx5c2lzX3ZhcnMiIGlzbid0IHRoZSBuYW1lIG9mIGEgY29sdW1uIGluIHBvbGljZS4KCldoYXQgaWYgd2Ugd2FudCB0byBzZWxlY3QgY29sdW1ucyBvZiBhIGNlcnRhaW4gdHlwZSAtLSBmb3IgZXhhbXBsZSwgb25seSB0aGUgbnVtZXJpYyBjb2x1bW5zPyAgCgpgYGB7cn0Kc2VsZWN0KHBvbGljZSwgd2hlcmUoaXMubnVtZXJpYykpCmBgYAoKYGlzLm51bWVyaWNgIGlzIHRoZSBuYW1lIG9mIGEgZnVuY3Rpb24uICBXZSBqdXN0IHVzZSB0aGUgbmFtZSB3aXRob3V0IGAoKWAuICBUaGlzIGZ1bmN0aW9uIGlzIGFwcGxpZWQgdG8gZWFjaCBjb2x1bW4sIGFuZCBpZiBpdCByZXR1cm5zIFRSVUUsIHRoZW4gdGhlIGNvbHVtbiBpcyBzZWxlY3RlZC4gIExpa2UgYWJvdmUgd2l0aCB1c2luZyBhIHZlY3Rvciwgd2Ugd3JhcCB0aGUgZnVuY3Rpb24gd2Ugd2FudCB0byB1c2UgaW4gYHdoZXJlYCB0byBtYWtlIGl0IGNsZWFyIHRoYXQgd2UncmUgdXNpbmcgYSBmdW5jdGlvbiwgbm90IGxvb2tpbmcgZm9yIGEgY29sdW1uIG5hbWVkICJpcy5udW1lcmljIikuICAKCmB3aGVyZWAgY2FuIGJlIHVzZWQgd2l0aCBhbnkgZnVuY3Rpb24gdGhhdCByZXR1cm5zIGEgKnNpbmdsZSogVFJVRSBvciBGQUxTRSB2YWx1ZSBmb3IgZWFjaCBjb2x1bW4uICAKCgojIEZpbHRlcjogQ2hvb3NlIFJvd3MKClRoZSBgZmlsdGVyKClgIGZ1bmN0aW9uIGxldHMgdXMgY2hvb3NlIHdoaWNoIHJvd3Mgb2YgZGF0YSB0byBrZWVwIGJ5IHdyaXRpbmcgZXhwcmVzc2lvbnMgdGhhdCByZXR1cm4gVFJVRSBvciBGQUxTRSBmb3IgZXZlcnkgcm93IGluIHRoZSBkYXRhIGZyYW1lLiAgUmVjYWxsIGZyb20gbGFzdCBzZXNzaW9uOgoKYGBge3J9CmZpbHRlcihwb2xpY2UsIGRhdGUgPT0gIjIwMTctMDEtMDIiKQpgYGAKCldlIGNhbiBkbyBjb21wbGV4IGNvbmRpdGlvbnMgYXMgd2UgY291bGQgZG8gd2l0aCBgW11gCgpgYGB7cn0KZmlsdGVyKHBvbGljZSwgc3ViamVjdF9yYWNlID09ICJoaXNwYW5pYyIgJiBzdWJqZWN0X3NleCA9PSAiZmVtYWxlIikKYGBgCgpJZiB3ZSBpbmNsdWRlIG11bHRpcGxlIGNvbW1hLXNlcGFyYXRlZCBjb25kaXRpb25zLCB0aGV5IGFyZSBqb2luZWQgd2l0aCBgJmAgYW5kLiAgU28gdGhpcyBmb2xsb3dpbmcgaXMgZXF1aXZhbGVudCB0byB0aGUgYWJvdmUuCgpgYGB7cn0KZmlsdGVyKHBvbGljZSwgc3ViamVjdF9yYWNlID09ICJoaXNwYW5pYyIsIHN1YmplY3Rfc2V4ID09ICJmZW1hbGUiKQpgYGAKCgojIyMgRVhFUkNJU0UKCjEuIEZpbHRlciBgcG9saWNlYCB0byBjaG9vc2UgdGhlIHJvd3Mgd2hlcmUgbG9jYXRpb24gaXMgNjAyMDEgb3IgNjAyMDIKMi4gRmlsdGVyIGBwb2xpY2VgIHRvIGNob29zZSB0aGUgcm93cyB3aGVyZSBsb2NhdGlvbiBpcyA2MDIwMSBvciA2MDIwMiBhbmQgc3ViamVjdF9zZXggaXMgIm1hbGUiCgpIaW50czoKCiogVGhlICJvciIgb3BlcmF0b3IgaXMgYHxgOyB0aGUgImFuZCIgb3BlcmF0b3IgaXMgYCZgCgpgYGB7cn0KCmBgYAoKIyMgSW5jbHVkaW5nIFZhcmlhYmxlIFRyYW5zZm9ybWF0aW9ucwoKV2hlbiBmaWx0ZXJpbmcsIHdlIGNhbiBpbmNsdWRlIHRyYW5zZm9ybWF0aW9ucyBvZiB2YXJpYWJsZXMgaW4gb3VyIGV4cHJlc3Npb25zLiAgVG8gc2VlIHRoaXMsIHdlJ2xsIHVzZSB0aGUgYnVpbHQtaW4gYG10Y2Fyc2AgZGF0YXNldCwgd2hpY2gsIHVubGlrZSB0aGUgYHBvbGljZWAgZGF0YSwgaGFzIHNvbWUgbnVtZXJpYyB2YXJpYWJsZXMuCgpIZXJlJ3Mgd2hhdCBgbXRjYXJzYCBsb29rcyBsaWtlOgoKYGBge3J9Cm10Y2FycwpgYGAKCk5vdywgbGV0J3MgZmlsdGVyIHRvIHNlZSB3aGljaCBjYXJzIGhhdmUgYWJvdmUgYXZlcmFnZSBtcGc6CgpgYGB7cn0KZmlsdGVyKG10Y2FycywgbXBnID4gbWVhbihtcGcpKQpgYGAKCk9yIHdoaWNoIGNhciBoYXMgdGhlIG1vc3QgaG9yc2Vwb3dlciAoaHApOgoKYGBge3J9CmZpbHRlcihtdGNhcnMsIGhwID09IG1heChocCkpCmBgYAoKCiMjIyBFWEVSQ0lTRQoKVXNpbmcgYG10Y2Fyc2AsIGZpbmQgdGhlIGNhciB3aXRoIHRoZSBtaW5pbXVtIChgbWluYCkgZGlzcGxhY2VtZW50IChkaXNwKSB2YWx1ZToKCmBgYHtyfQoKYGBgCgoKIyBCb251czogc2xpY2UgdmFyaWFudHMKCkxhc3Qgc2Vzc2lvbiwgd2Ugc2F3IGBzbGljZSgpYCBicmllZmx5IGFzIGEgd2F5IHRvIGNob29zZSB3aGljaCByb3dzIHdlIHdhbnQgYnkgdGhlaXIgaW50ZWdlciBpbmRleCB2YWx1ZS4gIEJ1dCwgdGhlcmUgYXJlIHNvbWUgdXNlZnVsIHZhcmlhbnRzIG9uIHRoZSBgc2xpY2VgIGZ1bmN0aW9uIHRoYXQgaGVscCB1cyBzZWxlY3Qgcm93cyB0aGF0IGhhdmUgdGhlIG1heGltdW0gb3IgbWluaW11bSB2YWx1ZSBvZiBhIHBhcnRpY3VsYXIgdmFyaWFibGU6CgpgYGB7cn0Kc2xpY2VfbWF4KG10Y2FycywgaHApCmBgYAoKQnkgZGVmYXVsdCBpdCBqdXN0IGdpdmVzIHVzIG9uZSByb3csIGJ1dCB3ZSBjYW4gYXNrIGZvciBtb3JlIHRoYW4gb25lIGJ5IHNldHRpbmcgdGhlIGBuYCBhcmd1bWVudDoKCmBgYHtyfQpzbGljZV9tYXgobXRjYXJzLCBocCwgbj0zKQpgYGAKCldlIGdvdCA0IHJvd3MgYWJvdmUgYmVjYXVzZSB0aGVyZSB3YXMgYSB0aWUgYXQgcG9zaXRpb24gMy4gIFRoZXJlJ3MgYW4gb3B0aW9uIGB3aXRoX3RpZXNgIHRoYXQgY2FuIGNoYW5nZSBob3cgdGllcyBhcmUgaGFuZGxlZC4gIAoKVGhlcmUncyBhbHNvIGEgbWluaW11bSB2ZXJzaW9uOgoKYGBge3J9CnNsaWNlX21pbihtdGNhcnMsIGRpc3ApCmBgYAoKCiMgTXV0YXRlOiBDaGFuZ2Ugb3IgQ3JlYXRlIENvbHVtbnMKCmBtdXRhdGUoKWAgaXMgdXNlZCB0byBib3RoIGNoYW5nZSB0aGUgdmFsdWVzIG9mIGFuIGV4aXN0aW5nIGNvbHVtbiBhbmQgbWFrZSBhIG5ldyBjb2x1bW4uICAKCldlIG5hbWUgdGhlIGNvbHVtbiB3ZSdyZSBtdXRhdGluZyBhbmQgc2V0IHRoZSB2YWx1ZS4gIElmIHRoZSBuYW1lIGFscmVhZHkgZXhpc3RzLCBpdCB3aWxsIHVwZGF0ZSB0aGUgY29sdW1uLiAgSWYgdGhlIG5hbWUgZG9lc24ndCBleGlzdCwgaXQgd2lsbCBjcmVhdGUgYSBuZXcgdmFyaWFibGUgKGNvbHVtbiBpcyBhcHBlbmRlZCBhdCB0aGUgZW5kIG9mIGV4aXN0aW5nIGNvbHVtbnMpLiAgCgpgYGB7cn0KbXV0YXRlKHBvbGljZSwgdmVoaWNsZV9hZ2UgPSAyMDE3IC0gdmVoaWNsZV95ZWFyKSAlPiUKICBzZWxlY3Qoc3RhcnRzX3dpdGgoInZlaGljbGUiKSkgICMganVzdCB0byBwaWNrIGEgZmV3IGNvbHVtbnMgdG8gbG9vayBhdApgYGAKCldlIGNhbiBwdXQgbXVsdGlwbGUgbXV0YXRpb25zIGluIHRoZSBzYW1lIGNhbGwgdG8gbXV0YXRlLCB3aXRoIHRoZSBleHByZXNzaW9ucyBzZXBhcmF0ZWQgYnkgY29tbWFzOgoKYGBge3J9Cm11dGF0ZShwb2xpY2UsIAogICAgICAgdmVoaWNsZV9hZ2UgPSAyMDE3IC0gdmVoaWNsZV95ZWFyLAogICAgICAgb2xkX2NhciA9IHZlaGljbGVfeWVhciA8IDIwMDApCmBgYAoKCldpdGhpbiBhIGNhbGwgdG8gbXV0YXRlLCB3ZSBjYW4gcmVmZXIgdG8gdmFyaWFibGVzIHdlIG1hZGUgb3IgY2hhbmdlZCBlYXJsaWVyIGluIHRoZSBzYW1lIGNhbGwgYXMgd2VsbC4gIEhlcmUsIHdlIGNyZWF0ZSB2ZWhpY2xlX2FnZSwgYW5kIHRoZW4gdXNlIGl0IHRvIGNyZWF0ZSB2ZWhpY2xlX2FnZV9ub3JtOgoKYGBge3J9CnBvbGljZSAlPiUgCiAgbXV0YXRlKHZlaGljbGVfYWdlID0gMjAxNyAtIHZlaGljbGVfeWVhciwgCiAgICAgICAgIHZlaGljbGVfYWdlX25vcm0gPSBpZmVsc2UodmVoaWNsZV9hZ2UgPCAwLCAgIyBpZmVsc2UgdGVzdCBjb25kaXRpb24KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAwLCAgIyB2YWx1ZSBpZiB0cnVlCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdmVoaWNsZV9hZ2UpICAjIHZhbHVlIGlmIGZhbHNlCiAgICAgICAgICkgJT4lICAKICAjIGJlbG93IGlzIGp1c3QgbWFraW5nIGl0IGVhc2llciBmb3IgdXMgdG8gc2VlIHdoYXQgd2UgY2hhbmdlZAogIHNlbGVjdChzdGFydHNfd2l0aCgidmVoaWNsZSIpKSAlPiUKICBmaWx0ZXIodmVoaWNsZV9hZ2UgPCAwKQpgYGAKClNpZGUgbm90ZTogdGhlcmUgaXMgYSB0aWR5dmVyc2UgdmVyc2lvbiBvZiBgaWZlbHNlKClgIGNhbGxlZCBgaWZfZWxzZSgpYCB0aGF0IHdvcmtzIGdlbmVyYWxseSB0aGUgc2FtZSBleGNlcHQgaXQgaXMgc3RyaWN0ZXIgYWJvdXQgY2hlY2tpbmcgZGF0YSB0eXBlcy4KCmBtdXRhdGUoKWAgY2FuIGFsc28gY2hhbmdlIGFuIGV4aXN0aW5nIGNvbHVtbi4gIFRoZSBsb2NhdGlvbiBjb2x1bW4gaW4gdGhlIGRhdGEgY29udGFpbnMgemlwIGNvZGVzLCB0aGF0IHdlcmUgcmVhZCBpbiBhcyBudW1lcmljIHZhbHVlcy4gIFRoaXMgbWVhbnMgdGhlIGxlYWRpbmcgemVybyBvbiBzb21lIHppcCBjb2RlcyBoYXMgYmVlbiBsb3N0LiAgQ29udmVydCBsb2NhdGlvbiB0byBjaGFyYWN0ZXIgZGF0YSwgYW5kIGFkZCBiYWNrIGluIHRoZSBsZWFkaW5nIDAgaWYgaXQgc2hvdWxkIGJlIHRoZXJlLgoKSGVyZSBJJ2xsIGNoYW5nZSB0aGUgbG9jYXRpb24gY29sdW1uIHR3aWNlIGluIHRoZSBzYW1lIGNhbGwgd2l0aCB0d28gZGlmZmVyZW50IHRyYW5zZm9ybWF0aW9uczoKCmBgYHtyfQpwb2xpY2UgJT4lCiAgbXV0YXRlKGxvY2F0aW9uID0gYXMuY2hhcmFjdGVyKGxvY2F0aW9uKSwgICMgZmlyc3QgY29udmVydCB0byBjaGFyYWN0ZXIsIHRoZW4gcmVjb2RlIGJlbG93CiAgICAgICAgIGxvY2F0aW9uID0gaWZlbHNlKG5jaGFyKGxvY2F0aW9uKSA9PSA0LCAgIyBpZmVsc2UgdGVzdCAodmVjdG9yIG9mIFRSVUUgYW5kIEZBTFNFKQogICAgICAgICAgICAgICAgICAgICAgICAgICBwYXN0ZTAoIjAiLCBsb2NhdGlvbiksICMgdmFsdWUgaWYgVFJVRQogICAgICAgICAgICAgICAgICAgICAgICAgICBsb2NhdGlvbikpICU+JSAgIyB2YWx1ZSBpZiBGQUxTRQogIHNlbGVjdChsb2NhdGlvbikgJT4lICAjIHNlbGVjdGluZyBqdXN0IHRoZSBjb2x1bW4gd2UgbXV0YXRlZCB0byBsb29rIGF0CiAgZmlsdGVyKHN0YXJ0c1dpdGgobG9jYXRpb24sICIwIikpICAjIHNlbGVjdGluZyBhIGZldyByb3dzIHRvIGxvb2sgYXQgdGhlIGNoYW5nZQpgYGAKClJlbWVtYmVyIHRoYXQgd2hlbiB1c2luZyBgbXV0YXRlKClgLCB5b3UncmUgb3BlcmF0aW5nIG9uIHRoZSBlbnRpcmUgY29sdW1uIGF0IG9uY2UsIHNvIHlvdSBjYW4ndCBzZWxlY3QganVzdCBhIHN1YnNldCBvZiB0aGUgdmVjdG9yIGFzIHlvdSB3b3VsZCB3aXRoIGBbXWAuICBUaGlzIG1lYW5zIG1vcmUgZnJlcXVlbnRseSB1c2luZyBmdW5jdGlvbnMgbGlrZSBgaWZlbHNlKClgIG9yIGhlbHBlciBmdW5jdGlvbnMgc3VjaCBhcyBgbmFfaWYoKWAsIGByZXBsYWNlX25hKClgLCBvciBgcmVjb2RlKClgLiAgCgpgbmFfaWZgIHJlcGxhY2VzIGFuIGV4aXN0aW5nIHZhbHVlIHdpdGggYE5BYC4gIGByZXBsYWNlX25hYCBkb2VzIHJvdWdobHkgdGhlIG9wcG9zaXRlOiByZXBsYWNlcyBgTkFgIHdpdGggYSBuZXcgdmFsdWUuCgpgYGB7cn0KbXV0YXRlKHBvbGljZSwgdmVoaWNsZV9tYWtlID0gbmFfaWYodmVoaWNsZV9tYWtlLCAiVU5LIikpCmBgYAoKYG5hX2lmKClgIGNhbiBvbmx5IGNhbiBjaGVjayBhbmQgcmVwbGFjZSBvbmUgdmFsdWUgYXQgYSB0aW1lOyBpdCBhbHNvIGNhbid0IGJlIHVzZWQgd2l0aCBhbnkgZXhwcmVzc2lvbnMgKGB4IDw9IDFgKSAtLSBvbmx5IHNpbmdsZSB2YWx1ZXMuCgoKIyMjIEVYRVJDSVNFCgpJZiBiZWF0IGlzICIvIiBvciAiQ0hJQ0FHTyIsIHNldCBpdCB0byBgTkFgIGluc3RlYWQgdXNpbmcgYG11dGF0ZSgpYC4gIAoKSGludDogaXQncyBvayBpZiB5b3UgdGFrZSB0d28gc3RlcHMgdG8gZG8gdGhpcy4gICAgCgpgYGB7cn0KCmBgYAoKCgoKCiMgUmVjYXAKCllvdSBub3cgY2FuIHVzZSBgc2VsZWN0YCBhbmQgYGZpbHRlcmAgdG8gc3Vic2V0IHlvdXIgZGF0YSBpbiBhIHdpZGUgdmFyaWV0eSBvZiB3YXlzLCBhbmQgYG11dGF0ZWAgdG8gdXBkYXRlIHZhcmlhYmxlcyBvciBjcmVhdGUgbmV3IG9uZXMuICAKCk5leHQgc2Vzc2lvbjogdGhlIHRocmVlIG90aGVyIGNvbW1vbiBkcGx5ciAidmVyYiIgZnVuY3Rpb25zIGZvciB3b3JraW5nIHdpdGggZGF0YSBmcmFtZXM6IGBncm91cF9ieWAsIGBzdW1tYXJpemVgLCBhbmQgYGFycmFuZ2VgLiAgCgo=