```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE, cache = TRUE, autodep = TRUE,
  fig.align = "center")
```


::: {.callout-note}
You can download the [Quarto Markdown file](https://gedeck.github.io/DS-6030/homework/templates/module-10.qmd) and use it to answer the following questions.

If not otherwise stated, use _Tidyverse_, _Tidymodels_, and _Tidyclust_ for the assignments.
:::

::: {.callout-warning}
## LLM use
You may use LLMs (ChatGPT, Claude, Copilot, etc.) on this assignment. If you do, you must:

1. **Disclose** which LLM you used and roughly what for (concept clarification, code generation, prose review).
2. **Include the prompts** for any output you used substantially. Paste them into a "LLM use" appendix at the end of your submission.
3. **Verify the output** --- LLMs frequently make small but plausible-sounding errors (wrong variable names, made-up `tidymodels` functions, wrong claims about dataset properties). Check anything you keep against the course material or the actual data.

The grader will spot-check disclosure. Undisclosed LLM use is treated as a citation failure.
:::

## 1. Analyzing the ANES 2022 Pilot Study - PCA {-}
The ANES 2022 Pilot Study is a cross-sectional survey conducted to test new questions under consideration for potential inclusion in the ANES 2024 Time Series Study and to provide data about voting and public opinion after the 2022 midterm elections in the United States. Information about this study is available on [the ANES website](https://electionstudies.org/data-center/2022-pilot-study/).

(A) Setup:

Load the data from [ANES data anes_pilot_2022_csv_20221214.csv](https://gedeck.github.io/DS-6030/datasets/anes_pilot_2022_csv_20221214/anes_pilot_2022_csv_20221214.csv)

The dataset contains information about the respondent's profile (e.g. birthyr, gender, race, educ, marstat, ...) and answers to 235 questions from different categories.

Load and preprocess the data. The data contain answers to 235 questions (see PDF [questionnaire](https://electionstudies.org/wp-content/uploads/2022/11/anes_pilot_2022_questionnaire_20221114.pdf)). The dataset also contains a number of variables that are not questions, but rather contain information about how the survey was conducted (see [user's guide and codebook](https://electionstudies.org/wp-content/uploads/2022/12/anes_pilot_2022_userguidecodebook_20221214.pdf)).

**(1.1)** Identify the feeling thermometer questions. These questions ask respondents to rate their feelings toward a number of groups on a scale from 0 to 100. The questions are listed in variables starting with `ft...`. Identify the names of all feeling thermometer questions ignoring the `ftblack` and `ftwhite` questions as these were only asked based on race of the respondent and therefore contain a large number of missing values. Make sure that you exclude timing (e.g. `ftjourn_page_timing`) and order variables from your analysis. You should have 16 columns left. (1 point - coding)

**(1.2)** If a respondent did not answer a feeling thermometer question, the value is coded as a negative number. Replace the negative values with `NA` and remove all rows that have NA values for _any of the selected feeling thermometer questions_ (see `drop_na` function). You should have about 1560 data points left. (1 point - coding)

(B) Principal Component Analysis (PCA):

You should now have a data frame that is suitable for a principal component analysis of the feeling thermometer responses.

Perform a principal component analysis of the feeling thermometer responses using `step_pca`.

**(1.3)** Create a scree plot of the eigenvalues. How many components should be considered? Justify your decision. (1 point - coding/discussion)

**(1.4)** Create a biplot using the first two components. You will need to multiply the loadings by a factor to improve the readability of the biplot.  (1 point - coding)

**(1.5)** Interpret the first two components. What do they represent? Check the questionnaire for the questions that were asked.  (1 point - discussion)

(C) Explore dataset:

**(1.6)** The ANES 2022 Pilot Study is a rich data set. We can map the respondent's profile and responses to other questions onto the principal component scatterplot. We start with the respondent's profile. (2 points - coding/discussion)

Select the following profile data:

- gender
- educ (education level)
- marstat (marital status)

Add steps to convert the columns into factors in your data processing pipeline.
See the questionnaire for the meaning of the different factor levels.

Combine the data set with the transformed PCA values.

Create scatterplots of the first two components, add a `geom_density2d` layer, and use `facet_wrap` to create a separate plot for each factor level. Inspect the level counts before faceting; consider collapsing rare levels if any panel has too few respondents to interpret (this may or may not be necessary).

Interpret the results. Can you see patterns?

**(1.7)** As an extension of **(1.6)**, we now focus on the answers to the actual questions.  Select one of the question categories (your choice), formulate a hypothesis and see if you find a correlation with the PCA analysis.

The categories are:

- 2022 Turnout and choice (6-20)
- Retrospective turnout and choice 2020 (21-24)
- Prospective turnout (25)
- Participation (26-33)
- Global emotion battery (34-40)
- Presidential approval (41-43)
- Party identification (44-50)
- Ideology (51)
- Economic performance (52-54)
- Inflation (55-64)
- Issue importance (65-79)
- Issue ownership (80-92)
- Climate change (93-94)
- Trust in experts (95-99)
- Political disagreement (100-101)
- Abortion (102-114)
- Abortion emotions (116-122)
- Transgender attitudes (123-126)
- Guns and crime (127-129)
- Immigrant emotions (130-136)
- Democratic attitudes / misinformation (137-146)
- Electoral integrity (147-159)
- Political efficacy (160)
- Feeling thermometers (161-179)
- Racism (180)
- Feminist attitudes (181-185)
- Racial resentment (186-190)
- Political tolerance (191-193)
- Racial stereotypes (194-206)
- Identities (207-208)
- Identity importance (210-220)
- Role of schools (221-226)
- Great replacement (227)
- Racial privilege (228-235)

Use a similar approach to **(1.6)** for the analysis. (2 points - coding/discussion)



## 2. Analyzing the ANES 2022 Pilot Study - Clustering {-}
We continue with the analysis of the ANES 2022 Pilot study and cluster the respondents based on their answers to the feeling thermometer questions.

(A) Hierarchical clustering:

**(2.1)** Create a hierarchical clustering using the feeling thermometer data with the `tidyclust` package. Explore four of the linkage methods supported by hier_clust (e.g. single or complete; [see `hier_clust` documentation for details](https://tidyclust.tidymodels.org/reference/hier_clust.html)). How many clusters should be considered? (2 points - coding)

(B) k-means clustering:

**(2.2)** Use k-means clustering to cluster the respondents based on their answers to the feeling thermometer questions. Use the `tidyclust` package.  (2 points - coding)

- Create a k-means clustering with 5 clusters.

- Combine the dataset, the results from the PCA, and the k-means clustering in a tibble. [_Hint_: add to result from **(1.6)**]

- Create a scatterplot of the first two principal components and color the points by the cluster assignment. Describe your observations.

- Applying the `tidy` command to the fitted k-means model extracts the cluster centroids. Visualize the cluster centers in a parallel coordinate plot and interpret the different clusters. It can be helpful to order the variables for the visualization (use `scale_x_discrete(limits=c("fttrans", "ftfem", ...))` where the order is defined by the `limits` argument).

(C) Explore dataset:

Characterize the different clusters.

**(2.3)** Use the profile data from **(1.6)** to characterize the different clusters. You can for example visualize the distributions of the different factor levels in a stacked 100%-bar plot (`geom_bar(position="fill")`). How does the distribution of the different factor levels differ between the clusters? Are your observations in agreement with the analysis from **(1.6)**? (2 points - coding/discussion)

**(2.4)** Now use the questions from **(1.7)** to characterize the different clusters. How does the distribution of the different factor levels differ between the clusters? Are your observations in agreement with the analysis from **(1.7)**? (2 points - coding/discussion)
