Sociology seminar – Thursdays

Time: 12:00 pm – 1:30 pm 
Date: 16th May 2024
Place: room 3105

ZOOM LINK: https://zoom.us/j/97751150432?pwd=NUpGdDg1OW9uSFNxZWNxUTR0ZjRNUT09

Gaël LE MENS (Universitat Pompeu Fabra) – Scaling Political Texts with ChatGPT

Abstract:

We use GPT-4 to obtain position estimates of political texts in continuous spaces. We develop and validate a new approach by positioning British party manifestos on the economic, social, and immigration policy dimensions and tweets by members of the US Congress on the left-right ideological spectrum. For the party manifestos, the correlation between the positions produced by GPT-4 and experts is 93% or higher, a performance similar to or better than that obtained with crowdsourced position estimates. For individual tweets, the positions obtained with GPT-4 achieve a correlation of 91% with crowdsourced position estimates. For senators of the 117th US Congress, the positions obtained with GPT-4 achieve a correlation of 97% with estimates based on roll call votes and of 96% with those based on campaign funding. Correlations are also substantial within party, indicating that position estimates produced with GPT-4 capture within-party differences between senators. Overall, using GPT-4 for ideological scaling is fast, cost-efficient, and reliable. This approach provides a viable alternative to scaling by both expert raters and crowdsourcing.

Organizers: Annina Cleasson, Paola Tubaro, Patrick Präg (CREST Sociology unit)

Sponsors: CREST