Before we start

Outline

The Politics of Policymaking

Often, as political scientists we are interested in:

• what political actors such as interest groups, government agencies, legislators, and scientific experts believe about or want a policy to be like;

• how these actors organize into political coalitions to influence policymaking;

• what (competing) policy ideas emerge in a discourse and how they are organized and expressed in frames, narratives or similar discursive structures;

• how (elements of) policy ideas or political coalitions change, evolve or vary over time, across venues, institutional contexts, or in response to certain events.

‘Policy process theories or frameworks usually combine some of these elements to explain why, how, and when policy change happens’ (Leifeld 2020, 180).

The Study of Policy Networks

DNA & Theories of the Policymaking Process

Discourse Network Analysis (DNA) is an empirical approach to studying the structure and dynamics of actor-based debates, such as policy debates or political discussions.

• While DNA is compatible with many theories of the policymaking process, there are greater degrees of compatibility or ‘elective affinities’ with certain such theories. However, different theoretical starting points should be reflected in the concrete empirical implementation. For example:

  • The advocacy coalition framework (Jenkins-Smith et al. 2017) puts special emphasis on ‘policy core policy preferences’, i.e., preferred solutions for addressing a policy problem (Leifeld 2017, 303).
  • Argumentative discourse analysis posits that political conflicts are structured by ‘discourse coalitions’ which are defined as ‘a group of actors who share a social construct’ (Hajer 1993, 45).

• These differences will lead to subtle differences in how coding schemes are constructed and what material is collected (e.g. focus on instruments or fundamental beliefs, public statements or technical papers). Theory and empirical approach should thus be aligned (see generally Hall 2003).

How does DNA work and what is it good for?

‘DNA is an attempt at measuring actors’ policy beliefs and discourses systematically using text sources and molding them into a data format that is compatible with policy network analysis’ (Leifeld 2020, 180).

• DNA’s core innovation is to combine category-based content analysis and network analysis, which allows one to analyse discursive dynamics (clustering at the level of concepts) and patterns of coalition formation (clustering at the actor level) at the same time. Other than in traditional content or discourse analysis, one thus records both concepts (e.g. frames or narrative elements) and the actors who advance them.

• This allows one to more systematically explore, describe, and draw inferences about discursive dynamics on both the actor and the content level. This, in turn, can not only illuminate political dynamics in specific policy areas but also shed light on the politics of policymaking more generally, e.g., the relationship between policy networks and discursive networks (Schaub and Metz 2020), endogenous dynamics of coalition formation through reciprocity or repulsion (Leifeld and Brandenberger 2019), the evolution of framing over time (Leifeld and Garic 2026), or the ‘stickiness’ of ideas in discourse (cf. Cox 2004).

What are Networks?

‘At its core, a network consists of nodes (which represent the actors, whether individuals or organizations) and edges (which signify the relationships or connections between these actors)’ (Schoch and Shafie 2025).¹

• Nodes and edges can carry attributes. Some come from outside the network (an actor’s type, the weight of a tie), others are computed from its structure (centrality). Ties can be directed or undirected.
• One-mode networks have one set of nodes; two-mode networks have two (e.g., actors and the concepts they refer to), with ties only between the sets.
• A two-mode network can be turned into one-mode networks: the core move of DNA.

Visualizing the intuition behind DNA

Visualizing the intuition behind DNA II

Normalization

Normalization changes absolute values; major network patterns usually remain similar.

Data & Methods

Example: Transformation of EU Trade Policy?

From a quote to a statement

Christian Democrat MEP Markus Weber, by contrast, said his group would support postponing the high-risk obligations, “as long as the protection of fundamental rights is not watered down”.

Who is speaking, and what do they claim?

• Actor: Markus Weber (Christian Democrats, EP group)
• Claim: The obligations for high-risk AI systems should be postponed → agrees
• Claim: AI systems threaten fundamental rights and need strict oversight → agrees

Our codebook for today

Now imagine we had coded a few more statements like this and developed the following codebook for the debate on postponing the AI Act’s rules for high-risk systems.

Each statement: actor (person, organization) × concept × agreement (yes/no) (+ further variables if the research design needs them, e.g., venue or type of argument)

DNA walkthrough

1. GitHub repository: releases, Java, issue tracker
2. Open DNA: file-based vs. server-based databases; open sample.dna (password: sample)
3. Interface: documents, text, statements (similar to MAXQDA etc.)
4. Click on a statement and explain the pop-up window
5. Create a new statement (select text, Ctrl+1); NPF element and annotation
6. Import documents: Document → Import from CSV… (new in 3.1.2), or Import from directory
7. Edit statement types (Statements → edit statement types)
8. Change attributes (Statements → attributes manager)

Sketching out the Challenge

Today we follow the lower path (news reports). With attributed documents, such as consultation responses or press releases, the extraction step falls away.

Challenges of Extraction

• Sometimes, documents have attributed authorship—the author is the speaker. Here, we already have the first two parts of the actor-statement-concept-stance tuple (e.g., position papers, press releases, consultation responses).

• In other cases, documents contain statements by multiple actors and we first need to extract structured actor-statement pairs (e.g., newspaper articles, parliamentary debates).

• Sometimes, simple rules (regex) suffice for extraction (e.g., transcripts with consistent formatting). But often we need models that can parse natural language to identify who said what.

• This is especially the case for newspaper articles, which generally select statements with a view to balancing different (prominent) viewpoints (Hertzum 2022).

• The task then essentially becomes a quote extraction task (Zhang and Liu 2021), which recent work suggests LLMs should perform quite well on.

Comparing codings: stances

Validation

• The six networks look broadly similar: promising for a first look at a developing approach, but no substitute for formal validation.
• What is the gold standard? Human coders disagree too (see the hand-coding exercise), and our coding is one contestable reading (e.g., Weber’s conditional support for C3, or Novak’s “trustworthy AI”).
• Ensembles combine several models, e.g., by majority vote, or by averaging position estimates (Benoit et al. 2025). They remove the errors of single models, not the ones they share: a majority of our five models matches 13 of the 17 stances and adds only 2, but all five miss the Commission’s implicit rejection of C1.
• With attributed documents (e.g., consultation responses), only the classification needs validating; with news articles, the extraction too.
• Extraction is hard to score: exact string matching is too strict. A passage that is almost verbatim, or a little longer or shorter, is a minor error rather than a miss, which overlap-based (fuzzy) matching can grade.
• How good does the coding need to be for the network to be valid?

Human in the loop: back into DNA