This class is part of the
New Geographies, New Pedagogies project at the Institute of International Studies; UC Berkeley. Funded by the Ford Foundation.
This class is part of the
New Geographies, New Pedagogies project at the Institute of International Studies; UC Berkeley. Funded by the Ford Foundation.
Syllabus: Political Science 239, Section 002
Prof. Marcus Kurtz
Department of Political Science
Tuesday, 10-12
749 Barrows Hall
This course is structured around the construction and implementation of multi-level research designs. It will emphasize the particular methodological issues that arise in the pursuit of such multi-level research strategies. The goal will be to combine the construction of such a design with the collection of preliminary cross-national data that will be necessary for its implementation.
There are two written requirements for the course. First, students will construct a multi-level research design that they will distribute to all members of the course in March. These designs will become the basis for discussion and critique in March. Then, at the end of the course, students will present the preliminary large-N findings for the class for similar discussion and critique. By the end of the course students will be expected to submit a polished research design and a preliminary analysis and critique of their large-N results.
As can be seen from the outline below, this class is 'modular' in its format. Depending on enrollment, students may be interested in participating in only one portion of the course -- e.g., getting help with research design issues, or in the development of specific data sets. Thus students could register for the entire course, audit, or participate in those modules of particular interest.
Introduction:The Benefits of Multi-Level Design
This class will focus on the debates between proponents of small-N and large-N research strategies with an eye to understanding the synergies and complementarities of combined approaches. The idea is to move beyond the rejectionism that has sometimes characterized both sides.
Discussion includes:
Beginning sources for cross-national data sets
The Perils of Pre-Collected data
What data are available? What utility do they have?
The focus here is first in getting a sense for what types of data are readily available at the cross-national level. Then we will consider the potential pitfalls involved in using data collected by others.
Discussion includes:
Specifying Hypotheses Amenable to Multi-Level Assessment
Here there is a lot of weight put on what might be called "theory specification." A frequent line of conflict between small-N and large-N research programs is that theory is often cast in different causal language. The most simple versions of this include the tendency for large-N researchers to use linear and additive models of causality, while small-N scholars pursue complex (and multiple) conjunctural causation. For multi-level analysis to be effective, theory must be similarly specified at both levels of analysis. But doing this is far more difficult that might initially appear, and can place very strong demands on the data.
Discussion includes:
Practical Data Sources
Given different thematic interests, where are the big repositories of information? If you have to put together the dataset yourself, where might you go? How do you make national data sources comparable in a cross-national frame?
Measuring Across Levels
Once you have small-N and large-N data on the relevant independent and dependant variables, efforts must be made to ensure that one is in fact measuring the same thing at both levels. This is much more complicated than it might seem given the tendency for small-N scholars to use complex conceptual structures, and for large-N scholars to be restricted in the available indicators of such concepts. This problem is manifest in cross-national measurement of "democracy" but also in such basic indicators as socio-economic development, size of the economy, or growth rates.
Hypothesis and Testing and Specification of the Universe
Unlike many large-N public opinion studies, the relevant "universe" of cases in cross-national quantitative research is not necessarily obvious, and the consequences of utilizing the wrong universe are severe. Here it is important to consider the theoretical claims that are being made and the relevant range of cases to which they are thought to apply. Undershooting the relevant universe can lead to selection bias. Overshooting can lead to inaccurate results in efforts to evaluate hypotheses.
Analysis and Critique of Research Designs
Here the first half of the class will present preliminary versions of a multi-level research design, to be followed by discussion and critique by the rest of the class.
Analysis and Critique of Research Designs
Here the first half of the class will present preliminary versions of a multi-level research design, to be followed by discussion and critique by the rest of the class.
Practical Problems in Implementing Designs: Bad and Missing Data
This begins the second half of the course, which focuses less on design and more on the practicalities of implementing a research design. We begin this section with a discussion of the all too common problem of "bad data." Here this can be data that only partly measures the phenomenon of interest, high levels of measurement error, differing levels of measurement error across cases, and data that are missing (e.g., years in a time series). Part of the discussion will involve efforts to use the insights of small-N work to improve the use of problematic cross-national data.
Practical Problems in Hypothesis Testing
Here we will consider exactly what "hypothesis testing" tests. Part of the leverage gained from multi-level designs is the ability to test theoretical propositions in a variety of very different ways. What do we learn from cross-national quantitative tests? How can we be sure that they are well structured? What do we learn from theory evaluation in small-N work? Does it test the same aspects of causal theory that large-N work does? How can the two most profitably be combined?
Practical Problems in Data Analysis -- Time and Place
It is frequently the case that small-N research has a dynamic (or longitudinal) component. It is also commonly the case that cross-sectional data are used in cross-national large-N research. This section examines whether cross-time data are required to answer questions, the distinctive issues that arise in using cross-sectional/time-series approaches, and the added difficulties that using such data entail. Similarly, we will examine the practical limits of what we learn from cross-sectional approaches, and how small-N information can help keep us from going astray.
Don't assume that you know more than you do
When the results of small-N and large-N work disagree, there can be a tendency to "believe" the large-N results, relegating the small-N findings to the status of 'outliers.' While this may be the case, it also may not. Here we will examine some classic cross-national work that has produced widely-replicated findings that are nonetheless heavily disputed by small-N scholars. Then we will examine how more recent work has called some of the large-N work into question, and suggested that undue reliance on quantitative results can be a serious pitfall.
[open session: topic to be determined]
Presentations III: Preliminary Results
Students will present their preliminary findings to the class, for discussion and critique.
Presentations IV: Preliminary Results
Students will present their preliminary findings to the class, for discussion and critique.
© Copyright 2000, Regents of the University of California