Overview

This course provides a framework for how analysts can create and evaluate quantitative measures. Consider the many tricky concepts that are often of interest to analysts, such as health, educational attainment and trust in government. This course will explore various approaches for quantifying these concepts. The course begins with an overview of the different levels of measurement and ways to transform variables. We’ll then discuss how to construct and build a measurement model. We’ll next examine surveys, as they are one of the most frequently used measurement tools. As part of this discussion, we’ll cover survey sampling, design and evaluation. Lastly, we’ll consider different ways to judge the quality of a measure, such as by its level of reliability or validity. By the end of this course, you should be able to develop and critically assess measures for concepts worth study. After all, a good analysis is built on good measures.

Syllabus

Fundamentals of Measurement

Scientists and social science researchers are tasked with measuring concepts of all types. Some are quantitative, like blood pressure, while others are qualitative, like support for a policy proposal. Moreover, some measures are at the individual level while others are aggregated. This module will focus on distinguishing between different types of measures, different types of units and the advantages and disadvantages of these differences. By the end of this module, I think you'll better appreciate the amount of judgment and decision making that is required to perform a statistical analysis, an in particular, to create and select measure of the concepts you care about studying.

Building a Measurement Model

Researchers are often tasked with analyzing concepts that are difficult to measure. As an example, think about the issue of immigration. A full consideration of policies related to immigration requires analysts to measure concepts such as border security, competition for jobs and employer needs. While there is broad agreement in most governments about the need for border security, there is disagreement about the extent to which their borders are currently secure. How could you measure border security? You could collect data on the amount of money spent on border security, the number of border patrol agents, the amount of physical barriers in place, the apprehension rate of undocumented immigrants and the deportation rate of undocumented immigrants. Some of these measures are inputs while others are outputs. All of these measures capture some aspect of border security. Which measure would you use? Why is this the most appropriate measure? Could you use a combination of measures? This module will explore these questions by examining the measurement process. We'll discuss conceptualization, operationalization and how to create full measurement models based on operationalized concepts.

Survey Research

V. O. Key, a renowned public opinion scholar, wrote that “to speak with precision of public opinion is not unlike coming to grips with the Holy Ghost.” But although survey research is challenging, it is an essential tool in both government and business. Survey research allows researchers to measure opinion and behavior in a wide range of areas, build an understanding of how abstract concepts are perceived, develop and test theories of opinion formation and formulate policy recommendations. In the public sphere, surveys give public officials insight into the views of their constituents and allow citizens to hold their governing officials accountable. In the private sphere, companies use surveys to measure their customers' experiences and levels of satisfaction with goods and services. This module will cover the three fundamental pillars of survey research: sampling, design and evaluation.

Evaluating Measurement Models

After you have designed a measurement tool, whether it is a survey, test, data collection technique or some other approach, it is essential to evaluate that tool. It is inevitable that a measurement tool will suffer from some degree of measurement error. If the error is well-understood and not overwhelmingly large, there are statistical techniques a researcher can use to address that error in an analysis. This module will introduce the concepts of systematic and random measurement error and explore how this error affects the reliability and validity of a measurement tool.