Agenda-Setting: The means by which media communicate to readers or viewers the importance of various issues.
Anomie: A social condition characterized by instability, the breakdown of social norms, institutional disorganization, and a divorce between socially valid goals and available means for achieving them. Introduced into sociology by Emile Durkheim in his study Suicide (1897), anomie also refers to the psychological condition-of rootlessness, futility, anxiety, and amorality-afflicting individuals who live under such conditions. The importance of anomie as a cause of deviant behavior received further elaboration by Robert K. Merton.
The Columbia Electronic Encyclopedia, Sixth Edition Copyright © 2003, Columbia University Press. Licensed from Columbia University Press. All rights reserved. www.cc.columbia.edu/cu/cup/
ANOVA (Analysis Of Variance): Similar to a t-test, which determines if variation between two groups is statistically significant, ANOVA is a test that allows comparison among multiple groups. Multiple t-tests are less desirable here because, as the number of groups increases, the number of needed comparisons grows quickly. For instance, for seven groups, there are twenty-one t-tests. If we test twenty-one pairs, it would not be surprising to find something that happens only 5 percent of the time, or outside the usual 95 percent confidence level.
Approval Bias: A possible source of error in a survey that results from respondents' trying to gain approval by giving what they perceive to be the answer the interviewer wants.
Area Probability Sampling: A type of probability sampling, usually done in multiple stages (multistage area probability sampling), conducted within or among geographical areas. “Multistage area probably first requires sampling a set of geographic regions. Next, a subset of a geographic area is sampled within each of those regions, and so on. The chance of an area being included increases with the number of people in it.” (Weisberg, Krosnick, and Bowen, An Introduction to Survey Research, Polling, and Data Analysis, 3rd edition, p. 47)
Average Absolute Error: The average of the absolute value of each separate error. The absolute value disregards the sign or direction of the individual error.
Back-Translation: The process of translating a document that has already been translated into a foreign language back to the original language—preferably by an independent translator… Back translation can improve the reliability and validity of research in different languages by requiring that the quality of a translation is verified by an independent translator translating back into the original language. Original and back translated documents can then be compared. (Source: http://www.asiamarketresearch.com/glossary/back-translation.htm)
Bivariate Correlation:A measure of association (strength) or relationship between two variables. Correlations are useful when researchers want to compare the attitudes, beliefs, or behavior of different groups.
Bonferroni Comparisons: A method that allows many comparisons, similar to ANOVA. The Bonferroni Correction is a statistical adjustment for multiple comparisons to avoid false positives.
Boxplot: A box and whisker plot is a way of summarizing a set of data measured on an interval scale. It is often used in exploratory data analysis. It is a type of graph which is used to show the shape of the distribution, its central value, and variability. The picture produced consists of the most extreme values in the data set (maximum and minimum values), the lower and upper quartiles, and the median. (Source: http://www.cas.lancs.ac.uk/glossary_v1.1/presdata.html#box)
Callback Survey: A survey in which people previously interviewed are contacted again. This method is often used to see if the attitudes or behavior of a particular sample have changed over time, or as a way of interviewing people who have said in the earlier interview that they intended to do something the researcher wants to investigate. For instance, respondents in the first sample might be asked if they intended to watch a speech, debate, or television show. Calling back those who said “yes” saves time trying to locate viewers after the event.
Categorical Variable: A variable that has two or more categories, but with no intrinsic ordering to the categories. For example, gender is a categorical variable having two categories (male and female), and there is no intrinsic ordering to the categories. Hair color is also a categorical variable having a number of categories (blonde, brown, brunette, red, etc.) and, again, there is no agreed way to order these from highest to lowest. A purely categorical variable is one that simply allows you to assign categories but you cannot clearly order the categories. If the variable has a clear ordering, then that variable would be an ordinal variable. (From UCLA Academic Technology Services, http://www.ats.ucla.edu/stat/mult_pkg/whatstat/nominal_ordinal_interval.htm)
CATI (Computer-Assisted Telephone Interviewing): An interviewing method in which questions appear on a computer screen and answers are entered directly into the computer. The method reduces interviewers' clerical errors and speeds up data processing (Bradburn and Sudman, 1988).
Chi-Square: A test of “goodness of fit”—that is, how well experimental data fit expectations.
Chi-Squared Automatic Interaction Detection (CHAID): CHAID is a statistical procedure, computed using SPSS Answer Tree 3.1, which examines a defined set of either categorical or continuous independent variables and selects the single best predictor of a categorical dependent variable. It then uses this predictor to divide a population into two or more subsets characterized by significantly different scores on the dependent variable. The process then repeats, choosing the best predictor of the dependent variable among the elements of each subset defined on the preceding step. The result is a dendrogram that arranges the significant predictors in a hierarchy based on increasingly small subsets until no additional significant predictors of the dependent variable can be found among the elements of any of the subgroups.
Thanks to Kenneth R. Blake, Robert O. Wyatt, and Holly Warf, Middle Tennessee State
“Clash of Civilizations”: Refers to Samuel P. Huntington’s assertion that in the post-Cold War era, the major conflicts were to occur between different “civilizations,” or primordial groupings based around the complex of ethnicity, religion, and culture.
Cleaning Table: Criteria-based filters set up at the data-cleaning stage of a project to catch respondents who do not meet certain criteria, usually by looking for certain combinations of response data in the file.
Closeness of Fit: Statistical models are typically evaluated in terms of how well their output matches data, that is, in terms of model accuracy. A model can match data in several ways, including precision, the absolute "closeness of fit" between model predictions and data.
Cluster Design: “[A] hierarchical type of sampling in which the elementary units are at least one step removed from the original sampling clusters… It is frequently adopted when it is not feasible to compile sampling frames of all enumeration units for the entire population, a prerequisite for simple random sampling or stratified random design. In such situations, sampling frames can be constructed that identify groups or clusters of enumeration units without listing explicitly the enumeration units. As a result, sampling can be performed from such frames by first taking a sample of clusters and then obtaining a list of enumeration units only for those clusters that have been selected in the sample.” From Stanley Lemeshow and Amy Ferketich. 2005. “Cluster Sampling,” in Polling America: An Encyclopedia of Public Opinion, ed. S. J. Best and B. Radcliff. Westport, CT: Greenwood Press, 88-92.
Cohort Effect: An effect that reflects the unique reaction of a cohort, or generational group, to an historical event, or that was experienced uniquely by the cohort.
Computer-Assisted Personal Interviewing (CAPI): An interviewing method in which the interviewer carries a lightweight portable computer into a household. The advantage is that the computer carries out branching and editing commands and reduces interviewer clerical errors (Bradburn and Sudman, Polls and Surveys).
Confidence Limit: The upper and lower values of a statistical estimate. The 95 percent confidence limit is the most widely used in polling. This means that the sampling procedure used had a 95 percent chance of producing a set of limits that encloses the proportion that would be found if the entire population had been asked. For instance, a poll might find that 65 percent of the public favored a policy, and the confidence limit (at the 95 percent level) could be 62-68 percent (paraphrased from Bradburn and Sudman, 1988).
Context Effect: The sensitivity of survey responses to variations in questionnaire form, such as question order, question wording, the format of the responses, and so forth.
Contract with America: Crafted by Representative Newt Gingrich (R-Ga.) during the 1994 congressional campaign, the "Contract with America" was a statement of principles signed by Republican candidates who pledged action on several pieces of legislation, all of which were said to have majority support from the American public. Many analysts believe that the Contract served as a catalyst for a victory that gave Republicans control of the U.S. House for the first time in decades.
Contrast Coding Method: As defined by McKee McClendon, contrast coding, like the more ubiquitous dummy coding, is a process whereby g-1 variables are created to present g categories within a nominal variable. However, contrast coding differs from dummy coding in that contrast coding can define a single group or subset of groups and compare them to another group or subset of groups within the nominal variable. There are two mathematical restrictions to the procedure, namely, that the numerical designations given to each group within each created variable must sum to zero. The second restriction is that the sum of the products for each pair of contrast coded variables must equal zero. These restrictions ensure the creation of a set of uncorrelated variables.
Control: A variable within which a relationship can be analyzed:
"When only two variables are crosstabulated, we call the resulting table a two-way table. However, the general idea of crosstabulating values of variables can be generalized to more than just two variables. For example [in an analysis comparing preferences of men versus women for soda Brand A and Brand B], a third variable could be added to the data set. [In this example, the control is] information about the state in which the study was conducted (either Nebraska or New York )."
"The crosstabulation of these variables would result in a 3-way table:"
STATE: NEW YORK
Source: Quoted from StatSoft Electronic Textbook.
Convenience Sample: One of the most common types of nonprobability sample is called a convenience sample—not because such samples are necessarily easy to recruit, but because the researcher uses whatever individuals are available rather than selecting from the entire population. Because some members of the population have no chance of being sampled, the extent to which a convenience sample—regardless of its size—actually represents the entire population cannot be known.
Correlation Coefficient: A correlation coefficient is a number between -1 and 1 which measures the degree to which two variables are linearly related. If there is perfect linear relationship with positive slope between the two variables, we have a correlation coefficient of 1; if there is positive correlation, whenever one variable has a high (low) value, so does the other. If there is a perfect linear relationship with negative slope between the two variables, we have a correlation coefficient of -1; if there is negative correlation, whenever one variable has a high (low) value, the other has a low (high) value. A correlation coefficient of 0 means that there is no linear relationship between the variables. (Valerie J. East and John H. McCall, Statistics Glossary, http://www.stats.gla.ac.uk/steps/glossary/paired_data.html#corrcoeff)
Coverage Error: “The error in surveys that can occur when the sample frame population and the target population for a survey do not correspond. The extent of the bias… depends on both the magnitude of the noncoverage and the differences between respondents included in the sample frame on the particular statistics involved in any analysis.” From Chase H. Harrison. 2005. “Coverage Error,” in Polling America: An encyclopedia of public opinion, ed. S. J. Best and B. Radcliff. Westport, CT: Greenwood Press, 2005), 134-40.
Cronbach's Alpha: Cronbach's alpha is a test for a model or survey's internal consistency. Sometimes called a "scale reliability coefficient."
Cross-Sectional Study: A study in which data from particular subjects are obtained only once. Contrast with longitudinal studies, in which a panel of individuals is interviewed repeatedly over a period of time. (ICPSR, University of Michigan, http://www.icpsr.umich.edu/DDI/glossary.html).
Crosstabulation: A test to determine whether there is a relationship between two variables. For instance, we may hypothesize that support for abortion is higher among younger people than older people. One could run a crosstabulation (crosstab) that has an abortion question as the dependent variable and age as the independent variable to see if young people give significantly difference responses from older people.
Deliberative Discussion: A process in which a group of participants is brought together to discuss, deliberate, and reflect on issues. The discussion, usually guided by a trained facilitator, is based on information in a briefing package, on video, presented by speakers advancing arguments for opposing positions, or presented by subject-matter experts.
Dependent Variable: The observed variable in an experiment or study whose changes are determined by the presence or degree of one or more independent variables. http://www.bartleby.com/61/17/D0141700.html
Designated Sample Point: Location where the sampling design of a survey says an interviewer should go to conduct an interview.
Disproportionate Stratified Sampling Design: A method used to reach a higher proportion of a target group efficiently while still representing those in the target group who live in areas with a lower density of that group.
The disproportionate stratified sample provides a highly accurate sampling frame, thereby reducing the cost per effective interview. Typically, all telephone exchanges within a target area are listed in descending order by concentration of the target population. Exchanges are then divided into strata based on the incidence of the target population. Each stratum generally contains the same number of target population households. For example, roughly 25 percent of households served by telephone exchanges with the highest incidence are placed in the first stratum, followed by those with the next largest incidence, and so on, with a fourth stratum containing the 25 percent with the lowest incidence.
At this point, most sampling designs employ an optimal allocation scheme. This "textbook" approach allocates interviews to a stratum proportionate to the number of target population households, but inversely proportionate to the square root of the relative cost, the relative cost in this situation being a simple function of the incidence. As such, the number of completed interviews increases as you move from a lower incidence stratum to higher incidence strata. This is a known, formulaic approach to allocation that provides a starting point for discussions of sample allocation and associated costs.
Thus, sample generation within each defined stratum utilizes a strict EPSEM sampling procedure, providing equal probability of selection to every telephone number. However, at that point numbers that reside in higher incidence strata are more likely to be dialed, and telephone numbers in the lowest incidence stratum are least likely to be interviewed. This procedure can double, or even triple, the incidence of reaching a target household as compared to the general, RDD incidence of that target population. The disproportionality of the sampling scheme is later taken into account with weighting, balancing the population back to its true parameters.
This process does have one principal cost, and that is on the design effect of the study. Simply stated, the design effect is the measure of the precision that is lost in any complex probability design, compared to what the precision would have been had the study been conducted using simple RDD methodology. Any stratified or other complex sampling design, "pound for pound," will increase the standard errors of all estimates, which can also be represented by the number of effective interviews, which is the number of unweighted interviews divided by the design effect.
Thus, the larger the design effect, the smaller the number of effective interviews, and therefore the larger the standard errors associated with the study. The size of the design effect in this type of study is determined by the amount of disproportionality introduced into the design. A design that roughly doubles the incidence of the target population will typically create a design effect of somewhere around 1.5a small price to pay considering the cost savings associated with a doubling of survey incidence.
For an example of disproportionate stratified sampling in action, see the methodology page of the 2004 National Public Radio/Kaiser Family Foundation/Kennedy School of Government Immigration Survey. Thanks to David Dutwin and Melissa Herrmann, ICR/International Communications Research, for this discussion of disproportionate stratified sampling design.
Double Opt-In: The recommended procedure for subscribing email recipients to an email list or newsletter. Once a person requests to subscribe to a list, a confirmation email message is automatically sent to the supplied email address asking the person to verify that he or she has, in fact, requested to be included in future mailings. (Source: www.lsoft.com/resources/glossary.asp)
Double-Punch Entry: This term dates from the era of punch cards for data recording and is still used widely today to describe the process where data is entered twice and then compared to identify human error in recording.
Downweighting: The practice of assigning a smaller mathematical value to certain types of survey respondents who are overrepresented in the sample. This technique is used to ensure that the overall survey results are reflective of a total population (see “weighting”).
Dummy Variable: "A variable that marks or encodes a particular attribute. A dummy variable has the value zero or one for each observation, e.g., 1 for male and 0 for female." (Source: http://economics.about.com/library/glossary/bldef-dummy-variables.htm )
Elaboration Likelihood Model (ELM): Borrowing from social-psychological theory, the elaboration likelihood model demonstrates that once an issue is elaborated by individuals, further information, even if it should conflict with the individual’s opinion, is unlikely to change that opinion.
EPSEM Samples: EPSEM samples are probability samples where each observation in the population has the same known probability of being selected into the sample (EPSEM stands for equal probability of selection method sampling; see Kish 1965, for a comprehensive discussion of sampling techniques). EPSEM samples have certain desirable properties; for example, the simple formulas for computing means, standard deviations, and so on can be applied to estimate the respective parameters in the population.
Exit Poll: “Exit polling is the practice of interviewing voters on Election Day as they leave (exit) the building in which they have cast ballots. Such polls have two functions. First, they allow election forecasters to gather information that may help them to predict who is likely to win an election long before the votes are tabulated and official results released. Second, they provide valuable information about voters and their preferences, so as to make it easier to understand the outcome” (Radcliff 2005).
Factor Analysis: Factor analysis tells us what variables group or go together. Factor analysis boils down a correlation matrix into a few major pieces so that the variables within the pieces are more highly correlated with each other than with variables in the other pieces. Factor analysis is actually a causal model. We assume that observed variables are correlated or go together because they share one or more underlying causes, called factors.
527 Organization: Named after a section of the United States tax code, a 527 group is a tax-exempt organization that is created primarily to influence the nomination, election, appointment, or defeat of candidates for public office. Although political action committees are also created under Section 527 of the Internal Revenue Code, the term is used to refer to political organizations which are not regulated by the Federal Election Commission and not subject to the same contribution limits as PACs. (Source: http://en.wikipedia.org/wiki/527_organization)
Framing: See "media framing."
Framing Effect: The effect of emphasizing one subset of potentially relevant considerations rather than another, leading to different responses. This effect can be seen in polling about several issues, for example, campaign finance reform (free speech or corruption) and abortion (rights of the mother or unborn child).
General Social Survey (GSS): The General Social Survey, conducted since 1972 by the National Opinion Research Center (NORC), consists of a large variety of important social indicators. Many of the questions have been asked for a number of years, which makes the GSS useful for measuring trends. Moreover, the large number of interviews in the cumulative dataset make it possible to learn about the attitudes and beliefs of small demographic groups.
Global Opinion Theory: The view that nations advance interpretations of world opinion whose structure and content are favorable to their interests and values, and that a process of negotiation then takes place among the involved nations to resolve these interpretations.
Guttman Scale: A measurement scale that assumes that when you agree with a scale item you will also agree with items that are less extreme.
Horserace Question: See "trial heat question."
House Effect: Variations in survey results that occur because of differences in procedures between the survey houses conducting the surveys.
Incented Panel: A group (panel) of available respondents who are given incentives to be on the panel or to participate in a survey.
Incidence Rate: The proportion at which a group or characteristic occurs within a population.
Independent Samples T-Tests: An independent samples t-test is used when you want to compare the means on a dependent variable (e.g., SAT score) for two independent groups (e.g., men and women).
Independent Variable: A manipulated variable in an experiment or study whose presence or degree determines the change in the dependent variable. http://www.bartleby.com/61/99/I0099950.html. In survey research, this is often a demographic variable, such as age, education, or gender.
Intercoder Reliability: The percentage of agreement between two coders coding the same material.
Interval Formula: A sampling formula according to which interviewers interview every nth person or unit (e.g., household) in a population.
Interviewer Effect: This term is most often used to refer to the effect of the interaction between the interviewer and the respondent, which can introduce bias. For instance, the respondent may believe that the interviewer would prefer a particular response. The term is more broadly used to refer to errors in data quality introduced by those implementing the survey process.
Issue Constraint: The consistency of views that a person or group holds about different issues, e.g., the degree to which a person has consistently liberal or conservative views.
Kish Grid: A table of numbers, named after the statistician who invented it. The number of people in the household is discovered, and a random number from the table is chosen to select a particular person. (Source: http://www.sysurvey.com/tips/sampling.htm)
Leaner: 1. A survey respondent who does not make a choice among alternatives in an initial question, but makes a choice once asked if he or she leans toward one of the alternatives. 2. A survey question that asks respondents who do not initially make a choice between alternatives if they lean toward one of them. "Leaners" occur most often in questions about election choices.
Leaner Question: A follow-up question used to encourage initially undecided respondents to choose between alternatives, usually political candidates.
Likely Voter: A survey respondent who is estimated, by a variety of means, to be likely to vote in a coming election. Survey firms use different methods of determining the likelihood of voting, usually including a scale of several items in a poll, such as current voter registration status, past history of having voted, and self-described likelihood of voting.
Linear Regression: A method estimating the conditional expected value of one ("dependent") variable given the values of some other ("independent") variable(s). For instance, if we want to determine the relationship between height and weight for a sample of people, linear regression attempts to explain the relationship with a straight line fit to the data.
Likert Scale: The most widely used scale in survey research, the Likert scale measures the level at which respondents agree or disagree with a given statement. The most Likert common scale ranges from 1 to 5, with the choice of responses being, for example, 1=strongly disagree, 2=disagree, 3=not sure,4=agree, and 5=strongly agree.
List Samples: With list samples, potential respondent names come from records or lists which are generally supplied by the clients. For example, for a survey of patrons of local libraries, the sample may begin with a list of persons who have library cards or who have used library services.
Samples drawn from such lists usually are generated by a random selection process. Using lists often makes it possible to link information from records (e.g., employer records or service records) with information in the survey. In addition, targeted respondents can be reached efficiently when working from current, comprehensive lists, thus keeping costs down.
Literary Digest Disaster: A poll conducted by the Literary Digest called the 1936 presidential election for Alf Landon, when in fact Franklin D. Roosevelt won reelection in a landslide. The survey ballot had been mailed to Literary Digest subscribers and certain other listed groups, which resulted in the poll being unrepresentative of the voting population.
Logistic Regression: Part of a category of statistical models called generalized linear models. This broad class of models includes ordinary regression and ANOVA, as well as multivariate statistics such as ANCOVA and loglinear regression….Logistic regression allows one to predict a discrete outcome, such as group membership, from a set of variables that may be continuous, discrete, dichotomous, or a mix of any of these. (http://userwww.sfsu.edu/~efc/classes/biol710/logistic/logisticreg.htm)
Longitudinal Study: a study that involves the repeated observation or examination of a set of subjects over time with respect to one or more study variables. (http://www.everythingbio.com/glos/definition.php?word=longitudinal+study)
Margin of Error: A bound that we can confidently place on the difference between an estimate of something and the true value.
Maslow's Hierarchy: The idea that that people must meet their most basic physiological needs before they can pursue their full potential as unique individuals.
Mean Squared Error: The average of the square of the difference between a desired response and an actual response. Since the definition of the mean is the point about which the average error is zero, we square the errors to eliminate the positive and negative signs and get the point where the average error is as low as it can be.
Measurement Error: All measurements are subject to error, including the imprecision of the method of measurement (e.g., reading a thermometer, understanding a question).
Media Framing: The process by which an issue is portrayed in the news media. Media frames provide boundaries around a news story and determine what is and is not newsworthy or notable. From Communications Terms and Concept, UCLA Center for Communications and Community.
Method Effect: Differences in survey results related to the method by which the data are gathered. For instance, the same question may yield different responses when asked on a telephone versus an online survey.
Metropolitan Statistical Area (MSA): An MSA is a county or group of contiguous counties that contains at least one city with a population of 50,000 or more or includes a Census Bureau-defined urbanized area of at least 50,000, with a metropolitan population of at least 100,000. In addition to the county containing the main city or urbanized area, an MSA may contain other counties that are metropolitan in character and are economically and socially integrated with the central counties. In New England, cities and towns, rather than counties, are used to define MSAs.
MSAs are defined by the U.S. Office of Management and Budget (OMB). The MSA standards are revised before each decennial census. When U.S. Census data become available, the standards are applied to define the actual MSAs.
Miscommunication Queues: Taken from communications theory, this term refers to one of the three forms of communication breakdown that can occur in conversations. Essentially, it means the listener does not understand the speaker's meaning.
Mode Effect: A difference in response caused by the mode by which the data are collected. For instance, the same question asked on both a telephone survey and an online survey may yield difference results because the respondent interprets or responds to the question differently when presented by one mode versus the other.
Most Recent Birthday Method: A way to choose one respondent randomly in a household by asking to interview the eligible person who had the most recent birthday.
Multi-Stage Sample: A sample that is selected in stages, where the sampling units at each stage are subsamples from the previous stage.
Multivariate Analysis: A statistical analysis of the simultaneous relationships among three or more (some would say two or more) variables; the analysis of several variables simultaneously.
National Election Pool (NEP): A consortium of ABC News, CBS News, NBC News, Fox News, CNN, and the Associated Press. Edison Media Research and Mitofsky International conducted the 2004 national exit poll for NEP.
1936 Presidential Election: In 1936, methods of polling pioneered by George Gallup, Elmo Roper, and Archibald Crossley were put to the test of predicting the outcome of the Franklin D. Roosevelt-Alf Landon presidential election. The new methods involved interviewing relatively small samples in person and paying attention to the demographic composition of the samplein those days achieved through quota samplingin contrast to the Literary Digest poll, which had used huge mail-in samples to predict the outcomes correctly in every presidential election since 1920. All three of the new polls correctly predicted a substantial victory for Roosevelt, while the Literary Digest, whose methods had been criticized prior to the election by George Gallup, wrongly forecast a Landon win. According to Bradburn and Sudman 1988, upon which this account is based, "The 1936 election led... to the almost overnight acceptance of public opinion polls by politicians and the general public" (p. 19).
1948 Presidential Election: In the context of polling, the 1948 presidential election is considered one of the profession’s greatest failures. President Harry Truman (D) defeated Governor Thomas Dewey (R), while preelection polls showed Dewey winning. A National Academy of Sciences committee investigated the reasons for the polls’ failure, and several polling practices were changed as a result.
Nonresponse Bias: The bias that arises in when those who do not respond give different answers to survey questions than those who do respond.
Nth Selection: A statistical means of taking a given number of names or units equally selected over the full population of study. The nth number interval is derived by dividing the total number of units by the sample number desired. Also known as interval. (http://www.quirks.com/resources/glossary.asp#LetterN)
Null Hypothesis: "Because a research hypothesis cannot be proved but only disproved, scientists have developed the notion of the null hypothesis. usually the opposite of the research hypothesis. [For] example, "Republicans are as likely as Democrats to vote for Democratic candidates." Most often, the null hypothesis states that no relationship exists between two variables or that one variable does not affect another variable. After stating a null hypothesis, researchers try to disprove or reject it. Disproving a null hypothesis offers some support for the research hypothesis." (From Herbert F. Weisberg, Jon A. Krosnick, Bruce D. Bowen, An Introduction to Survey Research, Polling, and Data Analysis.)
Odds Ratio: The odds ratio is a way of comparing whether the probability of a certain event is the same for two groups. An odds ratio of one implies that the event is equally likely in both groups. An odds ratio greater than one implies that the event is more likely in the first group. An odds ratio less than one implies that the event is less likely in the first group. http://www.childrensmercy.org/stats/definitions/or.htm
Omnibus Polls: Regularly conducted polls (e.g., weekly or monthly) on which clients can place their own questions for a standard, usually per-minute, cost. Omnibus polls usually have relatively short field periods, and the demographic questions are shared in common, so such polls are an inexpensive way to obtain data quickly.
Open Box: Text field in which an answer must be typed.
Ordinary-Least-Squares (OLS) Analysis: A type of regression analysis that uses sample data to estimate the true population relationship between two variables by minimizing the sum of the squared errors.
Oversample: A sampling procedure designed to give a demographic or geographic population a larger proportion of representation in the sample than the population's proportion of representation in the overall population. Oversamples are often used to study the attitudes or behavior of groups that make up a small proportion of the total population. For instance, one might oversample African Americans for a study on discrimination, or people ages 65 and over for a study about Medicare.
Panel Sampling: Panels represent sample units who have agreed to answer questions again and again over a period of time.
Panel Survey: A survey in which similar measurements are made on the same people at different points in time.
Paper-and-Pencil Interview (PAPI): An in-person interview in which responses are written by hand, rather than with computer assistance.
Party Identification: A self-description by respondents of which political party they identify with. In American polls, the response categories almost always include Democrat, Republican, and Independent. Some organizations add "or what," "some other party," or "something else" as a response. The question is usually framed in one of two ways: "In politics today, do you consider yourself a...," or "Generally speaking, do you usually think of yourself as a..." The former tends to measure at-the-moment identification, the latter a longer-term identification.
Here are some examples:
In politics, as of today, do you consider yourself a Republican, a Democrat, or an Independent? (Gallup)
In politics today, do you consider yourself a Republican, Democrat, or Independent? (Pew Research Center/Princeton Survey Research Associates)
Generally speaking, do you usually think of yourself as a Democrat, a Republican, an Independent, or what? (ABC News/Washington Post, CBS News/New York Times, American National Election Survey)
Generally speaking, do you think of yourself as a Democrat, a Republican, an Independent or something else? (NBC News/Wall Street Journal)
In some polls, those who identify with one of the two parties are asked whether they consider themselves strong or not very strong (Democrats/Republicans). Often those who say they are independents are asked whether they lean Democratic or Republican. If both of these follow-ups are used, a nine-fold scale results (strong Democrat, not very strong Democrat, Democratic leaner, true Independent, Republican leaner, not very strong Republican, strong Republican).
Peripheral Cue: described by Diana Mutz and Travis Rideout as a credible and persuasive information shortcut that does not lead to in-depth thought and is used by individuals to adjust their opinions. See "source cue."
Positivity Bias: The tendency of respondents who do not have strong opinions to give a positive rather than a negative response if pushed to make a choice.
Post-Materialist: A term coined by political scientist Ronald Inglehart in 1970 to describe an economic philosophy whereby once a society has people’s needs relating to their physical well-being, they begin to take survival and prosperity for granted, and turn their attention to fulfilling new expectations of their physical and social setting, such as environmental matters or gender equality.
Post-Stratification. Classification and weighting to correspond to external values of selected sampling units by a set of strata definitions after the sample has been selected. (http://nces.ed.gov/nationsreportcard/glossary.asp#s) See also “post-stratification weighting."
Post-Stratification Weighting: A method of stratification which is carried out after the sample data have been collected rather than at the time of sample selection, which makes it possible to use a much wider range of variables for stratification. This process adjusts marginal distributions of the sample data to be the same as the known distribution in the population. (Source: http://www.iser.essex.ac.uk/ulsc/bhps/doc/vola/wgimp.php)
Precinct: Generally the lowest-level (smallest) political unit, where adults are assigned a single place to vote in elections.
Predicted Probability: An estimate, based on several factors, of how likely an outcome is to occur.
Presidential Approval: An overall measure of how well Americans think the president is handling his job. In most cases, the question is presented as a simple choice, approve or disapprove.
Here are some examples:
Do you approve or disapprove of the way (name) is handling his job as president? (Gallup, CBS News/ New York Times )
In general, do you approve or disapprove of the job that (name) is doing as president? (NBC News/ Wall Street Journal )
Often a follow-up question is asked about how strongly the respondent approves or disapproves:
Do you approve or disapprove of the way (name) is handling his job as president? (If Approve/Disapprove, ask:) Do you approve/disapprove strongly or somewhat? (ABC News/ Washington Post )
Some organizations use a different rating scale.
How would you rate the job... President George W. Bush is doing--excellent, pretty good, only fair, or poor? (Harris Interactive)
Similar questions are used to measure how well Americans think the president is handling particular issues, such as the economy, foreign policy, health care, or racial relations.
Primacy Effect: The tendency of respondents to remember and/or choose the first item on a list. This is contrasted to recency effect, the tendency of respondents to remember and/or choose the last item.
Priming: A psychological process by which the triggering of certain ideas or concepts can make people more responsive to other, related ideas and concepts.
Probability Sampling: Any method of sampling that utilizes some form of random selection of participants from a population. Each possible participant in the population has an equal chance of being selected to be in the sample. Simple random sampling, stratified random sampling, cluster sampling, and systematic sampling are examples of probability sampling methods. Drawing names from a hat is also an example of probability sampling.
Probit: Probit is part of a category of statistical models called generalized linear models. A Probit model is a maximum likelihood estimation technique used for predicting outcomes of dichotomous dependent variables. Ordered probit is a form of a probit equation used for categorical variables that form an index or ranking with more than two values. It assumes the values are not important, only the ranking of the categorical values. http://en.wikipedia.org/wiki/Probit_model
Projection Model: A model that combines exit polls and actual votes to make projections in an election.
Projective Questions: Questions that present a hypothetical situation that requires a decision. (Source: http://www.mtsu.edu/~socwork/sw263g.html)
Projective Tests: In psychology, examinations that commonly employ ambiguous stimuli, notably inkblots (Rorschach Test) and enigmatic pictures (Thematic Apperception Test) to evoke responses that may reveal facets of the subject's personality by projection of internal attitudes, traits, and behavior patterns upon the external stimuli. (Encyclopaedia Britannica Online)
Propensity Score Adjustment in Weighting Data: Although proper sampling is typically the first step in achieving representativeness, data can be adjusted to resemble a general population more closely through a technique known as weighting . Most often, simple demographic weighting is used to bring proportions of respondents in line with the proportions as they exist for age, gender, or region of country. This is known as simple demographic or rim weighting.
Additional weighting can sometimes be useful to take into account not only demographic factors but also attitudinal ones. Taking the attitudinal factors into account when weighting forms the basis of a propensity score adjustment approach to weighting.
Some people (in the United States and elsewhere) who are not online have characteristics that lead survey researchers to think they should be online. By the same token, some people who are online have characteristics that would lead researchers to believe that they should not be online. A propensity score is a single, summary measure of whether one is likely to be a participant in a telephone survey rather than an online survey, given their characteristics. Propensity score adjustment makes it possible to balance efficiently the characteristics, beyond demographics, that differentiate online respondents from telephone respondents. Propensity score adjustment is a statistical technique that minimizes error associated with internet-based panel samples and the learning effects associated with participating in multiple surveys. By taking into account attitudinal differences between online and phone respondents, it is used in conjunction with standard data weighting techniques in order to produce reliable, valid data that can be projected to populations of interest, whether they are large, general populations or smaller, more specific groups.
Thanks to George Terhanian and John Bremer, H arris Interactive, for this description of propensity score adjustment in weighting data.
Proportional Representation System: Any election system which ensures a proportionally representative result of a democratic election, [that is,] x% of votes should be represented by x% in the democratic institutions, parliament, or congress. In practice, this is often only possible in various multi-winner electoral systems which try to ensure that the proportional support gained by different groups is accurately reflected in the election result. Proportional representation is also used to describe this (intended) effect. In practice this usually involves ensuring that political parties in parliament or legislative assemblies receive a number of seats (approximately) proportional to the percentage of vote they received. This is known as party-list proportional representation. (Source: http://en.wikipedia.org/wiki/Proportional_representation)
Push Poll: A form of negative campaigning disguised as a political poll that is designed to change opinions, not measure them. Legitimate polls accurately describe the candidates and issues in order to understand voter reactions. Push polls frequently present distorted information in order to influence voters. Push polls go beyond the ethical boundaries of political polling and bombard voters with problematic statements about candidates or issues in an effort to manufacture negative voter attitudes. (Source: AAPOR Statement on "Push Polls," May 19, 2004, http://www.aapor.org/pdfs/2004/pushpolls.pdf).
Question Order Effect or Question Order Bias: The tendency for responses to a survey question to be influenced by questions that appear earlier in the survey.
Quick Counts—In exit polling, the early reports of actual vote returns collected right after the polls close from a sample of precincts. Also known as a parallel vote tabulation (PVT), “a quick count is a… method of monitoring election day developments… During a quick count, observers watch the voting and counting processes at [a random statistical sample of] polling stations, record key information on standardized forms, and report their findings (including the polling station’s vote count) to a central data collection center. Quick count leaders use this information to evaluate the overall quality of election day processes and to project, or verify, official results based on precise analysis of polling station data.” (http://www.accessdemocracy.org/library/1417_elect_quickcounthdbk_1-30.pdf)
Quota Sample: A sampling procedure that includes specified numbers of respondents having characteristics known or believed to affect the subject being researched. Selection is by nonprobability means (Quirk’s Marketing Research Review, http://www.quirks.com/resources/glossary.asp#LetterS).
Radio Button: A circular check "bubble" next to a response category or value that is filled or clicked on to choose that value.
"Rally 'Round the Flag" Effect: The tendency of the public to express support for leaders and institutions at the time of a threat to national security. The most visible example is a rise in presidential approval, since the president is commander-in-chief. But the rallying effect often extends to a rise in ratings of other institutions and in the proportion of people expressing patriotic sentiments.
Random Digit Dialing (RDD): The selection of telephone numbers for a telephone sample by computer generation from the list of working telephone exchanges. RDD procedures have the advantage of including unlisted numbers, which would be missed if numbers were drawn from a telephone book. (Source: Norman M. Bradburn and Seymour Sudman, Polls and Surveys: Understanding What They Tell Us. San Francisco: Jossey-Bass, 1988.)
Random Error: Error that results from chance.
Random Route Method: In order to ensure random selection within a sampling unit for in-person surveys, a random route is chosen for the interviewer to take after finding the starting address. The route chosen gives every household in the cluster an equal chance of being selected for the survey.
Random Sampling: A sampling technique where we select a group of subjects (a sample) for study from a larger group (a population). Each individual is chosen entirely by chance, and each member of the population has a known, but possibly unequal, chance of being included in the sample. (Source: Valerie J. East and John H. McCall, Statistics Glossary, http://www.stats.gla.ac.uk/steps/glossary/paired_data.html#corrcoeff)
Ratings Battery: A series of questions used to evaluate institutions, businesses, people, products, advertisements, and so forth, in which respondents are asked to select one response from a scale to indicate the degree of their opinion.
Refusal Conversion: An attempt to convince potential respondents to cooperate in answering a survey after they refuse to do so in an earlier contact.
Regression Analysis: See “Logistic Regression.”
Regression Model: In a linear regression, which is designed to study the relationship between one (dependent) variable and several other (independent) variables, the regression model defines the variables being used.
Reliability: The degree to which multiple measures of the same behavior or attitude agree. These multiple measures may be over time or at the same time (Source: Bradburn and Sudman, 1988).
Cronbach's alpha assesses the reliability of a rating summarizing a group of test or survey answers which measure some underlying factor (e.g., some attribute of the test-taker). A score is computed from each test item, and the overall rating, called a "scale," is defined by the sum of these scores over all the test items. Then reliability is defined to be the square of the correlation between the measured scale and the underlying factor the scale was supposed to measure.
Religiosity: The quality of being religious. In common usage, the term sometimes has the implication of exaggerated or affected religious zeal. In the field of public opinion, the term more often refers to measures of how religious a person is (such as attendance at religious services or frequency of prayer), as distinct from their religious affiliation.
Response Order Effect or Response Order Bias: The order in which answer choices are presented to respondents can affect their answers. For instance, when answer choices are read aloud to respondents, they are more likely to choose an alternative at the end of the list. In order to reduce this bias, answer choices are sometimes rotated or randomized. See also question order effect .
Reverse Scoring or Reverse Coding: The process of rescoring items (i.e., survey questions) in a scale that are negatively worded in a positive direction, in order to match the other items in the scale that are positively worded (or vice versa).
Reweighting: Often used interchangeably with "weighting," reweighting can also mean applying a different weight than the one originally used.
Right Direction/Wrong Track: This question, first asked in the early 1970s and frequently asked since the 1980s by various polling organizations, is generally asked at the beginning of a survey to measure the public's general mood about the state and direction of the country (or state, or other political entity). The most common forms are, "Do you (think/feel) things in this country are generally (going/heading) in the right direction, or (are they/have they gotten) (pretty seriously) off on the wrong track?"
Rolling Cross-Sectional Survey: “With this method, random samples of respondents are interviewed each day of [a political] campaign period in such a way that the samples are comparable from one day to the next. Specifically, the composition of each day's interviews is balanced on various demographic characteristics. Daily interviews can thus be used to identify trends and points of change in the public's reactions to political events as they unfold over the course of the… campaign.” From the National Annenberg Election Survey. See tracking poll.
R-Squared: The percentage of variation in the response variable that is explained by the regression line. (http://www.stat.duke.edu/~jerry/sta101/faq.html)
Salience Effects: The tendency of people exposed to news coverage to adjust their issue agendas in response to that exposure. For instance, those who frequently see or read stories about an issue or event such as a war would be more likely to name war as an important issue or problem.
Sampling Error: An error arising from the fact that it is not statistically possible, short of having a 100 percent sample, to select a sample which corresponds perfectly to the population from which it is selected. As the size of a sample increases, the magnitude of the sampling error decreases. Sampling errors differ from other kinds of statistical errors in that they occur at random and are unbiased. Nonsampling errors, on the other hand, are errors that can be attributed to mistakes in data collection, tabulation, analysis, and so forth. (Source: www.nhes.state.nh.us/elmi/s_glossary.htm)
Sampling Frame: The source from which a sample is drawn. A list of all those within a population who can be sampled, which may include individuals, households, or institutions. (Source: http://en.wikipedia.org/wiki/Sampling_frame)
Screening Questions: Questions used to determine who will be included in and excluded from the sample. For instance, a preelection survey might use a screening question to exclude people who are not registered to vote, or a survey about Medicare might screen by age in order to get a sample of people ages 65 and over.
Selection Probability. The chance that a particular sampling unit has of being selected in the sample. (http://nces.ed.gov/nationsreportcard/glossary.asp#s)
Show Cards: A type of prompt material in the form of cards with images that are shown to participants in research studies.
Signed Error: Signed error is directional error; that is, error that is positive if it favors one candidate and negative if it favors the other.
Skewness: An asymmetrical frequency distribution in which the values are concentrated on one side of the central tendency and trail out on the other side. If the trail is to the right, or positive, end of the scale, the distribution is said to be positively skewed. If the distribution trails off to the left or negative side of the scale, it is said to be negatively skewed. (Source: www.brendan.com/Glossary.htm)
A measure of the degree of asymmetry of the data around the sample mean. If the data are distributed symmetrically around the mean, the skewness is 0.0. From a graphical point of view, negative skewness indicates that the data are more spread out to the left of the mean than to the right of it. The reverse is true for positive skewness. (Source: www.wizkid-trading.com/Freesamples/floyd.htm)
Skip-Out Option: An explicit option not to answer a question, but to continue on to another question.
Skip Pattern: the sequence of questions asked and skipped in a survey. For instance, if a respondent answers a question that indicates she does not own a car, the interviewer can opt to "skip" items regarding what kind of car she owns. (http://www.surveyshare.com/help/skippattern.html)
Snowball Sample: A nonprobability sampling method used to develop a sample for a population whose desired characteristic is relatively rare, or whose members are difficult to reach (e.g., drug users, mountain bikers). Existing members of the sample recruit or nominate others from among their acquaintances to build the sample. While this technique can reduce search costs, the selection method introduces a number of biases into the sample.
Social Capital: The central premise of social capital is that social networks have value. Social capital refers to the collective value of all "social networks" (who people know) and the inclinations that arise from these networks to do things for each other ("norms of reciprocity"). The term social capital emphasizes not just warm and cuddly feelings, but a wide variety of quite specific benefits that flow from the trust, reciprocity, information, and cooperation associated with social networks. Social capital creates value for the people who are connected and-at least sometimes-for bystanders as well. http://www.bowlingalone.com/index.php3
Social Desirability Bias: The tendency of the responder to give what he or she considers the socially acceptable answer, whether or not it is accurate.
Socioeconomic Status (SES): A measure constructed out of one’s education, income, and occupational status.
Source Cue: An endorser or message source that affects how people make a judgment, often without diligent thinking on the issue-relevant argument in a relatively objective manner (see peripheral cue). A source cue, therefore, is a more important determinant of attitudes when the people are less involved in the message.
Spearman's Rho: A measure of the linear relationship between two variables. It differs from Pearson's correlation only in that the computations are done after the numbers are converted to ranks. When converting to ranks, the smallest value on X becomes a rank of 1, etc. Consider the following X-Y pairs:
Converting these to ranks would result in the following:
The first value of X (which was a 7) is converted into a 2 because 7 is the second lowest value of X. The X value of 5 is converted into a 1 since it is the lowest. Spearman's rho can be computed with the formula for Pearson's r using the ranked data. For this example, Spearman's rho = 0.60 Spearman's rho is an example of a "rank-randomization" test.
Split Ballot: A procedure in which a sample is divided into two halves, and each half receives a slightly different questionnaire. The use of the split-ballot technique enables the comparison of results received using different versions of the same questions (see split sample, below).
Split Sample: Different parts of the sample are sometimes asked different questions in the same place in a survey. Generally this is done either to test the effect of some difference in question wording about the same topic, to avoid respondent fatigue in answering two long questions with multiple items, or simply to make it possible to ask more questions in the same survey.
SPSS—originally called Statistical Package for the Social Sciences, this software system was developed in at Stanford University in 1968 by Norman H. Nie, C. Hadlai (Tex) Hull, and Dale H. Bent for the purpose of analyzing “volumes of social science data gathered through various methods of research.” Now produced and distributed by SPSS Inc., SPSS continues to be widely used by social scientists, including public opinion researchers, to analyze the results of surveys and polls.
Standard Deviation: Ameasure of variability (or dispersion) of a distribution equal to the square root of the variance. See Standard Error.
Standard Error: A measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. See Standard Deviation. (Source: http://nces.ed.gov/surveys/frss/publications/92130/7.asp)
Statistical Significance: Statistical measures are used to test hypotheses that two (or more) estimates are really different from one another or that the estimate is really different from zero-that is, that the differences obtained in the survey are not the result of chance variation. When the outcome of a statistical test has statistical significance, the investigator is willing to say that the estimated differences between two groups (or example, in the percent supporting some policy) are real and not chance differences. Statistical significance is usually stated as being at some level-for example, at the 95 or 99 percent level (paraphrased from Bradburn and Sudman, 1988).
Stratified Probability Design: Where the population embraces a number of distinct categories, the frame can be organized by these categories into separate strata or demographics . In random sampling, also known as probability sampling, every combination of items from the frame, or stratum, has a known probability of occurring, but these probabilities are not necessarily equal. (Source:http://en.wikipedia.org/wiki/Sampling_(statistics)#Stratified_sampling)
Stratified Sample: The division of the total population into subgroups (strata), some of which are sampled at higher proportions. Major gains in efficiency (either lower sample sizes or higher precision) can be achieved by varying the sampling fraction from stratum to stratum. This is done to reduce the sampling error for subgroups of interests, such as African-American women or voters over age seventy. Sampling is done randomly within each stratum.
Subgroup Analysis: An examination of the responses or characteristics of distinct categories of respondents, which might be defined demographically, e.g., age (18-29, 30-49, 50+), race (white, African American), or gender (men, women), or by responses to other classifying questions in the survey, such as party identification or religion. This technique is often used in order to determine if responses given by mutually exclusive groups in the population differ.
Telephone Attempt Disposition: The outcome of each call attempt in conducting a survey, for example, whether the call resulted in a completed interview, a refusal, a callback, and so forth.
Tocqueville, Alexis de (1805-59): French historian and author of Democracy in America, a penetrating study of the American polity.
Total Design Method (TDM): A method of ensuring high-quality telephone and mail surveys at a low cost. (TDM is described in Don A. Dillman, Mail and Telephone Surveys: The Total Design Method, 1978.) The following description is quoted from: http://www.sesrc.wsu.edu/sesrcsite/methodologies/tdm.html
The theory underlying the TDM is Social Exchange, which suggests the likelihood that individuals will respond to a survey questionnaire is a function of how much effort is required to respond, and what they feel they are likely to get in exchange for completing the questionnaire. The basic elements and procedures of the TDM are:
- Minimize the burden on the respondent by designing questionnaires that are attractive in appearance and easy to complete; printing mail questionnaires in booklet format; placing personal questions at the end; creating a vertical flow of questions; and creating sections of questions based on their content.
- Personalize all communication with the respondent by printing letters and envelopes individually, using blue ball point pens for signatures and a first class stamp on outgoing and return envelopes; and constructing a persuasive letter.
- Provide information about the survey in a cover letter to respondents, interviewers, and clerical personnel. If possible, also send out letters in advance informing respondents that a survey is forthcoming.
- Follow-up contacts of non-respondents is essential:
For telephone: conduct between 3 to 8 call backs make refusal conversion attempts make appointments with respondents
For mail: postcard mailing one week after first mailing 2nd mailing to non-respondents two weeks later telephone follow-up if necessary
Topline: A document showing the overall responses (frequencies) for each question in a survey.
Tracking Poll: A series of one-day polls used to track changes in public opinion over the course of a specified period, generally the final weeks of a political campaign. The daily samples are combined over several days into a “rolling sample,” with the most recent day's interviews added to the sample and one earlier day's interviews dropped from it.
Treatment Effect: “In the language of experiments, a treatment is something done to a person that might have an effect. In the absence of experiments, discerning the effect of a treatment like a college education or a job training program can be clouded by the fact that the person made the choice to be treated. The outcomes are a combined result of the person's propensity to choose the treatment, and the effects of the treatment itself. Measuring the treatment's effect while screening out the effects of the person's propensity to choose it is the classic treatment effects problem.
“A standard way to do this is to regress the outcome on other predictors that do not vary with time, as well as whether the person took the treatment or not.”
Trial Heat Question (sometimes called “horserace” question): A question that asks respondents to choose between two or more candidates for a party’s nomination or for an elective office. Typically, such questions are worded, “If the election were held today, would you vote for x or y?”
T-Test, Independent Samples: A statistical test that compares the mean scores of two mutually exclusive groups to see whether or not they differ.
Type I and Type II Errors: "When a researcher decides whether to reject a null hypothesis, two types of errors can be made. First, a true null hypothesis may be rejected by mistake. Falsely rejecting a true null hypothesis is known as a Type I error. Second, a null hypothesis may be accepted when it is false. Accepting a false null hypothesis is known as a Type II error." (From Herbert F. Weisberg, Jon A. Krosnick, and Bruce D. Bowen, An Introduction to Survey Research, Polling, and Data Analysis.)
Upweighting: A method used to make survey results more representative of the population in question (see weighting). Certain groups of the population (e.g., people with low incomes) are difficult to reach and therefore likely to be underrepresented. To compensate, responses from people in underrepresented groups are given greater weight (mathematically) than those of overrepresented groups.
Variance: A measure of variability (or dispersion) of a distribution equal to the mean of the squared deviations of all values from the mean. (Source: http://www.esomar.org/web/show/id=137136)
Voter News Service (VNS): A consortium of the national television networks and major newspapers that conducted exit polls in state and national elections between 1994 and 2002.
Voter Research and Surveys (VRS): A consortium of the national television networks and major newspapers that conducted exit polls in state and national elections between 1990 and 1992.
Weighting: The adjustment of sample results to account for sampling procedures and possible sample biases caused by non-cooperation and incomplete data. Weighting assumes that universe estimates are available from the U.S. Census Bureau or elsewhere.
Working Block: A common way to generate telephone numbers for random-digit-dial (RDD) samples is to randomize the last two digits of a possibly working number. For instance, starting with area code (617), exchange (432), and working block (20), one could randomly select the last two digits (xx): 617-432-20xx, where xx could then be any number between 00 and 99.