So now you’ve developed a research question and selected an appropriate methodological approach, the next challenge is deciding what exactly will be examined and how it can be observed.
Research questions often contain broad or abstract concepts such as power, democracy, trust, inequality, diplomatic influence, or national identity. Before these concepts can be studied systematically, the researcher must determine what they mean and what evidence would indicate their presence, absence, or variation.
This is the problem of measurement. Measurement is the process of connecting abstract concepts to observable evidence. Although measurement is particularly important in quantitative research, where concepts often need to be converted into variables, the same basic problem occurs in qualitative research.
A researcher studying national identity through speeches, for example, must still decide what they mean by national identity and what they will look for in the evidence.
A research question identifies what we want to understand. Measurement determines how the important concepts within that question can actually be examined. Poor measurement can undermine an otherwise strong research design because the evidence collected may not accurately represent the concept the researcher intended to study.
The measurement process can be divided into three broad stages:
Conceptualization - what do we mean?
The researcher defines the concept being studied and identifies its important characteristics or dimensions. For example, what exactly do we mean by power, democracy, or diplomatic influence?Operationalization - how will we observe it?
The researcher decides how the concept will be identified or measured using evidence. Diplomatic influence, for example, might be examined through leadership positions in international organisations, participation in negotiations, or the ability to shape particular outcomes.Assessing measurement - does the measure work?
Finally, the researcher evaluates whether the chosen measure actually captures the concept accurately and consistently. This involves questions of validity, reliability, and the limitations of the chosen indicators.
Good measurement does not mean that complex social phenomena can always be measured perfectly. Rather, the researcher should make clear what is being measured, how it is being measured, and why that measure provides a reasonable representation of the underlying concept.
Sampling
Once researchers have decided what they want to measure and how they will measure it, they must decide where the evidence will come from. In many research projects, it is impossible or unnecessary to examine every person, document, event, country, speech, or other possible source of evidence. Researchers therefore select a smaller number for study. This process is known as sampling.
Sampling involves selecting cases or observations from a larger population. The population is the complete group relevant to the research question, while the sample is the smaller group that will actually be studied. For example, a researcher interested in South Korean attitudes towards the United States might define the population as all South Korean adults but collect survey responses from 1,000 people. Similarly, a researcher examining diplomatic speeches might select 100 speeches from several thousand delivered during a particular period.
Sampling follows directly from measurement. Once researchers know what evidence could represent their concepts, they need to decide which evidence they will actually collect and analyse.
The way a sample is selected affects the conclusions that can be drawn from the research. A poorly chosen sample may provide a distorted picture of the wider population, while a carefully chosen sample can provide useful evidence even when only a relatively small proportion of the population is examined.
Sampling generally involves four decisions:
Define the population - what people, cases, documents, events, or observations are relevant to the research question?
Identify the sampling frame - what population can the researcher realistically access?
Choose a sampling method - how will cases be selected from that population?
Determine the sample size - how many cases or observations are required?
The appropriate sampling strategy depends on the research question, methodological approach, available evidence, and practical constraints. There is a difference between quantitative and qualitative sampling.
Quantitative research often seeks samples that allow researchers to make generalisations about a larger population.
Qualitative research may instead deliberately select particular cases because they provide detailed information or insight into the phenomenon being studied.
The central questions that examiners or readers will ask in sampling is simple: Why have these cases been selected, is the method appropriate, and what conclusions can reasonably be drawn from studying them?
Convenience sampling
Convenience sampling is a non-probability sampling method in which participants are selected because they are easy for the researcher to access. Unlike probability sampling, individuals are not randomly selected and not everyone in the population has an equal chance of being included.
Convenience sampling is particularly useful when researchers have limited time, resources, or access to participants. For example, a researcher studying university students might distribute a survey to students in their own classes or approach students on campus. The convenience sampling process follows five steps:
Define the population - identify the broader population relevant to the research.
Identify accessible participants - determine which members of the population can be easily reached.
Determine the sample size - decide how many participants are needed.
Recruit participants - invite available and willing individuals to participate.
Form the sample - use those who agree to participate as the final sample.
The main strength of convenience sampling is that it is quick, simple, and inexpensive. Its main limitation is selection bias. People who are easiest to reach may differ from the wider population, meaning that the sample may not be representative and the findings may be difficult to generalise.
Purposive sampling
Purposive sampling is a non-probability sampling method in which cases are deliberately selected because they are particularly relevant to the research question. Rather than selecting cases randomly, the researcher uses specific criteria to identify people, documents, events, countries, or other cases that can provide useful evidence.
Purposive sampling is especially common in qualitative research, where the aim is often to gain detailed understanding rather than to make statistical generalisations about a larger population. For example, a researcher studying diplomatic negotiations might interview officials who participated directly in those negotiations. A researcher examining changes in foreign policy might select speeches by particular political leaders during important periods. The purposive sampling process follows five steps:
Define the population - identify the broader group of cases relevant to the research.
Establish selection criteria - decide what characteristics make a case relevant to the research question.
Identify potential cases - find cases that meet those criteria.
Select the sample - choose the cases that are most appropriate and practical to study.
Justify the selection - explain clearly why these cases were chosen and how they help answer the research question.
The main strength of purposive sampling is that it allows researchers to focus their resources on the most relevant and informative cases. Its main limitation is that the researcher’s judgement influences which cases are included, creating the possibility of selection bias. Researchers should therefore make their selection criteria clear and justify their choices.
Snowball sampling
Snowball sampling is a non-probability sampling method in which existing participants help the researcher identify or recruit additional participants. The sample therefore develops through referrals, with one participant leading the researcher to others who may be relevant to the research question.
Snowball sampling is particularly useful when the population is difficult to identify, locate, or access. There may be no available list of potential participants, or membership of the relevant group may not be publicly known. It can also be useful where trust is important and an introduction from an existing participant makes access easier. For example, a researcher studying diplomatic negotiations might begin with one former diplomat who then introduces the researcher to other officials involved in the process. The snowball sampling process follows five steps:
Define the target population - identify the type of participants needed for the research.
Identify initial participants - find a small number of suitable participants who can provide a starting point.
Request referrals - ask participants to identify or introduce other people who meet the research criteria.
Expand the sample - continue recruiting through these referrals as the network develops.
Determine when to stop - end recruitment when the required sample has been reached or additional participants are providing little new information.
The main strength of snowball sampling is that it can provide access to populations that would otherwise be difficult to reach. Its main limitation is that participants tend to refer people within their own networks. The resulting sample may therefore overrepresent people with similar backgrounds, experiences, or views. Researchers should clearly explain how the snowball process began and recognise the possible effects of these networks on their findings.
Quota sampling
Quota sampling is a non-probability sampling method in which the researcher divides a population into relevant categories and selects a specified number of participants from each category. The aim is to ensure that important groups are included in the sample, without requiring participants to be selected randomly.
Quota sampling is particularly useful when researchers want their sample to reflect important characteristics of a population but cannot conduct probability sampling. For example, a researcher examining attitudes towards an alliance might establish quotas based on age, gender, region, or political orientation. If 30 percent of the population belongs to a particular age group, the researcher might aim for approximately 30 percent of the sample to come from that group. The quota sampling process follows five steps:
Define the population - identify the broader population relevant to the research.
Choose relevant characteristics - decide which characteristics, such as age, gender, region, or occupation, should be represented.
Establish quotas - determine how many participants are required from each category.
Recruit participants - select participants until the quota for each category has been reached.
Assess the sample - check whether the completed sample meets the established quotas and consider any remaining biases.
The main strength of quota sampling is that it ensures important groups are represented in the sample. It can also be faster and less expensive than probability sampling. Its main limitation is that individuals within each category are not randomly selected. The sample may therefore appear representative while still containing important selection biases.
Simple random sampling
Simple random sampling is a probability sampling method in which every member of the population has an equal chance of being selected. Participants are chosen randomly from the entire population rather than being selected according to particular characteristics or groups.
Simple random sampling is particularly useful when researchers have access to a complete list of the population and want to minimise selection bias. For example, a researcher studying the attitudes of students at a university might obtain a list of all enrolled students and randomly select 500 students to participate. The simple random sampling process follows five steps:
Define the population - identify the broader population relevant to the research.
Create a sampling frame - obtain a complete list of the individuals in the population.
Determine the sample size - decide how many individuals should be included in the study.
Randomly select participants - use a random method, such as a random number generator, to select individuals from the list.
Form the sample - use the selected individuals as the final sample for the research.
The main strength of simple random sampling is that every member of the population has an equal chance of selection, which helps reduce selection bias. Its main limitation is that researchers need access to a complete and accurate list of the population. It may also be impractical when the population is very large or geographically dispersed.
Stratified sampling
Stratified sampling is a probability sampling method in which the population is divided into relevant groups, or strata, and participants are randomly selected from each group. Unlike quota sampling, the selection of individuals within each group is random.
Stratified sampling is particularly useful when researchers want to ensure that important groups within a population are adequately represented. For example, a researcher examining attitudes towards an alliance might divide the population according to age, gender, region, or another relevant characteristic and then randomly select participants from each group. This can prevent smaller but important groups from being overlooked in the sample. The stratified sampling process follows five steps:
Define the population - identify the broader population relevant to the research.
Choose the strata - divide the population according to relevant characteristics, such as age, gender, region, or occupation.
Determine the sample for each stratum - decide how many participants should be selected from each group.
Randomly select participants - use random selection to choose participants within each stratum.
Combine the groups - bring the selected participants together to form the final sample.
The main strength of stratified sampling is that it combines random selection with representation of important groups. It can therefore produce a sample that better reflects differences within the population. Its main limitation is that researchers need sufficient information about the population beforehand to identify the strata and randomly select participants within them.
Cluster sampling
Cluster sampling is a probability sampling method in which the population is divided into naturally occurring groups, or clusters, and a number of these groups are randomly selected for study. Rather than randomly selecting individuals from across the entire population, the researcher first selects groups and then studies individuals within those groups.
Cluster sampling is particularly useful when the population is large or geographically dispersed, making it difficult or expensive to sample individuals directly. For example, a researcher studying university students across South Korea might treat universities as clusters, randomly select several universities, and then collect data from students within those universities. The cluster sampling process follows five steps:
Define the population - identify the broader population relevant to the research.
Identify the clusters - divide or identify the population according to naturally occurring groups, such as schools, universities, cities, or districts.
Randomly select clusters - choose a number of clusters using random selection.
Select participants - study either all individuals within the selected clusters or randomly select individuals from within them.
Combine the results - use the selected clusters to form the final sample and analyse the collected data.
The main strength of cluster sampling is that it can make research involving large or geographically dispersed populations more practical and less expensive. Its main limitation is that individuals within the same cluster may be similar to one another. If the selected clusters differ significantly from the wider population, the resulting sample may be less representative.
The guided academic paper
You should now apply measurement and sampling to your guided academic paper. First, work through the measurement process:
Conceptualize - define the main concepts in your research question.
Operationalize - decide how each concept will be observed or measured.
Assess - consider whether your measures are valid and reliable, and identify any limitations.
Next, define the population relevant to your research and choose an appropriate sampling method. Decide what your sample will include and explain why this approach is suitable for your research question. Record this in your research diary.
