Communication Institute for Online Scholarship
CommunicationInstitute for Online Scholarship

 
ComAbstracts ComAnalytics Electronic Journal of Communication ComVista COSMIC

About Measurement

This page supplies an orientation to the controlled dictionaries of classification terms used in COSMIC's descriptions of measures. Familiarity with these sets of terms can help you to find what you are looking for (introduction of a new measure, critique of an existing measure, revision of a measure, etc.) more easily. Before listing the terms, we provide a short overview of a few key ideas relevant to the development of scales and measures.

1. Key Ideas in the Theory of Measurement

Measurement is fundamental to advancing knowledge in every scientifically oriented field of inquiry, including human communication and journalism. Measurement is central to the evaluation of theory and, more generally, to the assessment of what's "out there" in the world: how often something is happening, or how much of something there is. Because so much depends on the quality and accuracy of measurement, this key activity is often controversial and always a primary point of concern in the critique of a research study. How good were the measures? Were they appropriate to the task? Were they reliable? Are they powerful enough to detect distinctions at the level required? Were they valid? Were they free of bias?

For all forms of measurement, the most important two concepts are validity and reliability. These are vital properties of every measurement procedure. No measurement procedure -- quantitative or qualitative -- can be said to be scientifically adequate if it is not valid and reliable.

Validity has several senses that are assessed in standard ways. At its most basic, validity refers to the accuracy and representativeness of the measurement procedure: On the face of it, does the measurement system seem to measure what it is said to measure? This is referred to as face validity. But beyond that, does the measure adequately encompass all of the phenomena it is intended to measure? Does the measure relate to other established measures as it should (referred to as concurrent validity and discriminant validity)?

Reliability also has several senses. Most commonly it refers to the stability of a measure over time. A measure of television viewing that classifies people as high users should, barring some intervention, continue to classify them as high users on subsequent occasions. The type of reliability is assessed by repeating the measurement at two points in time, not so close together that respondents could easily remember their initial responses, and not so far apart that respondents' might have grown into new people. This type of reliability is usually assessed with a test-retest reliability study.

Reliability can also refer to the extent to which the items in a multi-question assessment consistently reflect the quality that the test is designed to measure. This is called internal consistency reliability.

In observational studies or content analysis studies, reliability is assessed through analysis of the extent of agreement between multiple coders. This form of reliability is referred to as interjudge agreement.

All studies aiming to evaluate the validity and reliability of a measure are affected by the quality of the study design (e.g., by the representativeness and size of employed samples, and by the care exercised in controlling bias).

2. Classification Terms Used in the COSMIC Database

In addition to being coded with terms from the ComAbstracts database, records in the COSMIC database have additionally been coded with terms from two dictionaries specific to COSMIC's exclusive focus on measurement and research methodology to make it easier to find materials that are pertinent to different purposes. The first of these dictionaries comprise "MethodTerms", which are tags that help to standardize the focus of the resource as it pertains to the typical range of concerns in measurement and research methodology. Thus, a search for "Sampling-Technique" will retrieve all resources in COSMIC that report innovations or critiques in the area of methods of obtaining samples.

The second dictionary is comprised of "FocusTerms". These tags indicate which of the large divisions of theorical and research activity (e.g., Health communication, Political communication, Journalism, etc.) are pertinent for the resource.

These two dictionaries are presented below.

Dictionary of MethodTerms

Bibliometric analysis: a statistical evaluation of databases, books, articles or other textual documents used to identify trends, research collaborators and sources or to measure the weighted impacts of researchers or fields of study.

Bio-physical measures: measured biological-physiological changes over a period of time following the introduction of a stimulus.

Categorization scheme: groupings of individual pieces of data into categories designed to reduce the number of data components and reveal themes and patterns.

Checklist: a list of attributes of interest.

Content analysis: a qualitative examination undertaken to describe the presence and number of concepts, themes and terms; or the systematic, objective, and quantitative description of written, visual or oral content.

Diary: a written record of reflections or observations.

Dimensionality: an expression of interrelated attributes or variables fundamental to a whole (e.g., time, number, size).

Ethnographic technique: immersion in a group or situation being researched to facilitate systematic documentation of behaviors and interactions.

Generalizability: applying conclusions from a studied group or data set to a larger population.

Index: a type of questionnaire, usually using a "yes or no" response format, that are scored to indicate the respondent's degree of belief, feeling or attitude.

Indirect: observation of some attribute that can be measured that is taken to reflect an attribute that for some reason can't be measured directly (e.g., taking the number of people visiting a food distribution center as an indirect indicator of the degree of poverty in the area).

Internet: studies that incorporate research garnered from the internet or the effects of internet use.

Interview: person-to-person direct question and answer.

Inventory: a list of items and their definitions pertinent to a research inquiry.

Likert-type-scale: standard scale item response categories such as strongly disagree, disagree, neutral, agree, and strongly agree.

Longitudinal: a study design involving the (re)collection of data at different points in time.

Measurement-intro: introduction of a new measure.

Measurement-critique: critique of an existing measure.

Measurement-revision: revision of an existing measure, adapting it for the particulars of a study.

Measurement-technique: a unique approach to measurement that incorporates chosen scales and procedures.

Multi-method-research: using two or more sources of data or measures for a research study.

Q-methodology: the systematic description of subjectivity, opinion and qualitative aspects of human behavior using statistical techniques and often using card sorting.

Qualitative-method: analysis of non-numerical data, such as discourse analysis, case study, open-ended interview, participant observation and focus groups.

Quantitative-method: analysis of numeric data such as statistics and mathematical scores on questionnaires.

Reliability: reliability refers to the assessment of internal consistency, agreement among observers, or stability of measurement over a variety of conditions.

Sampling-technique: a specific process that determines how the subjects within the sample are chosen (common methods of selection may include probability, random, systematic, stratified, and convenience).

Scale: a type of composite measure that is composed of several items that have a logical or empirical structure.

Software: a computer program, such as R, SPSS, SAS, QUANAL, WORDSTAT, etc. that assists with the quantitative or qualitative analysis of data.

Statistical-technique: method for the statistical analysis of research data.

Survey: standardized questionnaire or interview used to collect data from subjects.

Test-retest: measures the consistency (reliability) of results when you repeat the same test on the same sample at different points in time.

Typology: a set of descriptive names (types) used when describing groups of people or other units in a sample displaying characteristic clusters of behaviors, attitudes or perceptions.

Unitization-scheme: used in content analysis to assign units of analysis to categories.

Unobtrusive: objective observation of study subjects' behavior without affecting it.

Validity: the degree of accurate measurement of the specific concept that the researcher is attempting to measure (often assessed in terms of predictive validity, discriminate validity, face validity, or convergent validity).

Dictionary of FocusTerms

Education: Related to learning, most often in the context of formal institutional instruction. Characteristic journal: Communication Education.

Religion: Related to spiritual experience or organized practices of religion. Characteristic journal: Journal of Media and Religion.

Law: Related to matters of civil regulation and enforcement, often in the context of media regulation. Characteristic journal: Communication, Law and Policy.

Critical theory/Cultural studies: Related to matters of societal bias and social justice. Characteristic journal: Critical Studies in Media Communication.

Digital and social media: Related to the Internet as a medium and to the direct and indirect impacts of the introduction of new hardware or software systems of digital connectivity such as cell phones or content hubs like Facebook. Characteristic journal: Computer Mediated Communication.

Gender: Related to differences in male/female communication practices. Characteristic journal: Women's Studies in Communication.

Health: Related to communication practices relevant to doctor/patient interaction, health information campaigns, or institutional practices in health organizations. Characteristic journal: Health Communication.

Intercultural/International: Related to the assessment of differences among communication practices across or within national boundaries. Characteristic journal: Journal of Intercultural Communication Research.

Interpersonal: Related to the assessment of psychological or interactional qualities of individuals or small social groups as they pertain to communication. Interpersonal scholarship is the broadest and most active area of the field, and highly cross contextual, spanning virtually all of the field's divisions. There is no single journal that serves as an exemplar for interpersonal research. In fact, the area is so basic, foundational, and ubiquitous that there is no disciplinary journal that contains the word "interpersonal" in its title, as is commonly the case when the field spawns a significant new area of activity and a set of journals to represent it.

Intimacy, marriage, and family: Related to psychological and interactional processes in intimate relationships. Characteristic journal: Journal of Family Communication.

Journalism: Related to the journalism profession, its practices, processes, problems, and impacts. Often focuses on newspaper journalism but has rapidly diversified with the digital revolution. Characteristic journal: Journalism Studies.

Language, conversation, and discourse: Related to areas of scholarship that examine practices of written and oral communication, such as conversation analysis and discourse analysis. Characteristic journal: Research on Language and Social Interaction.

Mass: Related to the practices of communication that seek to impact mass audiences for any of a variety of purposes including persuasion, entertainment, or information spread, and the evolution of relevant technologies. Characteristic journal: Mass Communication Research.

Organizational and business studies:

Related to communication practices in large groups, often for-profit or non-profit organizations with a business or public service purpose but also informal organizations. Characteristic journal: Management Communication Quarterly.

Persuasion:

Related to the assessment of social influence. Like the interpersonal area, persuasion research and theory stands as a cross-contextual foundation of the field, which means that persuasion research is to some degree apparent in all of the field's divisions. Like interpersonal, persuasion research is so central and so ubiquitous that there is no disciplinary journal that contains the word "persuasion" in its title as would be the case if persuasion were a sub-specialty of the field.

Political:

Related to political practices, processes, and institutions. Characteristic journal: Political Communication.

Public relations: Related to practices of image management, generally, but not always, for formal organizations. Characteristic journal: Public Relations Review.

Race Related to communication practices and problems as mediated across demographic boundaries of race and ethnicity. Characteristic journal: Howard Journal of Communication.

Radio, television, and film studies: Research specific to these media. Characteristic journal: Historical Journal of Film, Radio, and Television.

Research methodology: Contributions to research techiques and practices, methods of data analysis, innovations in experimental design and the like. Characteristic journal: Communication Methods and Measures.

Rhetoric, writing, argumentation, and public address: Related to classical and modern rhetorical theory, criticism, and practice, particularly as manifest in practices of writing, public speaking, and argumentation. Characteristic journal: Rhetoric Society Quarterly.

Science communication: Related to the communication of scientific information or the rhetorical aspects of science practices. Characteristic journal: Science communication.

Visual communication: Related to the analysis and critique of displays of visual information. Characteristic journal: Visual Communication.

Other: Other areas of research specialty with sufficient frequency to be visible. Examples include conflict, cognition, nonverbal, emotion, and deception.