Module Components

• Learning Activities

Topics

• Sampling Overview
• Sampling for Quantitative
• Sampling for Qualitative
• Measurement
• Levels of Measurement
• Reliability and Validity of Measures
• Estimating Reliability and Validity
• Modue 3 Self-Review

NRSG 790 - Methods for Research and Evidence-Based Practice

Module 3: Research Methods

Measurement

Sound measurement principles and practices are an essential component of a well-designed research study, especially when the goal is to employ research results as a basis for utilizing evidence in practice. A major criterion for assessing quality in a study is the reliability of the instrument used to measure particular attributes. Validity is another important criterion when evaluating an instrument. As a reader and consumer of research one must always ask the following when critiquing a research study: (1) How consistently does the instrument measure what it measures? (Reliability) and (2) Does the instrument measure what it is supposed to measure? (Validity). The following topics will be reviewed in this module: Levels of Measurement of measurement, errors that impact reliability and validity, factors affecting reliability and validity, and estimating reliability and validity.

Measurement

• A process of assigning numbers to objects to represent the kind and/or amount of attributes or characteristics possessed by those objects.
• For example, a tape measure is a measurement rule that allows one to represent a person’s height,
• Thus, using a tape measure to quantify a person’s height is one type of measurement.

Instrumentation

• The process of selecting or developing data collection tools and methods appropriate for measuring an attribute or characteristic of interest.
• In research articles the heading “instrumentation”, is the section of the report where the instrumentation process and data collection methods are described.
• Ideally, a data collection method should measure the construct or concept of interest in a manner that is relevant, credible, accurate, truthful, and sensitive.

Reliability

• Refers to the consistency with which a tool or method assigns scores to subjects.
• For example, a tool or method is said to demonstrate evidence for reliability if it yields consistent results when used on two or more occasions. It is unreliable when if repeated different results are obtained.

Validity

• Refers to the determination of whether or not a tool or method is useful for the purpose for which it is intended, that is measures what it purports to measure.
• For example, if an instrument is employed to measure hopelessness, it is important to obtain evidence that resulting scores reflect the construct of hopelessness and not something else like depression.

Relationship between reliability and validity

• Reliability is a necessary prerequisite for validity.
• If a measure does not assign scores consistently, it cannot be useful for the purpose for which intended.
• Thus, a tool that is not  reliable cannot be valid.
• Reliability, however, is not a sufficient condition for validity. Because a measure consistently measures a concept does not ensure it measures the concept of interest.

In addition to instrument reliability and validity, attention needs to be given to reliability and validity of the measurement process per se. To increase the probability that the measurement process will yield reliable and valid information,

•  Whenever possible multiple tools or methods should be employed to measure any given variable and
• Information about any given variable should be obtained from a number of different perspectives or sources.

Accuracy

• Refers not only to the reliability of a measure but also encompasses aspects of validity
• For example, in the case of biophysiologic measures accuracy is evaluated by comparing test outcomes with a putative gold standard.

Precision

• Characterizes measurement tools that yield numerical scores that allow for the expression of finer detail in the observation