Feature Article

Data Analysis and the IRB: A Guide for BUMC Investigators

By Don Allensworth-Davies, MSc
Statistical Manager, Data Coordinating Center, BUSPH
IRB Reviewer, Panel Blue
Issue: March, 2006

Author has nothing to disclose with regards to commercial support.

Educational Objectives:

• Explain how data analysis relates to human subjects' protection
• Describe the six components that the IRB reviews in a data analysis plan
• Describe the two different ways of justifying sample size on the IRB application
• Identify some common study designs that require special analytic methods

PRINT | CLOSE WINDOW

• Introduction
• Data Analysis and Human Subjects' Protection
• What the IRB Considers in Reviewing an Analysis Plan
1. A sample size that is large enough to provide sufficient data without placing more subjects at risk than is necessary
2. An appropriate justification for the sample size
3. A description of the specific variables that will be analyzed
4. A description of the comparisons that will be made
5. Statistical methods that are appropriate for the study design and the type of information collected
6. How the investigator will know from the analysis whether the research objectives have been met
• Characteristics of a Good Analysis Plan
• Quiz

Introduction

“To write it, it took three months; to conceive it three minutes; to collect the data in it — all my life.” F. Scott Fitzgerald, referring to his novel This Side of Paradise in a letter to the Booksellers’ Convention, April 1920.

All research studies begin with an idea. The idea might be possible risk factors for disease scrawled on the back of an envelope or a series of research hypotheses that evolve over several years. Once conceived, a study protocol is then written either as part of a grant application, an IRB application, or both. Finally, IRB approval is obtained and data collection begins. While, hopefully, your study will not require a lifetime of data collection, having a well-designed study and analysis plan is an important part of the research procedures. The purpose of this article is to describe what the IRB considers in reviewing a data analysis plan (i.e., Section G of the BUMC IRB application) and tips for preparing the analysis plan for your study.

Data Analysis and Human Subjects' Protection

Federal regulations define research as a “systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” (45 CFR 46.102(d)). Analysis is the means by which individual data points are summarized to contribute to scientific knowledge and to benefit society.

Federal criteria for IRB approval of research mandate that “risks to subjects are minimized by using procedures which are consistent with sound research design and which do not unnecessarily expose subjects to risk” (45 CFR 46.111). Often investigators do not realize that the research procedures include the data analysis. Conducting an analysis that is inappropriate for the study design or for the type of data collected can bias the results of the research. While a small amount of bias may not affect the final study findings, a large amount can lead to incorrect conclusions. There are ethical implications of using an inappropriate study design or analysis. Both can result in bias that may lead to incorrect scientific conclusions of little benefit to either science or society, thereby exposing subjects to unnecessary risk. For more information on this topic see Weinberg and Kleinman, “Good Study Design and Analysis Plans as Features of Ethical Research with Humans”, IRB: Ethics and Human Research, September/October 2003.

What the IRB Considers in Reviewing an Analysis Plan

There are two questions that the IRB asks for each analysis plan: 1) Is the sample size adequate to answer the research question? And 2) Is the analysis plan clearly described and adequate to answer the research question? Inherent in these questions are six components:

1) A sample size that is large enough to provide sufficient data without placing more subjects at risk than is necessary
2) An appropriate justification for the sample size
3) A description of the specific variables that will be analyzed
4) A description of the comparisons that will be made
5) Statistical methods that are appropriate for the study design and the type of information collected
6) How the investigator will know whether the research objectives have been met

1) A sample size that is large enough to provide sufficient data without placing more subjects at risk than is necessary

Sample sizes for each study will be different depending on the study design and objectives. For example, to detect a small difference between one group of adults that receives a research counseling intervention and a second group that receives standard of care counseling, a hundred or more subjects may need to be enrolled. Conversely, in a laboratory study where cells are being observed, blood samples may only need to be obtained from a very small number of subjects.

The IRB also thinks about sample size differently than investigators. In addition to knowing how many subjects will be recruited locally at BUMC and how many worldwide, the IRB wants three types of subjects to be included in the sample size totals:

• Subjects who consented and participated until the end of the study
• Subjects who consented but withdrew, dropped out, or were terminated before the end of the study
• Subjects who were screened but not enrolled IF they were placed “at risk” during the screening process (e.g., blood was drawn or identifiable information was collected)

The same subject types that are included in your sample size totals should also be included in the annual Progress Report/ Final Report enrollment tables (Sections PR1 and PR2).

Sample Size Tips:

• The total sample size reported in the Progress Report must not be greater than the total approved by the IRB
• If you need to change your sample size, you must submit an amendment to the IRB in INSPIR describing the reason for the change
• The local sample size must always be equal to (if BUMC is the only study site) or less than (if BUMC is one of many study sites) the worldwide sample size
• If you are conducting research and having contact with human subjects in another country, then these subjects "belong to BUMC" and must be reported in both the local and worldwide sample sizes
• If data are being collected about both the subject and about another person through the subject, then both must be counted in the sample size (e.g., mother-child, partners’ or twins’ studies)
• If a study involves interviewing or surveying multiple groups (e.g., patients, physicians, and nurses) ALL SUBJECTS must be included in the sample size

2) An appropriate justification for the sample size

There are two ways that a sample size is considered by the IRB to be appropriately justified. The first, and preferred, method is a statistical justification. This may be through a power calculation or confidence interval approach appropriate to the study design to determine how many subjects will be needed to answer the research question(s). In describing the calculation in the IRB application, any assumptions that were made and the statistical parameters are included.

In some studies, the sample size may be based on feasibility. This justification may be based on:

the investigator’s knowledge of the number of subjects that can be realistically recruited;

Example: The investigator would like to study syphilis and recruit research subjects from an STD clinic that treats an average of 15 syphilis cases per year. The study is expected to enroll subjects for five years; the maximum feasible sample size is 75.

OR the resources available to the investigator to conduct the research.

Example #1: A funding agency requests an exploratory study before deciding whether to fund a larger study. The investigator receives enough funding to enroll 25 subjects.

Example #2: An investigator is conducting a sleep study and has three rooms equipped with sleep monitors available for two nights per week. A maximum of six subjects can be enrolled per week.

**IMPORTANT: Simply stating that the research is a pilot study is NOT adequate justification of sample size

3) A description of the specific variables that will be analyzed

Examples: sex, age, education, income.

4) A description of the comparisons that will be made

Examples: Comparing proportions between two groups, pre-/post- measurements, rates in exposed to rates in unexposed.

5) Statistical methods that are appropriate for the study design and the type of information collected

Investigators are strongly encouraged to consult with someone who has expertise in study design and statistical analysis to identify the methods that are most appropriate for the study. Factors such as whether the variables to be analyzed are nominal, ordinal or continuous, normally or non-normally distributed, and how much data are missing, will also determine which methods are best for your analysis.

Some study designs require special methods for analysis. Examples include:

• Longitudinal/case-crossover studies: methods for repeated measurements/pairs.
• Matched case-control studies (e.g., matching controls to cases based on age or gender): methods that take the matching into account.
• Focus groups and unstructured subject interviews: qualitative methods.

6) How the investigator will know whether the research objectives have been met

The IRB has reviewed very detailed analysis plans, with detailed description of the variables to be analyzed, elegant statistical methods, regression equations, and even graphs. Yet sometimes investigators so lose themselves in the methods that they forget to tell the IRB how each part of the analysis plan relates to the study objectives. How will the investigator know if the research question has been answered? Will this be determined by a difference between baseline and follow-up measures, a statistic, or a specific outcome such as subjects remaining free of disease? Organizing the analysis plan by the study objectives is one way to clearly communicate this and facilitate IRB review.

Example: Study Objective #1, Identify Demographic Differences in Unsafe Sexual Behavior: To test for differences in proportions between different groups classified by sex, age, education, and income, we will use chi-square or Fisher's exact tests (if data are sparse). Conventional criteria (p < 0.05) will be used to identify statistically significant differences.

Characteristics of a Good Analysis Plan

A good analysis plan includes information on what comparisons will be made to evaluate each study objective, the methods that will be used, and how the researcher will know if the objectives of the research are met. Ultimately, a well-designed analysis will reward the investigator by increasing the accuracy and validity of the research results while reducing bias.

To learn more about this topic, click here.

The author wishes to thank Christine E. Chaisson, MPH and Janice M. Weinberg, ScD for their editorial assistance with this article.

Top

Click here and close window if you are a BUMC researcher
and would like to take the quiz now.

Close Window