Decoding FDA DTC Policy — Part 2: What are user comprehension studies? And are pharmacogenic test report requirements different than previous FDA regulations?

by Scott D. Crawford, MA; Shawn Fayer, MSc, MS; and Robert C. Green, MD, MPH

This is the second in a five-part series that considers recent FDA actions on issues related to direct-to-consumer (DTC) genetic testing. The original posts appeared on the SoundRocket LaunchPad blog. Read Part 1 of the series here.

What Are User Comprehension Studies and Why are they Required by the FDA?

Unlike tests that are overseen by health care professionals where a conversation may take place about the meaning of results, direct-to-consumer tests bypass the traditional patient-doctor relationship and deliver results directly to patients (or consumers). A well-known example of a direct-to-consumer test is a pregnancy test that one may purchase at any pharmacy or similar retail store.

The FDA is charged with identifying and reducing risks to consumers with such direct-to- consumer tests. To do that, they require test manufacturers to conduct “controls” or “special controls” prior to marketing a test for a specific purpose — all designed to reduce risk. One common special control is the user comprehension study. Such studies are designed to ensure that the information being presented in the test report is done so in a way such that the average consumer understands and interprets the results accurately.

User comprehension studies are defined as a special control in all FDA DTC authorization processes. Most recently, in the Genetic Health Risk (GHR) protocol, the FDA identifies risks to the user of obtaining these reports as including “incorrect understanding of the device and test system” and “incorrect interpretation of the test results.” To help mitigate these risks, they propose special controls for the test manufacturer to take — Special Control #3 focuses on the comprehension of the test results.

Specifically, they require that naive, untrained users must be used to conduct comprehension testing. The testing must include sufficient numbers of cases and represent diversity of age and education level.

The user comprehension testing must evaluate a representative sample of the contents that are generated as the report, which must include a public facing website including test details, definition of terms, a pre-purchase page, a frequently asked questions page, technical details, as well as the test report itself.

The GHR authorization specifies that participants must be evaluated to ensure that they are capturing 90% comprehension on items in the following five domains (topics):

• Test limitations
• Test purpose
• Appropriate actions (next steps)
• Other factors that may have an impact on the test results
• Test results

According to the FDA, the details of what concepts should be included in the report within these domains is left to a “physician and/or genetic counselor that identifies the appropriate general and variant-specific concepts contained within the material tested in the user comprehension study.”

For example — the purpose of the test is a domain that the FDA requires. However, they do not indicate what the specific purpose for the test should be. Using 23andMe’s publicly presented example report for Parkinson’s Disease, we can find several pieces of information that describe the purpose of the test. The concepts of two genetic variants, increased risk of developing the condition, as well as specific gene variants that the test looks for are examples of content that relate to the test purpose. The decision to include these concepts (and not others) was made, presumably, by a physician and/or genetic counselor who understands how to communicate these kinds of results to patients.

The FDA leaves the specific questions and methods for how to capture comprehension among these domains to the professionals (presumably to the expert questionnaire designers/researchers) involved in designing the study. However, they do provide some details that they feel are important things to consider in the design. We recommend you talk with someone who has experience in such designs to learn more about these details. However, if you would like to drink it directly from the FDA fire hose, most of these details above can be found on pages 20–22 in the GHR DEN160026 Reclassification Order.

User Comprehension for Personal Genome Service Pharmacogenetic Reports (PGSPR)

This brings us to the recent authorization for 23andMe’s PGSPR. There is no difference in primary concept to the previous GHR protocol described above — however, one change in language does show up that may have an impact on those involved in user comprehension studies for these new reports.

Where the GHR authorization defines the domains of the special control user comprehension study (limitations, purpose, next steps, other factors, and results), the language used in the PGSPR authorization around user comprehension is that it must cover “all result comprehension concepts that are critical for safe use of the device.” They do not specifically reference the domains of comprehension previously used in the GHR protocol.

It is too early to tell what this may actually mean. More details may be revealed once the FDA Decision Summary on the PGSPR is released. The Decision Summary is a follow-up document that comes after an initial authorization and often includes more complete details that relate to the specific authorization given to 23andMe.

Scott D. Crawford, MA is an entrepreneur and founder of SoundRocket (on twitter @SoundRocket), a social science survey research firm located in Ann Arbor, Michigan. He is trained as a Survey Methodologist (University of Michigan, 2000) and has been involved in user comprehension research (among other fields) for government, academic and commercial customers since 2000. In recent years, he has been involved in several user comprehension studies implemented to support DTC genetic test submissions to the FDA.

Shawn Fayer, MSc, MS, CGC is a certified genetic counselor with many years of human genetics research experience. He received his genetic counseling training at Brandeis University and worked in the Adult Genetics Clinic at Brigham and Women’s Hospital in Boston, Massachusetts for two years. Mr. Fayer also worked as the project manager for the BabySeq Project, an NIH-funded randomized trial of whole exome sequencing in the newborn population. Mr. Fayer is currently a PhD Candidate at University of Washington.

Robert C. Green, MD, MPH is a medical geneticist and Professor of Medicine at Harvard Medical School, and directs the Genomes2People Research Program in translational genomics and health outcomes at Brigham and Women’s Hospital and Broad Institute. He conducts empirical research on the medical, behavioral and economic outcomes around the implementation of genomic medicine. Dr. Green is directing some of the first trials to explore sequencing in adults (the MedSeq Project) newborn infants (the BabySeq Project) and active duty military personnel (the MilSeq Project). Follow him on twitter @RobertCGreen.

The contents of this post are not intended as, nor should be considered legal, regulatory, health, or any other form of advice for any specific situations — they simply represent the opinions of these authors.