Innovative Methods to Assess Psychotherapy Practices (imAPP)


This project compares two methods of assessing the quality of cognitive behavioral therapy (CBT) that do not involve directly observing sessions: 1) adherence checklists embedded in clinical notes, and 2) rating the quality of worksheets that are completed with therapist guidance during sessions. It also examines whether ratings of worksheets completed on a mobile app are reliable and valid quality measures. This information can inform strategies to monitor and enhance CBT quality, which can ultimately improve the quality of care and clinical outcomes.

Study Start Date

April, 01 2017

Estimated Completion Date

December 2020


  • Behavioral: Cognitive Behavioral Therapy

Study ID

Palo Alto Veterans Institute for Research -- WIS0003AGG



Trial ID


Study Type


Trial Phase

Not Listed

Enrollment Quota



Palo Alto Veterans Institute for Research

Inclusion Criteria

    A. Clinician eligibility:
  • Private practice or employment at an agency at which the administration agrees to allow the recruitment and participation of their providers in research related activities
  • English Speaking
  • No anticipated plans to leave their current agency for at least 18 months
  • Willingness to allow their CBT sessions, worksheets, surveys, and interview data, and clinical notes to be used for research purposes
  • Carry a caseload that typically includes patients who experience depression, anxiety, or PTSD, with whom they regularly conduct individual therapy sessions and/or capacity to increase the proportion of such patients (e.g., clinic sees a substantial number of individuals with PTSD or depression)
  • Must be trained (worksheet or web-based training and consultation) or in training for Cognitive Processing Therapy (CPT) for PTSD or CBT for depression or anxiety that uses worksheets Does not include CBT-i Can include aspects (i.e., worksheets) of CBT for substance use as long as depression or anxiety is primary diagnoses
  • Must anticipate at least 3 eligible patients
  • Must be willing to record sessions and provide worksheets and symptom measures to the study
  • Must have computer and internet access
  • Must be willing to use a mobile app on a tablet or mobile device B. Patient eligibility:
  • Must be 18 yrs. of age or older
  • Experience one or more of the following (both a diagnosis and cut off score): Clinician diagnosis of primary PTSD (PTSD-Checklist-5 score of 33 or more) Depressive disorder (e.g., major depressive disorder, dysthymia
  • PHQ of 10 or above) Or an Anxiety Disorder (Beck Anxiety Disorder score of 22 or above)
  • Note that if a patient has a score close to the cut-off score, it's up to the therapist digression
  • Must be willing to allow the team to collect session recordings, measures, notes and worksheets
  • Must be able to read and write at a sixth-grade level or above
  • Able to participate in sessions conducted in English, or Spanish (if working with bilingual clinicians in community or private practice)
  • Must be willing to engage in CBT/CPT
  • Therapist considers the treatment with the individual patient to be "mostly" CBT or CPT Cannot have*:
  • Imminent risk of suicide or homicide (requiring hospitalization) that require immediate treatment In need of detoxification (can be enrolled when substance abuse treatment is not the primary treatment target) Active psychosis or manic episode unless well controlled by medication and not the primary focus of treatment Cognitive impairments that preclude any participation in therapy C. Administrator eligibility:
  • Must have a support or managerial/supervisory position at the clinic(s) from which A and B are being recruited from
  • Must also be willing to complete a packet of study measures/partake in an interview




18 Years and older

Accepts Healthy Volunteers

Accepts Healthy Volunteers

Study Locations and Contact Information (4)

Study Location Distance Name Phone Email
Massachusetts General Hospital - Chelsea, Massachusetts 4.5 miles Kaylie Patrick MPH 617-887-4063 None
NCPTSD VA Boston HCS - Boston, Massachusetts 5.4 miles Caroline Canale BA 857-364-3206
Aaron T Beck Psychopathology Research Center - Philadelphia, Pennsylvania 270.8 miles Nicholas Affrunti PhD 215-573-0209
VA Palo Alto Health Care System - Palo Alto, California 2,694.6 miles Shannon WiltseyStirman PhD 650-493-5000 SHANNON.WILTSEY-STIRMAN@VA.GOV provides clinical trial listings in an easy to view format. All clinical trial information is pulled directly from This website does not guarantee acceptance into any clinical trial, and is not responsible for adverse events that may be incurred from a clinical trial.