Testing Task Force Quarterly Update

This article originally appeared in The Bar Examiner print edition, Fall 2018 (Vol. 87, No. 3), pp 46–47.

For the latest Testing Task Force news—and to sign up to receive updates with the latest Task Force news, research, and blog posts—visit www.testingtaskforce.org.

The Testing Task Force has selected two independent research consulting firms—ACS Ventures, LLC, and American Institutes for Research (AIR)—to support its comprehensive, future-focused study of the bar examination. Although a detailed research plan is still being developed in consultation with ACS and AIR, broadly speaking the study will proceed in phases, with each successive phase building on the previous ones.

Initially, the study will focus on assessing the current bar examination and gathering stakeholder feedback both about the bar examination and about changes and trends taking place in the profession. The results of this research will help inform the next phase of the study, a future-­focused practice analysis to identify the job activities of newly licensed lawyers and the knowledge, skills, and abilities (KSAs) required to perform them. Finally, the study will build on the information gathered in the first two phases to explore the multitude of issues related to program design and test design. The goal is to develop a set of recommendations that are supported by empirical data and that take into consideration logistical and psychometric requirements along with stakeholder feedback.

The Testing Task Force aims to study the legal profession as it is practiced by new lawyers today and to anticipate what newly licensed lawyers of the relatively near future will need to know and be able to do. Although the Task Force’s study is limited to the bar examination, it must take into consideration recent changes, as well as potential future changes, to the profession and the regulation of the profession; such changes could affect the work performed by newly licensed lawyers, which in turn could affect what should be tested on the bar examination.

Regardless of possible changes to the content or format of the bar examination, the exam’s purpose remains the same: to determine whether individuals seeking a license to practice have the KSAs to practice safely and effectively. The Testing Task Force must consider what a bar examination that fulfills this purpose while serving a dynamic professional and regulatory landscape should look like.

One of the tasks of ACS and AIR will be to help the Task Force gather, analyze, and weigh the wealth of data, ideas, and opinions on these topics and others.

In the meantime, the Task Force has started gathering articles, studies, reports, and information about what others have identified as the necessary competencies for lawyers, both in the United States and in other common-law countries such as Canada, Australia, and the United Kingdom. The Task Force has also been looking at how other professions, such as medicine, dentistry, accounting, and architecture, have identified the competencies that are assessed on their licensing examinations. A reading list of selected research relevant to the Task Force’s study, and a summary of licensure and certification in various professions in the United States, are available on the Task Force’s website.

Ultimately, the Testing Task Force must come up with a set of recommendations for a bar examination that continues to satisfy the three foundational principles of testing—validity, reliability, and fairness—to support the needs of jurisdictions as they make the licensing decisions of the future.

Contact us to request a pdf file of the original article as it appeared in the print edition.