This article originally appeared in The Bar Examiner print edition, Winter 2024-2025 (Vol. 93, No. 4), pp. 21-23.By Kara McWilliams, PhD
The NextGen bar exam marks an important milestone for NCBE, not just in the form and content of the exam and the move to a fully digital format but in how we approach assessment development. Among the transformative changes NCBE has introduced is the creation of a dedicated User Experience (UX) division, led by Jaclyn Stanislaus, NCBE’s Director of User Experience in Product Development. This division represents our commitment to making the NextGen bar exam not only rigorous and fair but also accessible, intuitive, and equitable for all test takers.
UX is a critical component of any assessment’s overall validity narrative. A seamless, well-designed experience eliminates barriers that might otherwise interfere with test takers’ ability to demonstrate their true competencies. By foregrounding UX, we enhance not only the usability of the exam but also the defensibility of the inferences drawn from candidate performance.
Why UX Matters to Validity
When validity pioneers such as Samuel Messick and Michael Kane introduced the multidimensional construct of validity1 and laid the foundation for defensible validity arguments,2 digital assessments weren’t part of the equation. Today, test takers interact with technology in ways unimaginable decades ago—navigating digital platforms, entering keycodes, or completing drag-and-drop tasks. These interactions have a profound impact on test-taker performance and the validity of our inferences.
As Messick and Kane emphasized, assessments should measure what they claim to measure, with minimal interference from the data-collection tool. Subpar UX in the form of clunky interfaces or inaccessible platforms introduces “construct-irrelevant variance,”3 shifting test takers’ efforts from demonstrating their skills to navigating flawed design. This not only undermines the fairness of the assessment but also jeopardizes its validity.
Practical Integration of UX
NCBE has taken a deliberate approach to integrating UX into our validity framework so that usability and accessibility are considered at every stage of development. By prioritizing test-taker experience, we are not just creating a better exam—we are reinforcing the assessment’s foundational validity.
Here are five key ways UX impacts validity:
- Cognitive Load Reduction: Poor UX wastes cognitive energy, skewing performance data and introducing errors. Intuitive designs ensure test takers can focus on exam content.
- Accessibility as Inclusivity: Beyond legal compliance, accessibility makes the exam fair for all, regardless of ability, internet access, or device setup.
- Construct Relevance: Good UX eliminates irrelevant barriers, ensuring the exam measures competency—not test takers’ ability to navigate poorly designed interfaces.
- Face Validity: Experience shapes test takers’ perceptions of fairness. A well-designed platform fosters trust and confidence in the exam process.
- Data Integrity: Clean, reliable data are essential for psychometric analysis. Poor UX creates noise, complicating efforts to draw meaningful inferences.
The Four UX Pillars
NCBE’s UX efforts are built on four key components:
- UX Design: This focuses on crafting seamless interactions that make the exam easy to navigate and intuitive to use. By prioritizing clarity and simplicity, we ensure test takers can focus on their performance rather than the platform.
- User Interface (UI) Design: This involves creating the platform’s visual and interactive elements. Thoughtful UI design ensures the exam is aesthetically pleasing and functional, guiding test takers effortlessly through their tasks.
- UX Writing: This emphasizes using clear, concise, and supportive language throughout the exam platform. Intuitive instructions, prompts, and error messages help reduce test-taker anxiety and offer a smooth experience from start to finish.
- UX Research (UXR): This involves gathering insights into how test takers interact with the platform. Through surveys, interviews, and focus groups, we identify pain points and areas for improvement, allowing continuous iteration and refinement.
Following the NextGen prototype exam, we conducted extensive UXR, including a survey of all examinees and interviews and focus groups with smaller groups of examinees. A comprehensive UXR report will be published, but this article provides some early highlights.
NextGen Prototype Exam Insights
The October 2024 prototype exam provided invaluable examinee feedback across 32 jurisdictions and 36 test sites. (Visit here for more information on the prototype exam participants.) The insights gained underscore the importance of our UX efforts. Some highlights from the 752 survey respondents include:
- Experience Ratings: 47% rated the prototype assessment as very good or excellent, and 41% rated their overall experience as good.
- Digital Comfort: 68% felt comfortable or very comfortable on the digital platform, and 54% found the experience slightly or much better than paper-based testing.
- Confidence: 69% were fairly or completely confident taking the bar exam digitally.
- Clarity and Usability: 73% found exam instructions clear or very clear, 67% rated response input as mostly or very intuitive, and 68% agreed time management was appropriate and clear.
- Future Support: 74% agreed or strongly agreed that they would like to use this prototype as the bar exam, and 68% were likely or very likely to recommend the NextGen exam to classmates.
Overall, examinees were very positive, with one examinee sharing: “As someone with major test anxiety for standardized testing, this exam was much more digestible and took a lot of stress off of me while taking. This exam seemed to be more accurate as far as actual law practice [and] welcoming to the practice.”
Conclusion
As we advance NextGen bar exam development, our commitment to UX will remain central to our mission. UX and usability are not secondary considerations—they are foundational to the validity of any modern assessment. By designing platforms that reduce cognitive load, enhance accessibility, and ensure fairness, we are building assessments that truly measure legal competency.
The current prototype results underscore the power of integrating UX into every stage of development. When examinees can trust the platform and focus solely on demonstrating their skills, the validity of the exam’s inferences is strengthened. This alignment of usability and validity upholds NCBE’s mission of supporting a competent, ethical, and diverse legal profession.
Notes
- Samuel Messick, “Validity,” in Robert L. Linn (ed.), Educational Measurement (3rd ed., American Council on Education and Macmillan, 1989) at 13–103. (Go back)
- Michael Kane, “Validation,” in Robert L. Brennan (ed.), Educational Measurement (4th ed., American Council on Education/Praeger, 2006), at 17–64. (Go back)
- Messick, supra note 1. (Go back)
Kara McWilliams, PhD, is Chief Product Officer for the National Conference of Bar Examiners.
About the Next Generation of the Bar Exam
Set to debut in July 2026, the NextGen bar exam will test a broad range of foundational lawyering skills, utilizing a focused set of clearly identified fundamental legal concepts and principles needed in today’s practice of law. The skills and concepts to be tested were developed through a multiyear, nationwide legal practice analysis, focused on the most important knowledge and skills for newly licensed lawyers. Designed to balance the skills and knowledge needed in litigation and transactional legal practice, the exam will reflect many of the key changes that law schools are making today. NCBE is committed to ensuring a systematic, transparent, and collaborative implementation process, informed by input from and participation by stakeholders, and guided by best practices and the professional standards for high-stakes testing. For more information, visit nextgenbarexam.ncbex.org/.
Contact us to request a pdf file of the original article as it appeared in the print edition.
