Bend or Snap: Embracing or Banning ChatGPT and its Future in Legal Education
By Eveline Komrij
In the first month of 2023, unprecedented, powerful AI technology has been lighting up headlines and causing quite a stir in the educational community in the form of ChatGPT. ChatGPT is a powerful AI natural language processing (NLP) tool that can be used to generate original, “profoundly human-like” creative content.[1] As described by one professor—who asked ChatGPT to author a tweet in his voice—ChatGPT may be “the greatest cheating tool ever invented.”[2] As universities scramble to deal with this new powerful AI reality,[3] it is important to consider whether ChatGPT and similar language models that follow may simply be reduced to cheating tools. Some school districts have already banned ChatGPT, blocking it from their networks.[4] Select universities and individual professors have reverted to forcing students to write multi-hour, in-person essays longhand in blue books.[5] Alternatively, other professors have elected to embrace AI tools in their curriculum to combat cheating and teach students relevant technology skills they will need to be competitive in the labor market.[6] A chasm thus opens, leading to educational incongruencies between institutions that choose to fight the seemingly inevitable, and those who choose to embrace and harness its power. After all, built-in grammar and spelling checks—features now taken for granted—were once novel AI tools also fought or embraced.[7]
In the law school realm specifically, several scholars utilized ChatGPT in a study testing ChatGPT’s performance on four law school exams at a highly-selective law school.[8] These exams were graded against those of real students on a typical law school curve.[9] While ChatGPT did finish in the bottom of almost every class—earning a C+ average—scholars predicted its scores were sufficient to earn a J.D. from this highly-selective law school.[10] The study demonstrated that ChatGPT in its current state struggled to issue spot, a critical skill on most law school exams.[11] ChatGPT also struggled with critical, in-depth analysis, failed to go into sufficient factual detail, and often refused to render a probable conclusion.[12] This demonstrated lack of critical lawyering skills illustrates that ChatGPT needs human intervention and revision if it is to be implemented successfully and meaningfully in content generation. Law school exams typically emphasize and allocate more exam points toward critical analysis[13] over legal rule regurgitation, and many exams are already administered open-note or open-book style. Thus, law school professors should have few concerns regarding ChatGPT and extremely successful cheating at this time.
Law schools may choose to reshape their honor codes around these language models, restructure exams, and allow less technology in classrooms and while taking exams.[14] Alternatively, and perhaps more wisely, law schools should embrace this new technology that lawyers will undoubtedly leverage in their careers post-law school.[15] Instead of resisting change and envisioning a grim future filled with laziness and cheating, law schools can instead choose to prepare students to utilize these tools in practice while still emphasizing the importance of student’s honing their own research and writing skills.[16] After all, a C+ average may earn a J.D., but it is highly unlikely to impress any legal employer, nor would it be considered quality work product. Students should learn to leverage this technology while also obtaining the requisite legal training and knowledge to check it, filling any gaps and adding their own analysis where necessary. As one professor stated, “ChatGPT can create David . . . referring to the famous Michelangelo statue. ‘But his head is too big and his legs are too short . . . it’s our job to interrogate the evidence and improve on what it gives us.’”[17]
Redesigning law school curricula to embrace new technology—like ChatGPT—gives both students and law schools a competitive leg up. Society is not technologically regressing, and ChatGPT has the potential to contribute to more efficient, lower-cost client service. As we begin to see more alternative legal service providers and experience some shift away from the traditional firm model, embracing this technology becomes the future of law schools and the legal service industry.
Eveline Komrij is a J.D. Candidate in Vanderbilt Law School’s Class of 2024. She studied business administration, concentrating in business analytics, and plans to work in corporate transactional law in New York.
You can download a copy of Eveline’s post here.
[1] Clare Williams, Hype, or the future of learning and teaching? 3 Limits to Ai’s ability to write student essays, LSE Impact Blog, (Jan. 12, 2023), https://blogs.lse.ac.uk/impactofsocialsciences/2023/01/12/hype-or-the-future-of-teaching-and-learning-3-limits-to-ais-ability-to-write-students-essays/.
[2] Douglas Belkin, Professors Turn to Chat GPT to Teach Students a Lesson, Wall St. J. (Jan. 25, 2023, 10:00 AM), https://www.wsj.com/articles/professors-turn-to-chatgpt-to-teach-students-a-lesson-11674657460.
[3] Id.
[4] Id.
[5] Id.
[6] Id.
[7] Williams, supra note 1.
[8] Jonathan H. Choi, Kristin E. Hickman, Amy B. Monahan & Daniel Schwarcz, Chat GPT Goes to Law School 2 (Jan. 23, 2023), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335905.
[9] Id. at 3.
[10] Id. at 5, 11.
[11] Id. at 9.
[12] Id. at 10.
[13] Williams, supra note 1.
[14] Choi et al., supra note 8, at 12.
[15] Id. at 13
[16] Id.
[17] Belkin, supra note 2.