Skip to Content Top

A Journey from Socratic Dialogue to AI-Generated Exams: On Human Reasoning in the Age of Artificial Intelligence

Written by Sally Vazquez-Castellanos, Esq. Wife, Mother, Shareholder, Attorney, Editor and Author of Perspectives. Written with the assistance of ChatGPT, grounded in life experience and critical reflection.

You should know that I used ChatGPT to write this post. It’s not because I don’t have anything to say, but because this, in a way, is something of a test on innovation. Technology can be a rather useful tool, but we all must be wary of the tidings that stem from the world of innovation and the people that use these products and services. Part of what makes transparency notable is that we can begin to see that at the heart of what we do requires a human touch.

Ordinarily, we cannot know nor control the actions or intentions of others, but we certainly can begin to test the efficacy and the boundaries of technological innovation. This post tells part of a life story. Some of my educational journey. It’s mine. It’s shaped by years of struggle that began in elementary school, middle school, and high school. It includes two Puerto Rican parents who worked hard to try to ensure that I got a fair shot in life. It also sends a message to you. Use it well.

Never the stellar student in any regard, I spent years dedicated towards building an education at NYU, Suffolk University Law School, Southwestern Law School, Pepperdine’s Straus Institute for Dispute Resolution and Caruso School of Law. All of those years, while not perfect, culminated with the emotional and intellectual weight of sitting for-and not passing-the California Bar Exam on my first try.

I think it’s a tale that resonates with many applicants. It may also resonate with ordinary folks-and so we begin…

Introduction

At NYU’s Gallatin School of Individualized Study, I was taught not to simply memorize facts, but to learn to synthesize ideas and to think critically. The interdisciplinary nature of my concentration allowed me to combine philosophy, ethics, history, and classical legal theory into a coherent intellectual fabric. This provides the foundation for analytical thought.

My studies at NYU culminated not on a Scantron sheet, the prevailing technology of my youth, but in a conversation-an oral defense before a faculty panel where I articulated and defended the ideas that shaped my education. This was my academic capstone: not a test of what I knew, but a unique test of how I thought.

Years later, I found myself on the other side of the country, preparing for the California Bar Exam. I approached it with the same intellectual fervor and dedication. But the bar exam is not a conversation–it’s a closed-book, high-stakes, high-pressure endurance test, which lasted several days- three days to be exact. With miniature Yoda on my desk, gifted to me by my parents, I passed the exam on my fourth attempt.

This made me consider the 2025 administration of the California Bar Exam, where the experience for many hopeful applicants was not just an academic challenge, but which reportedly resulted in a huge technological failure and systemic breakdown.

The Gallatin Experience: Embracing Individualized Learning

At NYU’s Gallatin Division, each student crafts their own academic concentration. Mine was anchored in the philosophy of justice, law, business, entertainment and media. The flexibility of Gallatin’s program allowed me to study across departments-from legal theory in the General Studies Program at the School of Continuing Education to Film Theory and Production at Tisch School of the Arts.

I eagerly pursued internships and gainful employment with leading companies in the field of entertainment. To this day, I am still proud of those incremental steps towards building a career. I may not have been top of the class. I was not Harvard. I was not Yale. I was not Princeton. I was awkward. I was unusual. I was never comfortable in an academic setting. But I always managed to reinvent myself. Somehow, that always seemed to be enough.

The interdisciplinary approach towards my education culminated in a thesis of sorts: a reflective, oral defense where I presented a curated list of texts, responded to faculty questions, and demonstrated the coherence and depth of my academic journey. It’s the same journey where Professor Marc Crawford pushed me to confront the many unique challenges often characterized by the Nuyorican experience-caught between cultures, languages, and histories.

The California Bar Exam: Bias and Some of the Challenges of Innovation in the Legal Community

According to an article I recently read in the Daily Journal, widespread technological failures prevented thousands of bar applicants from completing the exam and ended up disrupting the entire process. University of California, Berkeley, Law School Dean Erwin Chemerinsky reportedly denounced the February 2025 bar exam as “stunning incompetence” and urged the California Supreme Court to intervene.

The controversy centers around an admission made by the California State Bar that they used AI-generated questions on the exam. As I engaged more deeply with Chat-GPT, somehow the same concerns come to fore. That time old tale of a lack of equality deeply embedded within the exam. AI-generated content replete with inaccuracies and bias. In a country with a history of slavery and still reckoning with systemic inequality, some wonder whether these exams will be used as a tool for exclusion and discrimination in all forms.

It’s also been reported that bar applicants were confronted with a number of problems with the administration of the exam, including issues with the vendor used to administer the exam. This situation has not only sparked debate, but there’s discussion of a federal class-action lawsuit and legislative inquiry into the matter.

Comparative Analysis: California’s Technological Leap vs. New York’s Traditional Approach

As California copes with the path forward after a reported failure using artificial intelligence in the context of the California Bar Examination, I thought I would consider the use of technology in the most recent New York State Bar Examination. California and New York administer the toughest bar exams in the nation. So it does seem appropriate.

New York is a state that launched the Excelsior Pass during the Covid crisis, which was the first government-issued digital vaccine passport. Reuters reported that New York is adopting a more technologically integrated approach towards taking the bar exam. So what does the Excelsior Pass have to do with administration of bar exams?

Technology.

All of this is occurring while a New York State Bar vendor named ExamSoft, recently settled a $2.1 million settlement in 2015, due to problems applicants had with uploading their answers. Meanwhile, as of July 28, 2023, the Excelsior Pass program was apparently discontinued due to decreased demand following Covid-but again I digress.

According to Reuters, over 167 million individuals had their healthcare data compromised in 2023, which is reportedly a substantial increase from previous years. According to the Federal Trade Commission, the agency has intensified its oversight of mobile applications, which includes health-related apps, in response to allegations about unauthorized data sharing and privacy violations.

If any of the information provided is inaccurate or if you wish to offer clarification or additional insight, please feel free to share your thoughts in a constructive and respectful manner.

We are living in an age of increased threats. Perhaps this is important information for all of us to consider as society rushes towards innovation, which includes the adoption of artificial intelligence. For most of us, we simply have no choice but to adapt to these products and services. The reality is that technology does help simplify things in many ways. Smartphones are necessary communication tools used in government, business, education, and in our daily lives. We are all being forced to implement these products and services-but we also must consider the risks.

Governments and social media platforms are confronting national security challenges dealing with international commerce and trade. Businesses are learning to build privacy programs and conduct data impact assessments to protect consumer data. We are all dealing with government compliance and cybersecurity measures to help the nation address complex global challenges.

We have all heard the term Surveillance Capitalism coined by Harvard professor Shoshana Zuboff where she describes a new economic system where personal data is harvested without our consent. That data is then used to exploit, objectify, predict, influence and monetize human behavior.

It’s hard not to feel like we’re living in a Brave New World.

What Gallatin Got Right and Why It Matters Here

What Gallatin taught me-through oral defense and an individualized study program is that the ability to reason, adapt, and communicate is not only at the heart of professional competence but it is at the heart of what it means to be human. I am not so sure I want to turn all of these skills over to a machine. That does not mean that innovation is necessarily a bad thing but it does mean that maybe we need to reevaluate our approach because moving fast without those guardrails in place may not always be the wisest course of action. In the context of some professions, the dangers of not understanding how to use the technology could lead to even greater problems or misuse.

Conclusion

So yes, this post was written with the help of AI. But it’s grounded in real experience: in the intellectual freedom of NYU, in the frustration of a failed exam, and in the belief that legal licensure should be about more than test scores and algorithms. While that may be difficult in the context of a state bar examination, I do think it tells us something about the application of technology in the legal profession.

I wish all of the applicants the best of luck going forward.

Citation: The Little Engine That Could

Author Biography:

My legal journey began in entertainment, serving in-house at FOX and DMX Music, where I supported commercial sales marketing teams and business and legal affairs executives. Over the years, I developed a working command of privacy and data protection law, copyright and trademark law, commercial agreements, and business transactions.

In recent years, I expanded my focus to include digital privacy law and child safety in online environments-issues I consider urgent and foundational in our evolving legal landscape. My published work explores the implications of GDPR, CCPA, and COPPA, as well as national security law and the emerging legal frameworks needed to safeguard children’s mental and emotional well-being in digital spaces.

If you would like to learn more about this attorney/author, please visit the following websites to learn more about the Editor and Author of Perspectives: Technology, Privacy and Global Data Protection Law and It’s Personal: Children, Privacy, Technology and the Law.