ARTIFICIAL INFLATION? – Since ChatGPT was first made available to the public this past December, it’s been pretty much all legal tech circles can talk about. The excitement is palpable–but is it overblown? James Michalowicz, senior manager of legal ops business performance at TE Connectivity, told Law.com’s Isha Marathe reports that while he acknowledges that generative AI is a remarkable innovation in computing, it is not a silver bullet. But the market around it and perhaps the susceptibility of some consumers to such promotion, set the stage for an unfortunate result: “It’s just not going to meet the expectations [vendors] have built up.” For law firms and legal departments jumping into the sea of 2023′s AI offerings, parsing through the tools, and their use cases, is likely to be a difficult feat. “It’s very easy to say it’s out-of-the-box or say it’s alright, it’s ready to use, but what I’ve learned is the time necessary to build a training set—training, testing and scoring—that investment, I think that’s what’s being undersold,” Michalowicz said.
BAR CODE – For more than a century, law students have run themselves ragged trying to pass the written bar exam. Earlier this month, artificial intelligence did it without a struggle–and without the student debt. On March 14, when the much-anticipated news broke that OpenAI had released GPT-4, its most powerful AI model to date, with it came the news that GPT-4 had passed the Uniform Bar Exam (UBE), getting nearly 76% of the answers correct, which would place it in the 90th percentile of test-takers. For context, GPT-4’s predecessor model, GPT-3.5, was only tested on the multiple-choice (MBE) portion of the bar exam. It got just shy of 50% of the questions right, which would put it at the 10th percentile of human test-takers. It only received a passing score in one area. “I don’t even really care about the bar exam, per se,” researcher Daniel Katz told Law.com’s Stephanie Wilkins reports. ”This crystallizes what is happening for people in a way [that says], here’s some tasks that lawyers do, and it does it marginally better.”
