lunes, 8 de abril de 2024

☕ Jury’s still out

New paper questions OpenAI bar exam claims.
April 08, 2024

Tech Brew

Nebula

It's Monday. OpenAI claims ChatGPT can pass the bar, but a new research paper suggests that you'll likely still want a flesh-and-blood attorney if you find yourself on the wrong side of the law. Tech Brew's Patrick Kulp has the deets, plus the second in his series of profiles of chief AI officers. (ICYMI, here's the first.)

In today's edition:

Patrick Kulp, Annie Saunders

AI

Lawyer up

A statue of Lady Justice with scales in front of an array of code. Pitiphothivichit/Getty Images

In what sounds like a budding lawyer's stress dream, a new research paper picks apart OpenAI's claim that GPT-4 managed to pass the bar exam with flying colors.

The paper, called "Re-evaluating GPT-4's bar exam performance" and published this week in Artificial Intelligence and Law, questions both the large language model's (LLM) score and its interpretation, as published in a pair of reports last year. It comes as the legal field has forged ahead with turning LLMs into research tools, but AI's use in the legal profession has also led to some high-profile mistakes.

What the study says: OpenAI's original attention-grabbing claim about GPT-4's legal prowess traces to a technical report published by the company last year, which boasted that the LLM placed "around the top 10% of test-takers" on a "simulated bar exam." A separate paper published before OpenAI's report found that GPT-4 scored a 297 on the Uniform Bar Exam, high enough to pass in all jurisdictions.

But the Artificial Intelligence and Law research, authored by Eric Martínez, a PhD student at MIT and a graduate of Harvard Law School, argues that the first above claim is compared to a February batch of Illinois State Bar test-takers that "appears heavily skewed toward repeat test-takers who failed the July administration and score significantly lower than the general test-taking population."

Keep reading here.—PK

   

PRESENTED BY NEBULA

The world's best independent streaming service

Nebula

What's this? A streaming service that has:

  • All the best, tech, engineering, science, travel, (and much more) video creators?
  • Nile Red, Real Engineering, Wendover Productions, Downie Live? With no ads nor ad reads?
  • Exclusive and highly produced, interesting documentaries, series, and even films that you can't get anywhere else?
  • Early access to videos from dozens of your favorite channels?
  • Morning Brew Daily & Good Work with Dan Toomey now available ad free on the brand new curated News section?

Could it really exist? Yes it could. In fact, it does.

Subscribe to Nebula for less than $3 / month and get all of the above and much, much more. All while supporting creators making the best, entertainingly informative videos out there, now on the world's best independent video streaming platform.

AI

CAIOs for all?

A robot dressed in a business suit with a grid of binary code behind it Francis Scialabba

The notion of selling pickaxes during a gold rush is a business cliché, but what about selling slide decks about gold-mining best practices?

That's part of how Accenture has landed $600 million in new bookings from the generative AI craze in the first few months of this year, despite a diminished forecast for the consultancy's year ahead overall, according to its recent quarterly earnings report. One of the executives at the center of this surge is Lan Guan, who was elevated to the position of Accenture's chief AI officer last year after previously leading the company's data and AI practice.

Accenture is far from the only company to umbrella booming AI operations under this trendy new C-suite title. In a survey published in CIO magazine last fall, 11% of midsize to large companies said they had hired a CAIO, and another 21% were seeking one.

But what do these AI-devoted execs actually do? And is this a position every company needs to fill? This is the second in a series of CAIO profiles in which Tech Brew looks into these questions.

In Accenture's case, Guan said the creation of her perch was an acknowledgement of how much the company could stand to gain from client companies' appetite for generative AI. Around the same time that she was promoted, the consultancy pledged to invest $3 billion in AI over the next three years and double its AI-trained workforce to 80,000 through hiring, acquisitions, and training.

"We definitely saw an enormous amount of opportunities in front of us to use generative AI together with classical AI to drive a lot of enterprise reinvention, especially post-pandemic," Guan told Tech Brew. "So I think, simply put, that was the key, direct catalyst to create this role."

Keep reading here.—PK

   

READER SPOTLIGHT

Coworking with Mario Queiroz

Graphic featuring a headshot of Hinge Health's Mario Queiroz. Mario Queiroz

Coworking is a weekly segment where we spotlight Tech Brew readers who work with emerging technologies. Click here if you'd like a chance to be featured.

How would you describe your job to someone who doesn't work in tech?

As chief product officer, I lead the definition of Hinge Health's product vision and strategy, software and hardware development, user experience design, and consumer marketing. My job is to work with a talented group of inventors, designers, clinicians, and technologists to build a product that puts a physical therapist in your pocket. Basically, we help people experiencing muscle or joint pain by connecting members with physical therapists and health coaches who help design a personalized program that is accessible from a phone.

What's the most compelling tech project you've worked on, and why?

Beyond my work at Hinge Health, one of the most exciting projects I ever worked on was the invention, development, and mass-market success of Chromecast at Google. In addition to the product being innovative in terms of technology, simplicity, and user value, it was also the seed of Google's consumer hardware business.

Keep reading here.

   

BITS AND BYTES

Stat: 11.6%. That's how much less plastic packaging Amazon claims to have used in global shipments in 2022 compared to 2021, Grist reported, then citing data from Oceana that showed that, over the same time period, the e-commerce giant was responsible for 208 million pounds of plastic packaging trash in the US, 10% more than in 2021.

Quote: "If the problem is hard today, it will be much harder next year…It will be almost impossible in five years."—Soheil Feizi, a researcher at the University of Maryland, to the Washington Post in a story about detecting deepfakes created by AI image generators

Read: What really happened when Google ousted Timnit Gebru (Wired)

JOBS

Stop waiting for your dream job to find you. CollabWORK puts you in front of the right opportunities in the online spaces you frequent. Get discovered by top companies and take control of your career path.

SHARE THE BREW

Share Tech Brew with your coworkers, acquire free Brew swag, and then make new friends as a result of your fresh Brew swag.

We're saying we'll give you free stuff and more friends if you share a link. One link.

Your referral count: 0

Click to Share

Or copy & paste your referral link to others:
emergingtechbrew.com/r/?kid=b3968dba

         
ADVERTISE // CAREERS // SHOP // FAQ

Update your email preferences or unsubscribe here.
View our privacy policy here.

Copyright © 2024 Morning Brew. All rights reserved.
22 W 19th St, 4th Floor, New York, NY 10011

No hay comentarios: