Thanks to a post at Instapundit, I found this article about an AI-generated service that supposedly detects whether a snatch of text was written by an AI or not. Since John McCrarey had recently asked ChatGPT to write a blog post for him, I decided to run that post through this newfangled AI-checker. Here's the text I had checked (from John's blog here):
Last night, I had the pleasure of visiting three of my favorite bars in Barrio Barretto. My evening started out at IDM bar, where I ran into my friends Dave and Chris. We chatted over drinks and shared our thoughts on the latest victims of the scamdemic. Our server, the lovely Juliet, made sure we were well taken care of.
Next, I headed to BarCelona, a rooftop bar with a fantastic ambiance. The views of the city from the third floor were breathtaking and the perfect backdrop for my drink. I had the pleasure of chatting with my server, Jade, over a lady drink. It was a nice way to spend some time getting to know her.
My last stop of the night was Wet Spot, where I had a “buy one, get one” discount coupon. I chatted with the owner, Dave, and enjoyed some discounted drinks with the latest news and views. I had enough to drink, so I decided to call it an early night and went home.
I must have been more intoxicated than I realized because I woke up on the couch later that night. But, it was all worth it for the great company, drinks, and views. If you’re ever in Barrio Barretto, be sure to check out these bars for a night you won’t forget.
Here's a link to the AI-checker.
And here's the result I got:
The classifier considers the text to be unlikely AI-generated.
So this could mean any of the following:
- The AI text was so human-like that it fooled the checker.
- John lied and wrote the piece himself (which is why I noted "McCrareyisms" when I responded to him in the comments to a later post).
- The AI-checker is awful and can't be trusted.
Right now, I'm thinking my best bet is option (3), but a version of option (2) is also possible: not that John lied, per se, but that his original input data were so specific that ChatGPT, in generating the blog post, adhered closely enough to John's choice of words that all it was doing was merely aping John instead of generating a totally original post.
So I'm wavering between (2) and (3).
Meanwhile, don't trust any of these bots.
Interesting. I'm leaning toward #2, and that's the truth! When I asked the AI to write a blog post about my night on the town, it asked me to provide some info about my activities. I did so, and much of what the AI wrote was just a regurgitation of my submission, changing words here and there, but nothing really unique.
ReplyDelete#3 could also come into play if you consider that the AI-generated blog post was such a fraud; even the AI fact-checker couldn't identify the words as being written by AI.
Anyway, both programs have a long way to go to be truly useful IMHO.