By now, there’s little question about the growing influence of AI in content creation— we’re riding that wave.
As magical as it appears, this newfound affinity could also lead us into minefields of plagiarism, inappropriate outputs, or a general lack of human essence we readers seek.
An unsettling twist in the plot is how AI-centric content has started leaving footprints in the SEO landscape.
In an ideal scenario, you’d argue AI tools are plagiarism-proof. Can machines err?
Heck, they do. And so, our worries about getting authentic content remain. Furthermore, the challenge of human-like fluency and authenticity persists in AI-generated text.
That’s where an AI content detector comes in. These tools can help you scrutinize and ‘de-robotize’ your text by identifying AI-created material. They work on machine learning algorithms trained on AI-generated and human-written texts, learning patterns to differentiate the two.
There was something oddly surprising when I first used an AI detector and found it was flagging off content authored by a ‘human.’ Couldn’t help pondering: Do these tools work?
In this blog, we’ll test some AI content detectors and see if they are reliable.
The prep
To put these tools to the test, I’ve prepared a pair of content pieces – the first written by me several years back when AI writing wasn’t popular. I used ChatGPT to generate the second piece on the same topic.
Here’s an excerpt from the human-written piece:
Clothing and colors are two incredibly powerful mediums of expression of our individuality.
The colors we wear are often a reflection of our mood, individual preferences, and personal identity. For the same reason, we have used colors to symbolize different aspects of humanity throughout our existence.
One such empowering symbol is the rainbow flag for gay pride, social equality, and individuality. The rainbow colors red, orange, yellow, green, blue, and violet, represent the diversity and pride of the LGBTQ community.
To celebrate this beauty of human diversity, we designed the transforming rainbow dress.
Every feature of this dress signifies the essence of the special human identities and transformation.
Firstly, the color palette of the dress represents the pride flag. The horizontal stripes of red, orange, yellow, green, blue, and violet honor the wide spectrum of gender identities.
Secondly, this dress is a versatile clothing item denoting diversity. A single piece can morph into many different styles – from an elegant full-length slit dress to a fun one-legged trouser outfit.
Moreover, the transformation capabilities of this dress are again a sublime portrayal of the transsexual identity. It marks the freedom to choose and express your unique self.
Last but not least, the rainbow dress is handmade with 100% organic cotton. The choice of fabric celebrates the LGBTQ identity as the gift of nature.
Not only is organic cotton natural, but also biodegradable and renewable. Furthermore, cotton fabric is the embodiment of comfort. In our design, it is an inspiration for you to feel comfortable in your skin (literally and figuratively).
Creating the rainbow dress is a matter of pride for us, and owning it will be yours!
Here’s the piece generated by ChatGPT:
Now, let’s uncover how some popular AI detection tools measure up.
1. Writer AI Content Detector
First on our list is Writer – the popular generative AI, grammar, and content alignment tool. Writer offers a free AI content detector where you can analyze up to 1500 characters at a time.
While Writer is excellent at its core offerings, its AI detector probably has a long way to go. For both the human-written and AI-generated content pieces, it detected them to be 100% human-generated.
Human-written text result
AI text result
2. Copyleaks
Copyleaks is an award-winning plagiarism and AI content detection tool. The company is devoted to protecting content authenticity and offering military-grade security.
It claims 99.1% accuracy in AI content detection. Along with AI-generated content, it can also detect AI-generated content that’s been paraphrased.
Moreover, it can detect across AI models, including GPT-4, ChatGPT, and Bard. It can also detect the mix of AI and human content, highlighting the probable AI text in red.
For our testing pieces, it was pretty accurate in identifying ChatGPT content. It also identified the human-generated content but assigned it only a probability of 55.7%.
Human-written text result
AI text result
3. Contentdetector.ai
Contentdetector.AI is another free AI text detection tool that can identify GPT, GPT-3, and GPT-4 content.
It gives a probability score for how artificial your content sounds, highlighting the sentences more likely to be AI-generated in red. It asks to check above 250 words for better accuracy.
This tool identified both our testing pieces as entirely human. In fact, it assigned a higher probability of being artificial to the human text.
Human-written text result
AI text result
4. Content at Scale
Content at Scale is an AI content detector for ChatGPT, Bard, GPT-4, and others. It claims a 98% accuracy in detecting AI-generated text, checking up to 25000 characters at a time.
Analyzing the text, it gives an overall probability of whether the content sounds like a human. It also color-codes each sentence based on how likely it is to be human or AI-generated.
For our testing pieces, it pretty accurately identified both texts.
Human-written text result
AI text result
5. Sapling
Sapling is an AI assistant offering products like grammar checker, chat assist, and AI detector. Its AI detection tool identifies AI text generated by models like GPT-3.5 and ChatGPT.
It gives a probability score for the likelihood of the content being ‘fake,’ i.e., AI-generated. It also breaks down each sentence and highlights it in shades of red as per the likelihood of being AI.
However, Sapling makes clear that the system can give you false-positive results. They are working on the accuracy of the system.
It detected both our example pieces as not fake with close to 0% probability of being AI-generated, which, of course, is not the case.
Human-written text result
AI text result
6. Crossplag
Crossplag is a multilingual plagiarism-checking tool. Taking their commitment to ensure content authenticity, they built the free AI content detector.
It ranks your content on a scale of ‘human’ to ‘mix’ to ‘AI’ in its AI Content Index. You can check up to 3000 words at a time.
During our testing, it classified both pieces as human.
Human-written text result
AI text result
7. Smodin
Smodin is another free AI content detector that can identify content generated by ChatGPT or other machine-learning models.
It gives an overall probability percentage for the likelihood of the content being human-written or AI-generated. Further, it gives a percentage likelihood of AI content for each paragraph.
During our testing, it identified both pieces as likely human-written, with the overall probability of the ChatGPT content being AI-generated as lower.
Human-written text result
AI text result
8. DupliChecker
DupliChecker has several SEO-focused tools, including plagiarism checker, grammar checker, paraphrasing tool, reverse image search, and AI content detector.
Its AI detection tool can identify GPT-3 and GPT-4 content with 100% accuracy (as it claims). You can check up to 2000 words at a time. The results are presented as the percentages of human-written and AI-written content.
In our test, it classified both pieces as 100% human-written.
Human-written text result
AI text result
9. GPTZero
GPTZero is another AI detector that can detect AI-generated content across AI language models such as ChatGPT, GPT-4, GPT-3, GPT-2, LLaMA, etc.
It claims to classify 99% of human-written text and 85% of AI-generated content correctly. You can check up to 5000 characters at a time, and it predicts the probability of it being AI. It also highlights the sentences likely written by AI.
So, how did it fare?
It detected our test pieces being around 50% likely written by AI.
Human-written text result
AI text result
10. WinstonAI
WinstonAI is an AI detector that identifies AI-generated content from ChatGPT, GPT-4, Bard, Claude, and other large language models.
It claims a 99.6% accuracy rate in AI content detection.
To check your content using WinstonAI, you need to sign up. Its free version offers up to a 2000-word scan, giving a human score to the content in question.
In our test, it accurately detected ChatGPT content but assigned only an 8% human score for the human-written text.
Human-written text result
AI text result
11. Originality.ai
The last on our list is Originality.ai, an AI checker, plagiarism checker, and fact checker. It claims to be the most accurate AI detector, with a 99% accuracy on GPT-4 and 83% on ChatGPT.
It also claims false positives to be only around 2%.
Once you sign up, you will receive 50 free credits to scan your content. For AI detection, every hundred-word scan costs 1 credit. It color codes every sentence according to the probability of it being AI-written.
Disappointingly, it flagged both our test pieces as AI-written.
Human-written text result
AI text result
Also, I had to use Originality.ai for a project because the client wanted it. It consistently flagged human-written content.
So, I dug deeper to find out why. Turns out, according to its false positive guide, using other AI tools like Grammarly or AI word processors can mess with its accuracy.
The verdict
Apart from the results from Content at Scale, no other AI content detection tool was accurate enough to differentiate the ChatGPT and human-written content.
It’s evident that these systems are still evolving, and the effectiveness of these tools can vary significantly.
One of the biggest limitations of these tools is false negatives and false positive results; that is, they might flag genuine, human-created content as being AI-generated or vice versa.
This is bound to happen, in my opinion, as AI writing tools are, after all, trained on human-written content databases.
While the accuracy rate of AI detectors is questionable and debatable, this presents an opportunity for writers to understand the patterns in their writing and distinguish them from AI.
So, it might be a good idea to experiment with these tools but not consider their results as the ultimate truth.
