{"id":78,"date":"2023-07-11T10:33:12","date_gmt":"2023-07-11T15:33:12","guid":{"rendered":"https:\/\/www.eastcentral.edu\/free\/?page_id=78"},"modified":"2025-02-17T09:18:36","modified_gmt":"2025-02-17T15:18:36","slug":"detecting-ai-generated-text","status":"publish","type":"page","link":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/","title":{"rendered":"Detecting AI-Generated Text: Things to Watch For"},"content":{"rendered":"\n

(Updated 2\/17\/2025) First and foremost, DO NOT rely solely on AI-text detection software to catch student usage.<\/strong> These tools are notoriously unreliable, providing large numbers of false positives and false negatives. The absolute best these detectors have performed is to correctly identify AI-generated-text 80% of the time (below is a screenshot from a study that helped to demonstrate this<\/a>). That means it’s wrong on one paper out of every five it looks at! AI-detection tools have infamously identified human-written documents like the US Constitution<\/a> and parts of the Bible<\/a> as having been generated by AI. They are also discriminatory against students who don\u2019t natively speak English, giving false positives for these students up to 70% of the time (!).\u00a0<\/a><\/p>\n\n\n

\n
\"Excerpt<\/figure>\n<\/div>\n\n\n

<\/p>\n\n\n\n

Instead of relying upon faulty detection tools to tell if a piece of content has been generated by an AI, you should rely more upon various factors in the technical aspects of the writing. Here are things to look for and tools to use:<\/p>\n\n\n\n

    \n
  1. Compare to Writing They\u2019ve Done in Class<\/strong>: The old methods remain the best, and will be the most resilient to new AI capabilities as they develop. So if you’re not interested in updating your detection methods as you keep constant track of developments in the field, it’s worth utilizing methods like this one, number 2, and number 3. <\/em>Students often have a unique writing style and voice that they maintain across their assignments. When reviewing a student’s work, look for significant deviations from this established style. Does the vocabulary, sentence structure, or complexity of ideas feel significantly different? Does the tone match the student’s usual approach? If the student normally struggles with grammar and spelling but suddenly hands in a flawless paper, it might be a sign of AI-generated content. Also, AI-generated text may lack the specific mistakes or idiosyncrasies that a student typically makes.<\/li>\n\n\n\n
  2. Quiz the Student on the Content of Their Work<\/strong>: If you suspect that a piece of work has been generated by AI, one effective way to confirm your suspicion is by quizzing the student on the content of their work. Ask them to explain complex points, the reasoning behind their arguments, or the meaning of specific words or phrases they used. If they are unable to provide satisfactory answers or seem confused by their own work, it could be an indication that they didn’t write it themselves. Be aware, though, that students might also struggle with this if they’re nervous or if they completed the assignment a while ago and no longer remember the specifics.<\/li>\n\n\n\n
  3. Use Google Docs’ Version History feature to detect human-like writing behavior:<\/strong> Google Docs has a feature that saves after each small change is made in a document. Having your students compose exclusively in Google Docs and share an editable copy of their drafts is a good way to track whether they’re pasting content wholesale from another source, or if they’re working bit by bit in a more human fashion. Here’s some instructions on how to use the feature.<\/a><\/li>\n\n\n\n
  4. Unusual or Over-Complex Sentence Construction:<\/strong> AI might sometimes generate sentences that are grammatically correct but unusual or overly complex, especially when it’s trying to generate text based on a mixture of styles or genres.<\/li>\n\n\n\n
  5. Overuse of bullet points:<\/strong> Some (not all!) AI text-generation tools tend to use a lot of bullet points to communicate in a concise, corporate way.<\/li>\n\n\n\n
  6. Not Referencing Current Events (or lying about\/misrepresenting them):<\/strong> Many AI models do not have real-time access to the internet and cannot provide up-to-date information or comment on current events. If an AI-generated text refers to events after its training cutoff, it’s likely extrapolating from patterns it learned and might not be accurate. Keep in mind, though, that many tools now have access to internet search tools and can understand uploaded documents in order to be “brought up to speed” even on events that happened after their initial training.<\/li>\n\n\n\n
  7. Lack of Depth\/Analysis:<\/strong> Another way to tell if an essay was written by an AI is if it lacks complex or original analysis. This is because machines are good at collecting data, but they’re not so good at turning it into something meaningful. As an example, try to get ChatGPT to give feedback on a student essay; it\u2019s terrible at it! We\u2019re nearing the point where AI is able to start to analyze writing, but its current responses are still very “robotic.\u201d You’ll notice AI-generated writing is a lot better for static writing (like about history, facts, etc) compared to creative or analytical writing. However, be aware: this is a thing that AI tools are getting increasingly good at, so this detection method is getting less useful.<\/em> Once Deep Research from OpenAI becomes more widely available, expect this detection method to be entirely irrelevant.<\/em><\/li>\n\n\n\n
  8. Inaccurate Data or Quotes:<\/strong> If a prediction machine (which is what these AI text generators are) doesn’t know something but is destined to give an output, it’ll predict numbers based on patterns (which aren’t accurate). So, if you’re reading student homework and you spot several discrepancies between the facts and the numbers, this might have been generated with AI.<\/li>\n\n\n\n
  9. Do the Sources Exist? Do They Say What the Writer Claims They Do?:<\/strong> While this is a thing that students have already done for a long time, many AI text generators will create sources out of thin air that appear convincing. You should check every single source that your students cite if you\u2019re concerned about they\u2019re having used an AI text generation tool. However, be aware: this is a thing that AI tools are getting increasingly good at, so this detection method is getting less useful.<\/em><\/li>\n\n\n\n
  10. Look Out for Weird Paraphrases:<\/strong> Many students use tools like Quillbot to take existing academic essays that their friends or classmates have written and \u201cparaphrase\u201d them into not being caught by Turnitin or other plagiarism checkers. This can result in turns of phrases being \u201ctranslated\u201d literally and in a way that doesn\u2019t make sense, among other odd changes.<\/li>\n<\/ol>\n\n\n\n

    Works Cited<\/strong><\/p>\n\n\n\n

    Gluska, Justin. \u201cHow to Check If Something Was Written with AI.\u201d Gold Penguin<\/em>, 6 June 2023, https:\/\/goldpenguin.org\/blog\/check-for-ai-content\/<\/a>.<\/p>\n\n\n\n

    Needle, Flori. \u201cAI Detection: How to Pinpoint AI Generated Text and Imagery [+ Detection Tools].\u201d HubSpot Blog<\/em>, 23 June 2023, https:\/\/blog.hubspot.com\/marketing\/ai-detection<\/a>.<\/p>\n\n\n\n

    \u201cWhat are some distinctive features about AI-generated text that a professor might be able to use to differentiate from their students’ own writing\u201d prompt. ChatGPT 4.0<\/em>, 24 May version, 28 June 2023, https:\/\/chat.openai.com\/share\/52c2ecd9-4155-4d16-9952-43548019761c<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

    (Updated 2\/17\/2025) First and foremost, DO NOT rely solely on AI-text detection software to catch student usage. These tools are notoriously unreliable, providing large numbers of false positives and false negatives. The absolute best these detectors have performed is to correctly identify AI-generated-text 80% of the time (below is a screenshot from a study that […]<\/p>\n","protected":false},"author":98,"featured_media":0,"parent":28,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_eb_attr":"","footnotes":""},"class_list":["post-78","page","type-page","status-publish","hentry"],"yoast_head":"\nDetecting AI-Generated Text: Things to Watch For - Faculty Resources for Educational Excellence<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Detecting AI-Generated Text: Things to Watch For - Faculty Resources for Educational Excellence\" \/>\n<meta property=\"og:description\" content=\"(Updated 2\/17\/2025) First and foremost, DO NOT rely solely on AI-text detection software to catch student usage. These tools are notoriously unreliable, providing large numbers of false positives and false negatives. The absolute best these detectors have performed is to correctly identify AI-generated-text 80% of the time (below is a screenshot from a study that […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/\" \/>\n<meta property=\"og:site_name\" content=\"Faculty Resources for Educational Excellence\" \/>\n<meta property=\"article:modified_time\" content=\"2025-02-17T15:18:36+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.eastcentral.edu\/free\/wp-content\/uploads\/sites\/82\/2023\/08\/AI-Detection-Tools-Dont-Work-1.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/\",\"url\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/\",\"name\":\"Detecting AI-Generated Text: Things to Watch For - Faculty Resources for Educational Excellence\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/www.eastcentral.edu\\\/free\\\/wp-content\\\/uploads\\\/sites\\\/82\\\/2023\\\/08\\\/AI-Detection-Tools-Dont-Work-1.jpg\",\"datePublished\":\"2023-07-11T15:33:12+00:00\",\"dateModified\":\"2025-02-17T15:18:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/wp-content\\\/uploads\\\/sites\\\/82\\\/2023\\\/08\\\/AI-Detection-Tools-Dont-Work-1.jpg\",\"contentUrl\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/wp-content\\\/uploads\\\/sites\\\/82\\\/2023\\\/08\\\/AI-Detection-Tools-Dont-Work-1.jpg\",\"width\":624,\"height\":484,\"caption\":\"Excerpt from book about failure of detecting AI-generated text\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/detecting-ai-generated-text\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI Essentials for Â鶹Éç Faculty\",\"item\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/ai-faculty-resources\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Detecting AI-Generated Text: Things to Watch For\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/#website\",\"url\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/\",\"name\":\"Faculty Resources for Educational Excellence\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.eastcentral.edu\\\/free\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Detecting AI-Generated Text: Things to Watch For - Faculty Resources for Educational Excellence","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/","og_locale":"en_US","og_type":"article","og_title":"Detecting AI-Generated Text: Things to Watch For - Faculty Resources for Educational Excellence","og_description":"(Updated 2\/17\/2025) First and foremost, DO NOT rely solely on AI-text detection software to catch student usage. These tools are notoriously unreliable, providing large numbers of false positives and false negatives. The absolute best these detectors have performed is to correctly identify AI-generated-text 80% of the time (below is a screenshot from a study that […]","og_url":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/","og_site_name":"Faculty Resources for Educational Excellence","article_modified_time":"2025-02-17T15:18:36+00:00","og_image":[{"url":"http:\/\/www.eastcentral.edu\/free\/wp-content\/uploads\/sites\/82\/2023\/08\/AI-Detection-Tools-Dont-Work-1.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/","url":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/","name":"Detecting AI-Generated Text: Things to Watch For - Faculty Resources for Educational Excellence","isPartOf":{"@id":"https:\/\/www.eastcentral.edu\/free\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/#primaryimage"},"image":{"@id":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/#primaryimage"},"thumbnailUrl":"http:\/\/www.eastcentral.edu\/free\/wp-content\/uploads\/sites\/82\/2023\/08\/AI-Detection-Tools-Dont-Work-1.jpg","datePublished":"2023-07-11T15:33:12+00:00","dateModified":"2025-02-17T15:18:36+00:00","breadcrumb":{"@id":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/#primaryimage","url":"https:\/\/www.eastcentral.edu\/free\/wp-content\/uploads\/sites\/82\/2023\/08\/AI-Detection-Tools-Dont-Work-1.jpg","contentUrl":"https:\/\/www.eastcentral.edu\/free\/wp-content\/uploads\/sites\/82\/2023\/08\/AI-Detection-Tools-Dont-Work-1.jpg","width":624,"height":484,"caption":"Excerpt from book about failure of detecting AI-generated text"},{"@type":"BreadcrumbList","@id":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/detecting-ai-generated-text\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.eastcentral.edu\/free\/"},{"@type":"ListItem","position":2,"name":"AI Essentials for Â鶹Éç Faculty","item":"https:\/\/www.eastcentral.edu\/free\/ai-faculty-resources\/"},{"@type":"ListItem","position":3,"name":"Detecting AI-Generated Text: Things to Watch For"}]},{"@type":"WebSite","@id":"https:\/\/www.eastcentral.edu\/free\/#website","url":"https:\/\/www.eastcentral.edu\/free\/","name":"Faculty Resources for Educational Excellence","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.eastcentral.edu\/free\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/pages\/78","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/users\/98"}],"replies":[{"embeddable":true,"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/comments?post=78"}],"version-history":[{"count":12,"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/pages\/78\/revisions"}],"predecessor-version":[{"id":639,"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/pages\/78\/revisions\/639"}],"up":[{"embeddable":true,"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/pages\/28"}],"wp:attachment":[{"href":"https:\/\/www.eastcentral.edu\/free\/wp-json\/wp\/v2\/media?parent=78"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}