5.1 Common Failure Patterns¶
What you will learn on this page
- Common failure patterns in using generative AI and why they happen
- How severe each failure is and how easy it is to detect
- The reality and limits of AI detection tools
- A minimum pre-submission checklist
Common failure patterns¶
1. Full outsourcing to AI¶
Situation: You give only a Japanese outline and ask AI to write the entire paper.
Problem: The content becomes too generic and loses specificity. When reviewers ask, “What exactly is the author’s contribution?”, you cannot explain it convincingly.
Countermeasure: Use AI for diagnosis, options, and checks, not for generating the whole manuscript.
2. Fabricated citations¶
Situation: You asked AI to write a literature review, and it included papers that do not exist.
Problem: Fabricated citations are direct research misconduct. DOI links may point to non-existent pages.
Countermeasure: Do not allow AI to generate bibliographic information at all. Ask it only to suggest what types of sources you should look for. See: 4.2 Fact Checking and Reference Management
3. Mixed tense¶
Situation: AI-generated sentences and your own sentences are mixed, and the tense becomes inconsistent.
Problem: Readers cannot tell “when” you are talking about.
Countermeasure: Ask AI to run a tense check by section.
4. Over-politeness and wordiness¶
Situation: AI output becomes verbose and roundabout.
Problem: Academic English values conciseness. Filler phrases such as It is important to note that accumulate.
Countermeasure: Add an explicit instruction such as “Remove filler and use more direct phrasing.”
5. Escalating dependence¶
Situation: You started with grammar checking only, but gradually began relying on AI for full drafting.
Problem: Your own writing ability does not develop, and you become unable to write without AI.
Countermeasure: Define the scope of AI use in writing and review it regularly. See:
4.3 AI-use Disclosure and Work Logs
6. Style chimera (inconsistent voice)¶
Situation: Paragraphs you wrote and paragraphs AI generated are mixed, and the style becomes inconsistent.
Problem: Vocabulary level and hedging frequency suddenly rise or fall, making reviewers feel “different people wrote this.”
Countermeasure: Run a whole-manuscript style consistency check before finalizing. See:
3.5 Revision Techniques
7. Uncritical acceptance¶
Situation: You accept every AI suggestion without applying your own judgment.
Problem: You import AI errors, and learning benefits become zero.
Countermeasure: Treat AI suggestions as options. Compare at least one alternative of your own before adopting a change.
8. Security and confidentiality risks¶
Situation: You paste unpublished data or participant personal information into an AI tool.
Problem: Confidential data may be transmitted to external servers. This can violate privacy law or IRB rules.
Countermeasures: - Anonymize personal information (names, student IDs, etc.) before input - For sensitive core data, test with dummy data first, then proceed carefully - For highly sensitive content, consider local LLM options
Severity matrix¶
Not all failures have the same severity. Classifying them by impact and detectability helps you prioritize prevention.
| Failure pattern | Severity | Detectability | Priority |
|---|---|---|---|
| Fabricated citations | ★★★ (misconduct) | ★★ (DOI searches can reveal) | Top priority |
| Security risk | ★★★ (legal/ethical) | ★ (hard to notice at input time) | Top priority |
| Full outsourcing to AI | ★★★ (ethical) | ★★ (uniform style can raise suspicion) | High |
| Uncritical acceptance | ★★ (quality loss) | ★ (hard to self-notice) | High |
| Mixed tense | ★★ (readability loss) | ★★★ (easy to check with AI) | Medium |
| Style chimera | ★★ (impression loss) | ★★ (detectable by rereading) | Medium |
| Over-politeness/wordiness | ★ (verbosity) | ★★★ (mechanically detectable) | Low |
| Escalating dependence | ★★ (long-term loss) | ★ (hard to self-notice) | Regular reflection needed |
The reality and limits of AI detectors¶
It is important to understand AI detection tools (GPTZero, Turnitin AI Detection, Originality.ai, etc.) accurately.
For the ethical background and expert detection approaches, see:
1.2 Academic Integrity and Plagiarism Prevention
For concrete vocabulary/style characteristics and practical self-check prompts, see:
4.1 Grammar Checks and Style Consistency
Current state of AI detectors:
- False positives: Human-written text can be judged as AI-generated. Non-native English writing is especially vulnerable.
- False negatives: AI-generated text can be judged as human-written.
- Accuracy variation: Accuracy differs widely by tool and by text type.
- OpenAI discontinuation: OpenAI discontinued its AI Text Classifier in 2023, citing insufficient accuracy.
Do not overtrust AI detectors
Detector results are only “reference information.” What matters most is keeping work logs and being able to explain your writing process. See:
4.3 Work logs
If your detector score is high:
- Confirm that you can explain your authorship with your work logs
- If you have logs, the score itself is not decisive
- Reduce overly uniform “AI-like” style by intentionally inserting your own phrasing
- If the target venue uses detectors, consider explaining your AI-use scope proactively in your cover letter
Minimum pre-submission checklist¶
This checklist reflects the failure patterns above. A more detailed, comprehensive final checklist is provided in:
3.5 Revision Techniques: Final pre-submission checklist
- I verified that all cited sources exist and that DOIs are correct
- I prepared an AI-use disclosure statement
- I checked tense consistency by section
- I can explain why I wrote key sentences the way I did
- I confirmed consistent spelling and terminology across the manuscript
- I re-checked the target venue’s Author Guidelines
- If anonymization is required, I confirmed that self-identifying citations/info are removed
- I prepared a cover letter
Prompt: self-diagnose failure patterns
Please read the manuscript below and check whether it shows any of the following failure patterns.
Checklist:
(1) signs of full AI outsourcing (generic statements without specificity)
(2) inconsistent style (sudden shifts in vocabulary level or hedging frequency)
(3) AI-typical filler expressions (e.g., It is worth noting that, plays a crucial role)
(4) tense inconsistency
(5) possible fabricated citations (unnatural author-year combinations)
For each item, quote suspicious parts and explain why they need checking.
[Manuscript]