AI Frustrations: Why the Hype Rarely Matches Reality
A university educator recently voiced a frustration that many professionals share:
"I feel like I'm going insane. I work at a university, and the number of people so catastrophically pilled with this tech keep telling me, 'It's getting better and better.' And I HAVEN'T seen a single improvement in output quality for more than a year and a half!"
I've heard this sentiment from many people - in fact at my training workshops I always ask who the AI sceptics are in the room (they're often the best at understanding how to use AI effectively).
Many component parts of AI aren't evolving as fast as the hype suggests - and that gap is frustrating.
Why AI Feels Radical to Some, But Useless to Others
AI opinions are sharply divided. Some find it useful, while others - especially in education, creative industries, and technical fields - remain unimpressed.
A key reason is the gap between claimed progress and actual, measurable improvements in quality.
Another educator explained how AI tools were forced into their university's workflow without consulting the people who actually use them. The result was more frustration than innovation.
AI Is Built for Some Tasks, But Not Others
Christopher Penn offers a useful way to explain why AI seems brilliant in some areas but terrible in others. He highlights the distinction between two types of tasks:
Probabilistic tasks - Creative, open-ended, multiple possible "right" answers (brainstorming, first drafts, idea generation).
Deterministic tasks - Structured, rule-based, with clear right and wrong outcomes (final drafts, fact-checking, technical precision).
AI excels at probabilistic tasks. It's great at generating ideas, summarising information, and automating repetitive work. But on deterministic tasks - where precision, nuance, and accuracy matter - it falls short.
When working with clients, I've found that understanding which specific tasks AI handles well versus where human expertise remains essential allows for more successful adoption.
As Christopher puts it:
"We used to say, 'Write drunk, edit sober.' Now, it's 'AI writes, humans edit.'"
This explains why many academics and professionals feel frustrated. Their work isn't just about creating something plausible - it's about getting it right. AI isn't built for that.
Has AI Improved?
Has AI improved in the last 18 months? The answer is yes and no - depending on what you measure.
AI Advancements:
✔️ Multimodal capabilities (text, image, audio, video)
✔️ Code generation
✔️ Following complex instructions
✔️ Mathematical reasoning
✔️ Custom models for specific industries
Areas with Minimal Progress:
❌ Factual accuracy
❌ Deep understanding of specialised fields
❌ Consistent adherence to style guidelines
❌ Subtle error detection
❌ Adaptability to unique educational needs
But beyond performance issues, AI faces another major challenge: the quality of the data it learns from.
The Training Data Problem
Training data presents two major issues:
1️⃣ Quality and Accuracy Issues
Many AI models are trained on data that is:
Incomplete or outdated
Filled with biases and inaccuracies
Lacking crucial context in specialised fields
Wrong in many instances
When AI confidently presents misinformation, the results range from mildly annoying to dangerously misleading. This is especially problematic in academic and professional settings.
2️⃣ Ethical Concerns About Intellectual Property
Many AI models have been trained on massive datasets - including copyrighted books, articles, and creative works - without proper permission or compensation.
For those in creative and academic fields, this is an ethical issue as well as a technical one. Universities, for example, promote academic integrity, yet many AI tools they adopt are built on scraped content from researchers, authors, and artists without consent.
As an author myself, I've seen one of my books appear in a repository of stolen IP like LibGen without my permission. The thought of AI being trained on my work - without credit or compensation - by corporations worth billions is frustrating.
If organisations don't address these issues, they risk undermining their own values.
A Smarter Way to Use AI: Human-First, Not Tech-First
The problem is how it's being implemented.
Too often, organisations introduce AI simply because it's trendy, without considering whether it actually helps. A better approach:
1️⃣ Start with real human needs
2️⃣ Be honest about limitations - AI has strengths and weaknesses.
3️⃣ Use AI selectively - Let it handle the probabilistic tasks where it shines.
4️⃣ Keep humans in charge - AI can assist, but it shouldn't replace human judgement.
5️⃣ Design systems that combine AI + human expertise
When AI is forced into environments where it doesn't fit - like an education system that requires precision and context - it's no surprise people feel let down.
This human-first approach focuses on specific areas where AI creates value: improving productivity, extending capabilities, enhancing decision-making, and accelerating learning.
Finding the Balance
I understand why educators and professionals are frustrated. AI tools are often introduced without considering real-world challenges, and their capabilities are oversold.
At the same time, when used appropriately, AI can deliver real value. Despite the frustrations, I've seen firsthand how effective AI can be when applied correctly.
- In one project, I used coding-focused AI tools to complete in a single day what would have previously taken me 2-3 weeks of manual development.
- I've seen AI sift through vast amounts of text data, uncovering insights that would have taken months or years to document.
- I helped a blood testing laboratory implement AI automation that cut processing time by 92% and saved over €250,000 annually - all with a tiny initial investment and without disrupting their scientific work.
I've also seen that the risk of waiting too long to adopt AI can often outweigh the risk of measured adoption itself.
But this is only true when that adoption follows a thoughtful, human-centred approach - not the hasty deployment without proper assessment that we're seeing in many settings.
Undeniable Progress & The "Jagged Edge"
- AI has made remarkable strides in the past 18 months. Models today can handle tasks that were impossible just two years ago.
- Frustration with AI is entirely valid, but it might be field-specific. In some types of work like coding and data analysis, today's AI tools vastly outperform those from even a year ago.
AI is improving, but we're experiencing the jagged edge of AI improvement. Progress isn't smooth or uniform; it's dramatically uneven across different domains and use cases and different technologies. And the pace of actual improvement doesn't match the hyperbolic claims of AI marketers and fanboys.
What's Your Experience?
Have AI tools actually helped you, or are you still waiting for the 'big breakthrough'?
If you're working through the challenges of AI implementation in your organisation, you might find my Complete AI Toolkit valuable. This free resource requires no email opt-in, and it includes three practical guides:
🧭 Opportunity and AI Adoption - Helps you identify where AI can add value versus where it's just hype
⚠️ Risk and AI Adoption - Provides a framework for understanding and mitigating the risks we've discussed
🗺️ AI Adoption Roadmap - Offers a structured approach to implementation that puts human needs first