5 AI Basics Every Executive Should Know

Last week, I was ​coaching​ a CEO who stopped me mid-session with a straightforward question: "Sorry, but what does that actually mean?" after I had mentioned LLMs for the nth time.

It was a perfect reminder that while I try to avoid technical jargon, every leader should understand some foundational AI concepts to leverage this technology effectively.

So today, I'm breaking down five essential AI concepts in plain language – the building blocks you'll need to speak about and interact with AI in your role or organization confidently.

1. Large Language Models (LLMs): The Engine Behind Modern AI

Large Language Models (LLMs) are why we have the AI we know today. These systems, trained on massive datasets, power tools like ​ChatGPT​, Claude, and Gemini.

LLMs memorized nearly all the public information, including every web page, books in the public domain, Wikipedia and even Reddit articles. This lets them quickly retrieve, summarize, and even create content based on patterns they've learned.

(This is the massive amount of data GPT-3 was trained on. It’s believed that GPT-4o was trained on a dataset that’s several times larger.)

What makes LLMs powerful isn't just their knowledge—it's their ability to make connections across different domains and generate new insights. And to let you interact with them without needing a Computer Science degree: regular human language will do. (See “Prompting” below.)

2. Tokens: The Building Blocks of AI Language

Tokens are how AI actually processes language—breaking words, parts of words, and even punctuation into smaller pieces that it can work with mathematically.

For example, the phrase "I need help with my presentation" might be broken into tokens like "I", "need", "help", "with", "my", "present", and "ation." Each token gets translated into a numeric code, which is how the AI model actually processes language.

(The sentence above breaks down into 28 tokens. Most words are one token, but tokenization is two.)

Why Tokens Lead to Hallucinations

Here's where things get tricky: because AI is fundamentally a mathematical model without true comprehension, it doesn't genuinely understand the meaning behind these tokens. It's just incredibly skilled at predicting the next likely token based on patterns.

This explains why even the ​best AI models​ can sometimes confidently deliver completely fabricated responses—a phenomenon known as "hallucination."

I gave an example to a group of ​Lead with AI​ students I coached last week: if an AI model's training data included the incorrect phrase "the sun is black" enough times, it might confidently state this as fact—not because it believes it, but because the tokens statistically align with its learned patterns.

When you're working with AI, remember this key insight: behind the scenes, it's playing a sophisticated numbers game with these tokens, not truly understanding content the way humans do.

3. Context Window: AI's Short-Term Memory

The ​context window​ is essentially how much information AI can "remember" during your conversation—and it's measured in tokens.

Different AI models have dramatically different memory capacities:

• ChatGPT-4: ~128,000 tokens
• Google's Gemini: ~2,000,000 tokens

When you're chatting, every message you send or receive consumes part of this context window. Eventually, if you hit the token limit, the AI begins to "forget" information from the start of your conversation—like a colleague who loses track of what you discussed earlier.

For practical work, this means choosing the right model for your task. If you're analyzing a large document or uploading substantial content, you'll need a model with a larger context window.

Otherwise, you'll quickly reach the AI's short-term memory limit, causing it to lose track of important information and potentially deliver unclear or incorrect outputs.

4. Prompt Engineering: Communicating Effectively with AI

All ‘tokens’ aside, the best way to get the most out of AI is by thinking about it as a new coworker – from a senior copywriter to a business coach tailored to your background and industry.

Prompting is how you give inputs to the AI model. However, given that AI is a coworker, you should think of “prompting” as giving clear instructions to a new team member. The better your instructions, the better the results.

If AI is considered a coworker rather than just technology, prompting becomes more like delegating or training. The most effective approach is what I call "​SuperPrompting​" with the CO-DO framework:

• C (Clarify): Define the AI's role clearly (e.g., "Act as an experienced marketing strategist")
• O (Objective): Specify exactly what you want to achieve
• D (Do's/Don'ts): Detail what should and shouldn't be included in its output
• O (Output): Describe the desired format and structure

This level of detail—setting a clear role, outlining specific goals, and providing relevant context—transforms AI from a basic search engine into a powerful collaborator. Leaders who master prompting get dramatically better results from the same AI models.

Advanced techniques such as "​chain of thought​" (guiding AI step-by-step through reasoning) and "few-shot prompting" (providing examples to illustrate exactly what you expect) can take your results even further beyond the already impressive outputs of basic SuperPrompting.

5. Building Your AI Team: Employees, Automation, and Agents

To truly leverage AI, you need to think beyond single-purpose tools and build a team of AIs around you.

In this context, the terms AI Assistants, Automation, and Agents get used interchangeably. But there are actually very crucial differences. In our ​AI courses​, I ask people to keep this in mind:

• AI Employees/Assistants: Digital coworkers that handle content creation, meeting summaries, and routine inquiries through direct interaction. For example, ChatGPT GPTs, or (confusingly named) Microsoft Copilot agents. By offloading these repetitive tasks, your human team members can focus on strategic priorities. Mastery of prompting skills significantly enhances an AI assistant's productivity.
• AI Automation: Systems that perform repetitive tasks entirely on their own without constant human oversight—such as sorting emails, entering data, and scheduling meetings. Tools like Zapier or Make facilitate this automation through workflows triggered automatically (e.g., checking for new documents every two minutes). The main requirement is clearly defined triggers and actions, potentially requiring an engineering mindset to implement effectively.
• AI Agents: The most sophisticated approach—these systems can independently make decisions, manage intricate workflows, and interact with various software tools with minimal intervention. They actively handle complex tasks, significantly boosting productivity and expanding your team's capabilities beyond standard automation.

(Especially the term Agent gets abused a lot. This slide from Allie K. Miller tries to delineate the messiness.)

The Bottom Line

Don't let AI terminology intimidate you.

These five concepts—LLMs, Tokens, Context Windows, Prompt Engineering, and AI Team Structures—provide the foundation to speak about and ​implement AI​ in your business confidently.

What AI concept would you like me to explain next, if any? Let me know here.