A conversation with AI

6.3. A conversation with AI#

LLMs have become a part of our daily lives, becoming more than just a fun chatbot we can ask simple questions to test its “intelligence”. Here, we would like to acknowledge that LLMs can aid learning; therefore, we will give you some insights gained from recent scientific literature on LLMs in education. We also discuss how we can make the most out of it, and point out the need for being careful not to become too dependent on LLMs. Through a strategic use of LLMs, we can speed up our learning process without giving up our authentic contribution and critical thinking abilities.

LLMs can help us by offering personalized learning materials and quick feedback. However, we should be aware of the unresolved challenges such as ethical usage (for example the negative effect on the environment due to its high energy consumption), data privacy, and algorithmic bias (LLMs may reflect societal, cultural, or linguistic biases present in training datasets, potentially leading to discriminatory outcomes). Another important consideration is LLMs’ long-term effect on our cognitive abilities such as critical and creative thinking.

From this point onwards, we can already start integrating LLMs in our discussion. For the rest of this chapter we will present you with prompts you can insert into a chatbot followed by a series of questions for you to begin to evaluate its response. For each prompt we will also include the response ChatGPT-4o produced when we inserted the prompt. However, you can use any version you prefer.

6.3.1. Prompt 1: LLMs in education#

Prompt: If you could explain your own purpose in education, how would you describe your responsibilities and limitations?

Critical thinking questions for you:

  1. Is there anything that surprised you in the answer?

  2. When you try to change the question and/or highlight different points, how does the answer change?

  3. What do you think about the difference between a human tutor and LLMs? And what about the LLM’s “opinion”?

  4. Did you notice any parts where it sounded overly-confident and/or human-like?

Our Answer to Prompt 1 (with ChatGPT-4o)

LLM’s Perspective: Purpose, Responsibilities, and Limitations in Education

Purpose in Education

My primary role in education is to support and enhance learning by:

  • Explaining concepts clearly and accessibly.

  • Offering guidance through questions and challenges.

  • Providing feedback, examples, and summaries on demand.

  • Encouraging exploration across disciplines.

I’m not here to replace teachers or original thought—I aim to be a catalyst for curiosity and understanding.

Responsibilities

While I don’t have consciousness or intent, I’ve been designed with ethical use in mind. That means my “responsibilities,” as guided by my design, include:

  • Providing accurate and contextually relevant information.

  • Adapting explanations to different knowledge levels and learning styles.

  • Encouraging critical thinking rather than delivering “absolute truths.”

  • Avoiding harmful, biased, or misleading content to the best of my ability.

Limitations

It’s also crucial to understand what I can’t do:

  • I do not understand or feel—my responses are generated based on patterns in data, not real comprehension.

  • I can’t verify facts in real time—I can be outdated or wrong.

  • I may reflect biases present in the data I was trained on.

  • I do not replace human judgment, empathy, or creativity—I can support, but not embody, those things.

Final Thought

Think of me as a powerful tool: one that’s only as effective—and as ethical—as the person using it. When used thoughtfully, I can amplify learning. When used uncritically, I might distort or mislead. Your curiosity and critical thinking are what bring our interaction to life.

6.3.2. Prompt 2: Brainstorm with LLMs#

Prompt:

#1 Do you agree with the following suggestions and can you give me a guideline to implement them while I learn programming?

#2 Can you challenge the following arguments?

Key components of AI-resistant critical thinking:

  1. Source Evaluation: Developing the ability to critically assess the origin and credibility of information, whether human-generated or AI-produced.

  2. Algorithmic awareness: Understanding the fundamental principles behind LLMs and their decision-making processes to better interpret and evaluate their outputs.

  3. Bias recognition: Honing skills to identify and mitigate both human and AI-induced biases in reasoning and decision-making.

  4. Logical reasoning: Strengthening deductive and inductive reasoning skills to complement and sometimes challenge AI-generated insights.

Integrating LLMs as tools for enhancing, not replacing, human thought:

  1. Collaborative problem-solving: Designing workflows that combine human intuition and creativity with AI’s data processing and pattern recognition capabilities.

  2. AI-assisted reflection: Using LLMs to generate alternative perspectives or counterarguments, prompting deeper human analysis and consideration.

  3. Scaffolded learning: Employing AI systems to provide personalized guidance and feedback in developing critical thinking skills, gradually reducing reliance as human competence increases.

  4. Metacognitive prompting: Utilizing AI to pose thought-provoking questions that encourage self-reflection and deeper engagement with complex issues.”

Critical Thinking Questions:

What other good practices do you think are essential for using AI responsibly?

Our Answer to Prompt 2 (with ChatGPT-4o)

AI-RESISTANT CRITICAL THINKING: GUIDELINES FOR IMPLEMENTATION

  1. Source Evaluation

    What to do:

    • Always cross-check my suggestions (e.g., code snippets, definitions, explanations) with trusted sources like official documentation, reputable tutorials (e.g., MDN, W3Schools, Python.org), or expert-reviewed platforms (e.g., Stack Overflow, GitHub Discussions).

    • Use prompts like: “Can you cite a reliable source for this method?” “What are alternative interpretations of this output?”

    Practice Tip: Before implementing the code I generate, ask yourself: Would this solution be accepted in a real-world codebase?

  2. Algorithmic Awareness

    What to do:

    • Learn about how LLMs work at a high level (e.g., how tokens and probabilities guide generation).

    • Use prompts like: “Explain why you suggested this specific syntax.” “Is this a commonly used pattern or inferred from training data?”

    Practice Tip: If you get strange or inconsistent results, investigate why—it builds insight into both AI and your own debugging mindset.

  3. Bias Recognition

    What to do:

    • Recognize that both code and explanations can reflect biases (e.g., defaults toward certain languages, frameworks, or practices).

    • Ask: “Could this answer be biased toward one framework or convention?” “What’s an alternative approach from another community or culture?”

    Practice Tip: If I favor Python, try asking how the same task is done in Rust, JavaScript, or another language—compare approaches.

  4. Logical Reasoning

    What to do:

    • Don’t just accept code—trace its logic. Predict what it should do before running it. Then test it.

    • Ask: “What are the assumptions behind this function?” “How would this fail or produce unexpected results?”

    Practice Tip: Use LLMs as a “rubber duck”—explain your logic and ask for critique.

ENHANCING HUMAN THOUGHT WITH LLMS: GUIDELINES FOR INTEGRATION

  1. Collaborative Problem-Solving

    What to do:

    • When stuck, explain your reasoning so far and ask for the next steps, not full solutions.

    • Prompt example: “Here’s my logic. What might I be missing?” “I’m stuck at this part—what are some strategies to proceed?”

    Practice Tip: Use LLMs like a teammate you can bounce ideas off—not just a code generator.

  2. AI-Assisted Reflection

    What to do:

    • After writing or reviewing the code, ask: “Can you suggest an alternative design pattern or refactor?” “What’s a trade-off between my approach and yours?”

    Practice Tip: Reflect on different solutions and why one might be better in a given context (performance, readability, scalability).

  3. Scaffolded Learning

    What to do:

    • Start with simple guidance, then gradually ask for less help. For example:

      • “Explain this algorithm step-by-step.”

      • “Give me a high-level overview; I’ll try the rest.”

      • “Just give me a hint.”

    Practice Tip: Create a “training arc” for yourself—more dependence early, more independence later.

  4. Metacognitive Prompting

    What to do:

    • Ask me questions that make you reflect. For example: “What questions should I be asking about this code?” “What misconceptions do beginner programmers often have about this topic?”

    Practice Tip: Pause and journal briefly after a session: What did I learn, and how did the LLM help (or hinder) that?

🎯 FINAL ADVICE TO STUDENTS

Using LLMs responsibly means recognizing both their power and their limits. They can help you learn programming faster and with more confidence—but they’re not magic. Your critical thinking, reflection, and curiosity are what transform this interaction into real learning.

Contents