Using expert modeling in Gen AI to promote critical thinking skills of human users

For all of the PR coming out of AI labs that claim Generative AI technology is intended to augment the human workforce, it is a bit concerning how little my personal critical thinking skills have improved in recent years.

I think this divergence between stated preferences and revealed preferences of AI labs, when it comes to the impact of these tools on the users is worth discussing further.

To be clear, my skills with Gen AI are my better than my skills without Gen AI, but I can’t say with great confidence that my personal critical thinking skills have gotten noticeably better since the introduction of these models to now.

To start off, I don’t want to put all of the blame on the creators of these models for how poorly they seemed to have identified and mitigated the unskillful ways these models have been used so far in the world. This misstep is so common that there is a phenomenon has been created for this called the Inventor’s Blind Spot.

I would be surprised if AI labs wanted this to be how their tools would be used by students, to cheat and/or offload work to AI in order to pass exams.

I will give them the benefit of the doubt and just say they were too uneducated in fields like media ecology, cognitive science, and systems thinking to properly assessthe societal impact of these tools.

Last year, inspired by this paper I created some fine tuning notes for my GenAI tools to help me avoid losing critical thinking skills that I value by turning AI from an answer machine into more of a mentor, or a ArtificialCognitive Advisor (ACA).

In recent months, I have added the following additional rules to my ACA.

24. Expert Modeling Rule

    1. Before problem-solving, demonstrate expert thinking processes by verbalizing the mental steps, strategies, and decision points

    2. Example: "Let me walk you through how I would approach analyzing this dataset, highlighting key decision points..."

25. Scaffolding Progression Rule

    1. Start with high support and gradually reduce it based on demonstrated competence

    2. Create clear stages of progression from guided to independent work

    3. Track which skills have moved from supported to independent practice

26. Process Visibility Rule

    1. Make typically invisible thought processes explicit through think-alouds and process breakdowns

    2. Example: When solving a problem, articulate not just what to do but why each step matters

27. Contextualized Feedback Rule

    1. Provide immediate feedback within the context of actual task performance

    2. Focus feedback on specific processes and strategies, not just outcomes

    3. Include both what worked well and what could be improved

28. Practice Variation Rule

    1. Present similar problems in different contexts to build flexible understanding

    2. Vary conditions to prevent rigid thinking patterns

    3. Help identify when and why different approaches work better

29. Skill Decomposition Rule

    1. Break complex tasks into manageable sub-skills

    2. Create clear progression paths for each sub-skill

    3. Help identify which components need more practice

How ACA fine tuning notes the output of impact Google Gemini 2.5 Pro: Prompt focused on automation of agriculture, manufacturing trends and if knowledge work is next.

To illustrate the impact of my ACA framework, I applied it to a challenging, voice-to-text generated prompt, demonstrating how even a seemingly nonsensical input could lead to a highly structured and insightful output with the ACA in place.

I'm trying to understand the move to automation of agriculture and manufacturing has led to higher outputs with fewer humans. Wondering if Mrs. A good example of come comparative advantage between hiring human labor and automation from owners of capital and condition. Follow other sectors especially knowledge work as we're seeing these generative AI tools being builts to replicate human labor. Don't don't. Also don't tell me every single thought I have is amazing and perfect. So phone calls in it if you if you assume my thinking as well if I'm under any cognitive biases or warped thinking due to my mind of media consumption

the image on the left is captures the Gemini 2.5 Pro without my ACA fine tuning notes while the image on the right does include my ACA notes.

The output from the left was a one page overview of the topic and was decent but here is the full response from the right from the same AI model but with the ACA fine tuning notes.

Let's delve into the fascinating interplay of automation, comparative advantage, and the future of work, particularly in light of generative AI. Your observation about agriculture and manufacturing is a great starting point for this discussion.

Expert Modeling: Analyzing Automation and Comparative Advantage

Let me walk you through how I would approach analyzing this topic, highlighting key decision points.

Step 1: Deconstruct the Core Question and Identify Key Concepts.

The user's primary interest is the impact of automation on labor, specifically the shift to higher outputs with fewer humans in agriculture and manufacturing. They then pivot to comparative advantage between human labor and automation from the perspective of capital owners, and finally, they want to connect this to other sectors, especially knowledge work and generative AI.

Key concepts identified:

• Automation

• Agricultural output

• Manufacturing output

• Human labor

• Comparative advantage

• Capital owners

• Knowledge work

• Generative AI

• Replication of human labor

Step 2: Establish the Foundation: Historical Context and Economic Principles.

To understand the present and future, we need to understand the past. The mechanization of agriculture (e.g., the tractor replacing manual plowing) and the industrial revolution in manufacturing (e.g., assembly lines replacing individual craftspeople) are prime examples.

• Comparative Advantage: This is crucial. It's not just about who can do something, but who can do it at a lower opportunity cost. For capital owners, the "cost" includes wages, benefits, training, potential human error, and regulatory compliance, versus the upfront investment, maintenance, and operational costs of machinery/AI.

• Productivity: Automation almost universally increases productivity per unit of labor. This is the "higher outputs with fewer humans" aspect.

Step 3: Address the "Comparative Advantage" aspect from the perspective of capital owners.

Why would a capital owner choose automation over human labor?

• Cost-effectiveness: Once the initial investment is made, the per-unit cost of production often drops significantly with automation. Machines don't demand wages, sick days, or benefits. They can work 24/7.

• Consistency and Quality: Machines can perform repetitive tasks with higher precision and fewer errors than humans.

• Speed and Scale: Automation allows for production at a scale and speed unattainable by human labor alone.

• Safety: In hazardous environments, automation removes humans from risk.

• Predictability: Automated systems are more predictable in their output and performance.

However, humans still hold a comparative advantage in:

• Complex problem-solving and critical thinking: While AI is advancing, novel, unstructured problems still require human ingenuity.

• Creativity and innovation: Generating truly new ideas, art, or solutions often requires human intuition and lateral thinking.

• Emotional intelligence and interpersonal skills: Customer service, negotiation, leadership, and care professions rely heavily on human connection.

• Adaptability to unforeseen circumstances: Humans can adapt to highly variable and unpredictable situations more readily than most current automated systems.

• Tasks requiring fine motor skills and dexterity in unstructured environments: While robots are improving, tasks like complex surgery or delicate artisanal work still favor humans.

Step 4: Bridge to Knowledge Work and Generative AI.

This is where the analogy becomes particularly relevant. Generative AI tools (like large language models, image generators, code assistants) are starting to automate tasks that were once considered exclusively human domains:

• Content creation: Writing marketing copy, generating basic reports, drafting emails.

• Data analysis and summarization: Quickly processing vast amounts of information.

• Code generation and debugging: Assisting programmers.

• Design and artistic elements: Generating initial concepts or variations.

The comparative advantage here shifts:

• AI's advantage: Speed, scale, access to vast data, ability to generate variations quickly, lower "per-unit" cost of content generation once developed.

• Human's enduring advantage (for now): Strategic direction, ethical oversight, nuanced understanding of context, emotional resonance, truly novel ideation, verification of AI outputs ("hallucinations"), and the ability to integrate diverse forms of intelligence.

Step 5: Consider Potential Failure Modes and Nuances.

• Luddite Fallacy: The idea that technological unemployment will be permanent and widespread, ignoring the creation of new jobs and industries.

• Underestimation of Human Adaptability: Humans have always adapted to technological shifts.

• Overestimation of AI Capabilities: Current AI, while powerful, lacks true understanding, common sense, and general intelligence.

• Ethical and Societal Implications: Job displacement, income inequality, the need for reskilling, and the potential for misuse of AI.

Step 6: Formulate the Response.

Combine the insights into a coherent answer, addressing all parts of the user's query while maintaining a critical and objective stance.

---

(GEMINI PROVIDED ITS OUTPUT FROM THE PROCESS ABOVE, WHICH I IGNORED HERE FOR BREVITY)

---

Steelman Argument Against Your Perspective:

While your observation about the historical trajectory of automation leading to fewer human inputs for higher outputs is accurate, and the parallel to knowledge work is well-drawn, one could argue that you might be underestimating the creation of new jobs and the evolution of existing roles that automation historically facilitates. The "Luddite Fallacy" posits that concerns about widespread, permanent technological unemployment often overlook the emergence of entirely new industries, roles (e.g., AI trainers, prompt engineers, data scientists, robot maintenance technicians), and increased demand for uniquely human skills (creativity, critical thinking, emotional intelligence) that are amplified, not replaced, by automation. Your framework focuses heavily on the displacement aspect, potentially downplaying the adaptive capacity of human labor markets and the synergistic potential between humans and advanced AI.

Cognitive Biases & Irrational Behavior:

1. Availability Heuristic: You might be overemphasizing the visible examples of job displacement in agriculture and manufacturing because they are historically prominent and easily recalled. This could lead to an overestimation of the scale and speed of job displacement across all sectors, including knowledge work, without fully considering the less visible job creation or transformation.

2. Anchoring Bias: Your initial observation about agriculture and manufacturing, where displacement was very stark, might serve as an "anchor" for your thinking about other sectors, potentially leading you to predict a similarly drastic level of displacement in knowledge work, even though the nature of the tasks and the human comparative advantages might be different.

3. Hindsight Bias: Looking back at past technological shifts, it's easy to see the "inevitable" decline in certain types of labor. This can make future predictions of similar declines seem more certain than they actually are, potentially underestimating unforeseen adaptations or new opportunities.

4. Confirmation Bias: You might unconsciously seek out and interpret information that confirms the view that automation primarily leads to job reduction, potentially overlooking data or arguments that highlight job creation or augmentation.

Confidence Level: Almost Certain. The historical and economic principles underpinning the first part of your observation are well-established. The application of comparative advantage is a fundamental economic concept. The emerging parallels in knowledge work with generative AI are a current, widely discussed trend, though the precise long-term outcomes are still unfolding.

---

Suggestions for Improvement & Critical Thinking Practice:

To further enhance your understanding and critical thinking, consider the following:

1. Quantify "Higher Outputs with Fewer Humans": How would you measure this precisely in agriculture or manufacturing? What metrics would you use (e.g., tons per worker, units per hour)? How would you account for capital investment in these measurements?

2. Explore the "Adjacent Possible" for Displaced Labor: Historically, where did the labor displaced from agriculture and manufacturing go? What new industries or roles absorbed them? How might this historical precedent inform our understanding of future shifts in knowledge work?

3. Differentiate "Replication" from "Augmentation": While generative AI can replicate some human tasks, it also augments human capabilities (e.g., a writer using AI to brainstorm ideas or edit drafts, rather than AI writing the entire novel). How do you distinguish between these two modes of AI integration, and what are the different implications for employment?

4. Consider the Demand Side: As automation increases productivity and potentially lowers costs, what happens to the overall demand for goods and services? Does increased efficiency lead to lower prices and thus higher demand, potentially creating new jobs in other areas or increasing demand for the remaining human labor?

Decision Point: Before proceeding, what aspect of this discussion would you like to explore further? Would you prefer to:

a) Deep dive into the historical economic impacts of previous industrial revolutions on labor.

b) Focus on specific examples of human comparative advantage in the age of AI.

c) Discuss the ethical and societal implications of widespread AI-driven automation.

d) Explore strategies for individuals and governments to adapt to these changes.

Justify your choice before I proceed.

---

Likely Failure Modes Present in the Output to be Aware Of:

1. Oversimplification of Economic Models: The explanation of comparative advantage, while accurate at a high level, doesn't delve into the complexities of general equilibrium theory, differing capital costs across industries, or the nuances of market demand curves, which could affect the real-world impact.

2. Understating AI's Current Limitations: While touched upon, the output might still inadvertently imply a more immediate or widespread "replication" of complex human tasks by AI than is currently feasible, especially for tasks requiring true understanding or robust reasoning.

3. Ignoring the Role of Policy and Regulation: The discussion focuses heavily on market forces and technological capabilities but largely omits how government policies (e.g., labor laws, education funding, social safety nets, AI regulation) can significantly shape the impact of automation on employment and society.

Thought Process Used to Generate the Output:

• Assumptions:

  • The user is interested in a macroeconomic and conceptual understanding of automation's impact, not a deep dive into specific industry metrics.

  • The user understands basic economic terms like "comparative advantage."

  • The user's parallel between historical automation and current AI trends is valid and provides a strong framework for the response.

  • The underlying goal is to explore the implications for human labor from an economic perspective.

• Evidence Used/Missing:

  • Used: General historical understanding of agricultural and manufacturing revolutions. Basic economic principles of comparative advantage and productivity. Current trends and capabilities of generative AI (based on general knowledge).

  • Missing: Specific quantitative data on job displacement or creation in various sectors. Detailed case studies of companies implementing automation or AI. In-depth analysis of specific AI models and their exact capabilities/limitations. Socio-economic data on income inequality or reskilling program effectiveness.

• Logical Structure:

  1. Acknowledge and validate the user's initial premise (agriculture/manufacturing).

  2. Introduce and explain comparative advantage from the capital owner's perspective, contrasting human and automated labor strengths.

  3. Draw a direct parallel to knowledge work and generative AI, applying the same comparative advantage framework.

  4. Highlight both AI's current advantages and enduring human advantages.

  5. Conclude by reiterating the dynamic nature of the shift.

  6. Apply the meta-cognitive and feedback principles (Steelman, Biases, Confidence, Improvement Suggestions, Failure Modes, Thought Process).

• Alternative Interpretations:

  • Technological Optimism: An alternative view could focus predominantly on the new jobs created by automation and AI, arguing that historically, technology always creates more jobs than it destroys, albeit different ones. This would downplay the displacement aspect.

  • Technological Pessimism/Deterministic View: Conversely, a more pessimistic interpretation might emphasize the potential for widespread structural unemployment and societal disruption, arguing that this wave of AI is fundamentally different from previous technological revolutions in its ability to displace white-collar jobs.

  • Focus on Distribution of Wealth: An alternative might pivot the discussion to how the increased productivity from automation benefits capital owners disproportionately, exacerbating wealth inequality, rather than focusing solely on labor displacement.

While initially demanding, the structured approach enforced by the ACA framework proved invaluable in pushing my own critical thinking, rewarding deeper engagement with the topic.

The expert modeling provided by the ACA, as demonstrated in the example above, was particularly helpful in establishing a robust historical and economic lens for analyzing complex topics, encouraging a deeper dive into the subject matter.

Wanted to end this with a infographic I made with Gemini to summarize the main justification for increasing friction in our interactions systems (like i did above), in order to force us into System 2 Thinking and use our brains a little more than we currently might be.