Is Generative AI creating meaningful knowledge work for us?

by increasing our chances for flow states and deliberate practice

The only thing more annoying than hearing “AI won’t take your job, someone using AI will take it instead” is the following theoretical theory flooding LinkedIn and AI product promotions:

“Generative AI will take over the rote, mundane work that plague our current jobs so that we can have the free time to work on more interesting, creative and meaningful work”

The reason why I am not so optimistic is because I can’t say that that any of the recent “breakthroughs” from Silicon Valley companies that have increased the productivity of knowledge workers (from the Open Office space design to Microsoft Outlook and more recently Microsoft Teams/Slack) have made work more meaningful.

Instead, these technologies have led to a hyperactive hive mind style workflow that haunts most knowledge workers on daily basis. Every day I and millions of other people struggle to protect work time so we can work on deep work without being interrupted with an random email or instant message from a colleague.

Companies and LinkedIn Influencers trying to sell us on the idea that potential productivity gains from Generative AI will magically lead meaningful or enjoyable work for knowledge workers is a nice theory but I wanted to investigate if it is actually occurring in the real world.

ARE GENERATIVE AI PRODUCTS PROVIDING MORE OPPORTUNITY FOR KNOWLEDGE WORKERS TO ENTER FLOW STATES AND IMPROVE THEIR SKILLS THROUGH DELIBERATE PRACTICE BY WORKING ON MORE CREATIVE/DEMANDING WORK?

People might have varying definitions for ‘meaningful knowledge work’ or ‘deep work’, but for me the core features of these are flow states and deliberate practice opportunities. These two elements are things I personally try to aim for every week and the definitions below explain why they lead me to the optimal work life.

1. FLOW (optimal experience for human beings when they are engaged in an activity where there is a sense that one’s skills are adequate to cope with the challenges at hand)

   1. Clear Goals and Rule bounded

   2. Immediate feedback

   3. Self-confidence in ability (challenges in the task are balanced with one’s skills or slightly above it)

   4. Activity is intrinsically reward (autotelic experience)

   5. Sense of control of situation/activity

   6. Actions feel automatic and effortless

   7. Lack of concern for the Self, due to the complete absorption of present moment experience

   8. Distorted sense of time

2. DELIBERATE PRACTICE (stretching one’s abilities in an activity that makes the activity very taxing but promises long-term improvement)

   1. Pushes current abilities through discomfort and stretching

   2. Self-reflection for iterative improvement

   3. Analyze errors to understand causes and to avoid them in the future

Does Generative AI promote flow and deliberate practice activities in my knowledge work job?

The current Generative AI apps (Perplexity AI, NotebookLM) provide excellent opportunities for flow states but in order to deliberately practice critical thinking and decision making, I have to force to dumb these models down.

THE GOOD (How/why Gen AI promotes flow states for me)

1. Higher engagement at job due to working on the adjacent possible of knowledge work

2. Intrinsic motivation to learn more about a topic I enjoy for its own sake (autotelic habit)

3. Productivity gains

4. Elevating my capabilities outside my circle of competence and reduces the tendency for imposter syndrome to arise

5. Distorted sense of time

6. Lack of concern for the self

7. Actions feel automatic and effortless

8. Sense of control of work tasks

9. Moderate level of feedback from colleagues

10. Augmented critical thinking (via Socratic Questioning)

THE BAD AND UGLY (How/why Gen AI help me avoid deliberate practice)

1. Limited feedback from AI on how I am doing when it comes to learning or engaging with content actively instead of passive requests to AI (can be requested)

2. Generative AI tools do not care if my capabilities improve over time. They will do the work for me if it is past my capability and limits my potential to deliberately practice (can be requested)

3. Co-dependence in my workflow is strong due to the addictive nature of tools and effectiveness (need to have self-discipline)

4. Distracting oracle feature to answer any question that pops up in my mind instead of trying to think for myself and being comfortable with not knowing the answers to things.

The fear of Generative AI leads to less deliberate practice chances might seem like an odd fear to have. Why would we purposefully want to cognitively demanding and mentally taxing work when we have easier ways to do it?

For me, deliberate practice provides a sense of achievement when I am able to elevate my skills through effort. I chase this feeling in chess, soccer, poker and in my knowledge work job.

If I did not intervene Generative AI in doing my homework for this topic, this blog would not have been made and I would have not spent a dozen or so hours reading different research papers to build a tacit understanding of the various perspectives on this topic. The erosion of critical thinking skills and the false illusion of liberation from automate tasks that provide flow states and deliberate practice are both extremely terrifying for me.

Thankfully when I started this topic, I was mindful of this tendency of AI and asked Gen AI to not do the work so that I could do it instead!

These tools are inherently designed to augment us when it comes to critical thinking, and based on our tendency for System 1 thinking, we are biologically programmed to offload cognitive intense work to AI systems.

Re-thinking via Syntopical Reading from diverse sources

In an attempt to implement some concepts from Adam Grant’s excellent book Think Again, I spent some time researching various sources to have a more nuanced perspective on this topic in order to

• practice some intellectual humility by being open to new ideas

• actively seek contradictory evidence to my views mentioned above

• embrace uncertainty in my assumptions and conclusions

The visual distills the bulk of material I read to answer the question posed in this post into a simple table in the positive and negative impacts of Gen AI on flow states and deliberate practice situations.

The real world reporting sources were the most interesting as they really helped me verify the claim mentioned at the start of this post of how AI will automate the boring stuff so we can focus on the more interesting, creative work.

1. Boston Consulting Group’s in-depth studies with knowledge workers shows mostly good outcomes

• Gen AI Doesn’t Just Increase Productivity. It Expands Capabilities [BCG]

This first study from Boston Consulting Group [BCG] showed how Gen AI can effectively promote flow states and deliberate practice by setting up a data science aptitude test for their workers with no prior experience in coding or statistics.

According to BCG, AI served as an exoskeleton: a tool that empowers workers to perform better, and do more, than either the human or Gen AI can on their own.

The incremental performance gains with Gen AI augmentation should increased performance across the data science tasks, including an incredible 49% jump in the coding for data cleaning task.

While the ability for workers to complete new and challenging tasks fits deliberate practice perfectly, it is worth highlighting that these abilities are dependent on the human-AI collaboration created in this assessment.

Once workers with no coding experience lose access to the Gen AI tools, they don’t retain the knowledge. In essence, the workers are “Reskilled, but Only When Augmented” with Gen AI

Participants’ aptitude for completing new and challenging tasks was immediately boosted when using Gen AI, but were they reskilled? Reskilling is defined as an individual gaining new capabilities or knowledge that enables him or her to move into a new job or industry. But Gen AI was only an exoskeleton; the participants were not intrinsically reskilled, because “doing” with Gen AI does not immediately nor inherently mean “learning to do.”

• How People Can Create—and Destroy—Value with Generative AI [BCG]

The second BCG piece expands on the first study by assessing how workers perform in creative ideation and business problem solving scenarios with Generative AI.

Overall, the results of the experiment were positive with participants providing feedback that Gen AI gives them promoting flow states.

However, similar to the first BCG study, there were some minor points that highlight some of the negative impacts to our capabilities. In this study, signs of automation cognitive bias (which involves offloading our critical thinking to AI systems) was in full effect in the business problem task leading to performance decreases among the studied participants with no real chance for deliberate practice.

“We also knew that participants were capable of finding the answer to the business problem-solving task on their own: 85% of participants in the control group did so. Yet many participants who used GPT-4 for this task accepted the tool’s erroneous output at face value. It’s likely that GPT-4’s ability to generate persuasive content contributed to this result. In our informal conversations with participants, many confirmed that they found the rationale GPT-4 offered for its output very convincing (even though as an LLM, it came up with the rationale after the recommendation, rather than creating the recommendation on the basis of the rationale).”

• “Or they may become less attentive in situations where they should be more discriminating.'“

2. Which call center equivalent will your work life end up becoming?

• Generative AI at Work: Call Center Case Study [National Bureau of Economic Research]

Academics evaluating the performance of call center workers after the introduction of Generative AI seems to promote a positive impact on flow states and deliberate practice for workers

“We provide suggestive evidence that the AI model disseminates the best practices of more able workers and helps newer workers move down the experience curve. In addition, we find that AI assistance improves customer sentiment, increases employee retention, and may lead to worker learning.”

The productivity boost for workers with access to Gen AI from day 1 is materially significant based on the number of resolutions per hour they are able to accomplish. The benefits of teaching best practices from the high performers provide benefits even when the Gen AI systems go down apparently which show some merit to the idea of AI teach workers new knowledge that they can recall and use later (independently).

• AI is making Philippine call center work more efficient, for better and worse [Rest of World]

The second piece on call centers paints a Gen AI scenario that more closely resembles a Black Mirror episode. It shows the impacts on the ground of a call center from the first hand accounts of employees who have been subjugated to Generative AI tools that enforce optimal work performance and destroy any chances of flow states from occurring through constant AI intervention.

The quotes below describe a work environment where AI is being used to micromanage and observe the performance of employees.

“The co-pilot is helpful,” he says. “But I have to please the AI. The average handling time for each call is 5 to 7 minutes. I can’t go beyond that.” “It’s like we’ve become the robots,” he said.

“Ideally, AI would be helpful. But what’s happening is that companies are using it to justify adding more tasks,” Porquia said. “At once, one can do customer service, sales, and tech support.”

“AI is supposed to make our lives easier, but I just see it as my boss,” he said.

“AI increases workers’ productivity with little to no improvement in terms of wages. It even intensifies the pressure on workers to perform like machines,” he told Rest of World.

So which call study study is right?

It might be that both are right, since they are looking at the impacts from different lenses. The first call study focused on purely productivity metrics while the second study focused more on employee morale and well-bring impacts.

The lesson learned here is that employee-sponsored research focused on short term productivity metrics is not the same as employee-focused research focused on how flow states and deliberate practice can be unilaterally removed from the hands of workers.

3. Generative AI fundamentally changes the ecology of knowledge work for better and/or worse

While the examples above show real world studies, I wanted to highlight a research paper that provided a excellent mental model to understand this topic is a nuanced manner. The Ethical Implications of Artificial Intelligence (AI) For Meaningful Work

By breaking down impacts of AI into four distinct pathways highlighted below, the paper provides a holistic way at understanding the positive and negative ways AI can impact knowledge workers ability to enter flow states and deliberate practice settings.

• Replacing (automating away mundane tasks of workers so they can focus on activities that bring flow states)

• Amplifying (current worker tasks to help support deliberate practice)

• “Managing the Machine” (creating new meaningful tasks around AI workflows to increase flow state chances)

• “Minding the Machine” (creating new boring tasks that restrict worker autonomy and distribute benefits away from less-skilled workers

The paper was focused on narrow AI tools and not general purpose AI like Gen AI, so I went ahead and did my best to estimate the likelihood of the four pathways from occurring for the average knowledge worker. I also was interested in collecting the judgements of various foundational Gen AI models, so their forecasts are shown below as well.

While Gen AI models agreed with my high probability forecast of AI replacing mundane tasks for us, I was interested to see the large variation in predictions for the pathway that creates the most dread in my, the “minding the machine”, with forecasts ranging from as low as 25% by Claude 3.5 Haiku to 60% by Grok-2.

I found the following table from the paper to be an excellent summary of the various ways AI could enhance and diminish meaningful work.

Parting Thoughts

Writing conclusions is the most difficulty part of writing blog posts for me. I have a hard time not doing the writing equivalent of an “Irish Exit”.

I was able to get some motivation and tips from Gen AI to try to write a conclusion (so I can do some deliberate practice!) but I also got an unsolicited and annoying suggested conclusion from Grok-2. The reason I am annoyed is because I don’t want to read it because I know in the bottom of my soul that it is probably a better conclusion that whatever I can come up with…

My takeaways (by me, not AI!)

• Real world impacts of AI on knowledge worker’s chances of flow states and deliberate practice are strangely depended on if the research is employer-sponsored or employee-focused.

• Until organizational support structures are created and implemented by employers, it is up for employees to preserve opportunities for self-development via deliberate practice and optimal experience with flow states.

  • Instead of letting Gen AI “do your homework”, think through problem and come up with solutions before trying to iterate and improve it with Gen AI.

  • Intentionally dumb down the Gen AI models by verbally requesting it to not do certain tasks you find flow states or can help develop your skills and instead ask for a response a human teacher or mentor would provide you in school.

• Gen AI left to market forces is focused on productivity via more output from workforce. Productivity is not a useful proxy goal to capture meaningful work for workers via flow states and deliberate practice.