What does ChatGPT mean for CTOs?

Computation shaped human silhouette overlaid on a picture of a human being (Image by Freepik)

A series of articles about ChatGPT and implications for technical leaders:

1. Can ChatGPT help CTOs with Technical Strategy?
2. Can ChatGPT help CTOs with Talent Management?
3. Can ChatGPT help with Operational Support?
4. Lessons learned from ChatGPT about marketing new technologies

Other topics discussed:

• What motivations guide ChatGPT’s training?
• Ideas to push the limits of ChatGPT
• ChatGPT and the problem of training bias
• ChatGPT: Ethical and philosophical considerations
• Does ChatGPT’s language model prioritise readability?

I have a professional interest both leadership and technology. This means that understanding the implications of ChatGPT, and AI in general, is now non-optional.

As part of a crash course on ChatGPT, I recently conducted an interview with the AI. Interacting over the course of three days, I was able to experiment freely with the model. However, despite this investment of time, it was still necessary to limit topics of discussion.

For the purpose of the exercise, I indicated to ChatGPT that I am a CTO, although my current role is more consultative in nature.

My general interview technique in this instance is as follows:

1. Start with a general question about a topic pertinent to technical leadership. This elicits a high-level answer from ChatGPT.
2. Respond by selecting part of the response for deeper discussion. At times this relates to using ChatGPT in a useful way. But, sometimes the deeper discussion is to understand the technology behind ChatGPT.
3. As ChatGPT answers questions, watch for unusual or notable features of the conversation. Ask reflective or meta-questions to better understand the mechanisms at work.

Pre-planning and setting up the conversation

In order to establish a rough agenda for the interview, I decided to brainstorm the topics which are most pertinent to a typical CTO. Although not part of the main interview, I also used ChatGPT to assist with this brainstorming process.

I started with the prompt: “What are the most important things for a CTO to focus on?”. This produced a list of eight points, which I asked ChatGPT to narrow down by picking the top three. The resulting list was:

1. Technology strategy: Developing and implementing a technology strategy that aligns with the overall business goals of the organization is crucial for the long-term success of the company.
2. Talent management: Attracting, retaining, and developing top technical talent within the organization is essential to ensure that the company has the right people in place to drive innovation and execute on the technology strategy.
3. Operations: Ensuring that the organization’s technology infrastructure and systems are reliable, efficient, and secure is critical to maintaining the integrity of the company’s data and systems, as well as ensuring continuity of operations.

We could debate this choice, but as there is no specific context provided, it’s a pretty good starting point. I use these three categories to structure the main interview with ChatGPT.

Introductions, and some ground rules

ChatGPT is capable of remembering context and applying it to an evolving conversation. Here’s how I introduced the subject to ChatGPT:

Interviewer ❯
Hello, I’d like to conduct a short interview. The intention is to publish it in an online forum such as LinkedIn and/or Medium. The intended audience is technology leaders including CTOs, Technical Directors and/or Heads of Engineering.

ChatGPT ❯
Of course! I’d be happy to help. What are the questions you would like to ask?

Andrew Gibson ❯
Before we start, can I suggest that we keep answers fairly short — perhaps two or three sentances?

ChatGPT ❯
Sure, I’ll do my best to keep my responses concise.

Notes:

1. In the first exchange, we see ChatGPT responding even without a direct question. This exhibits understanding of natural human communication flow. Typical chat bots struggle to respond without discrete prompts.
2. In the second exchange I ignore the question ChatGPT poses. Instead I suggest guidelines for the later conversation. ChatGPT is able detect the shift in conversation.
3. As agreed, ChatGPT produces concise responses for the next five or six exchanges. But, later in the conversation, it defaults back to long-form answers. This is a rather human-seeming characteristic.

The “Uncanny Valley” and the ELIZA effect

The ELIZA effect⁵ refers to a chatbot created in 1964. It was convincing enough that people attributed intelligence to it which did not exist. This has been a recurrent problem.

In a similar vein, over the last sixty years, technologists have explored the concept of the “Uncanny Valley”. This is the phenomenon of empathising with objects when they achieve human-like characteristics.

Arguably, artistic endeavour of all kinds exploit this trait. But, the phrase “Uncanny Valley” emerged in relation to robotics and similar technologies.¹

ChatGPT is able to exhibit human-like characteristics such as those noted above. This leads to an “uncanny valley”-like experience, especially given extended usage. Its facility with human language means that attributing human intelligence reduces cognitive load.²³⁴ This leads to a confusion on the part of the user about whether they are interacting with a human or a machine.

So, what’s different this time around?

Our roles as technologists are changing. This has always been true. But the advent of generally available Artificial Intelligence is a watershed moment. And companies like OpenAI (ChatGPT’s creator) are only getting started. They aim to create AIs more powerful than ChatGPT — the fabled AGIs (Artificial General Intelligence).

Technology is becoming a more active participant in our endeavours. The role of managing teams of humans might start to encounter hybrid teams of humans and machines.

So ignore Gartner’s hype cycle claims. We aren’t headed for the Trough of Disillusionment any time soon.

Gartner’s hype cycle for AI 2022 shows Natural Language Processing as entering the Trough of Disillusionment

Citations

¹ Misselhorn, C. Empathy with Inanimate Objects and the Uncanny Valley. Minds & Machines 19, 345 (2009). https://doi.org/10.1007/s11023-009-9158-2

² Sweller, J., Ayres, P. and Kalyuga, S. (2011) Cognitive load theory. New York, NY: Springer New York.

³ Skelton, M. and Pais, M. (2019) Team topologies organizing business and technology teams for fast flow. California: IT Revolution Press.

⁴ Team Cognitive Load (no date) IT Revolution. Available at: https://itrevolution.com/articles/cognitive-load/ (Accessed: January 23, 2023).

⁵ Felton, J. (2022) The eliza effect: How a chatbot convinced people it was real way back in the 1960s, IFLScience. IFLScience. Available at: https://www.iflscience.com/the-eliza-effect-how-a-chatbot-convinced-people-it-was-real-way-back-in-the-1960s-64155 (Accessed: January 31, 2023).