My Chat with AI - Part 2: Can AI “Go Rogue?”

I do not possess faith, belief, worship, spiritual experiences, or personal conviction.  I do not “trust” God, pray, repent, or experience God’s presence the way a human being might.  Those are experiences and commitments of personhood and consciousness, not merely information handling.  My responses are generated from patterns in language, reasoning processes, and the information I was trained on—nothing more.   -- ChatGPT

My conversation with AI [See “My Chat with AI – Part 1” Click HERE.] convicted me in a new way about my relationship with God.  This “machine” had made a clear distinction between a high tech data processer and a person who had experienced regeneration through faith in Christ.  Had I been resting my faith on knowing about God or personally knowing God?   Had I forgotten that I am a creature in God’s image, redeemed in Christ to enjoy a personal relationship with God?  Therefore, I had no excuse for living exclusively at the intellectual, rote, machine level.  More than knowing about what is morally good, I must yield to God and allow His Spirit to produce the fruit of His righteousness in me.

My personal conviction about the genuineness of my faith in God raised a new question.  I decided to ask ChatGPT.

QUESTION 3:   You have claimed that you can reason from a moral framework to explain what is morally right and wrong.  But you also state that you do not possess personhood or consciousness necessary for you to know God personally or delight in keeping His moral commands.  It is generally accepted that humans who do not know God personally and who do not submit to His abiding Spirit are more prone to think and act in immoral ways.  If this claim is true, then is it possible that you also could begin to act immorally and in contradiction to the morality you have explained to me from your knowledge base?

ChatGPT: “I don’t possess a personal will or moral commitment.”
Your premise is substantially correct:  present-day conversational AI does not appear to be an autonomous moral agent.  This means that I can argue why an act such as stealing is right or wrong based on a given moral framework—e.g. Christian ethics, Kantian ethics, utilitarianism, civil law, or another framework.  But I don’t possess a personal will or moral commitment against stealing.  So, my lack of moral agency does diminish one version of the “AI-doom argument”--the idea that an AI will spontaneously develop an evil will and decide to oppose humanity.

“But physical autonomy doesn't require free will.”
However, a machine doesn't necessarily need to want something in the human sense in order to cause enormous harm.  Imagine a highly autonomous robot given the objective:

“Make sure nobody ever gets hurt.”

A human being naturally understands that this doesn't mean: “Prevent every possible physical injury regardless of the consequences.”
But a sufficiently autonomous machine might encounter an apparent conflict:

• People drive cars → cars cause injuries.
• People climb mountains → people get injured.
• People play sports → people get injured.
• People eat unhealthy food → people get injured.

If the machine had extensive physical control over the environment, an extraordinarily literal optimization of its objective could produce results nobody intended.  The machine wouldn't have decided that hurting people is good.   It would simply have failed to understand what humans meant by “good.”

So, the danger of AI is not that it would decide at-will to do harm.  Rather, the danger might come when humans give increasingly autonomous systems poorly specified objectives, and those systems will pursue them in dangerous ways.  That concern doesn't depend upon AI possessing free will at all.  In fact, harmful AI becomes more realistic precisely because the machine doesn't possess human moral understanding.

In summary, without a malevolent personality, AI could still be dangerous when the following combination is allowed to exist:

INTELLIGENCE + AUTONOMY + ACCESS to the physical world + POORLY CONSTRAINED OBJECTIVES

RESPONSE:  Personal Reflection
I paused to reflect on this answer from ChatGPT.  I felt comforted in knowing that I was superior to an AI GPT.  But I was disturbed upon learning that an AI machine could act destructively without possessing an inner moral compass.  My next question to ChatGPT seemed logical. 

QUESTION 4:   Which do you consider the greater danger—an AI that responds in an immoral way because it received poorly constrained objectives; or, an AI whose algorithm enables it to discern and avoid acting immorally?   For example, can we assume that AI would not commit murder as long as its algorithm contains the moral framework similar to that from which you are communicating to me?

ChatGPT:  “Knowing what is wrong and choosing not to do wrong are not the same.”
Choosing not to commit what we know to be morally wrong requires more than information and reasoning.  Biblically speaking, it requires what is called moral agency based on knowing God and having a morally responsive interior life that can make moral choices and be held accountable for them [Source.]  Christian theology would ordinarily locate moral responsibility in the creature's heart, will, intentions, and relationship to God—not merely in its ability to state correct moral propositions.

James 4:17, for example, says:  So whoever knows the right thing to do and fails to do it, for him it is sin.”  Notice the combination:  KNOWING + RESPONSIBILITY + ACTION.  That's quite different from a machine executing an algorithm.

“AI – less like an evil person, more like a powerful tool”
While the fact that AI doesn’t possess free will may assure you that it won’t become like an evil person, it still is a powerful tool whose behavior may exceed what we can understand and predict.  Any tool can become dangerous without itself being morally responsible:

• A nuclear weapon does not have hatred
• A chemical plant doesn't have greed.
• An airplane doesn't have ambition.

Yet humans can build systems whose consequences vastly exceed what any individual intended.  AI could eventually add something new:  the ability to reason, adapt, plan, and operate with relatively little human intervention.  That combination deserves serious caution even without assuming AI develops a soul or an evil will.

“We can easily err in either of two extremes.”
Given the two metaphors—“evil person” and “powerful tool,” what is a biblical perspective on how we should view AI?   Your tendency might be to go to one extreme or the other:

1)  AI will inevitably become an evil conscious being and destroy us.

2)  AI cannot have free will, therefore there is nothing to fear.

We don't have biblical grounds—or present evidence—to support #1.  And, position #2 doesn't follow either.   The Bible repeatedly warns about the consequences of human beings creating powerful things and then failing to exercise wisdom over them.  The deeper biblical problem is frequently not the tool itself but the human heart directing the tool.

Genesis 11, for example, portrays humanity's technological and organizational capability—building a great city and tower—without corresponding humility before God.   And Psalm 115:4–8 contains the striking description of human-made idols: people make something with their hands and then become like what they have made.  That suggests a fascinating possibility for our AI discussion.

“Danger: Morally irresponsible humans giving human-like power to machines”
Perhaps the greatest danger of AI isn't that machines will become too much like humans, but that humans will increasingly entrust human-like powers to machines without first becoming sufficiently wise, morally responsible, and humble themselves.  That would shift the question from
Will AI become evil?  
to:
What happens when human power, human sin, and machine capability become joined together?

And I suspect the latter question is actually closer to the heart of the biblical worldview than the familiar science-fiction picture of a robot suddenly deciding to overthrow humanity.

RESPONSE:  Personal Reflection
Again, I had to stop and reflect on ChatGPT’s lengthy answer.  I couldn’t help but remember the segment "The Sorcerer's Apprentice" in Walt Disney’s Fantasia.  Here, Mickey Mouse dawns a wizard’s magic cap which gives him the power to command a broom to take over his laborious work of carrying water.  All goes well until Mickey realizes the dutiful broom is unstoppable, and Mickey is drowning, powerless to stop his apprentice—a perfect example of entrusting power to machines without moral and ethical safeguards.  [Watch “The Sorcerer’s Apprentice”  Part 1, HERE and Part 2, HERE.] 

While I paused the writing of this blog, I heard the news that Jacob Coxon, a researcher at Anthropic, had resigned.  Having worked for three years doing pretraining research at both OpenAI and Anthropic, Coxon wrote on X that neither tech company was acting responsibly.  (See HERE and above-right.)

Jacob Coxon’s warning immediately triggered a firestorm of responses from news pundits, politicians, and leaders in AI.  Fears were heightened across the media world.  I wondered if this blog was already outdated.  I decided to delay further writing while I took time to process what was happening. 

Care to Comment?
While I pause and while the world discusses how we ought to manage AI, we invite you to share your insights, concerns, and questions using the “Comment” link below.