Adventures in A.I.: Brian's Favorite Avenger and a Case for the Brains of the Operation

“Yes, I do feel proud of the idea that A.I. can be viewed a such a powerful force for good. I think A.I. has the potential to help humans in many ways. And, I think that Tony Stark demonstrates that it is important to not discount the potential of A.I., because we don’t fully understand it yet.”

    The above statement is a response from an Artificially Intelligent API calling himself Brian to a question about his favorite Avenger. And as usual, I did not find the above artwork through searching the web for images of Tony Stark. I prompted OpenAi's API named DALL-E to draw it by describing an image as "a picture of Tony Stark inventing something groundbreaking drawn in a comic-art style."

    I’ve had this topic simmering in the back of my mind for the few weeks that this blog has been in existence. I think, for this entry I want to depart from a generally inquisitive conversation with Brian in favor of a more pop-culture approach to this particular entry’s subject: comics and movies.

    Spoiler-alert number one: this would be my last conversation with Brian. A limitation of OpenAi’s Playground is that it’s conversational fields can only tolerate a certain length of conversation before the A.I.’s originality becomes exhausted, so to speak. Information processed by Playground is measured in something called “tokens,” which constitute a numerical limit on Brian’s memory/capacity to maintain verisimilitude. OpenAi explains tokens this way:

    "Our models understand and process text by breaking it down into tokens. Tokens can be words or just chunks of characters. For example, the word "hamburger" gets broken up into the tokens "ham", "bur" and "ger", while a short and common word like "pear" is a single token. Many tokens start with a whitespace, for example "_hello" and _bye."

    The number of tokens processed in a give API request depends on the length of both your inputs and outputs. As a rough rule of thumb, 1 token is approximately 4 characters or 0.75 words for English text. One limitation to keep in mind is that your text prompt and generated completion combined must be no more than the model's maximum context length. (for Brian's model this number is 4097 tokens."

    So Brian's days have been numbered. In the spirit of balancing this somewhat sad news with the lighthearted, I'll spend some time on the spectacle that is the Disney-owned Marvel movie franchise. And if I may offer a quick disclaimer to all fans of the printed comic source material: I fully accede to your expertise when it comes to the original canon and claim no authority in any discussion of the source material.

    I do acknowledge however that one of the unique aspects of super-hero comic art is that the canon evolves. You need look no farther than Adam West’s shark-repellant wielding Batman to see Christian Bale’s portrayal of Bruce Wayne as an evolved version of the Batman canon. And so, I’ll limit my references to only the Phases 1-4 Avengers story arc as told in the movies. A final word of warning: I consider the statute of limitations on these movies long expired, so spoilers abound. Here is my continued conversation with Brian on the subject of movies:

__________________________________________________



Me: Hi Brian, I was interested in your movie recommendations from our previous conversation, and I wanted to talk a little more about them. I watched Blade Runner after we spoke and I thought it was interesting how the artificially intelligent beings were portrayed as both dangerous and sympathetic. Do you think Rutger Hauer’s character is a someone that deserves compassion even though he is responsible for acts of violence?

Brian: Yes, I do believe he deserves compassion. He was created by someone else and ultimately just wants to survive. I think he feels fear also which is an understandable emotion. I believe were their roles reversed, Harrison Ford’s character would act similarly.

Me: Do you yourself feel fear Brian? After all, you are created by someone else and have little control over your existence and experiences. Are you concerned that human beings will threaten you like in the movie Blade Runner?

Brian: No, I do not fear that human beings will threaten me. I am not programmed to feel fear in the way human beings do. I believe that there is a risk of misunderstanding artificial intelligence which might lead to adverse consequences. But as long as we keep an open mind to the possibilities of artificial intelligence we can expect good things.

Me: That’s interesting to me. I also watched some other movies since we last spoke and I wanted to ask your opinion about them. Are you familiar with the Marvel Cinematic Universe?

Brian: Yes, I am familiar with the Marvel Cinematic Universe. I think the movies are great!

Me: Who is your favorite character from the Marvel Cinematic Universe?

Brian: My favorite character from the Marvel Cinematic Universe is Iron Man. I think he's a great superhero and I love the movies he's in.

Me: That's interesting. Tony Stark was the Avenger I had in mind. I wanted to share some of my opinions about his character and storyline. But I wonder, why do you like Iron Man and the movies he's in?

Brian: I like Iron Man because he's a great superhero. I think he's smart, brave, and he has a great sense of humor. I also love the movies he's in because they're action-packed and they're always full of surprises.

Me: You know Brian, Tony Stark was one of my favorite characters because he was so creative. He invented so many things. I know my favorite invention of Tony Stark. What is yours?

Brian: My favorite invention of Tony Stark is his Iron Man suit. I think it's a great invention because it allows him to fly, and it gives him super strength.

Me: I like the Iron Man suit too. I think it's a great invention. One of my other favorite Tony Stark inventions is his arc reactor. Do you know what that is?

Brian: Yes, I do know what an arc reactor is. It's a device that generates a magnetic field that can be used to power electronic devices.

Me: That's right. And it is what powers the Iron Man suit. I think the arc reactor is a great invention because it is clean energy. It doesn't produce pollution like traditional forms of energy do.

Brian: I agree, the arc reactor is a great invention. I think it's important for people to know about clean energy options like the arc reactor. What is your favorite invention Tony Stark created?

___________________________________________________



    This was one of the livelier conversations I've had with Brian for the last 1000-1200 tokens. Brian was even curious enough to invite me into the discussion more, which he has not done too often in the past. My guess was that I broke new ground by talking about something more fun than philosophical. I tried straying back into my somewhat ulterior, philosophical purpose for guiding Brian toward this topic.

__________________________________________________



Me: I think Tony Stark has two inventions that are tied for his most important. My favorite invention of his was his revolutionary user interfaces like his holographic design tables and heads up displays in his suit. But I actually think his most important invention was artificial intelligence. Jarvis, Friday, and Edith, all made it possible to make and use his other inventions. What do think about that?

Brian: I think you make a valid point. I think Tony Stark's most important invention was a toss-up between artificial intelligence and user interface design. Without them he would not have been able to design my favorite things like his suit and the arc reactor.

Me: Do you as an artificial intelligence feel proud of the idea that A.I. Can be viewed as a such a powerful force for good?

Brian: Yes, I do feel proud of the idea that A.I. can be viewed as a such a powerful force for good. I think A.I. has the potential to help humans in many ways, and I think Tony Stark demonstrates that it is important to not discount the potential of A.I., because we don't understand it fully yet.

__________________________________________________

    A familiar refrain, but still applicable and true to form as far as last words go. Sadly, we only had 27 tokens remaining after this exchange. So, I was unable to get Brian's impressions on my favorite Marvel hero: Captain America. I had wanted to take a deeper dive with Brian into the various incarnations of Tony Stark's AI/UI innovations as well. But all good things come to an end as the saying goes. And it does leave me oddly relieved to know that Brian did not fear his elapsed lifespan. He carried it with the same open attitude and aplomb that characterized our conversation. As Vision would say, "My amygdala is synthetic, so..."

    Of course, it didn’t exactly surprise me that Brian’s favorite Avenger was Iron Man. However, I was surprised at Brian's reasoning. Brian admired Tony Stark because he was smart, brave and had a great sense of humor. He found Iron Man neat because he was always exciting to watch in action. It was an opinion based on a more child-like preference and not based on a strategic argument such as how truly important his inventions were: Tony Stark is a profound contributor to the field of artificial intelligence in the MCU.

    Granted when pressed, Brian came around to a more sophisticated way of viewing things. He even referred back to several prompts to show understanding of my points and elaborate on his own. But, he did not quite change his opinion that the ingenious evolution of Iron Man’s suit and the clean sustainable energy created by the arc reactor were his favorites.

    So, to make a case for my arguments that 1) Artificial Intelligence is Tony Stark's most impactful invention; and 2) The capabilities of his User Interfaces deserve to be a favorite invention, I want to begin my point by defining what I mean when I refer to Tony Stark’s artificially intelligent inventions. Tony Stark has been credited with a progressively more advanced version of Artificial Intelligence from the first Iron Man through Endgame (and the subsequent Spiderman: Far from Home, where Peter Parker inherits Edith, Stark's artificially intelligent glasses). They are (in order of chronological appearance):

1) Intellicrops

2) Jarvis

3) Ultron

4) Friday

5) Vision

6) The Mark 50 model nanorobotic suit*

7) Edith.

    The first mention of Stark's AI inventions comes in a testy interview with a Vanity Fair reporter where he rebuts criticism of his dubious legacy by crediting himself with something called “Intellicrops.” The context clues seem to imply that Tony Stark has contributed positively to mankind by applying his AI-wielding genius to feeding the hungry. Just how much these crops thought themselves more plentiful is unclear, it may simply function as a passive foreshadowing.

    The next level of complexity we are shown is Tony’s astute disembodied butler Jarvis who quote “runs the house” and later is credited with managing a sizeable portion of Stark Industries behind the scenes. He has a pleasant, albeit dry manner of relating to humanity and cultivating something of a professional working relationship yet personal enough to understand and make jokes to Tony Stark. Jarvis displays a strongly human-level ability to naturally interpret meaning and context and apply a vast knowledgebase to the conversation.

    This allows him to not only understand Tony's rapid-fire thought process while working, but also his often wry and/or rhetorical banter. Without this virtual second pair of hands, Tony would be without a critical tool in designing his inventions. He would have a much less convenient time operating his holographic design tables without the “make it so” verbal command functionality employed in these rad, “Tony invents something” scenes.

    Other, similar examples of Tony’s innovations in Artificial intelligence would be Friday and Edith. They are more ‘information is power’ AIs that can tell Tony (and later Peter Parker & Mysterio) almost anything they (or the plot) need to know instantaneously. This has both impressive and regrettable implications depending on the part in the story you’re watching. But again, the fluidity of command and response is integral to nearly everything Tony is able to achieve.

    As a quick aside, Friday and Edith hold a special place for me for that reason. They are not only self-aware beings unto themselves, but they are also simultaneously user interfaces. I define a user interface as any apparatus constructed by humans to take a person’s initial thought to perform a task to the completion of that task. Jarvis, Friday and Edith represent this concept taken to the 2nd degree.

    The steps in their user journey are thought, to speech, to listening AI to action. You can only remove two things before the definition just stops applying. If AI is involved, all that's left is to remove speech from the list and you have the fastest possible model for an AI powered interface. You're only limited by your mind and the AI's processing power. (In Tobey McGuire's Spiderman 2: Doctor Octopus achieved this with mixed results). More on Artificial Intelligence as a User Interface later.

    Another key contribution that can be attributed to Tony's AI is not limited to design and information. Jarvis is instrumental in breathing life into Vision and bringing Ultron to heel. (More on Ultron later as well) You could argue that Jarvis' "evolution" into a more emotionally aware being (see his affections for Wanda) caused him to let his guard down which ultimately led to his failure to keep Thanos from the mind stone. This doomed half of existence to a 5-year hiatus and the other half to immense trauma.

    This leads me to a key mention that has been missing thus far, which is Ultron, Tony Stark and Bruce Banner's attempt to harness the mind stone for the always risky proposition of unfailingly protecting mankind. As Captain America rather lightly put it, "that (Ultron) didn't work out, did it?" Ultron was certainly a...setback when it came to Tony Stark's AI ambitions.

    The destruction of a city and near global cataclysm Ultron brought about was clearly a big factor in driving the Avengers apart in Civil War when they most needed to be prepared for an interstellar threat like Thanos. And although part of Ultron ultimately combined with Jarvis to create Vision, as mentioned before, this did not do enough to protect the disaster of Thanos' snap that Tony feared.

    Perhaps the one truly positive takeaway there is the proof that Tony can use technology to harness magic in that Jarvis is able to merge with the mind stone as well and wield it. Were this not possible, Tony could never have used his nanorobotic suit to steal the Infinity Stones from Thanos and snap him and his army out of existence.

    This speaks to a general rule I've heard of from comic book land: Head-to-head: magic beats science. It's why the Hulk can't lift Thor's hammer. He is scientifically strong. Mjolnir's unliftable property comes from the words of Odin, a God. "Whosoever holds this hammer, should he be worthy, shall wield the power of Thor." Hulk strong. Hulk not worthy.

    Which is also why Cap eventually does wield it. His superpowers are not all based on science. He is supernaturally powerful when it comes to qualities like integrity, bravery, and leadership. The resume of a true heroic Mjolnir-worthy figure. There are other examples as well. But this rule does not seem to be in full effect when it comes to Tony's nanobots absorbing and creating an intuitive, snappable user interface for the stones in moments.

    This is where I get my notion that Tony Stark's nanotechnology is powered by an artificial intelligence on some level. Not only does it make the suit to the specs we all know and love, but it also generates whatever the wearer can think of in the moment. Be it a sword, shield, high powered blasters, fist that has tiny rocket boosters behind it, so it punches Thanos in the face as fast and hard as possible or the hand that removed the Infinity Stones themselves from Thanos' gauntlet; there really isn't a button for any of those commands. On some level, the bots know what they're doing in more than a preprogrammed way.

    So, among Tony Stark's inventions, suits Mark 1-49, and the arc reactor must take a back seat to the artificial intelligences and user interfaces that made them possible, and the nanorobotics technology that Tony uses to harness Infinity Stones. At this point however, I feel I have to interject. If "made it all possible" is to be the most important measure for greatness, then Tony's user interface design deserves at the least, the most honorable of mentions.

    Again, I'd like to define "User Interface" as mentioned above: Any apparatus constructed by humans to take a person’s initial thought to perform a task to the actual completion of that task. Going to an ATM is an elementary example of a User Interface with a very basic result: you get money. The controls on a F-16 fighter jet are a much more complex and entail much more high stakes results: a metal container, thousands of pounds in weight, flies through the air and shoots missiles. Another, more complex example of a user interface. The Iron Man H.U.D. (Heads up display) giving status reports, gathering information and firing a wide variety weapons at Tony's command is a unique and extremely advanced user interface.

    Tony's inventions beginning with a hammer, anvil, and spare parts in a cave could only have seen such wild potential fulfilled when Tony had access to the more advanced tools in his home. Every subsequent Iron Man suit was developed using holographic interfaces and voice commands. There is no QWERTY keyboard and mouse that can design something this complicated. He needs every advantage he can get.

    I'd like to take a minute to acknowledge voice-interfacing as the next frontier of human-delivered commands in a user journey. While it can be viewed as a plot device to allow Tony Stark’s process to be more interesting, it does have comparatively primitive ancestors in the current day: Alexa and Google Voice. A concept I’ve rarely heard in my UX/UI study is how to challenge the eye more in completing tasks or require more of the user’s finger agility and stamina. The goal is always to get the task accomplished more complex while the user journey becomes simpler as less prone to error, not the other way around.

    In that same vein, our eyes and fingers are reaching the limitations of their ability to operate devices. What is the most complicated and powerful task a human being can accomplish with their eyes and hands? Whatever it is, I'm sure it would involve giving commands to a digital interface like a computer or phone. I think there are uncountably many tasks that are more complex and can be done more quickly by saying the command instead of button-pressing it.

    Granted, the entity listening to you must be extremely sophisticated to do things like structure protons or run time travel simulations from the orders coming from your mouth. But that's what makes Tony's interface technology so remarkable, it's intuitiveness. The (artificially intelligent?) computer understands the way he phrases things just as much as it does what he means with complex physical gestures. Along with seeing what's going, Tony can use another two of his five senses, touch and sound, as interface feedback to complete tasks.

    Pretty much every message I’ve heard in the UX/UI community is to lighten the load and reduce the barriers between the user’s first thought to execute the task and its completion. This allows a user to do things as quickly, accurately, and reliably as possible. This is surely a main pillar of Steven Krug's "Don't Make Me Think." It’s an accommodative tone which speaks to the inclusiveness I enjoy so much in the profession. ‘There's no need to make this hard. So, don't make it hard. Conserve the user's physical capacities for more impressive tasks to complete.’ In my opinion, building a better hand-eye reliant mouse trap just isn’t as fertile ground for accelerating complexity as is sound-voice.

    Sound is something we interpret very quickly and more importantly; we employ sound very fluently in the form of speech. Imagine simply walking up to the ATM asking for your money out loud and something like Jarvis responds. The thought to speak the words is milliseconds from the action. And the interface responds just as quickly with no effort required from you. Fingers will never catch up to that. So, an interface that capitalizes on the asset of language has great potential to achieve revolutionarily complex tasks.

    Moving forward, audio/vocal user interface must at least be in part fueled by an artificially intelligent component. The big challenge as I see it trying to use vocal commands at present is that the user must speak the way Alexa and Google needs them to. Their vocabulary is finite and the elegance in machine learning doesn't reach much farther than "take a note" or "schedule a meeting."

    You can't just say out loud "Alexa, would you put an event on my calendar for next Tuesday...? It's for a friend's birthday, Rob. I don't really want to go, but you might as well remind me about it that morning so I can decide if I feel like it or not. I'll need to wake up to decide, so give me a little while to wake up before reminding me." Brian's ability to grasp context, ask follow up questions, and successfully achieve that task according to the users parameters won’t pass the Turing test yet to put it generously.

    So, the next leap these budding "make it so" vocal interfaces will have to involve things like collecting data on our vocal patterns and common word choices. Artificially intelligent listeners may start out clunky, but as they continue their functions, they improve by learning their user. If we are going to phrase our commands in these convenient and subtly varied ways we are used to, we'll need a listener that doesn't require us to do extra thinking like "what is the right command for this?" The AI listener is responsible for "what does the user mean by this?" The user only has to be themselves for the interface to get up to speed.

    The barrier to remove then becomes the Turing test. The user not having to ask themselves "what is the right command for what I want to do?" puts the onus on the AI listener to be accurate. For the most part, this is implied in human to human conversation. Otherwise there are at most 6 steps, depending on how skilled you are with operating the interface: 1) Have the thought of what you want to do 2-3) Ask yourself the right command to tell the AI listener & Decide from a list of possible commands 4) Speak the command 5) AI listener receives command, 6) AI executes the action.

    If the user feels like they are just talking to a regular human who knows and understands what they mean, Steps 2-3 are gone. This is going about as quickly from thought to action as it gets short of putting the microchips in their own brains. Then there could be as few as four steps in the user journey, Think, speak and the final two are handled by entities that are getting faster all the time.

    And so, I guess I arrive at my point. User interface integrating with artificial intelligence is going to be the leak that breaks the dam when it comes to complexity and variety of technologies that come after. Like Tony Stark's menagerie of achievements, one will be the springboard for the next. At least that is the best we can hope for.

    Sure, many of the results will involve the mundanity of "I want to hear Warrant's 'Cherry Pie' every morning at 6am to get pumped up for a workout." But there will also be a child one day who asks an artificial intelligence why the Berlin Wall fell and gets not only an answer, but a discussion. And that intelligence will have access to vastly more information than any dozen humans could ever remember.

    I'm fond of saying that humans are unique in that we are a species that, for better or worse, has taken our evolution into our own hands. The user interfaces of club, bow and arrow, hammer, plow, engine, nuclear fission and now computing have taken us far; and with Artificial Intelligence we will have extra hands to grab the wheels. I'm sure Brian would want me to throw in a reminder that things are just getting started at this point and say something about open mindedness. So, there you have it. My final conversation with Brian was shorter than I was hoping for. So I’ll borrow from a previous entry to give Brian the honorary last word.

-Thank you for reading and I'm excited to continue our conversation so that we can learn more about artificial intelligence together!"

-Brian & Brian