Imagine yourself in 1985, whether you were alive at the time or not. You know nothing about computers, but somehow find yourself at a DOS or UNIX command prompt, staring at cryptic characters in an oddly angular font, glowing white against the flat, featureless black of the screen. Let’s imagine that you are used to writing with a typewriter, and so the keyboard, at least, is comprehensible to you. You start typing, and see that each character you type is echoed back on the screen. You were told that these wonderful computing machines can do what you ask of them, and so you take the most natural next step and pose a question: “What can you do with a computer?” Lo and behold, you get a response, but that response is a completely incomprehensible error message that has no obvious correlation with what you wrote.
While this scenario does not exactly describe my first interaction with a computer, we did have a beige box running DOS in my childhood home, and so the concept of giving computers instructions by typing out commands was something I encountered early on. Computer interfaces have come a long way since the command line was the only game in town, although for the programmers and sysadmins of the world, it never went anywhere. I pose this scenario to highlight the fact that while you can gain a fluency with computer interfaces, whether CLI or GUI, they are skill learned with patient practice and years of developing familiarity. Natural language, on the other hand, is obvious and familiar to just about anyone. The fact that natural language is now an effective way to give instructions to a machine is a major milestone in the evolution of our interaction with computers, one that I want to spend some time exploring in this post.
The Keyboard and the Mouse
There are two major modalities of computer interaction: graphical and textual. This distinction is of course somewhat arbitrary, and text is a kind of graphic. But there is enough difference in the way textual and graphical interfaces are used to control a computer that we can make some general distinctions.
It is easy to think of the graphical interface as the more user friendly version of the two. The operations possible with it correspond more closely with physical movement in a 2D space, so it has the advantage for anything having to do with movement in space. But some things are better expressed in the form of a language, which is a medium for encoding a set of instructions to a computer. Graphical programming languages have been tried, but so far none has had much adoption. Text also has the advantage of persistence, since a mouse interaction may be recorded for playback but generally isn’t reusable in the same way as a useful script is.
This distinction is also reflected in the fact that there are two main computer input devices, the keyboard and the mouse. The keyboard has the advantage of the number of inputs it can produce, which create an immense number of combinations. This level of sophistication means that it is the tool of choice for the professional and the power user, people who need a lot of options and are willing to put in the time to learn the many possibilities. There are, for example, the Emacs and Vim users who have memorized innumerable keyboard shortcuts in order to avoid having to move their hands away from the keyboard.
While the keyboard provides innumerable combinations, the mouse provides a simple set of interactions which can be used for innumerable actions. The button is a simple label for a clickable area that provides an affordance for clicking with few limitations, which can be put to all kinds of uses in all kinds of interfaces. We can include the touch interface of mobile devices in the same category, adding tap and swipe to click and drag as basic interactions, which any number of interfaces can make use of for their own unique use-cases. Swiping right and swiping left can be for navigation or indicating interest, depending on whether you are using a web browser or dating app.
Text and Graphics, Tortoise and Hare
We’ve looked at the various affordances provided by the two interaction modalities, but we also see a significant difference when comparing their histories. As computing has advanced and the machines become more and more capable with each iteration of Moore’s law, graphical capabilities have improved exponentially, with new hardware and APIs dedicated to producing more detailed, realistic images coming out year after year. Text interfaces on the other hand have remained largely similar to their predecessors. Although we no longer face the limitation of the 80-character screen width, many programmers today use text editors and terminal emulators that have not strayed too far from their origins in the 1970s.
It is not strictly true to say that all improvements have been on the side of graphical interfaces, as the desktop environments of today retain a basic form going back to the original ideas coming from Xerox PARC and adopted by Apple and Microsoft, and there certainly have been innovations in how text is handled, such as the adoption of Unicode as the universal character encoding. This is not to be doctrinaire about the differences between the two. But it can be said with confidence that outside of typographers, designers, and other specialists, the capabilities of the newest generation of GPUs is much more exciting to the average computer user than advances in text handling.
And this is not surprising, since we are visually oriented creatures. I certainly remember being impressed with the improved graphic capabilities of each new generation of video game console. Cutting-edge graphics are also closely associated with gaming, and while it is a huge market, not everyone is a gamer. But nearly everyone who uses a computer uses it to work with text, whether creating documents, chatting, or writing code. So text has this kind of steady position, never in any danger of becoming unnecessary but in no rush to advance. George R. R. Martin, author of A Song of Ice and Fire, the inspiration for the Game of Thrones TV series, notoriously still does his writing in WordStar 4.0.
The Revenge of the Text
Then in 2022, with the general availability of ChatGPT, that abruptly changed. All of a sudden, you could talk to your computer. Not just interact with it using a set of commands or device actions, but simply by writing or speaking to it. Natural language, in the form of prompts to large language models, has become a way to interact with computers on the same level as the command line and the GUI.
With this sudden major advance in the ability for machines to work with language comes a major advance in the textual interface, an advance that goes beyond the incremental improvements that we are used to. The chat interface, which has been with us since the days of IRC and AIM, and shares similarities with the command line, has emerged as a primary method of computer interaction. Of course, this advance in the usability of textual interfaces is really an advance in the usability of AI, which has gone from something studied in high-echelon universities and research labs to become as ubiquitous in daily life as the Internet.
The generalization of the famous Turing test is what you might call the “Duck Test”, by analogy with what is informally called “Duck Typing” in programming languages: if it walks like a duck and quacks like a duck, then you can call it a duck, even if beneath the feathers you find transistors and silicon. If our computers are now able to respond to our queries with comprehensible, relevant information, then you can treat it like a conversational partner rather than a very complex calculator.
Not that natural language interfaces, or AI in general, didn’t exist in the pre-ChatGPT world, just that they were never much more than a way to translate natural language into a specific set of commands. But barking orders at a subordinate is only one, very circumscribed kind of language use. These interfaces could not support a dialog that makes use of an implicit, shared set of expectations and which creates its own context as it progresses, something essential to the way language is actually used by people all the time. Far from the unsophisticated text parsers of prior days, we can now interact with our machines in language close to that which we would interact with a fellow air breather.
What LLMs make possible is that textual interfaces are now dynamic in a way that was never possible before. Previously, graphics were dynamic, capable of all kinds of transformations, animations, recording and playback, while text was relatively static, with content, documentation, labels and so on limited to what was anticipated by developers and content creators. Now text can be generated and updated just as quickly as you ask it to. All of a sudden, computers can produce understandable, relevant, useful text, and generate it on demand.
Returning to the scenario that started this post, in a way, computers are finally able to meet us where we are, rather than requiring us to think like a machine. The match isn’t perfect, of course, and I am not here trying to excuse any of the outcomes that the Pandora’s Box of AI unleashes on the world based on the usability improvements they make possible. The fact that computers can now converse with us rather than merely presenting a set of options brings with it many questions: what tone they should take; how friendly, neutral or even antagonistic; how much they should reveal about information that has the potential to be used harmfully. All big questions that will have to be asked while we are in the middle of the changes that are already happening. Here I just wanted to highlight one particular change that I’ve noticed: that with LLMs now able to generate useful text at the click of a button, whether information, instruction, recipe, or source code, text-based interfaces have gained a new place of prominence after decades of relative stasis.