I reveal no secret if I say that imagination comes from the image and, in fact, is the “faculty of the soul which represents images of real or ideal things”, in the beautiful expression of the fat book. Have you noticed if your thoughts run through images? Mine usually does, although I know there are people who say they think through language. My impression is that language comes later, when we already have in mind the comedy of what we remember, or what we are simulating, or what we are fabulating. The fact is, our knowledge of the brain is already advanced enough to see that speech bubble from outside the skull and translate it into sentences with shocking precision. It is a dictionary of the imagination.
The new technique means the end of the elephant in the room, that idea that is on everyone’s mind, but that no one expresses for fear of violating some taboo, irritating the boss or appearing unhappy. Because if the idea is in your mind, it can be seen from the outside. It requires a rather bulky gadget, but it is a technical problem, and we already know how quickly technical problems can be solved nowadays when there is a multi-zero incentive. It is not necessary to reach 12 zeros of Elon Musk’s new salary (one billion dollars). Half zeros (one million dollars) are often enough to stimulate a talented engineer. If the machine were simplified enough to fit into glasses, our train of thought would be an open book to the wearer.
The invention is called mental captions (“Subtitling the mind”) and is the work of computational neuroscientist Tomoyasu Horikawa, from the NTT laboratories in Kanagawa, Japan. Horikawa used a deep language model (different from great linguistic modelsLLM, such as ChatGPT and similar) to analyze the textual descriptions of 2,000 videos. He then showed the videos to six volunteers inserted into the MRI tube to see what happens in their brains. A second AI tool then inferred associations between brain activity patterns and what was said in the videos. That is, a dictionary of the imagination. The device is simple, but its results are amazing.
For example, a participant watches a video in which a person jumps from the top of a waterfall, and the machine first says “flow jump” and then improves its description until it reaches “fast waterfall” (step 10) and “a person jumps over a deep waterfall on the top of a mountain” (step 100). I can’t help but think how much Noam Chomsky would love that evolving language. As in its generative grammar, the text starts from images, builds a “deep structure” and makes it evolve towards the “surface structure” that we normally pronounce.
Another interesting point is that the person does not need to watch the video while using the machine subtitle your mind: you just have to remember it, or imagine it. The results are very similar. It’s not the first data to indicate that seeing something is associated with the same patterns of brain activity as remembering it, but it certainly demonstrates that general concept with remarkable eloquence. If I tell you to imagine an elephant, your brain does pretty much the same thing as if you actually saw it. Horikawa’s machine may not go so far as to reveal your private thoughts, but it’s obviously moving in that direction, and at a steady pace. Intimacy does not exist; It’s just a 20th century myth.
