Brainish - Formalizing a Multimodal Language for Intelligence and Consciousness

Explore the concept of Brainish, a multimodal language for intelligence and consciousness, in this 25-minute conference talk by Paul Pu Liang at the Models of Consciousness Conference. Delve into the framework of the Conscious Turing Machine (CTM) and learn how Brainish, comprising words, images, audio, and sensations, enables core cognitive functions. Discover the syntax and semantics of this innovative language and its operationalization through multimodal artificial intelligence. Examine the three-step process for learning Brainish, including unimodal encoders, coordinated representation space, and decoders. Understand the crucial role of Brainish in achieving consciousness within the CTM and its potential impact on advancing machine models of intelligence and consciousness. Gain insights from real-world implementations and evaluations of Brainish's multimodal intelligence capabilities across various datasets.