Applications
Components of an Augmented Cognition System
At the most general level, the field of Augmented Cognition has the explicit goal of utilizing methods and designs that harness computation and explicit knowledge about human limitations to open bottlenecks and address the biases and deficits in human cognition. It proposes to do this through continual background sensing, learning, and inferences to understand trends, patterns, and situations relevant to a user’s context and goals. At its most basic level, an augmented cognition system should contain at least four components - sensors for determining user state, an inference engine or classifier to evaluate incoming sensor information, an adaptive user interface, and an underlying computational architecture to integrate these components. In reality a fully functioning system would have many more components, but these are the most critical for inclusion as an augmented cognition system. Independently, each of these components is fairly straightforward. Much of the ongoing augmented cognition research focuses on integrating these components to “close the loop,” and create computational systems that adapt to their users. The figure below illustrates components that must be considered when developing such a closed-loop AugCog system.
Thus, the challenge with these systems is not the sensing component (although researchers are using increasingly complex sensors). The primary challenge with these systems is accurately predicting/assessing, from the incoming sensor information, the correct state of the user and having the computer select an appropriate strategy to assist the user at that time. As discussed in the first section, humans have well documented limitations in attention, memory, learning, comprehension, sensory bandwidth, visualization abilities, qualitative judgments, serial processing and decision making. For an augmented cognition system to be successful it must identify at least one of these bottlenecks in real time and alleviate it through a performance enhancing mitigation strategy. These mitigation strategies are conveyed to the user through the adaptive interface and might involve: modality switching (between visual, auditory, & haptic), intelligent interruption, task negotiation and scheduling, and assisted context retrieval via book marking. When a user state is correctly sensed, an appropriate strategy chosen to alleviate the bottleneck, the interface adapted to carry out the strategy and the resulting sensor information indicates that the aiding has worked – only then has a system “closed the loop” and successfully augmented the user’s cognition.
Applications of Augmented Cognition
The applications of Augmented Cognition research are numerous, and although initial investments in systems that explicitly monitor cognitive state have been sponsored by military and defense agencies, there is an interest from the commercial sector to develop augmented cognition systems for non-military applications. As mentioned earlier, closely related work on methods and architectures for detecting and reasoning about a user’s workload based on such information as activity with computing systems and gaze have been studied for non-military applications such as commercial notification systems and communication. There has also been interest from civilian agencies such as NASA on the use of methods for limiting workload and and managing information overload. Hardware and software manufacturers are always eager to include technologies that make their systems easier to use, and augmented cognition systems would likely result in an increase in worker productivity with a savings of both time and money to companies that purchased these systems. In more specific cases, stressful jobs that involve constant information overload from computational sources, like air traffic control would also benefit from technology. Finally, the fields of education and training are the next likely targets for this technology once it reaches commercial viability. Education and training are moving towards an increasingly computational medium. With distance learning in high demand, educational systems will need to adapt to this new non-human teaching interaction while ensuring quality of education. Augmented Cognition technologies could be applied to educational settings and guarantee students a teaching strategy that is adapted to their style of learning. Above all other domains, this application of Augmented Cognition could have the biggest impact on society at large.