Frequently Asked Questions
Who are the people who defined the intellectual parameters for the current field of AugCog?
Where do you think the science is going in, say, ten, twenty, thirty years?
How will AugCog systems be tuned to specific individual needs?
Has this been tested in the field? Doing what precisely? And if not how has it been tested?
What is the underlying purpose of this work? What benefits will it offer?
When could we reasonable expect AugCog systems to be deployed in real situations?
What's the next step in this research?
What is the long term vision for this kind of work? Where is it going?
Could brain/machine interfaces change the way we relate to each other, and to ourselves?
Who are the people who defined the intellectual parameters for the current field of AugCog?
Schmorrow: Most any discipline today associated with Human System Integration (HSI) and interactive computing systems (e.g., human (brain)-machine interfaces, the Internet, the gaming industry) and a vision to achieve human-computer symbiosis can be traced back to the 1940s-60s and the Advanced Research Projects Agency (ARPA). ARPA’s initial leaders, researchers, and associate scientists and engineers were the original pioneers in early cybernetics thinking regarding how to augment the human intellect with machines. The idea was to create machines that had a greater understanding of and synergy with the structure and function of the human mind as a neural-networked, problem-solving entity. This would allow the human to more effectively and efficiently solve problems with the aid of a tightly coupled, synergistic human-machine support system tailored to the needs of each individual human operator. For a nice yet brief review of these original cybernetics pioneers and their systems science thinking regarding how to augment the human intellect, we recommend our readers to check out the following recent citation:
Hunter Crowther-Heyck. "Mind and Network," IEEE Annals of the History of Computing, vol. 27, no. 3, pp. 104, 103, July-September, 2005.
Where do you think the science is going in, say, ten, twenty, thirty years? If it does percolate out into the hacker/hobbyist/developer communities, what do you anticipate happening?
Schmorrow: Firstly, I don’t think the issue is that the vision today is much different from the original pioneering cybernetics visions of the 40s, 50s, 60s and since. Rather, the difference is the almost limitless possibility that today’s and future technological innovations have to offer. We are currently at a very exciting juncture regarding where computing power, miniaturization, and mobility efforts are headed and with what we are discovering on basically a daily basis with brain imaging studies and how to augment human cognition with neurophysiological tools (e.g., EEG-based and functional near infrared [fNIR] cognitive state sensing devices). In ten years from now, we should see the results of these present efforts manifested as operational technologies in military and industrial applications and possibly entertainment. Twenty years from now, I see augmented cognition (AugCog) technology being pervasively woven into the fabric of our daily interactive computing lives. And, in 30 years and beyond—who knows? I do know, however, that the possibilities will be limited by only our own intellects and imaginations over these next few decades.
Secondly, regarding percolating into hobbyist/developer communities, I see direct parallels to most any computing technology progress we have witnessed in the past few decades. That is, once the utility of and enormous opportunities for industry, education and even entertainment are clearly demonstrated (where you have systems constantly being able to adapt to meet your specific cognitive capabilities, information needs, and likes), more people will be willing to invest time and money into developing AugCog technologies accessible and affordable to most any user. This will inevitably drive up the competition, drive down end-user costs, and spark even more innovation. Would you have ever thought last year that you would be able to buy an Ipod this year that allows you t o download and watch TV shows and movies on it? Maybe you did; maybe you didn’t—and maybe you don’t really care. However, you can’t deny that it’s innovative and that today’s market demands warrant it.
Various articles have mentioned how an AugCog system will “interpret a user’s cognitive, emotional, and physical state and then prioritize information through the system for that user" - could you explain precisely what this means? Are you basically saying each system will be "tuned" to each individual user?
Schmorrow: As addressed in many of our publications, I would say that a primary objective of a ‘closed-loop’ AugCog system would be a system that can be calibrated and tailored to meet the varying human information processing (HIP) strengths and weaknesses of any individual. So, yes, this can include a myriad of user types and conditions, such as: dismounted soldiers in the field, with whom Honeywell is experimenting; pilots in the cockpit, with whom QinetiQ is experimenting; command and control workstation operators, with whom LMATL and Boeing are experimenting; vehicle operators, with whom DCC is experimenting, and; numerous other application domains that are currently being investigated (e.g., training systems, intelligence analysis, etc.).
More specifics may be found in our latest Human Factors Chapter (53), Augmented Cognition in Human-System Interaction, which I coauthored with Stanney, Wilson & Young. Specifically, the section on Human-System Augmentation gives some concrete examples regarding adapting information presentation (e.g., presentation modality augmentations, scheduling, sequencing, and pacing) to meet a warfighters dynamically changing needs.
Has this been tested in the field? Doing what precisely? And if not how has it been tested?
Schmorrow: The recent Schmorrow & McBride overview article addresses the successes of the lab experiments reported in the IJHCI ’04 Special Issue (2004) (see also St. John et al., article in same issue).
For examples of successes thus far in the field, please see the Schmorrow & Kruse ACI/HCI 2005 Session Overview: Foundations of AugCog article, which references the experimental work in various field studies being conducted by Boeing, Lockheed Martin Advanced Technologies Laboratory [LMATL], DaimlerChrysler Corporation [DCC], and Honeywell.
What is the underlying purpose of this work? What benefits will it offer?
Schmorrow: As discussed in various of our publications and in many of the other FAQs here, basically, we want to improve the information assimilation and decision-making capabilities of human-computer operators. The emerging field of AugCog science is aimed at revolutionizing the way humans interact with computer-based systems by coupling traditional electromechanical interaction devices (e.g., mouse, joystick) with psychophysiological methods (e.g., respiration, heart rate, EEG, functional optical imaging), where human physiological indicators can be used in real time to drive system adaptation or a priori to assess potential design issues (e.g., system design features/functionality which may induce information overload or inefficient decision making).
When could we reasonable expect AugCog systems to be deployed in real situations? For example, in military domains would all soldiers be equipped with this equipment or would it only be for superior officers?
Schmorrow: Although multiple successes have been established to date through various lab and field studies, size, weight, portability, invasiveness, etc. are all major areas on which the AugCog science and technology (S&T) developers are continuously focusing. The integration of the most applicable AugCog technologies into an easily affordable, non-invasive, and comfortable headset that can be integrated with most any human-system computing environment is an ultimate vision for the applied research being conducted by the present AugCog scientists and developers. As I’ve said in previous interviews, how this manifests in terms of community-wide benefits has yet to be determined. The idea is that continued developments and demonstrated payoffs to multiple communities, in addition to military, will eventually lead AugCog technology development into a more technology-pull market rather than technology-push market, with the benefits of improved usability and reduced costs eventually reaching the every-day consumer.
Is there an acceptable level of accuracy for these systems, or some kind of target accuracy, to ensure that there aren't too many false negatives or false positives?
Schmorrow: As alluded to in other answers, AugCog S&T developers are continually improving their S&T, which also includes improving the measurement, assessment, and diagnostic capabilities for detecting HIP and overload conditions and when to intervene with mitigation strategies and what those mitigation strategies should be. There is no single answer to this question, as accepted accuracy levels will depend on the task domains and their respective cost/benefit analyses and potential consequences. For instance, a closed-loop cockpit system or training system where a pilot may be directing bombs to targets may require greater than 99% accuracy. Other domains (e.g., exploratory learning, entertainment systems, usability testing, etc.) would most likely not require such accuracy levels.
AugCog S&T developers that are currently working with Army, Navy, Marines, and Air Force applications have mandated metrics they must meet to satisfy their potential military customers.
What's the next step in this research?
Schmorrow: Other FAQ answers provide some insights to this question, as we do not really consider it as a “next step” per se but more of an emerging scientific field that is continuously evolving and merging S&T among numerous multi-disciplinary fields (e.g., neuroscience, cognitive neuroscience, psychology, physiology, applied materials and engineering, etc.). The continued collaboration among existing AugCog scientists and developers and their dissemination of results and lessons-learned back to the community will be key. This is a main reason why we established the Augmented Cognition International Conference last year and will continue to host it annually.
What is the long term vision for this kind of work? Where is it going?
A: As discussed throughout the responses here, various books and articles we’ve published, and recent interviews I’ve given, the eventual effectiveness and efficiency payoffs for being able to develop real-time, cognitively aware and adaptable (and eventually proactive) systems for many environments, including military and industry, are enormous. Imagine your personal computing devices (e.g., desktop or laptop computer, cell phone, Blackberry, Treo, etc.) actually being able to detect your information processing patterns, environmental surroundings, and goals for the day, and possibly even infer intent, in order to more effectively, dynamically adapt to your needs and assist you throughout your entire working day and even into your personal life.
What is clearly necessary to realize this vision is a unified field of augmented cognition as its S&T continues to progress and mature, with an increased cohesiveness between theory and its implications for engineering design. If the progress demonstrated thus far in the field of AugCog is sustained, order of magnitude increases in HIP and decision making capabilities could be realized in most any human-machine coordinated environment.
Could brain/machine interfaces change the way we relate to each other, and to ourselves? If so, how? Do you see science fiction as helpful in prompting us to think about this? What has influenced your own thinking?
Schmorrow: As alluded to in answers to the other FAQs, the eventual effectiveness and efficiency payoffs for being able to develop real-time, cognitively aware and adaptable (and eventually proactive) systems for many application environments (e.g., industry, military, entertainment) are enormous. Imagine owning a PDA that could actually sense your surroundings and goals for the day, and possibly even infer intent, in order to more effectively, dynamically adapt to your needs and assist you throughout your entire working day and even into your personal life. Imagine how much more free time you would have to relate to your colleagues, friends, and family members by not having to worry about every little line item on your ‘To Do’ list for the day.
I think we are definitely ready to reap the benefits of the current innovations emerging from the S&T efforts of the AugCog scientific field, which began with the Decade of the Brain efforts and continues to flourish with the aid of major DoD investment. It is up to national and supranational institutions (e.g., DARPA, NIH, NATO) to continue to foster the research and development with necessary funding ventures. It is up to the AugCog researchers and developers to continue to innovate and report research results and lessons learned back to the AugCog and other scientific communities.
I think this is only the beginning of the next big paradigm shift in interactive computing. Again, we are only bound by our own imaginations. Thus, one could argue that science fiction novels and movies are nothing more than manifestations of the creators’ imaginations, dreams, and desires in the absence of the technology necessary to make them actually possible in real life. Thus, if such creative thinking by some sparks innovative thinking in others, then more power to the science fiction buffs who can begin to build the things that others could only previously dream.
As far as influences on my own thinking, my current vision is too directly traceable back to the visions of the original cybernetics pioneers such as Licklider and Weiner and to my time in the Navy and my recent 4 year post at DARPA. It was my time at DARPA that enabled me to explore just what was capable by the year 2000 and where we could make leap-ahead improvements in augmenting human cognition and improving the human information processing capabilities of our warfighters. Knowing how many eventual lives could be saved with advances in AugCog S&T is something that I take very seriously and would hope is not something considered ‘fiction’ in anyone’s mind.