Thompson, C., Najjar, L. J., & Ockerman, J. J. (I997). Wearable computer based training and performance support systems. In S. Aronberg (Ed.), 19th Interservice/Industry Training, Simulation and Education Conference Proceedings (pp. 746-752). Arlington, VA: National Training Systems Association.
Chris Thompson, Lawrence J. Najjar, Jennifer J. Ockerman
Georgia Tech Research Institute
Atlanta, GA USA
Increasingly sophisticated technology is being deployed by the military throughout the world. It is often difficult or impossible to always have a highly trained technician available to maintain, repair, and operate this technology. In an effort to address this problem, a unique training/performance support system for technicians that combines a hands-free wearable computer with a multimedia electronic performance support system (EPSS) is presented. The wearable computer allows the technician to retrieve or enter information wherever he or she is, even while working in the field. Our software-based EPSS integrates multimedia information, tools, and methodologies to help users perform specific tasks. This wearable platform can be used to deliver computer based training (CBT) literally anywhere. Perhaps more importantly, the wearable computer and the EPSS combine to provide much needed follow-on expert support after a training event. Users are able to query the system when a question arises, much the same as an apprentice might query an accomplished expert technician. The system can respond with a suggestion, an illustration, a parts list, or even a movie detailing a procedure. Typical applications of this technology include maintenance, inspection, and operational support of aircraft, radar, heavy vehicles, and other complex electronic and/or mechanical systems.
Chris Thompson is a Senior Research Engineer and project director at the Georgia Tech Research Institute. He is currently pursuing a doctorate in Instructional Technology at Georgia State University in Atlanta. Prior to coming to Georgia Tech, Mr. Thompson served as a senior applications engineer for Pioneer Communications Corporation New Media Division. His current activities include the design and application of wearable computing systems for industrial information systems applications, development of interactive multimedia training materials for manufacturing related activities, and the use of World Wide Web technology by industry. He has worked in an R&D environment for over 17 years.
Larry Najjar is a Graduate Research Assistant at the Georgia Tech Research Institute. Larry helped design the user-interface for the next-generation US air traffic control system, including the digital flight strips, keyboard, work station, and user warnings. In 1989; he transferred to Atlanta, Georgia, where he designed and tested user-interfaces for a wide variety of commercial applications. Larry left IBM in 1993 to pursue his Ph.D. in engineering psychology at the Georgia Institute of Technology. He has performed task and function analyses, written training materials, prototyped user-interfaces, designed World Wide Web pages, and written on-line help. In his current assignment, he is working on a team to design and build a multimedia electronic performance support system and a wearable computer. For his dissertation, Larry plans to examine the effects of multimedia user-interfaces on learning.
Jennifer Ockerman is currently working on a Ph.D. in human-machine systems research in the School of Industrial and Systems Engineering at the Georgia Institute of Technology. She developed cost estimation software to be used by the advanced programs office and computer simulations for various manufacturing lines. After four years as a manufacturing engineer for the Electronic Systems Group of Westinghouse, she returned to Georgia Tech. For her master's thesis she developed a case-based design browser to aid developers in software reuse at NASA's Goddard Space Flight Center. She is currently designing and developing educational multimedia performance support systems for Georgia Tech Research Institute. Her areas of research interest are decision and performance support systems for manufacturing, focusing on the use of wearable computers.
Chris Thompson, Lawrence J. Najjar, Jennifer J. Ockerman
Georgia Tech Research Institute
Atlanta, GA USA
Due to shrinking budgets, the military is deploying increasingly sophisticated technology with fewer and fewer support personnel. As a result, it is imperative that the technicians operating and maintaining this equipment receive the best training possible before deployment along with follow-up expert support. Unfortunately, this is rarely possible in the real world. In general, training is most effective when the following conditions are met:
This paper describes a new kind of training/support system which attempts to address these challenges by combining two new technologies; a wearable computer and an electronic performance support system (EPSS). Combined, these technologies create a kind of "personal electronic trainer" wherever the technician goes; i.e., a flight line, a maintenance bay, or actual mission. No longer is learning restricted to a classroom or a desktop environment. Instead, training and follow-on support are always available when and where the learner needs them.
The first new technology is a belt-mounted wearable computer. Figure 1 shows a front view of the computer's major components. Figure 2 shows a back view. The wearable computer uses an Intel 486 chip running at 75 MHz with 24 MB of RAM, a 340 MB hard disk, full-duplex sound, SVGA video, a wireless network adapter, and two available PCMCIA expansion slots. To power the computer, we developed flexible, ergonomic, easy-to-replace, rechargeable, nickel metal hydride battery packs. The computer is equivalent to a desktop unit and runs any PC-compatible operating system such as Windows 95, UNIX, or DOS.
Figure 1. Front view.
Figure 2. Back view.
Information is entered into or retrieved from the computer using either voice, a hand-held pointing device, or a wrist-mounted keyboard. A head-mounted display (HMD) serves as the computer screen. For most applications, voice recognition serves as the primary means of control since it allows the users to interact with the world around them while operating the computer. The computer is attached to a small waist pack. Including all components, the wearable computer system weighs 2.4 kg (5.3 pounds). This weight may vary depending on the size of the battery and the accessories added to the system; i.e., digital video camera, display, GPS, etc. Figures 3 and 4 show an actual user of an early prototype of the system.
Figure 3. Photograph of early prototype.
Figure 4. Back view of prototype.
Wearable computers provide many advantages to a technician attempting to perform a technical task. For example:
The second new technology is called an electronic performance support system (EPSS) (Gery, 1991, 1995). Traditional training systems train the learner before the learner performs the task. An EPSS teaches and supports a learner while performing a task. To do this, an EPSS uses specialized software that integrates information, tools, and methodologies to help a user perform a specific task. An EPSS provides this relevant, just-in-time information when the user needs it and when the user asks for it. An Interactive Electronic Technical Manual (IETM) (Department of Defense, 1992a, 1992b, 1992c) is often a component of an EPSS. Other kinds of information can be included as well:
To be successful, an EPSS must organize information in a way that is obvious to someone who has not used the EPSS before. For example, we developed a simple EPSS demonstration that teaches a user how to fold a paper jumping frog (Najjar, Ockerman, Thompson, & Treanor, 1996). We organized the training information into the following intuitive categories:
Figure 5 shows the main menu of our prototype EPSS. This example serves merely to illustrate the possibilities of a "personal electronic trainer" while providing us with a framework to conduct research. Although the basic concepts are applicable to any task, the look and feel of the application is likely to change considerably in a military setting.
Figure 5. Main Menu Prototype EPSS.
A typical interaction consists of several steps. Any step is begun by speaking into the headset any word or combination of words visible on the screen. The system then recognizes the spoken phrase and calls up the requested information automatically. If for some reason the user does not wish to speak the command, a pointing device may be used instead to select an item by clicking on the desired button. Any visible words on buttons are possible commands. The user continues to navigate through the system using voice commands and may occasionally be required to enter some data as well.
In our prototype EPSS, we tried to use media in a way that helped the user to understand and learn the information (Najjar, 1995, 1996). For example, in the section that has step-by-step instructions, each step begins with a simple static drawing and auditory instructions. To understand the fold described by the step, the user can examine the drawing for as long as desired. However, since paper-folding requires that the user move both paper and fingers, we also provided an option that allows the user to see and hear a video of someone actually folding the paper. Since the display does not block the user's field of view, the user can look at the drawings, listen to the auditory instructions, or view an explanatory video while completing the prescribed task.
At Georgia Tech we have completed a preliminary laboratory study comparing the performance of personnel using a wearable computer with others using paper-based documentation. All of the participants were assigned a simple mechanical task; i.e., the folding of paper into an origami jumping frog. Initial results point to a reduction in the number of errors made while completing the task; however, the participants using the wearable computer took significantly longer (Ockerman, 1997). Future studies are planned dealing with more complicated tasks in an effort to understand the interaction between the technology and the human more fully.
Few systematic studies have yet been carried out documenting the effectiveness of wearable computers in real world applications. However, preliminary studies conducted by Carnegie Mellon University (Department of Defense, 1995) in conjunction with maintenance of heavy vehicles point the way to significant advantages including:
Wearable computer applications are currently being developed for a number of military applications. In the medical field, one research group is exploring the use of wearable computers to support medical personnel in the field. The system allows the medic to enter data concerning the condition of the patient into an electronic record using voice input. This critical data is relayed across a wireless network to a mobile communications center which in turn passes the information to the final treatment facility. In this way, the most current status information is available at all times so that the attending physicians can plot a treatment strategy well in advance of the patient's arrival. This advance information can save lives by reducing the time to diagnosis and subsequent treatment.
Another wearable computer application focuses on minimizing the time and personnel required to conduct a routine inspection of a piece of equipment. The wearable computer is used to guide the personnel through the steps of the inspection procedure while collecting whatever data is required. Advantages include less formal training requirements, enhanced accuracy of data collection, a reduction in inspection time, and a simpler method of updating and maintaining the required procedures.
Closely related to the inspection problem is one of supporting maintenance personnel in the debugging and subsequent repair of malfunctioning equipment. This support can be provided through simple documentation, an expert system assistant, or remote collaboration with an expert. As equipment becomes more and more sophisticated, the knowledge required to repair it typically increases as well. It is often not practical to distribute paper-based documentation to every site the equipment might be sent. Consequently, electronic documentation is becoming more common. Yet, until the advent of the wearable computer there has not been an effective means of delivering this information to a technician in the field.
These new technologies are not necessarily effective in all applications. Tasks with the following characteristics are perhaps best suited for wearable support:
Other tradeoffs in technology must be made as well including:
Wearable computer technology is rapidly advancing. Several commercial vendors have entered the market offering commercial off-the-shelf (COTS) technology upon which one can base a custom solution. These products are moving the technology out of the research laboratories and into the hands of real users. One such device is pictured in Figures 6 and 7. This device provides an advanced degree of comfort and ergonomic adjustments due to a flexible bus which conforms to the user as it is worn.
Additionally, we are investigating the use of a networked wearable computer equipped with a video camera and collaboration software to allow people in remote sites to work together. A technician can send live video and audio of a broken piece of equipment or problem to a remote expert. The remote expert can then collaborate through video and audio with the technician working on the problem to find a solution.
Figure 6. Back view of COTS system.
Figure 7. Side view.
Wearable computers and EPSSs are being deployed today. Solutions to real support and training problems for mobile technical personnel are emerging rapidly. In the future we expect wearable computers and electronic documentation to become a standard component of any advanced weapon or surveillance system. Other applications will emerge in the industrial and consumer markets. These commercial applications will reduce the cost of development and deployment in the same way that widespread acceptance of the home computer has benefited businesses.
Work described in this paper is funded by the state of Georgia as part of the ATRP Program at the Georgia Tech Research Institute.
Department of Defense (1992a). Data base. revisable: Interactive Electronic Technical Manuals, For the support of (MIL-D-87269). Washington, DC: Department of Defense. (NTIS No. PB93-145225/XAB)
Department of Defense (1992b). Manuals. Interactive Electronic Technical: General content, style, format, and user-interaction requirements (MIL-M-87268). Washington, DC: Department of Defense. (NTIS No. PB93-144616/XAB)
Department of Defense (1992c). Quality assurance program: Interactive Electronic Technical Manuals and associated technical information, Requirements for (MIL-Q-87270). Washington, DC: Department of Defense. (NTIS No. PB93-144608/XAB)
Department of Defense (1995). DARPA TlA-Smart Modules-Technology Programs, ETO August, 1995. http://eto.sysplan.com/ETO/SmartMod/Presentation/TIAtech/index.html
Gery, G. (Ed.) (1995). Electronic performance support systems [Special issue]. Performance Improvement Quarterly 8(1). Available World Wide Web: http://www.cet.fsu.edu/sy2000/PIQ/PIQContents.html
Gery, G. J. (1991). Electronic performance support systems. Boston, MA: Weingarten.
Najjar, L. J. (1995). A review of the fundamental effects of multimedia information presentation on learning (GIT-GVU-95-20). Atlanta, GA: Georgia Institute of Technology, Graphics, Visualization, and Usability Center. Available World Wide Web: http://www.cc.gatech.edu/gvu/reports/1995/index.html
Najjar, L. J. (1996). Multimedia information and learning. Journal of Educational Multimedia and Hypermedia, 5, 129-150.
Najjar, L. J., Ockerman, J. J., Thompson, J. C., & Treanor, C. J. (1996). Building a demonstration multimedia electronic performance support system. In P. Carlson & F. Makedon (Eds.), Educational Multimedia and Hypermedia 1996 (p. 794). Boston, MA: Association for the Advancement of Computing in Education. Available World Wide Web: http://mime1.marc.gatech.edu/mime/papers/edmedia1l.html
Najjar, L. J., Thompson, J. C., & Ockerman, J. J. (1977, March). Using a wearable computer to improve the performance of quality assurance inspectors in a food processing plant. Paper presented at the CHI'97 Workshop on Wearable Computers, Atlanta, GA. Available World Wide Web: http://mime1.marc.gatech.edu/mime/papers/CHI97_position_paper.html
Ockerman, J. J., Najjar, L. J., & Thompson, J. C. (in press). Evaluation of a wearable computer performance support system. Educational Multimedia and Hypermedia 1997. Available World Wide Web: http://mime1.marc.gatech.edu/mime/papers/edmedia97_eval.html
Ockerman, J. J., Najjar, L. J., & Thompson, J. C. (1996). Factory automation support technology (FAST). In D. Adelson & E. Domeshek (Eds.), International Conference on the Learning Sciences, 1996 (p. 567). Charlottesville, VA: Association for the Advancement of Computing in Education. Available World Wide Web: http://mime1.marc.gatech.edu/mime/papers/icls96.html
Ockerman, J. J., Najjar, L. J., Thompson, J. C., Atkinson, F. D., and Treanor, C. J. (1996). FAST: A research paradigm for educational performance support systems. In P. Carlson & F. Makedon (Eds.), Educational Multimedia and Hypermedia 1996 (pp. 545-550). Boston, MA: Association for the Advancement of Computing in Education. Available World Wide Web: http://mime1.marc.gatech.edu/mime/papers/edmedia2.html