Figure 1: A rifle-mounted tactile interface with chording sub-system.

Each sensor can be programmed to operate as a two- or three-state device (additional states are available but are difficult for the user to interface with). The UIUC group is currently incorporating vibratory feedback and designing human subject experiments to evaluate the efficiency of the system.

April 1 - June 30, 1997 (FY97Q3)

Goals: Develop ranges for the different specifications for the tactile sensor. Evaluate the ability of the human body to respond to tactile actuators in a noisy environment. Assemble a simple chording device and signal processing for use by a dismounted soldier. Model and design a flexible tactile device suitable for controlling a graphics display in a moving vehicle.

Progress: Ranges for the tactile sensor specifications were generated in this quarter. The ranges are intended to bracket the final values for each of the specifications on the previously constructed list. As a result of the generation of the ranges for the sensors, the specification list for the tactile actuator was made more complete. In future quarters, these ranges will be refined and final values will be associated with each specification. Also the list will be expanded by the addition of any specifications that were overlooked earlier.

The team at NCAT has continued to review and polish strategies for the experiments planned this summer. Previously, only a series of cognitive surveys were actually planned, which would serve to investigate the effectiveness and efficiency of euphemisms as cognitive cues, and to understand how tactile information is processed by humans in a noisy environment. This effort is being expanded following a decision to test the performance of the low voltage motor of a pager as a vibration source for tactile sensations. If this unit works as expected, experiments to be staged would include those to test a range of several physiological and psychophysical issues. Some of the issues relate to the number, size, type, frequencies, methods, and range of excitations, the associated pain sensations, and the impacts on the ability of humans to perceive them. We would also investigate the influence of body location on the vibration used and sensation perceived.

Tactile communication is effected and affected by the ability of humans to recognize the pressure or vibration characteristics being perceived, associate them with previously learned message codes, and to recall or recognize the actual information associated with the coded message. The ability to completely and accurately decode embedded messages and information depends to a large extent on the level of cognitive or mental task required. The more complex the coded information, the greater the cognitive load on the soldier, whether use of the tactile device occurs during learning, training, or in applied situations. One question that must be answered before training strategies are devised for tactile communication is whether tactile messages require recall and/or recognition efforts. To reduce the level of cognition required in performing a recall or recognition task, cognitive cues are often used. Such cues enhance memory, facilitate recall, and reduce time to recall or recognize events. Some of our experiments are designed to investigate how and whether common euphemisms or catch phrases that reference human body parts (locations of touch or tactile vibration) could serve as cognitive cues, to help with the recall or recognition of tactile messages. We are continuing to revise and refine these possible test implementations and scenarios.

A rifle-mounted chording system capable of tactile input and multiple feedback modalities was assembled at UIUC. The feedback was done using visual and/or vibratory tactile means. Preliminary, small sample size, human subject experiments have been performed.

A three-state three-element system configuration in which input states correspond to levels of finger pressure on sensors was used for the experiments. Tactile feedback was provided via stimulators vibrating in bursts on the palm and/or with visual feedback in colors. The input elements were mounted on one side of a wooden block (3.8 cm x 20 cm x 6 cm) with the vibrators mounted on the opposite side (3.8 cm away) of the block. Both input elements and vibrators were spaced approximately 2.2 cm and 3 cm center-to-center, respectively. The subject was told to grasp the block, using a natural grasp, such that the first three fingers (index, middle, ring) align with the input elements and the vibrators contact the palm.

Tactile feedback was provided via bursts of 250 Hz vibrations to the palm of the chording hand. One tenth second bursts were given at 2, 5, and 8 Hz corresponding to low, medium and high input states, respectively. Visual feedback for different force levels was provided by changing the colors (green, blue, red) of boxes on the screen.

Experiments were performed on three groups of subjects using the following techniques: visual, tactile, and combined visual and tactile feedback. Subjects were given the chord verbally. The three-level and three-sensor configuration gives a vocabulary of 27 unique chords. An input sequence received by the system is interpreted as a valid chord if the sequence consists of three individual inputs (one from each sensor). The inputs must be received in order (first inputs from the index finger, next inputs from the middle finger, and finally inputs from the ring finger). Upon receipt of the verbal instruction, the subject entered the chord via the tactile device. After completion of each attempt, the subject was told verbally "correct" or "incorrect".

The influence of feedback on the speed and accuracy with which the subjects could input information was examined. The preliminary results indicated that feedback within the tactile modality provides improved performance over cross-modality feedback (visual) or simultaneous visual and tactile feedback. This suggests a superiority of tactile feedback as a means of enhancing subjects' performance with the multi-state input device. Improvements and modifications to the system based on the results of these experiments are ongoing.

An initial design was completed for the flexible tactile device, including modeling the effects of variations in the structural dimensions and flexing of the device. The flexible tactile device is designed to generate a quasi-analog X and Y voltage corresponding to the position of a single pressure point on the device due to a finger or thumb. This is accomplished through the use of two interwoven arrays of contact switches that pick off two voltages from two resistive voltage dividers corresponding to X and Y positions. The switches are simple membrane switches and calculations were done for the decrease in the gap between the contacts as a function of the bending radius of the device, the initial gap, and the diameter of the membrane.

This information was then used to establish a design point for the as-constructed gap between the contacts. The effects of the diaphragm thickness and diameter were modeled to ensure reasonable pressures would fully deflect the diaphragm and cause contact closure. The diaphragm is designed to be constructed from polyimide with a gold contact bar attached to the underside of it. Since the width of the contact bar is small compared to the diaphragm diameter, the modeling indicates that the dimensions of the diaphragm (thickness and diameter) are the most important parameters to control. In the next quarter a mask set will be designed and fabricated that will be used to fabricate a set of test structures and small arrays of switches. A small range of dimensions for the switches will be included centered on values derived from the modeling work done to date.

July 1 - September 30, 1997 (FY97Q4)

Goals: Develop range of different specifications for tactile actuator. Working from the list of potential specifications, we will generate a range of values for each actuator specification. This will be done using the capabilities of the technologies and of the human body as a guide. Demonstrate a simple chording input system. This project will be funded directly by UIUC through internal funds. Design and fabricate masks for flexible tactile device.

Progress: Ranges for the tactile actuator specifications were generated this quarter. They are intended to bracket the final values for each of the specifications on the previously constructed list. In future quarters, these ranges will be refined and final values will be associated with each specification. Also, the list will be expanded by the addition of any specifications that were overlooked earlier. As a result of the generation of the ranges for the actuators, the specification list for the tactile sensor was made more complete.

MCNC continued to work with a set of motors that include an off-center weight for creating a vibration sensation. The initial testing of the motors indicated that varying the voltage supplied to them easily controlled them. Interface circuitry was designed, and began to be assembled to enable the control of multiple motors by a computer, which is also capable of recording the response of the subject. An initial system of five actuators will be tested in the second week of FY98Q1. This system will then be expanded to ten actuators.

The team at NCAT completed two cognitive surveys this quarter, in which two groups of subjects were used. One group had military training, while the other had none, but were in the age range of young military recruits. These two groups participated in a series of cognition-based tests, designed to investigate the effectiveness of euphemisms or catch phrases as cues for identifying actuator placement locations on the human body. Combined efforts of the surveys focus on using cognition as a cue for determining the most viable locations for the placement of a tactile actuator; thus, the actuator can provide a cue for the interpretation of the command information by virtue of its location alone on the body. This will reduce the level of cognition required in performing a recall or recognition task, and reduce time to recall or recognize events.

One of these surveys directly associates command terms with human body parts and derives placement location on the basis of general intuition; we characterize this as a function of some psychological goals and intentions of the individual subjects after having understood the intent of the command language. The modal location of all locations intuitively implicated describes the "best" placement for a tactile actuator representing the command intent.

The other survey, which associates the meanings of euphemisms with knowledge of the command intent, derives placement location by using the body part that is linked to the euphemism as a cue. We will analyze the results of these two surveys, separately and jointly, for the purpose of making statistical inferences, which relate the findings to possible placement locations for the actuators when guided solely by the psychological goals and intentions of users. Note that it is not necessary that we understand what such goals and intentions are, because we seek only to limit our interest to their effects.

An alternative criteria for the placement of the actuators has been developed. This criteria depends on results of psychophysical experiments, which combine the influences of body physiological reactions to the brain's responses (or interpretations of them) as a basis for selecting the design criteria and/or configurations for the tactile devices. The experimental design has been completed, based on the pager actuator system currently being assembled at MCNC. Once this system is ready, testing will commence as planned. Special considerations would be given to body sites implicated in the two cognitive surveys described above.

A rifle-mounted chording system capable of tactile input and multiple feedback modalities was transferred from a wooden rifle to a standard issue, Army training mock rifle at UIUC. This will facilitate more realistic testing of the system. Also, the testing software is being updated to incorporate the ability to provide feedback and instructions in all three modalities. This will allow for more extensive human subject testing that will map the performance of all modalities and investigate important cross modality issues, including distractions, training, and retention.

A mask set was designed and fabricated based on the design completed last quarter for the flexible tactile device. The flexible tactile device is designed to generate a quasi-analog X and Y voltage, corresponding to the position of a single pressure point on the device due to a finger or thumb. This is accomplished through the use of two interwoven arrays of contact switches, picking off two voltages from separate resistive voltage dividers that correspond to X and Y positions. The mask set included a set of individual test contact switches with a range of diaphragm diameters from 70 to 120 µm, multiple small arrays of switches (5x5), eight larger arrays of switches, (either 40x40 or 50x50), and some process diagnostic test structures. This range of dimensions is centered on the values derived from the modeling work done earlier.

Two types of contact switch cells were designed. One type has a single contact for each of the X and Y dimensions that tap the resistive voltage divider network. The second type has two contacts for each dimension. The two contacts are wired in a series and are intended to provide improved yield against the case of some switches becoming stuck in a closed position. By requiring both switches to be closed for the voltage divider to work, a few randomly stuck switches will not affect the operation of the device. Since the areal density of the contacts is high, the closure of two contacts simultaneously will be easily obtained in operation.

Figure 2 is a plot of the mask layout for a double-contact switch cell. The cells are designed to be arrayed by simply abutting adjacent cells together. Clearly visible are the four circular contacts in the cell. The top two contacts sense the Y position and the bottom two sense the X position. The cells contain two resistors with equal impedance, one for each direction.

The cell was designed so that changing the diameter of the contacts did not affect any other part of the cell layout.

Figure 2: A plot of the layout of a double-contact switch cell.

The fabrication run for the flexible tactile device began at the end of FY97Q4. A number of fabrication parameters will be reviewed, including the spacing between the contact elements, the thickness of the polyimide diaphragm, and the thickness of the different metal films.

MCNC delivered a tactile actuator system with five actuators to NCAT in FY98Q1. The system of actuators included a box with control circuitry and five push button switches for human response studies. The system was designed to use a PC printer port, which is controlled by the PC program written in C. The initial programs include a hardware test program and a reaction time program. The reaction time program randomly activates the different actuators with two different vibration levels, and with random intervals between the stimuli. The program records which button is pushed in response by the subject and the response time. The data is written to a text file that can then be analyzed by other programs, such as Excel. The actuators are pager motors which use an off-balance weight attached to the rotor to vibrate when the weight is rotated. The actuators are packaged in plastic cylinders approximately 1 cm in diameter and 4 cm long. The system will be used to test the ability of human subjects to sense vibrations at different locations on the body in different work and environmental conditions. Figure 4 shows a photograph of the completed system.

Figure 4: The actuator test system.

The team at NCAT continued work on two cognitive surveys done last quarter. During this quarter, data from survey results were summarized and preliminary analysis was started in preparation for the Advanced Display Symposium. Ryan Urquhart successfully defended his thesis proposal, Cognitive Cues For Placement of Tactile Interface Device. Preliminary results from analysis show that subjects preferred locations on the front and right side of body for the tactile interface. Work is continuing on the final plan to begin the psychophysical experiment, using a tactile stimulation device designed in collaboration with MCNC. This experiment will utilize information from Ryan's thesis work on placement cues, and the information on preferred locations from other analysis in choosing body sites for psychophysical tests.

The team at UIUC has completed the software and hardware changes necessary to provide instruction and feedback capability in all three sensory modalities (auditory, tactile, and visual) for the rifle-mounted chording system. The auditory stimuli is given as three different frequency tones: 100, 600, 1100 Hz, corresponding to low, medium, and high force levels, respectively. Similarly, the tactile stimuli are given by changing the burst frequency of the vibrators (1.5, 3, 6 Hz). The visual stimuli are given with different colors (green, blue, and red). The system automatically logs all the experimental data to files for further analysis. The system has been tested on three subjects and is now ready to be used in a full-scale human subject experiment.

In addition, the system was moved to human subject facilities in the psychology building at UIUC. This will facilitate the use of undergraduate volunteers in human subject experiments. These new experiments are aimed at investigating the performance of all modalities, and then investigating important cross modality issues, including distractions, training, and retention.

January 1 - March 31, 1998 (FY98Q2)

Goals: Fabricate first generation flexible tactile control device using MEMS technology in the MCNC cleanroom (Q198). Evaluate placement options of the tactile actuators on the human body. In a similar manner, the positioning and number of actuators will be considered. Perform psychophysical experiments to evaluate placement options of tactile actuators on the human body. Test and evaluate performance of a flexible tactile control device. Complete literature search on oral perception. Fabricate micro/milli-scale patterns for oral perception studies.

Progress:The placement of tactile actuators on the human body was investigated this quarter. The infantry soldier was the focus of the design. The actuators would be used to transmit information to the individual soldier from the computer system or from other soldiers. Some of the issues addressed were: specific locations that do not interfere with other equipment, locations that can be easily sensed by the soldier and distinguished from other locations, locations whose associated meanings can easily be remembered, the number of actuators, the use of multiple simultaneous actuators, and control over the level of the actuation.

The fabrication run of the flexible tactile device was completed this quarter. Due to two earlier delays in the processing, the completion date was pushed from FY98Q1 to FY98Q2. During the last steps in the processing this quarter, one problem was encountered and solved. This problem was an adhesion failure of the upper metal layer contact bar to the underlying sacrificial oxide layer. The adhesion failure resulted in some of the contact bars tearing away from the metal ends that adhere to the surrounding polyamide. The deposition was repeated with the addition of an aluminum adhesion layer under the thicker gold layer. This deposition had good adhesion to the underlying oxide. The additional thickness of the metal in some locations was not expected to significantly affect the performance of the device.

Testing began on the fabricated devices. The wafers contain test structures, individual switches and resistors, and small and large arrays of switches and resistors. The testing to date of the test structures indicates continuity of the metal layers and the via structures, although the longer, thin and narrow resistor structures have some electrical open circuits. The removal of the sacrificial oxide layer was completed successfully with HF, followed by a water rinse and a hot plate bake. The released structures were tested and some switch functionality was observed. However, a large number of structures exhibited behavior consistent with the switches being shorted together. This was measured on both released and unreleased devices. This result on the unreleased structures is very surprising since the device design used the sacrificial oxide as an insulating layer between the bottom metal contact disk and the upper metal contact bar. To understand the source of the problem, the exposed polyamide was etched away with reactive ion etching on an unreleased wafer fragment. These structures were examined in a SEM. Figure 5 shows a micrograph from that wafer piece.

Figure 5: SEM micrograph of a portion of a completed switch with the polyamide layers removed by reactive ion etching.

The figure shows a portion of a switch where the upper contact bar crosses the edge of the lower metal contact disk. The double metallization of the upper contact bar is clearly visible. The lower contact disk is visible as the light-colored triangle at the left of the figure, directly under the upper contact bar. Below and to the right of the contact disk is the sacrificial oxide layer that extends beyond the contact disk. Based on this micrograph and others, a possible cause of the shorting was identified. The oxide coverage on the sidewall of the bottom contact disk does not appear to be complete. That, combined with a misalignment of the polyamide window over the contact disk, could lead to the top metal contact bar making an electrical connection to the edge of the contact disk.

The edge of the polyamide window can be seen in the figure as the point where the top contact bar suddenly rises at a 30° angle above the underlying layers. The structure appears to have a minimal separation between the edge of the contact disk and the edge of the polyamide window, much less than the designed amount. Further testing is underway to determine if a direct correlation between electrically shorted structures and this misalignment can be made. If this correlation is found, the remaining wafers will be examined to determine which have the best alignment of these two levels, and those structures will be tested.

The team at NCA&T continued work on two cognitive surveys focusing on the testing and characterizing of the perceptual issues associated with the design and location of tactile actuators. Research shows that the location of the actuators on the human body could affect the physical structure and choice of how component parts of the device are configured. Physiological determinants of sensation are known and well documented in literature.

We wanted to know how perception would determine the choice of location. Two kinds of studies were conducted: psychological and psychophysical. Of interest in this study was the need to select locations for tactile interface based on some established cognitive relationship between the site and the messages to be delivered by tactile stimulation. We also wanted to know whether terms relating to the message of interest would evoke some cognitive relationship with some human body locations, and whether subjects would have distinct preferences in associating the human body locations with the command terms. The results showed, in some cases, statistically significant preferences were established, and such preferences were similar for two subject groups: those with and without military training.

Another study on this concept looked into a possible relationship between visual-based hand signals and human body location for tactile interface with euphemisms that mention the body parts serving as cues. Specially selected euphemisms were tested with two groups of subjects (those with/without military training) to see if they elicit some relationship with any of the signals. The intent, given that euphemisms are commonly known mediums of human expression, is the demonstration of strong relationship, implying that humans will be better able to recall and interpret the tactile code represented by placement location when sensors are placed on those body parts mentioned in them. A thesis report was submitted and defended by a student on this subject.

A preliminary investigation was conducted to demonstrate the capacity of intelligent systems to distribute vibrotactile stimuli, and the capacity of humans to perceive them at varying force levels and random locations. The objectives were to measure the extent to which computer-generated, random stimuli could be effectively and efficiently distributed and perceived at multiple human body locations, and the human capacity to passively maintain heuristic knowledge of the changing energy states of dynamic variables, if coded by intensity of vibration. The findings indicated that using vibrotactile displays to enhance visualization had a strong potential to increase the awareness of information that otherwise would have been lost through filtering and depth stacking. The research is exploring use of the vibrotactile displays for sensory substitution (visual-tactile) as a mechanism for increasing awareness when information must be filtered and stacked, particularly in a mixed or variable-time collaborative task environment. The result of an experiment to test the performance of a mechanism which models human-computer interaction during collaborative task was analyzed and the test data analysis was completed.

The team at UIUC began to investigate the use of an oral interface. The novel oral tactile interface may provide an innovative approach for information transmission and human-machine interaction by freeing hands for tactile tasks. Compared to other body locations, the oral cavity is a more stable environment to house portable tactile input and output mechanisms. To realize the oral tactile interface, we first need to study the oral tactile sensitivity, as compared to other body areas.

We completed a search of literature related to oral tactile sensation. Limited previous work investigated such areas as tactile thresholds, two-point discrimination, oral stereognosis, and pattern/form identification. On the oral tactile sensitivity and discriminating ability, oral areas most frequently investigated are tongue (tip and blade), lips, and palate. Although past research does not provide very consistent results on these areas, it does show that lips and tongue tip are usually more sensitive than other oral areas and hands. Tongue tip is believed to have the highest discriminating ability in two-point discrimination with better or same-level performance when compared to the finger. Vibrotactile and electrotactile stimulations were also reported on oral areas. Lips and tongue are ten times less sensitive to vibratory stimulation than the fingertip, which leads to the belief that vibrotactile stimulation works better on hard tissues. For electrical stimulation, anterior regions of all oral structures are more sensitive than their posterior counterparts.

We will perform a further study on the oral tactile sensitivity by using patterns fabricated with photolithographic processes. Fabrication of these patterns is in progress. The first batches of patterns were made with Epon SU-8 photoresist on 4-inch silicon wafers, and are about 0.6 mm high. Different thickness (50 - 1000 µm) can be achieved with this type of photoresist. Problems encountered in the process include adhesion and sharp pattern edges. More patterns, including two lines and grating patterns, are to be designed and fabricated in the next quarter.

April 1 - June 30, 1998 (FY98Q3)

Goals: Evaluate potential specifications for the tactile sensor. This effort will examine generated specifications and placement options for performance and technical feasibility. Test and evaluate performance of a flexible tactile control device. Design experiments to test the capabilities, usability, and applicability of tactile actuators using human subjects. Design second generation flexible tactile input device. Integrate chording demonstration into ISL. Fabricate micro/milli-scale patterns for oral perception studies.

Progress: The overall specifications for tactile sensors was evaluated this quarter. Earlier quarters had included the generation of performance specifications and placement options for the sensors. The infantry soldier was the focus of the design. The sensors would be used to transmit information from the individual soldier to the computer system or other soldiers. The overall specifications were examined to ensure they will meet the desired performance requirements and also to verify that the specifications are technically feasible. Changes were made to the specifications as a result of this re-examination due to an improved understanding of the requirements of the Army of the tactile sensors.

Testing has continued on the flexible tactile control devices which were fabricated in quarters 1 and 2. The focus of the testing was on determining the cause of the behavior consistent with the switches being shorted together. This has been measured on devices that have not been released from the fabrication substrate as well as on devices which have been released. Switch shorting on the unreleased structures is unexpected since the device design used a sacrificial oxide as an insulating layer between the bottom metal contact disk and the upper metal contact bar, therefore, there should be no direct electrical path in unreleased devices. By further testing, we hope to determine whether there is a direct correlation between electrically shorted structures and a misalignment between the polyimide window and the underlying oxide release layer. The results of this testing has so far been inconclusive. The misalignment of the two layers is still a likely source of the problem so the remaining wafers are being examined to determine which ones have the best alignment of these two levels, and those structures will be tested. Since determining the cause of the shorting is fundamental to the design of the second generation flexible tactile device, the device design has been delayed until the testing of the first run is completed.

During this quarter, the team at North Carolina A&T State University submitted two conference papers for peer review which were returned with suggestions for improvement. These two research reports were considered publishable next year if resubmitted with some of the improvements suggested. The suggestions for improvement centered around providing more details to help clarify the research focus. In all, however, we are encouraged by the fact that all reviewers viewed this research as innovative and needed. To respond to this criticism, therefore, we have focused our efforts on reviewing materials and data related to the two papers reviewed in preparation for revising and resubmitting them to journals during the fourth quarter. In addition, plans are put in place for revising aspects of the experiments that we have already completed, so as to address salient points raised by one of the reviewers. The main issue of interest is that humans are either the tactile information encoder or decoder (cognitively and/or physically). In the experiments already completed, the two functions were not expressly segregated. The report is currently being revised to emphasize the results of each aspect at a time, so as to make space for more detailed analysis and presentation of research results.

Emphasizing human characteristics as either a tactile information encoder or decoder will result in more specific human factors issues which have cognitive, physical, and practical or functional implications when specifications for related interface devices are drawn. The question and inquiry will change from what humans are capable of doing to how and why they actually do things. We have focused on research that investigates the former as a beginning point, and hope to move into the latter in subsequent efforts as resources permit. The rationale that is being followed is that understanding the human tactual capacity or tendencies will give the scientific platform for generating the specific input parameters needed to draw desired interface specifications. As such, we are continuing to explore knowledge on a) human perceptual preferences for placement of tactile interface devices and b) how tactile displays can be used in real-time, to substitute and/or augment visual displays in a "human-computer-human" communication task phenomenon.

Figure 6: Optical micrograph of a pattern structure for use in oral perception studies.

The team at UIUC has begun to investigate the use of an oral interface. The work this quarter has focused on developing appropriate fabrication processes to create precise raised pattern structures for use in oral perception studies. They have developed a process based on EPON SU 8 epoxy based photoresist to create tall (800 µm) structures. This process gives us good control over the in plane dimensions of the structure. The structures (see Figure 6 for example) will be used for oral perception experiments to explore the feasibility of oral-based tactile communication. Experiments will include line separation tests and pattern recognition experiments.

In collaboration with Art Kramer and Paul Acthley, multi-modal perception experiments are continuing using the rifle-mounted tactile interface system that was developed previously. This study is expected to be complete sometime during Spring 1999.

MCNC began the assembly of a second tactile actuator system. This system of actuators will be used at MCNC for improving and debugging software changes requested by Herbert Nwankwo at NC A&T. The implementation of the control system was changed from a wired approach to a PC board which was designed for this project. This PC board implementation simplifies the assembly while still allowing for changes in the actuator device and control circuitry.

July 1 - September 30, 1998 (FY98Q4)

Goals: Evaluate potential specifications for the tactile actuator. This effort will examine generated specifications and placement options for performance and technical feasibility. Test and evaluate performance of a flexible tactile control device. Conduct experiments to test the capabilities, usability, and applicability of tactile actuators using human subjects. Design second-generation flexible tactile input device. Begin fabrication of second-generation flexible tactile input device. Integrate chording demonstration into ISL. Complete preliminary oral perception studies.

Progress: The overall specifications for tactile actuators were evaluated this quarter. Earlier quarters had included the generation of performance specifications and placement options for the actuators. The infantry soldier was the focus of the design. The actuators would be used to transmit information from the computer system or other soldiers to the individual soldier. The overall specifications were examined to ensure they will meet the desired performance requirements and also to verify that the specifications are technically feasible. Changes were made to the specifications as a result of this re-examination due to an improved understanding of Army requirements for tactile actuators.

Testing has continued on the flexible tactile control devices that were fabricated in the first and second quarters of FY98. The focus of our testing was on determining the cause of the measured behavior. The measurements indicated large currents consistent with the switches being shorted together. This has been measured on devices that have not been released from the fabrication substrate as well as on devices that have been released. Switch shorting on the unreleased structures is unexpected since the device design used a sacrificial oxide as an insulating layer between the bottom metal contact disk and the upper metal contact bar, therefore, there should be no direct electrical path in unreleased devices. We designed our test procedure to determine whether there is a direct correlation between electrically shorted structures and a misalignment between the polyimide window and the bottom metal layer. The remaining wafers were examined to determine which ones had the best alignment of these two levels, and those structures were tested. Electrical testing of those wafers also revealed that the unreleased structures were shorted in the same manner as the devices tested earlier. Even on the best wafers there was not a clear optical separation between the edges of the structures that would rule out the misalignment of the structures being the cause of the shorting. A sample was then cross-sectioned and examined in the SEM at MCNC. Figure 7 shows a typical result from the examination. The top white band is the upper metal contact bar and the lower white band on the right half of the photo is the edge of the bottom metal contact disk. The edge of the polyamide window corresponds to the change in slope of the upper metal layer from the horizontal orientation on the right side to sloping upward just past the edge of the bottom metal. This photo clearly shows the misalignment between the two layers. Although there is not a clear metal to metal connection shown in this photo, the misalignment combined with the earlier photos that exhibit poor oxide coverage on the sidewall of the bottom metal, gives strong evidence that the misalignment of the levels was a cause of the shorting in the devices. The preparation of the sample could have led to no connection being visible in the photo.

Figure 7: SEM micrograph of the edge of a bottom metal contact disk with a misaligned polyamide window.

Based on these results, the mask set was redesigned for the second-generation device. The first change was to increase the overlap of various layers to ensure that the misalignment errors of the first run would not be repeated. The second change was to move the resistors used to create the variation in sensed voltages across the array, out from within each cell to locations around the periphery of the arrays. These reduced the cell size and improve the reliability of the switches. The resistors created redundant voltage dividers in the first generation arrays for the horizontal and vertical directions. To minimize the power consumption, the resistors were designed with a large number of squares that occupied a large fraction of each cell in the arrays. By moving the resistors outside the arrays in the second-generation design, the cell size was reduced, allowing a finer resolution of position by the array. Because there are fewer voltage dividers in parallel when the resistors are located outside the array, the overall power consumption of the new design will be less than the first generation devices. To maximize the resistance of the first generation cells, the resistors were designed with a small pitch and were fabricated with a very thin layer of chrome (25 or 40 nm). Testing of electrical test structures on the first generation of devices indicated that the resistors with the smaller pitches were prone to have breaks in the resistors. By moving the resistors outside the arrays, more area was available for each resistor, so only the largest pitch resistors were used, and the length of the resistors was increased to allow for a thicker chrome film and better reliability. Figure 8 shows the layout of a small five by five array of cells that is part of the new design. The first five mask levels of the new design have been manufactured in preparation for next fabrication run.

Figure 8: Layout of a small array of cells in the second generation flexible tactile input device.

This quarter the team at NCA&T continued to review and revise procedures for improving the experiments relative to placement of the tactile devices on a human body. They submitted and have secured approval from the university IRB to continue with the studies titled "Perceptual Preferences for Placement of Tactile Communication Interface Mechanisms" and "Vibrotactile Displays for Sensory Substitution during 'Human-Computer-Human' Communication Task". The second study will use the "tactilator," a tactile stimulator device conceived in conjunction with staff at MCNC. During this quarter a redesign and reprogramming request was made to MCNC as part of the procedural enhancement scheme stated earlier. MCNC modified the operating program and completed the assembly of a new "tactilator" based on a PC board assembly. The new version of the "tactilator" program was tested in early September as part of the redesign process, with further suggestions submitted to Dr. Palmer at MCNC for improvement. This redesign effort was not part of an existing milestone, but was necessary as a result of the research process. The consequence is that a delay was experienced in completing the planned quarter activity as stated above. The laboratory studies will commence as soon as this redesign and review process is complete in the next quarter.

In order to investigate the tactile discriminating ability of the tongue, the team at UIUC has been working on the fabrication of the static plastic patterns and the experimental setup. These patterns are fabricated through a photolithographic process using Epon SU-8 photoresist, a new type of photoresist capable of producing thick structures. Groove patterns with groove (line) widths of 4, 2, 1, 0.5, 0.25, and 0.125 mm have been made for the grating orientation experiment; Two-line patterns with two-line distance of 4, 2, 1, 0.5, and 0.25 mm have been made for the two-line separation experiment. A thickness of about 0.8 mm can be achieved with the process we are using. During the fabrication process, problems of sharp edges and excessive bowing of the patterns were encountered. The process parameters, especially soft-baking and hard-baking time and exposure time and intervals, have then been improved to solve these problems. Design and construction of experimental setup have also been carried out. The setup is expected to allow a subject to explore patterns either outside or inside his/her mouth using mouthpieces that can be quickly changed. The autoclaving method that they planned to use for sterilization of the mouthpieces was proven not appropriate as the high temperature destroys the patterns that were made. They are searching for an alternative chemical sterilant to achieve high-level disinfection and/or sterilization. A selection between two EPA-registered sterilants (Cidex and Sporox) will be completed soon with the help from the Division of Environmental Health and Safety at UIUC.

January 1 - March 31, 1999 (FY99Q2)

Goals: Complete fabrication of second-generation flexible tactile input device. Evaluate the functionality of second generation flexible tactile input device. Design experiments to evaluate second generation flexible tactile input device with human subjects. Develop a coding protocol for tactile interface device to allow insertion of these devices into the ISL. Integrate chording demonstration into ISL. Design oral perception and psychophysics experiments.

Progress: The fabrication of the second generation flexible input device was delayed due to delays in the establishing of a new clean room at MCNC and the requalifications of the unit process steps in it. The fabrication began at the end in the second quarter. The requalifications are still occurring and as a consequence the processing will move at a slower than normal rate through the first portion of the fabrication sequence. As a result, no testing of the devices was possible this quarter.

Five experiments were designed for testing of the flexible input tactile device after its fabrication to evaluate its performance with human subjects. These tests will evaluate the functionality of the device and its ability to control a graphics cursor in adverse environments including vibrations and large motion jolts.

In preparation for the symposium this quarter, further analysis was done on the data generated by the vibrotactile experiments. An experiment had been done earlier at NC A&T that involved measuring the response of subjects to the vibrotactile stimulation at five different locations on the arms and neck. An analysis of the time delay between the stimulation and the response by a subject (both correctly and incorrectly identified stimulation sites) revealed that there were statistical differences between the time responses for different conditions. The data indicate that the response time was less for high vibration levels (M = 0.840 sec) compared to low vibration levels (M = 0.935 sec) with a significance of p = 0.000. The longer response time for the lower vibration level could indicate that the subject needs a longer time to associate a vibration with a body location. Comparing the response time for correct answers (M = 0.869 sec) and incorrect answers (M = 0.988 sec) shows a significant difference (p = 0.020). A more detailed comparison of the four subsets defined by the vibration level and answers shows some other information about the responses. The means of the response times for the high vibration level, correct answer and high vibration level, incorrect answer are nearly the same (0.840 and 0.830 sec, respectively). This appears to indicate that the subject reacted in the same way in these two cases and possible only pressed the wrong key by mistake. The number of samples in these two cases, 296 correct and 9 incorrect, could support this idea. However for the low vibration level the means of the response times are quite different (0.920 sec for correct responses and 1.059 sec for incorrect responses) with a statistical significance of p = 0.037. This appears to indicate indecision by the subjects as to location of the stimulation at the low vibration level.

The experiment done at Sytronics using five actuators that were placed in a cloth vest on the upper chest, and controlled to indicate 18 different directions to provide directional information to a dismounted soldier, showed that the tactile actuators could be used. Figure 10 shows the mean absolute error between the desired direction and the subject indicated direction for the tactile and visual inputs. The plot of the visual input data shows the accuracy with which human subjects specify directions in the cardinal directions (with respect to their bodies), as is well known. The tactile input data has the same basic shape visual data with larger errors primarily at the directions that lie between the actuator locations.

Figure 10: A plot of the measured absolute error between the desired direction and the subject indicated direction as a function of direction.

This quarter the team at NC A&T completed two experiments to verify and validate results of earlier experiments. In these studies, we redesigned two experiments to test:

1. The ability of subjects to associate some euphemistic terms that mention human body parts with military communication terms for situation/status reports or messaging.

2. The ability to directly associate actual parts of the human body with military communication terms for situation/status reports or messaging.

In the previous experiment conducted with Mr. Urquhart, the treatment factors were not randomly presented. This may have influenced the results, as some subjects may have completed the test in sequential order. In the new design conducted with Mrs. Aikens, similar or equivalent subject groups were used. The terms were presented to the subjects in a completely randomized fashion. Rather than a list of terms on a single piece of paper, we listed each term from both treatment variables on separate cards. Each pack was thoroughly shuffled before being handed over to subjects.

The NC A&T team started work on completing a two phased experiment using the vibrotactile actuators. The objectives are to verify previous results that showed the subjects were able to sense and identify the spatially separated actuators, with a larger and less homogenous subject group; and to include a noisy task environment. The noisy environment will be structured as either a distracter from the primary activity of responding to the vibrotactile stimulation or as an engrossing primary task with the monitoring of the tactile messages serving as the secondary task. We want to understand whether the tactile messaging can still be effective with environmental distractions and if the tactile messaging will distract or enhance performance on a primary, engrossing task. In other words, can tactile messaging be useful in helping, say a C2V group engaged in HCI environment activity by increasing performance. It is a test of their capacity to use multi-modal sensing (Tactile, Visual, and Auditory), in which tactile interfacing will play major role.

Another thesis experiment for the validation of the term association results is planned to test and validate the ability to use the associations between body locations and commands. Under this plan, tests will be conducted to see how novice and trained subjects, who use selected body locations with high correlation to commands, will differ in their response when the interface locations serve as cues.

The team at UIUC has been evaluating the oral cavity's advantages as a site for a tactile interface for the purpose of communication and situational awareness. While tactile interfaces have been studied on body surfaces such as hands, back, abdomen, etc., only a few researchers have attempted to study the oral cavity. Nevertheless, the oral cavity is promising for electrotactile stimulation because of low current/low voltage simulation and low skin resistance. Recently we proposed an ultra-flexible electrotactile display design for the application of electrical stimulation ot oral structures. The first generation of the display has been made as shown in the figure. Because of the topological irregularity of the skin surface, like the roof of the mouth, the display has to be very flexible to fit into the local curvature of the surface. The high degree of the flexibility prevents the use of commercially available flexible circuit technology or fabrication of electrodes on flexible substrate directly. However, thin-film technology can be still utilized for constructing the display through a polyimide transfer process in which structural layers can be processes on thin polyimide file before it is peeled off from a glass substrate. The shape of the electrodes is another consideration in the design of the array. Compared with planar electrodes, three-dimensional, dome-shaped electrodes would allow better contact with the skin surface and larger electrode surface area within less display space. The structure is realized with an isotropic nickel electroplating process. A final cut on the polyimide film along the edge of the wafer results in an electrode array in an ultra-flexible polyimide film separated from the glass substrate. Figure 11 contains a photograph of a fabricated electrode array.

Figure 11: The nickel electrodes, 700 µm in diameter and 200 µm in height, are electroplated through an opening of diameter of 300 µm. excellent uniformity has been obtained. Test under mechanical force shows good adhesion of the electrodes to the bottom polyimide layer.

April 1 - June 30, 1999 (FY99Q3)

Goals: Complete fabrication of second generation flexible tactile input device. Evaluate the functionality of second generation flexible tactile input device. Integrate chording demonstration into ISL. Conduct experiments to evaluate second generation flexible tactile input device with human subjects. Develop experiments to examine training needs and tactile information retention using human subjects. Design and conduct experiments to test cognitive load and implications of coding protocol for tactile interface devices, using human subjects. Build stimulator for dynamic oral perception studies.

Progress: Fabrication continues on the second-generation flexible tactile devices. The wafers have had the bottom electrodes deposited and patterned as well as the sacrificial oxide layer. They are currently undergoing a polyamide etch to further define the contact areas. The fabrication run has been delayed due to various equipment issues that have been discussed in previous reports and will not be discussed here. Fabrication should be completed to allow for examination of the devices in the Fourth Quarter. The team at MCNC has been working with NCA&T to develop experiments to test tactile information cognitive loading on subjects performing tasks. MCNC is providing equipment to help NCA&T with these studies. Discussions will be held shortly with the staff at UIUC to transition some tactile equipment there for some multimodal experiments.

The NCA&T team's main effort went into designing the most fitting experimental alternatives for the next few months. We explored several scenarios for achieving the platform for conducting the planned experiments for a thesis being proposed by Ms. Aikens. One plan explored was that of integrating the tactilator system with over-the-counter type programmable video games, in ways that will enable the subjects use the games as primary tasks of interest, while the tactilator stimulation inject signals to instruct about tasks to be performed at some point. The message could be reporting a status information to the subject on systems. The subject while engaged in a game, gets a signal, if the subject perceived such signal, he/she would stop respond to it and proceed with the game. The game will serve as main activity being performed while the tactilator stimulation will serve as random message to which the subject must respond.

The thrust of this plan is to determine how the level of engagement (or distraction from the stimulation) affects perceptions of the various force levels of the stimulus. The result will produce understanding of the force thresholds required for the stimulus to be perceived when soldiers are fully engaged with some main or primary tasks in a battle field environment. In reality, soldiers in the field will be fully engaged in other primary tasks. The tactile communication activity will not be the main tasks occupying their attention. So, having the correct perceptual threshold for effective stimulation will provide insight as to what will be the most appropriate threshold level.

We also planned for summer activities. Several discussions were held with MCNC group to articulate and outline the design of experiments planned for this summer June-July. The experiments will integrate and test results of two previous studies. We worked to narrow this study due to limitations in time, space, and money to assemble the desired set of apparatus. We are continuing to review published materials to expand and increase our understanding of issues that are important to planned experiments.

The team at UIUC team is in the process of making a flexible 49-point electrode array as an oral electrotactile display to be used inside the mouth. The size of array will allow us to present static and dynamic patterns onto oral structures for psychophysical studies. The process is based on the earlier process to make a 3X3 array. The array uses polyamide as a supporting structural material for flexibility and the electrodes are electroplated to a dome-shape for conformity between the electrodes and the oral surface. The mask design of the display is shown as in the figure. The display includes a 7X7 array with lead lines and bonding pads integrated on a 3-inch glass wafer. We have fabricated several devices and are now optimizing the electroplating process to improve electrode uniformity. We expect to begin testing the device in the next few weeks. Several milestones are delayed due to the slow progress in fabricating the second-generation flexible tactile array. Work is continuing and being expedited to the extent that is possible.

Figure 12: Schematic of oral electrotactile device. The array uses a polyimide supporting structure and consists of a 7x7 array with lead lines and bonding pads integrated on a 3-inch glass wafer.

July 1 - September 30, 1999 (FY99Q4)

Goals: Complete fabrication of second generation flexible tactile input device. Evaluate the functionality of second generation flexible tactile input device. Conduct experiments to evaluate second generation flexible tactile input device with human subjects. Develop a methodology for inserting the flexible tactile input device into the ISL. Test the coding protocol with tactile hardware in the ISL. Conduct experiments to examine training needs and tactile information retention using human subjects. Preliminary study of the feasibility of the stimulator for the oral tactile application.

Progress: Fabrication was completed on the second-generation flexible tactile devices. The basic design of the device is an array of pressure sensitive switches that connects a voltage tap to a resistor voltage divider network. Separate arrays are included for X and Y position allowing the voltage sensed by the tap to indicate the complete position of the closed switches. Testing results from the first generation device indicated two design and fabrication difficulties that needed to be resolved. The first was a large incidence of shorting within the switches due to two factors: the failure of a deposited oxide to cover the sidewalls of the patterned metal features of the switches and the misalignment of a polyimide insulator layer designed to overlap the edges of the metal features. These factors led to the top metal features of the switch coming in contact with the bottom metal feature's edges. The second fabrication run used a more conformal oxide deposition process, plasma enhanced chemical vapor deposition, and modified design rules to increase the overlap between the polyimide and metal features. The second difficulty was in the formation of the resistors. The initial design had resistors in each cell of the array, with small dimensions to keep the cell size small and the resistance values high. The yield of those resistors was not high enough to produce working parts consistently. The second generation design was changed by consolidating the resistors and moving them to the edges of the arrays. There the size constraint was eased, allowing larger dimensions and improving the yield. Inspections of the completed devices indicate that the modifications to the design and fabrication process were successful. The metal shorting bar appears to be isolated from the contact disk and the resistors appear to have a higher yield. Electrical testing of the devices has begun. Preliminary results confirm the optical inspections of good isolation between the shorting bars and the contact disks and functioning resistor chains. One portion of a wafer has had the sacrificial oxide removed and the discrete test switches were closed mechanically with a micromanipulator probe resulting in the closed switch and wires electrically measuring around 100Ω. Further testing to characterize the devices will be done in the next quarter including some experiments with human subjects. To assist in the removal of the sacrificial oxide without the switches sticking shut during the drying of the devices, a supercritical CO₂ drying chamber is being assembled at MCNC. The system should be completed by the middle of November The chambers uses supercritical CO₂ to make the transition from a liquid to a gas without the formation of liquid/gas interfaces where surface tension can stick two surfaces together. Because of the delays in completing the fabrication of the devices, and the subsequent delays in their testing, development of the methodology of inserting the devices into the ISL has not been completed due to the lack of the results from the testing.

Figure 13: Microphotograph of a portion of an array of a completed flexible tactile input device. The disks act as the bottom electrode of the switches and the metal bars across the disks as the top electrodes.

The team at MCNC has been working on modifing the tactile actuator hardware to allow its operation with a range of power supply voltages and outputting a range of voltages to generate different stimulation levels. A commercially available integrated circuit was located that reads a potentiometer setting and converts the power supply voltage to a lower one. A digitally controller potentiometer will be used to allow the printer port of a computer to set the pager motor vibration. Once this setup has been tested, discussions will be held shortly with the staff at UIUC to transition some tactile equipment there for some multimodal experiments.

Several experiments were planned and implemented by the NCA&T team during this quarter, to verify and validate five body locations identified in previous studies through an association test between euphemistic terms and military messaging terms. We had previously hypothesized that if it could be determined that a euphemistic term which mentioned a human body part, had a substantial association level with a military messaging term, then it may be possible for the body part mentioned in the euphemistic terms to be used as a tactile interface location for communicating the military message. During this quarter we selected the five pairs of euphemisms and military terms that showed the highest common association across the subjects group, so the body parts could be directly tested against the military terms for verification. In one experiment, the subjects were tested to verify that tactile stimulation could be perceived at the five locations selected, under a simulated battlefield-type noise environment. The tactilators were used to generate and deliver stimuli at the five body locations identified, and subjects responded by identifying where the stimulus was located. In another study, the subjects completed a test designed to verify a previous association study that used an indirect approach to identify candidate body locations. This time the subjects completed an association task in which they directly matched military terms to the five body locations previously identified. A third experiment was started which would verify that when the tactilator is stimulated at the selected body part, the subjects will respond to it by selecting a military term that relates to the location.

Progress: Additional testing was done on the recently completed second-generation flexible tactile devices. The basic design of the device is an array of pressure sensitive switches that connects a voltage tap to a resistor voltage divider network. Separate arrays are included for X and Y position allowing the voltage sensed by the tap to indicate the complete position of the closed switches. Electrical testing of the devices confirms the optical inspections of good isolation between the shorting bars and the contact disks and functioning resistor chains. One wafer has had the sacrificial oxide removed and the discrete test switches were closed mechanically with a micromanipulator probe resulting in the closed switch and wires electrically measuring around 100Ω. Further testing of some of the small 5 by 5 arrays has been done with the measured resistances through the tap giving the expected monotonic increase in resistance. An example measurement is shown in Figure 14. There are four pairs of resistors between the five rows or columns. The closing of a switch changes the resistance seen by the tap from a parallel combination of zero pairs and eight pairs to a parallel combination of four pairs and four pairs. These combinations of resistances give the shape to the curve shown in the figure. The measurements of the resistance seen by the tap is a simpler version of the final measurement system where a fixed voltage will be placed across the resistor strings and a voltage will be measured with a high impedance voltmeter at the tap. That measurement will produce a straight-line variation in resistance as a function of position. Additional characterization of the devices will be done in the next quarter including some experiments with human subjects. To assist in the removal of the sacrificial oxide without the switches sticking shut during the drying of the devices, a supercritical CO₂ drying chamber assembly was completed at MCNC this quarter. The chamber uses supercritical CO₂ to make the transition from a liquid to a gas without the formation of liquid/gas interfaces where surface tension can stick two surfaces together. Because of the delays in completing the fabrication of the devices, and the subsequent delays in their testing, development of the methodology of inserting the devices into the ISL has not been completed due to the lack of the results from the testing.

Figure 14: Graph of the measured and calculated resistance measured at a tap on a five by five array of switches.

The team at MCNC has completed a modification to the tactile actuator hardware to allow its operation with a range of power supply voltages and outputting a range of voltages to generate different stimulation levels. A commercially available integrated circuit was located that reads a potentiometer setting and converts the power supply voltage to a lower one. A digitally controlled potentiometer was used to allow the printer port of a computer to set the pager motor vibration. This will allow the use of a single power supply and the ability to tune the vibration levels to where they are easily sensed. While copies of the hardware are constructed, some simple tests will be done to evaluate the sensitivity of subjects to variations in the vibration levels. One other possible application of the new hardware is to use changing levels of vibration as a second modality to the sonification of a Bayesian Belief Network

During this past quarter the NCA&T team completed testing additional subjects for the validation study started in the previous quarter, to verify and validate five body locations identified in previous studies through an association test between euphemistic terms and military messaging terms. We had previously hypothesized that if it could be determined that a euphemistic term which mentioned a human body part, had a substantial association level with a military messaging term, then it may be possible for the body part mentioned in the euphemistic terms to be used as a tactile interface location for communicating the military message. During the previous quarter we selected the five pairs of euphemisms and military terms that showed the highest common association across the subjects group, so the body parts could be directly tested against the military terms for verification. In one experiment, the subjects were tested to verify that tactile stimulation could be perceived at the five locations selected, under a simulated battlefield-type noise environment. The tactilators were used to generate and deliver stimuli at the five body locations identified, and subjects responded by identifying where the stimulus was located. We are continuing to compile and summarize data collected in this study for statistical analysis.

Since the initial testing of the oral electrotactile display on the oral structures showed positive results, the team at UIUC has started to build a system that integrates the tongue touch keypad (TTK) ( The NewAbilities Inc., CA) and the flexible oral electrotactile display. The system is intended to demonstrate the two-way communication via the oral tactile interface, information to the subject via the electrotactile display and information from the subject via the tongue touch keypad. Currently a computer user interface has been designed and implemented on PCs. The program is designed to respond to the inputs from the TTK, and meanwhile, allow a user to send out the tactile stimulation patterns onto the electrotactile display. The scenario that the program tries to simulate is a soldier moving to a designated position inside the jungle of forest with the silent tactile guidance in his mouth. The range of the field is shown on the computer screen as an image of the forest. After the commander clicks on the field, an image of a soldier is displayed showing his original location. The commander can then select the type of geospatial cues using the program. By pressing one of the arrow keys on the keyboard, the geospatial cue is sent out. On the screen, a visual array shows the movement of the tactile pattern presented on the electrotactile display. Currently two types of geospatial cues (arrow and line patterns) and four directions (left, right, up, and down) can be generated in the program. The system transmits the signal via the serial communication between the computer and the control circuit unit for the display. The demonstrator wearing a mouth retainer can then indicate his next position by pressing the keys on the TTK with the tongue. The signal is sent to the computer through wireless connection. The program responds to the signal by showing the new position of the soldier on the field.

A tactile experiment performed this quarter by SYTRONICS was a laboratory usability study, which examines the effectiveness of a tactile display in a plausible application. RSC is developing microsensors that detect the presence of humans and warn defenders. The experimental goal was to measure the effect of prompting, including tactile prompting, on defender's vigilance performance.

Subjects performed a visual vigilance task over 40 minutes where they watched for 0.5 second duration threatening pop-up intruders on four displays (12 different azimuths) which subtended 120 degrees of visual angle. In the baseline condition, no prompting was provided.

In the tactile condition, subjects wore four vibrating beeper motors mounted in a cloth vest on their chest. Each motor corresponded to a screen and was not activated 10% of the time when a threat occurred. Motors also activated on 10% of the trials that no threat occurred on the displays. These miscues were designed to prevent the prompt from making the task trivial and to mimic the less than perfect reliability, which a sensor system would likely have.

A third 3-D audio prompting condition was run as a further reference to tactile effectiveness. SYTRONICS provided assistance to ARL/HRED in a 3-D audio study and used ARL prerecorded 3-D audio wave files (recorded from a mannequin head) to prompt subjects about which screen contained a threat. The same 10% miscues were employed in this condition. The results for the prompting experiment are still being analyzed at reporting time.

January 1 - March 31, 2000 (FY00Q2)

Goals: Compare the effectiveness of visual, 3-D audio and tactile displays in guiding cross-country navigation. Conduct experiments to evaluate second generation flexible tactile input device with human subjects. Design and fabricate masks for third generation flexible tactile input device. Develop a methodology for inserting the flexible tactile input device into the ISL. Test the coding protocol with tactile hardware in the ISL. Design and test an experiment that evaluates the ability of tactile commands to be sensed during an engrossing second task. Analyze data gathered on the use of body location intuitive cues for tactile command recognition. Develop experiments for geospatial and dynamic stimuli presentation in the oral cavity. Conduct experiments to evaluate the use of tactile devices to communication of simple commands.

Progress: Further electrical/mechanical testing was done on the recently completed second-generation flexible tactile devices. The basic design of the device is an array of pressure sensitive switches that connects a voltage tap to a resistor voltage divider network. Separate arrays are included for X and Y position allowing the voltage sensed by the tap to indicate the complete position of the closed switches. Electrical testing of the devices confirms the observations made by optical inspections that there is good isolation between the shorting bars and the contact disks and functioning resistor chains. Testing of some of the small 5 by 5 arrays has been done with the measured resistances through the tap giving the expected monotonic increase in resistance. An example measurement of all 25 locations in an array is shown in Figure 15. There are four pairs of resistors between the five rows or columns. The closing of a switch changes the resistance seen by the tap from a parallel combination of zero pairs and eight pairs to a parallel combination of four pairs and four pairs. These combinations of resistances give the shape to the curve shown in the figure. The measurements of the resistance seen by the tap is a simpler version of the final measurement system where a fixed voltage will be placed across the resistor strings and a voltage will be measured with a high impedance voltmeter at the tap. That measurement will produce a straight-line variation in voltage as a function of position. Initial testing was done with a tungsten probe whose sharp tip is made for probing metal pads less than 40µm in size. That sharpness easily tore the polyimide if too much force was applied with the micromanipulator. A number of different probe materials and tips were tried in an effort to limit the damage to the polyimide. The focus was on a more rounded tip that was mechanically compliant, and was still small enough to be able to selectively probe individual switches for testing. The best results were obtained by dipping the tungsten tip in a hot glue (brand name Crystal Bond) to form a small ball at the tip when it cooled. It was also found that the use of a thin plastic film like Saran Wrap also prevented damage to the devices. We have not been able to easily probe the double switch cell arrays since the separation of the individual switches requires a larger radius ball of more compliant material than we currently have made. Some preliminary tests were made of the lifetimes of the switches by using the manipulator to repeatably depress a contact until electrical failure. The switches lasted approximately 20 to 40 cycles. Additional testing will be done to try to determine the failure mode. Additional characterization of the devices will be done in the next quarter including some experiments with human subjects. Further refinements to the supercritical CO₂ drying chamber assembly were done this quarter to improve the operation of it. The chamber uses supercritical CO₂ to make the transition from a liquid to a gas without the formation of a liquid/gas interface where surface tension can stick two surfaces together. Because of the delays in completing the fabrication of the devices, and the subsequent delays in their testing, development of the methodology of inserting the devices into the ISL has not been completed due to the lack of the results. The design and fabrication of a third set of masks for the final device generation was also delayed until sufficient data can be taken to determine the optimum device size and style from the options fabricated in this run.

Figure 15: Graph of the measured and calculated resistance measured at a tap on a five by five array of switches.

Two copies of the modified tactile actuator hardware were completed and distributed to Bill Marshak of Sytronics and Art Kramer of UIUC at the annual symposium. A third copy is being prepared for NCA&T. The modified hardware allows its operation with a range of power supply voltages and outputting a range of voltages to generate different stimulation levels.

The team at NCA&T reviewed materials and held discussions to decide on the structure and apparatus for the experiment to test the tactilator application under the situation with an engrossing task. This task was delayed because Ms Aikens was not available to continue as planned. However, we continued with meetings to consider the various possible platforms, such as video games and programmable joysticks with force feed-back for integration with the tactilator as a medium for the engrossing task. The scenario of interest at this point is to harness and organize these three components (joystick, tactilator and video games) so that the messages from the tactilator is used as a strategic information resource media for controlling activities to be performed by the subjects (video players). The purpose here is to achieve in the subjects the state of mind of being engrossed in the game, but being able to be interrupted randomly by the tactilator to provide cues and other control information which would be used in modifying the engagement strategy or level in the game. The dependent variable will be the engagement behavior, levels, or strategy being implemented as a result of the tactilator control messages (independent variables) being received. The planned process will use the newly modified tactilator such that fine adjustments in the force level of the tactilator stimuli can be made until several levels (3 to 4) are achieved that are desirable to the subjects in terms of ease of perception and comfort. These levels are then used in the main experiment for further analysis.

A prototype of an oral tactile interface with both input and output capability through the tactile channel has been implemented by the team at UIUC. A demonstration of two-way tactile communication using the oral tactile interface has been performed to show the potential application of the tactile interface for navigation guidance. The oral tactile interface is built into a mouthpiece that can be worn in the mouth. The flexible tactor array is mounted on top of the mouthpiece so that it is in contact with the roof of the mouth, while the tongue touch keypad (NewAbilities, Mountain View, CA) is located on the bottom side of the mouthpiece. An interfacing system is implemented to control both the tactor array and the tongue touch keypad. The system is programmed to simulate the scenario of navigation guidance with simple geospatial cues. The scenario that the program tries to simulate is a soldier moving to a designated position inside the jungle of forest with the silent tactile guidance in his mouth. The range of the field is shown on the computer screen as an image of the forest. After the commander clicks on the field, an image of a soldier is displayed showing his original location. The commander can then select the type of geospatial cues using the program. By pressing one of the arrow keys on the keyboard, the geospatial cue is sent out. On the screen, a visual array shows the movement of the tactile pattern presented on the electrotactile display. Currently two types of geospatial cues (arrow and line patterns) and four directions (left, right, up, and down) can be generated in the program. The system transmits the signal via the serial communication between the computer and the control circuit unit for the display. The demonstrator wearing a mouth retainer can then indicate his next position by pressing the keys on the TTK with the tongue. The signal is sent to the computer through wireless connection. The program responds to the signal by showing the new position of the soldier on the field. The demonstration shows that the simulated task of simple geospatial cues and confirmations can be adequately handled through the oral tactile interface. The team is currently developing experiments for geospatial and dynamic stimuli presentation in the oral cavity.

SYTRONICS completed the analysis of two experiments done last quarter with the MCNC Tactilator. Recall that participants detected vibrations of five chest-mounted stimulators at three different intensities against the vibratory noise generated by a massage chair. Typical data for one subject (Figure 16) shows how sensitivity increased as a function of the increasing signal-to-noise (SNR) at each of five positions. Positions 1 and 5 were nearly on top of the shoulders. Position 3 was above the sternum. Positions 2 and 4 were half way between on a radius around the neck. Calibration data indicated wide differences between the stimulators in their responses to the same voltage. SNRs were based on each devices individual calibration because of the variability.

Figure 16: Signal detection measure d' (sensitivity) as a function of stimulator position and SNR, the SNR is expressed in decibels.

Increasing voltages produced higher SNRs and better d'. Performance exceeds chance levels above a d' of approximately 2.0. Centrally located Tactilator stimulators (2-4) had greater slopes than did the peripherally located ones (1,5), indicating greater sensitivity and/or increased resistance to noise. Noise masking occurred despite very little overlap in the signal and noise energy distributions.

The next step will be employing the enhanced Tactilator in a cross-country navigation study as a directional indicator display. In section 4.0 Usability and Validation is a second report from SYTRONICS concerning the multimodal use of tactile, visual and 3D audio stimuli.

April 1 - June 30, 2000 (FY00Q3)

Goals: Conduct experiments to evaluate second generation flexible tactile input device with human subjects. Design and fabricate masks for third generation flexible tactile input device.

Fabricate third generation flexible tactile input device. Design experiments to evaluate third generation flexible tactile input device with human subjects. Develop a methodology for inserting the flexible tactile input device into the ISL. Test the coding protocol with tactile hardware in the ISL. Design and test an experiment that evaluates the ability of tactile commands to be sensed during an engrossing second task. Conduct an experiment that tests the ability of tactile commands to be sensed during an engrossing second task. Analyze data gathered on the use of body location intuitive cues for tactile command recognition. Design experiments to evaluate the use of intuitive cues versus training for tactile command recognition. Develop experiments for geospatial and dynamic stimuli presentation in the oral cavity. Oral geospatial cues demonstration.

Progress: Further electrical/mechanical testing was done on the completed second-generation flexible tactile devices. The basic design of the device is an array of pressure sensitive switches that connects a voltage tap to a resistor voltage divider network. Separate arrays are included for X and Y position allowing the voltage sensed by the tap to indicate the complete position of the closed switches. Electrical testing of the devices confirms the observations made by optical inspections that there is good isolation between the shorting bars and the contact disks and functioning resistor chains. Testing this quarter focussed on the precision testing of the double switch cells. These cells are designed to be more fault tolerant by requiring two adjacent contacts to be depressed and connected in a cell for a position to be sensed. The spacing between the contacts is approximately 220µm. To depress both simultaneously requires either a probe tip with a large flat tip or a soft probe tip that can conform to the surface. A tungsten probe tip was coated with a silicone film to create a soft tip. This was then successfully used to close pairs of contacts in the small five by five arrays. In actual operations of the flexible tactile array, the finger tip will be large enough and conformal to close pairs of contacts easily. However for this testing, we needed to be able to close specific pairs of contacts to verify the operation of the device. Additional characterization of the devices will be done in the next quarter including some experiments with human subjects. Because of the delays in completing the fabrication of the devices, and the subsequent delays in their testing, development of the methodology of inserting the devices into the ISL has not been completed due to the lack of the results. The design and fabrication of a third set of masks for the final device generation was also delayed until sufficient data can be taken to determine the optimum device size and style from the options fabricated in this run. Due to time commitments of some other programs at MCNC, less progress was made on this task this quarter than was planned at the start of the contract year. Increased efforts will be made on this task starting in August.

Two copies of the modified tactile actuator hardware are being assembled for NCA&T and Rockwell. The modified hardware allows its operation with a range of power supply voltages and outputting a range of voltages to generate different stimulation levels. The version sent to Sytronics was returned to MCNC to improve its operation over a wider range of power supply voltages.

A change in the research staff at NCA&T involved in this task occurred this quarter, and as a result no milestones were completed. A meeting is scheduled for the beginning of August between Scott Goodwin-Johansson, Dan Mountjoy and Celestine Ntuen to create a plan for the efforts at NCA&T.

A prototype of an oral tactile interface with both input and output capability through the tactile channel has been implemented by the team at UIUC. Some psychophysical experiments on static and moving patterns have been performed on six human subjects to study the tactile sensory characteristics on the roof of the mouth and its performance in identifying geospatial cues. The psychophysical studies also provide an objective evaluation about the performance of the oral electrotactile display and the potential of the device for two-way oral tactile communication. The experiments include threshold measurement, evaluation of spatial sensitivity, and identification of geospatial cues. In the experiment of threshold measurement, a static square and a dynamic square are used as the test patterns, and an adaptive method of 2-alternative forced choices (2AFC) is used to find the thresholds. Results show that the thresholds for electrotactile stimulation on the roof of the mouth are around 10-20V. Spatial sensitivity is evaluated with two-line patterns of varying gap and line shifting patterns of varying shift, and with static square patterns and dynamic square patterns in different size. The experiment shows that the spatial sensitivity depends on the pattern presentation mode, and the subjects show better spatial discrimination on moving or dynamic patterns. The experiment on geospatial cues is performed on two-choice geospatial cues and four-choice geospatial cues. Results indicate that identification on left or right-moving patterns is highly accurate while errors on forward or backward moving patterns vary considerably among subjects. This will likely be the final contribution from the efforts lead by David Beebe due to his relocation to the University of Wisconsin and the graduation of the student who worked on this project.

SYTRONICS was delayed in continuing the experiments with the MCNC Tactilator. To complete the calculation of the common metric for the tactile communication channel, it is required that the input levels of the tactilator be manipulated with respect to the noise level that is fixed. The voltage levels supplied by the system at SYTRONICS were over too large of a range for the tactilator to work properly and required the return of the tactilator to MCNC for a hardware adjustment to enable that voltage range. The modified tactilator was returned to SYTRONICS at the beginning of the fourth quarter.

July 1 - September 30, 2000 (FY00Q4)

Goals: Conduct experiments to evaluate second generation flexible tactile input device with human subjects. Design and fabricate masks for third generation flexible tactile input device. Fabricate third generation flexible tactile input device. Design experiments to evaluate third generation flexible tactile input device with human subjects. Conduct experiments to evaluate third generation flexible tactile input device with human subjects (SYT). Develop a methodology for inserting the flexible tactile input device into the ISL.

Insert flexible tactile device into the ISL. Design and test an experiment that evaluates the ability of tactile commands to be detected under varying levels of a visual or auditory workload. Conduct an experiment that tests the ability of tactile commands to be detected under varying levels of a visual or auditory workload.

Progress: Further electrical/mechanical testing was done on the completed second-generation flexible tactile devices. The basic design of the device is an array of pressure sensitive switches that connects a voltage tap to a resistor voltage divider network. Separate arrays are included for X and Y position allowing the voltage sensed by the tap to indicate the complete position of the closed switches. Testing this quarter focussed on preparations for testing of the larger arrays of switches and also for some quantitative testing of the pressure required to close the switches. These results will be used to evaluate whether any changes will be needed for the third fabrication run. A special probe assembly is being constructed to provide the force information. The assembly will utilize a spring located between the probe tip and the micromanipulator to convert the quantitative vertical displacement of the manipulator in to a force or pressure on the switch. Two designs are being pursued. In one the probe is supported by the spring alone, and the resulting requirement of a stiffer spring for supporting the probe mass requires the manipulator to have a precision measurement of the vertical displacement. The second design uses a pivoting arm to hold the probe that is balanced for gravitational forces, with a weaker spring used to provide the force and hence a less stringent requirement on the vertical displacement. Additional characterization of the devices will be done in the next quarter including some experiments with human subjects. Because of the delays in completing the fabrication of the devices, and the subsequent delays in their testing, development of the methodology of inserting the devices into the ISL has not been completed due to the lack of the results. The design and fabrication of a third set of masks for the final device generation was also delayed until sufficient data can be taken to determine the optimum device size and style from the options fabricated in this run. There was some delay in this task due to the final distribution of funds to MCNC of $64,549 not arriving until September 27, 2000.

The older version of the tactile actuator hardware located at NCA&T was repaired and returned. The modified version of the hardware at Sytronics was worked on several times to remove some operational bugs that appear to be associated with differences between the compliers used to generate the controlling software. The modified hardware allows its operation with a range of power supply voltages and outputting a range of voltages to generate different stimulation levels.

A change in the research staff at NCA&T involved in this task occurred in the third quarter, and the effort this quarter was spent reviewing past results and determining new directions. The re-furbished tactilator was received from MCNC in the last half of September. After reviewing the past studies involving the tactilator at NCA&T, it has been decided that an experiment more relevant to Army needs is necessary. In particular, past studies have only looked at the possibility of attaching meanings to various body locations, that when stimulated might readily evoke the intended response by the subject. A more fundamental, and perhaps more relevant question should be addressed should be first. Namely, can tactile signals be reliably detected under varying levels of visual and auditory workload? Further, if the detection is reliable, how much information content can be reliably passed through the tactile channel? It is possible that, at lower levels of workload, complex meanings/codes can be transmitted, but at higher levels of workload, only very basic commands can be understood. An experiment is currently being designed to examine these questions.

The basic design is a 2x2x3 factorial, including two difficulty levels of a visual search task (low/high), two difficulty levels of an auditory search task (low/high), and three levels of tactile information content (detection, simple commands, complex commands). Subjects will be required to monitor a visual scene and detect a signal from background noise. Likewise, the subject must concurrently monitor the auditory channel to detect a signal fom noise. During the same trial, at random time intervals, a tactile signal will be sent to the subject. When the signal is detected, the subject must verify the detection of the signal, and its associated meaning (if applicable in that test condition). Data will be analyzed based upon signal detection theory. The visual and auditory search tasks have not been concretely identified at the time of this writing, but these will be decided upon in the next week after talking to ARL and other consortia scientists.

A working version of the Tactilator with variable intensity has been received by SYTRONICS this quarter and software development to complete the tactile experiment is underway. We expect to conduct the experiment in the next quarter. Dan Mountjoy has taken NCA&T's effort to do tactile research and SYTRONICS will cooperate with him to facilitate their data collection.

Additionally, conversations are underway with Rockwell-Collins to conduct some joint experiments using tactile stimulation to direct soldiers. Some supplemental funding may be possible and SYTRONICS will pursue the opportunity.