Virtual Reality Tactile System For Access To Graphics

STATIC DISPLAY

It is widely recognized that blind people have a problem accessing any kind of information in graphical form. In the usual approach , tactile graphic "hard-copy" images are produced using swell paper or molded plastic. One advantage of this system is that there i s no cost associated with actually reading the image; the user merely scans with a finger over the surface. But there are a number of disadvantages:

• the images are static - they cannot be changed once produced;
• they are low resolution - with limited information content;
• labeling by Braille takes a lot of space;
• the choice of information is not selectable at the time of use;
• the images are not scaleable;
• they require special, expensive equipment to be produced;
• each image has a production cost per copy;
• the image is bulky, and unwieldy to carry around;
• an image, once -produced cannot be easily marked or annotated.

The information content of such hard-copy can be boosted by having audio output of text annotations stored in a computer, and mount ing the image on a touch-sensitive tablet. This is the ingenious approach of the system known as Nomad [Parkes 1988]. However other disadvantages remain, and there is now a cost associated with the reading system - the tablet and computer.

DYNAMIC DISPLAY

A dynamic display is one generated at the time the user wants to access the information - by direct output of the computer - where t he medium is re-used or refreshed for the display of successive images. For text display, the medium is typically an array of cells, reproducing a Braille pattern. The display corresponds to a window on the visual display - typically one line of text on the screen.

A dynamic display should not have any of the disadvantages of a purely hard copy image, as listed above. But the cost of dynamic display systems has been a problem, especially for a system with reasonable resolution. The cost is necessarily very high if a whole image is to be reproduced in the chosen medium. For ultimate resolution one would need a pin (or other moveable element) for each pixel of the graphic image.

VIRTUAL REALITY DISPLAY

In dynamic Braille, only a small part of the screen is displayed at any moment. Similarly tactile graphic systems have been propose d that only display limited tactile information at any moment, but they present this information directly to be felt by the user's h and. The "reality" of the graphics is in the computer, and the system tries to simulate this reality, hence the term virtual reality. As the user's hand moves over the simulated surface, a corresponding window moves over the image on the screen or stored in the computer. What the user feels is directly related to what is in that window.

There have been three approaches tried thus far. In the first approach, that of the Optacon, there is an array of a large number of small pins, each pin corresponding to a pixel in the window. The pins act over the surface of a finger-tip, while a hand-held camera is moved over the graphic image, and sees an area of the image, which is the window. The user needs to have a sensitive finger and a lot of training to detect shapes under the finger. The solution is expensive because of the number of pins and precision engineering involved, each pin being operated by a separate piezo-electric device.

A second approach has been to use a virtual reality display with a tactile glove incorporating position sensors and actuators. This has also proved very expensive.

In a third and recent approach, that of the device known as FeelMouse, the button on a mouse has been modified with tactile feedback , so that the user can feel a force related to what is at a point on the screen. This is a much cheaper solution, but provides only limited sensory input to the user - that is, the channel for information flow to the user is very narrow, and the user would require a long time to build up a mental image of a large or complex graphic image.

THE PROPOSED SYSTEM

The proposed system is a virtual reality display in which tactile information is presented by a small number of pins over the surface of the hand, for example the palm of the hand. A prototype of the hardware has been built using an array of seven pins: six in a hexagon around a central pin, and over 1 cm apart. The pins are operated by relays and can be easily felt even by an elderly person. The hardware is intrinsically cheap and robust. The pins are far enough apart to be easily distinguishable.

The array is mounted on a pointing device, such as a mouse. As the mouse is moved, the window is correspondingly moved over the virtual surface representing the graphics.

The proposed method of finding a particular object and its shape is as follows. Consider first a point object such as bus stop. While the window is not over the object, the pin or pair of pins closest to the object are activated periodically, with a period proportional to distance. The user can then move the window toward the object, and the frequency of activation increases as the object i s approached. When the window is directly over the point, the central pin is activated. The user can request a spoken description of the object.

Next consider a line object such as the center line of a pavement along a street. While the window is not over the object, the pin or pair of pins closest to the object are activated periodically. When the line is reached the pins over the line are activated. The user can then follow the line. While exactly over the line, the central pin is operated.

Now consider objects that have an area (not just points lines). The same procedure is followed to find the edge of the object. However if the window is moved inside the area, the central pin is continuously activated, and the pins nearest to the nearest edge are periodically activated.

Of course if a large number of different kinds of objects are to be displayed in the same image, the user will be confused. But the user can change the image at will, selecting different objects to be displayed, and gradually building a composite mental image of the whole graphics construction, zooming into detail when necessary.

OBTAINING IMAGE DATA

The images for tactile display can be easily obtained for mathematical functions or from computer graphics applications. Simple lin e drawings can be scanned in and displayed. However obtaining appropriate image data for maps presents a problem.

No map presentation device, however ingenious, has any value without appropriate maps to display. Of most interest to blind people are maps at a scale that can show pavements and routes to walk from point A to B. Map data with such detail is expensive. There is also a problem of obtaining maps in a suitable vector format, with objects and object descriptions. Ideally the data would be obta ined freely from the Web. Unfortunately there is no generally and internationally accepted vector format for publication of maps on the Web. However a promising emerging source of data employs VRML, which is suitable for two or three dimensional modeling, and i t is possible to project three dimensions onto two to obtain a map.

CONCLUSIONS

The virtual reality system can claim the following advantages over static systems:

• the images are dynamic - they can be changed at will;
• they are as high a resolution as required - with unlimited information content;
• no image space is taken by labeling, as the labels are spoken;
• the choice of information is selectable at the time of use;
• the images are infinitely scaleable;
• there is no expense to reproduce images;
• there is no hard-copy production cost;
• there is no hard-copy to carry around;
• an image can be easily marked or annotated.

A question still remains over speed of use. We still need to experiment to find how mental images can be most quickly built up by users of this type of display. But the system seems open to almost unlimited possibilities for the presentation of 2-D information.

REFERENCE

[Parkes, 1988] "Nomad - an audio-tactile tool for the acquisition, use and management of spatially distributed information by p artially sighted and blind people", Don Parkes, in Proceedings of the 2nd International Conference on Maps and Graphics for Visuall y Disabled People, editors Tatham and Dodds, pp24-29.