We would like to advise you to read the glossary first, as you may encounter
some difficulties considering medical terminology.
This presentation is broken into the following sections:
The cerebral cortex, the outermost layer of the cerebral hemispheres of the brain, responsible for all forms of conscious experience.
A multi-layered sheet of nerve cells at the back of each eye which converts light into electrical signals that are transmitted to the brain through the optic nerves and tracts. Below is a picture of the anatomy of the retina showing layers, vessels and nerves.
Image courtesy of CIBAVision
One of the 125 million nerve cells in the retina of each eye that emit electrical signals when activated by light of a particular wavelength. There are two types of photoreceptor cells in vertebrates: rods, used in low light, and cones, which respond in brighter light and to color.
Image courtesy of Howard Hughes Medical Institute
The intricate layers and connections of nerve cells in the retina were drawn by the famed Spanish anatomist Santiago Ram?n y Cajal around 1900. Rod (a) and cone (b) cells are at the top. Optic nerve fibers leading to the brain may be seen at bottom right.
Is a new display technology that scans modulated low energy laser light directly onto the viewer�s retina to create a perception of a virtual image.
As shown in the above figure the reflected rays of the pencil pass through
its lens. The inverted image of the pencil is directly projected onto the
retina. The rod and cone cells of the retina are responsible of tranmitting
signals to the brain through the optical nerve.
Another important factor in seeing is the stereo view capability. The images presented to the human vision must provide those characteristics that make them appear stereoscopic. That way, the viewer can optically calculate the distance of far seen objects. "The dumber the animal, the smarter its retina," observes Denis Baylor of Stanford Medical School.
Below is a graphical explanation of how the human vision generates stereoscopic images.
Image courtesy of Howard Hughes Medical Institute
The two eyes provide slightly different views of the same scene.
Information from the left visual field goes to the right side of
the retina in both eyes. At the optic
chiasm, half the nerve fibers from the left eye cross over to the right hemisphere and the rest stay uncrossed, so that all the information from the left visual field ends up in the right hemisphere. In this way, a given hemisphere gets information from the opposite half of the visual world�but each hemisphere gets input from both eyes. Thus, the future VRD product must have stereoscopic capabilities.
Image courtesy of Microvision
As shown above, the source image is directed into the drive electronics
device where the green, blue, and red light are modulated. Then each pixel
passes through the horizontal and vertical scanners (one moves from left
to right and the other from up to down) and produces a rasterized image
onto the retina. The retina has no persistence like
phosphorus. Moreover, there is no flickering and the produced image is
very bright with very high resolution. The time each pixel is projected
onto the retina is very small (30-40ns). Furthermore, these devices consume
very little amount of energy. They also provide a wider field of view.
A very sophisticated representation of how the beam is projected through the pupil can be found in the HIT Lab of Washington University (VRD animations).
The drive electronics control the acousto-optic modulators that encode the image data into the pulse stream. The color combiner multiplexes the individually-modulated red, green, and blue beams to produce a serial stream of pixels, which is launched into a singlemode optical fiber to propagate to the scanner assembly. The drive electronics receive and process an incoming video signal, provide image compensation, and control image display. For VGA projection, the electronics process over 18 Mpix/s. The virtual retinal display is capable of providing UXGA resolution of 1600 x 1200 or 115 Mpix/s.
The light source module contains laser light sources, acousto-optic modulators to create the pulse stream, and a color combiner that multiplexes the pulse streams. To provide sufficient brightness, full-color displays suitable for outdoor, daylight applications incorporate red diode lasers (635nm), green solid-state lasers
(532 nm), and blue solid-state or argon gas lasers (450-470 nm range). Systems designed for indoor use can incorporate LEDs; red, blue, and green devices currently under development for such systems are being tested. Generally, the energy levels are on the order of nanowatts to milliwatts, depending on display requirements. The levels of light involved are well within laser safety standards for viewing, as confirmed by analysis.
The scanner assembly contains two scanning mirrors. One 24 mm x 6 mm x 6 mm scanning mirror sweeps the beam horizontally at a high frequency, typically between 15.75 kHz and 18.9 kHz for 60-Hz non-interlaced refresh rates. This corresponds to one-half the VESA monitor-timing standard since the retinal scanning display can process and display pixels bidirectionally. A second scanning mirror sweeps the beam of laser light vertically at 60 Hz to complete the raster image.
Microvision is currently developing a microelectromechanical system (MEMS-) based full image scanner capable of bi-directional scanning. Unlike MEMS-based spatial light modulators used for projection applications, the millimeter-sized mirror in the virtual retinal display will provide the scan angles and rates necessary to provide the full raster image. The need to fabricate only one mirror significantly reduces the development effort for higher resolutions and associated yield problems. It also eliminates the potential for "dead" pixels due to inoperative mirror elements.
Nominally the entire image would be contained in an area of 2 mm2. The exit-pupil expander (not shown in Figure 2) is an optical device that increases the natural output angle of the image and enlarges it up to 18 mm on a side for ease of viewing. The raster image created by the horizontal and vertical scanners passes through the pupil expander and on to the viewer optics.
The viewer optics relay the scanned raster image to the oculars worn by the user. The optical system varies according to the application. In the case of military applications such as helmet mounted or head mounted display optics, the system incorporates glass and or plastic components; for medical applications such as image-guided surgery, head-mounted plastic optics are used. In industrial or personal displays, the optics might be a simple plastic lens.
|Resolution||The VRD produces very high resolution images. Its resolution is limited only by diffraction and optical aberrations in the light source and not by how small one can make an individual pixel element in a large array of pixel elements.|
|Contrast Ratio||The brightness of the VRD can be increased to very high levels or decreased to minimal levels. As a result, its contrast ratio is inherently high and far greater than that of standard flat panel displays or even conventional CRT monitors.|
|Luminance||Conventional electronic displays do not emit (or transmit) substantial amounts of light energy. As a result, they are primarily used in controlled lighting environments, and it is difficult to see them under bright ambient light conditions such as exist outdoors. In a see-through mode, the VRD can be controlled to allow the user to see an image that matches the brightness levels of the ambient light conditions.|
|Color Range||CRTs and miniature FPDs are able to reflect only a portion of the total palette of colors visible to the human eye, and are limited in the degree of saturation they can achieve. Because red, green and blue light sources used in VRD technology emit highly saturated, pure color, the VRD can produce a range of possible colors and color fidelity superior to any other electronic display technology.|
|Power Consumption||Because they do not efficiently convert electrical energy into light energy, both back-lit FPDs and CRTs draw substantial power and are among the biggest battery consumers in portable devices that use them. As a result, their brightness is relatively low, because most of their energy input is wasted. VRD technology, by contrast, conveys virtually all of its generated light onto the retina, allowing bright displays with minimum power requirements.|
|VRD technology is capable of improving performance more rapidly and with significantly lower research and development and tooling requirements than competing technologies.|
|Range of Application||Because the VRD delivers a truly general-purpose solution, without performance compromises or tradeoffs, it is appropriate for a broader range of applications than competing display technologies. This means that it can potentially achieve greater economies of scale necessary to drive costs downward and increase its appeal versus other display solutions.|
|Cost||Current flat panel displays use exotic and costly production techniques involving complex assemblies and critical tolerances. In contrast, the basic design of the VRD consists of subsystems that are very simple in their design and largely make use of established optical and electronic technologies. Investment in specialized manufacturing equipment is not required. Ultimately, VRD devices will be mass-produced at low cost.|
Image courtesy of HIT Lab, Washington University
The goals of the IVRD project are:
|Image courtesy of HIT Lab, Washington University||Image courtesy of HIT Lab, Washington University|
Prototype Characteristics: 640x480
50 degrees f-o-v
The emulator system requires a scanned light sources of the emulator to be modulated by passing the light through a 35mm slide. This provides us with the capability to vary the spot size at the retina, to vary the spacing between
horizontal lines, to vary the spacing between pixels etc.
More at HIT Lab
Head Mounted Displays (HMDs)
"Helicopter pilots require information to support time-critical (and often life-and-death) decisions," Swanson said. "If that information is presented in a graphical and intuitive fashion, it reduces the pilot's workload and can enhance visibility in degraded conditions. A helmet-mounted display capable of presenting full-color graphical information in both day and night flight operations has been the missing link to creating an effective pilot-data interface. That ultimately could save both lives and money."
Rick Rutkowski, Microvision's president and chief executive officer, said, "The Army has a powerful vision: the ability to overlay flight reference data, sensor imagery and weapons symbology on [images from] the outside world." Such a versatile display capability is expected to provide a significant performance boost to both aircraft and pilot. "When you can also enable a pilot to see the normally invisible 'bloom' of a radar signature, or to project a 'pathway in the sky' in front of him, and to superimpose wireframe or 3-D imagery onto the terrain, it becomes even more powerful," he said.
Microvision delivered a monochrome green binocular HMD system to the Army with a total field of view of 52� horizontal by 30� vertical with 1716 horizontal pixels by 960 display lines, and 1470 ft-Lm displayed raster luminance at the eye (see Figure 3). The system weighs approximately 2 lb, including two display engines (one for each eye), the helmet-mounting structure, and the optics to provide the large display format. The system will project the images in a "see through" fashion �the pilot will still be able to see the background scene, but has the option of focusing in on the presented information.
According to Rutkowski, the Army's vision of the
virtual cockpit also includes a "what you see depends on where you look"
concept. As the pilot looks up and out of the cockpit, various types of
targeting, navigational or terrain overlays would appear. When pilots look
in a downward direction, they may see "virtual" instruments projected onto
the eye that literally replace many of the existing dials and multifunction
displays that are in cockpits today. Microvision has modularized the system
designed to permit delivery of a full-color 1280 x 1024-pixel biocular
head-mounted system three months later, as well as an HDTV (1920 x 1080
pixels) HMD to the Air Force by the year 2000 (see Figure 4).
Wearable "augmented reality" displays Incorporated into eyeglasses, goggles or helmets, VRD technology will display an image that doesn't block the user's view but will instead superimpose a high-contrast monochromatic or color image on top of it. This ability can enhance the safety, precision and productivity of professionals performing complex tasks.
Wearable three-dimensional / interactive displays VRD technology can be incorporated into eyeglasses, goggles or helmets to create a stereoscopic, 3-d effect. These compact, high-resolution displays can further enhance the visual realism of the interactive experience to make the simulated environment more engaging.
Image courtesy of HIT Lab, University of Washington
Hand-held two-dimensional displays VRD technology will be integrated into cellular phones and pagers, allowing users to tap into business networks or the Internet to view e-mail, web pages, faxes and files as if on a full-size desktop monitor. This ability affords greater convenience and utility to users of these devices.
Image courtesy of Microvision
Medical applications (as referenced by Microvision Inc)
It has been estimated by the Community Services for the Blind and Partially Sighted that 2% of the population in the United States have low vision (20/400 or worse, legal blindness being 20/200). People with low vision are not completely blind, but many are unable to hold a drivers license and have a great deal of difficulty reading or watching television. Possible causes include damage to the retina - central or peripheral field degradation - and damage to the optical pathway - corneal damage/malformation or lens damage. The HIT Lab develops these VRD that can provide more visible images to people with low vision. Microvision capped 1998 by signing a contract with the Wallace-Kettering Neuroscience Institute (Dayton, Ohio) to collaborate on headgear for neurosurgery.
In its test screenings, "Patients without retinal damage saw moving particles throughout their field of vision, but patients with areas of retinal damage were actually able to paradoxically see their own blind areas and localize them," a Microvision spokesman said.
"Using a pen-based computer program, patients were asked to trace the borders of any visual disturbances," he said, "in effect creating a digital map of their own retina, which accurately detailed the locations of damaged tissue. The images were then compared with the 'gold standard' � photographs taken from a fundus camera � with outstanding results." A fundus camera captures images of the internal surfaces of hollow organs like the eye.
The VRD-based workstation can "screen for retinal damage in asymptomatic patients within virtually all of the visually significant retina," William Freeman, professor of ophthalmology and administrative director of the retina service at Shiley, reported at the meeting. "This rapid, non-invasive screening test has the potential to be an excellent addition to the clinical tool set of primary-care health providers to rapidly screen for retinal damage and diseases. It will also allow for community screening of underserved populations to detect vision loss." In the past year, a Shiley spokesman said, the workstation screened 58 patients with a variety of retinal diseases, including AIDS-related CMV retinitis, diabetic retinopathy, ocular melanoma, blood-vessel disorders, retinal detachment and macular degeneration.
Image courtesy of Microvision
The approach can also be adapted to image projection systems. The applications for VRD technology are varied�HUDs, color projections systems for entertainment or flight training simulators, etc. A key area for continued development is an image display system that can augment and enhance a person's task performance.