Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.
A63F 13/213 - Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
A63F 13/211 - Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
A63F 13/25 - Output arrangements for video game devices
A63F 13/428 - Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
A63F 13/65 - Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.
A63F 13/213 - Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
A63F 13/211 - Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
A63F 13/25 - Output arrangements for video game devices
A63F 13/65 - Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
The present disclosure relates to display systems and, more particularly, to augmented reality display systems. In one aspect, an adaptive lens assembly includes a lens stack configured to exert polarization-dependent optical power to linearly polarized light. The lens stack includes a birefringent lens and an isotropic lens contacting each other to form a conformal interface therebetween. The adaptive lens assembly is configured to be selectively switched between a plurality of states having different optical powers.
A display system aligns the location of its exit pupil with the location of a viewer's pupil by changing the location of the portion of a light source that outputs light. The light source may include an array of pixels that output light, thereby allowing an image to be displayed on the light source. The display system includes a camera that captures images of the eye and negatives of the images are displayed by the light source. In the negative image, the dark pupil of the eye is a bright spot which, when displayed by the light source, defines the exit pupil of the display system. The location of the pupil of the eye may be tracked by capturing the images of the eye, and the location of the exit pupil of the display system may be adjusted by displaying negatives of the captured images using the light source.
Examples of systems and methods for matching a base mesh to a target mesh for a virtual avatar or object are disclosed. The systems and methods may be configured to automatically match a base mesh of an animation rig to a target mesh, which may represent a particular pose of the virtual avatar or object. Base meshes may be obtained by manipulating an avatar or object into a particular pose, while target meshes may be obtain by scanning, photographing, or otherwise obtaining information about a person or object in the particular pose. The systems and methods may automatically match a base mesh to a target mesh using rigid transformations in regions of higher error and non-rigid deformations in regions of lower error.
Methods and systems for aligning head scans of a subject for a virtual avatar can be based on locating eyes of the subject in the scans. After one or more eyeball models are fitted to reference candidate points of a sclera of each eyeball of the subject in a reference head scan, an additional reference point can be inferred from the eyeball models. The eyeball models can be fitted to candidate points of the sclera of each eyeball of the subject in another head scan and an additional point can be inferred from the fitted eyeball models. An affine transformation can be determined between the head scans based on the eyeball models fitted to the candidate points in the reference head scan and the other head scan and the additional points inferred. The methods and systems can be used for rigging or animating the virtual avatar.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G09G 5/377 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of individual graphic patterns using a bit-mapped memory - Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
Described are improved systems and methods for navigation and manipulation of interactable objects in a 3D mixed reality environment. Improved systems and methods are provided to implement physical manipulation for creation and placement of interactable objects, such as browser windows and wall hangings. A method includes receiving data indicating a selection of an interactable object contained within a first prism at the start of a user interaction. The method also includes receiving data indicating an end of the user interaction with the interactable object. The method further includes receiving data indicating a physical movement of the user corresponding to removing the interactable object from the first prism between the start and the end of the user interaction. Moreover, the method includes creating a second prism to contain the data associated with the interactable object at the end of the user interaction with the interactable object.
A method is disclosed, the method comprising the steps of identifying a first real object in a mixed reality environment, the mixed reality environment having a user; identifying a second real object in the mixed reality environment; generating, in the mixed reality environment, a first virtual object corresponding to the second real object; identifying, in the mixed reality environment, a collision between the first real object and the first virtual object; determining a first attribute associated with the collision; determining, based on the first attribute, a first audio signal corresponding to the collision; and presenting to the user, via one or more speakers, the first audio signal.
A method of presenting audio signals in a mixed reality environment is disclosed, the method comprising the steps of identifying a first ear listener position in the mixed reality environment; identifying a second ear listener position in the mixed reality environment; identifying a first virtual sound source in the mixed reality environment; identifying a first object in the mixed reality environment; determining a first audio signal in the mixed reality environment, wherein the first audio signal originates at the first virtual sound source and intersects the first ear listener position; determining a second audio signal in the mixed reality environment, wherein the second audio signal originates at the first virtual sound source, intersects the first object, and intersects the second ear listener position; determining a third audio signal based on the second audio signal and the first object; presenting, via a first speaker to a first ear of a user, the first audio signal; and presenting, via a second speaker to a second ear of the user, the third audio signal.
A method of presenting an audio signal to a user of a mixed reality environment is disclosed, the method comprising the steps of detecting a first audio signal in the mixed reality environment, where the first audio signal is a real audio signal; identifying a virtual object intersected by the first audio signal in the mixed reality environment; identifying a listener coordinate associated with the user; determining, using the virtual object and the listener coordinate, a transfer function; applying the transfer function to the first audio signal to produce a second audio signal; and presenting, to the user, the second audio signal.
Disclosed is an improved systems and method for navigation and manipulation of browser windows in a 3D mixed reality environment. An improved approach is provided to view a user's windows, regardless of the current location for the user relative to one or more previously-opened windows. A method for displaying windows in a computing environment includes receiving an instruction to select multiple open windows. The method also includes retrieving information for the multiple open windows, where the multiple open windows are associated with different physical locations. The method further includes displaying a representation of the multiple open windows in a single user interface. Moreover, the method includes upon receiving a selection of a selected window of the multiple open windows, loading the selected window into a foreground of a field of view for a user.
Disclosed is a a method of localizing a user operating a plurality of sensing components, preferably in an augmented or mixed reality environment, the method comprising transmitting pose data from a fixed control and processing module and receiving the pose data at a first sensing component, the pose data is then transformed into a first component relative pose in a coordinate frame based on the control and processing module. A display unit in communication with the first sensing component is updated with the transformed first component relative pose to render virtual content with improved environmental awareness.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G01S 1/70 - Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmitters; Receivers co-operating therewith using electromagnetic waves other than radio waves
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
An image display system can include a plurality of light sources configured to emit uncollimated light, and an eyepiece waveguide having an input port configured to receive beams of light at differing angles. The image display system also includes a scanning mirror having a surface with positive optical power configured to receive light emitted by the plurality of light sources. The surface with positive optical power is configured to collimate light emitted by the plurality of light sources to form a plurality of collimated light beams and direct the plurality of collimated light beams to the input port.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
G09G 3/02 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes by tracing or scanning a light beam on a screen
14.
ECLIPSE CURSOR FOR VIRTUAL CONTENT IN MIXED REALITY DISPLAYS
Systems and methods for displaying a cursor and a focus indicator associated with real or virtual objects in a virtual, augmented, or mixed reality environment by a wearable display device are disclosed. The system can determine a spatial relationship between a user-movable cursor and a target object within the environment. The system may render a focus indicator (e.g., a halo, shading, or highlighting) around or adjacent objects that are near the cursor. When the cursor overlaps with a target object, the system can render the object in front of the cursor (or not render the cursor at all), so the object is not occluded by the cursor. The object can be rendered closer to the user than the cursor. A group of virtual objects can be scrolled, and a virtual control panel can be displayed indicating objects that are upcoming in the scroll.
A display system can include a head-mounted display configured to project light to an eye of a user to display virtual image content at different amounts of divergence and collimation. The display system can include an inward-facing imaging system images the user's eye and processing electronics that are in communication with the inward-facing imaging system and that are configured to obtain an estimate of a center of rotation of the user's eye. The display system may render virtual image content with a render camera positioned at the determined position of the center of rotation of said eye.
A wearable device may include a head-mounted display (HMD) for rendering a three- dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment vary for different users and may change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.
The present disclosure relates to display systems and, more particularly, to augmented reality display systems. In one aspect, a method of fabricating an optical element includes providing a substrate having a first refractive index and transparent in the visible spectrum. The method additionally includes forming on the substrate periodically repeating polymer structures. The method further includes exposing the substrate to a metal precursor followed by an oxidizing precursor. Exposing the substrate is performed under a pressure and at a temperature such that an inorganic material comprising the metal of the metal precursor is incorporated into the periodically repeating polymer structures, thereby forming a pattern of periodically repeating optical structures configured to diffract visible light. The optical structures have a second refractive index greater than the first refractive index.
Disclosed is an approach for managing and displaying virtual content in a mixed reality environment on a one-on-one basis independently by each application, each virtual content is rendered by its respective application into a bounded volume referred herein as a "Prism." Each Prism may have characteristics and properties that allow a universe application to manage and display the Prism in the mixed reality environment such that the universe application may manage the placement and display of the virtual content in the mixed reality environment by managing the Prism itself.
Disclosed is an approach for displaying 3D videos in a VR and/or AR system. The 3D videos may include 3D animated objects that escape from the display screen. The 3D videos may interact with objects within the VR and/or AR environment. The 3D video may be interactive with a user such that based on user input corresponding to decisions elected by the user at certain portions of the 3D video such that a different storyline and possibly a different conclusion may result for the 3D video. The 3D video may be a 3D icon displayed within a portal of a final 3D render world.
A method for calibrating a device having a first sensor and a second sensor. The method includes capturing sensor data using the first sensor and the second sensor. The device maintains a calibration profile including a translation parameter and a rotation parameter to model a spatial relationship between the first sensor and the second sensor. The method also includes determining a first calibration level associated with the calibration profile at a first time. The method further includes determining, based on the first calibration level, to perform a calibration process. The method further includes performing the calibration process at the first time by generating one or both of a calibrated translation parameter and a calibrated rotation parameter and replacing one or both of the translation parameter and the rotation parameter with one or both of the calibrated translation parameter and the calibrated rotation parameter.
G01B 21/04 - Measuring arrangements or details thereof, where the measuring technique is not covered by the other groups of this subclass, unspecified or not relevant for measuring length, width, or thickness by measuring coordinates of points
G06T 7/80 - Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
A visual perception device is described. The visual perception device has corrective optics for viewing virtual and real-world content. An insert for the corrective optics is attached using a magnetic set, pins and/or a nose piece. Interchangeable nose pieces allow for height adjustments to accommodate different users. The visual perception device has pliable components to absorb forces exerted on a nose piece and a protective barrier for limiting electric shock or ingress of dirt.
An eyepiece waveguide for an augmented reality display system may include an optically transmissive substrate, an input coupling grating (ICG) region, a multi-directional pupil expander (MPE) region, and an exit pupil expander (EPE) region. The ICG region may receive an input beam of light and couple the input beam into the substrate as a guided beam. The MPE region may include a plurality of diffractive features which exhibit periodicity along at least a first axis of periodicity and a second axis of periodicity. The MPE region may be positioned to receive the guided beam from the ICG region and to diffract it in a plurality of directions to create a plurality of diffracted beams. The EPE region may be positioned to receive one or more of the diffracted beams from the MPE region and to out couple them from the optically transmissive substrate as output beams.
A61B 3/10 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions
G02B 6/00 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
G02B 6/10 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
Examples of systems and methods for rendering an avatar in a mixed reality environment are disclosed. The systems and methods may be configured to automatically scale an avatar or to render an avatar based on a determined intention of a user, an interesting impulse, environmental stimuli, or user saccade points. The disclosed systems and methods may apply discomfort curves when rendering an avatar. The disclosed systems and methods may provide a more realistic interaction between a human user and an avatar.
An optical system for an augmented reality head mounted display eyepiece that is configured to deliver images to the eye wherein the optical system includes optics The optics are disposed so as to receive light output from the light source The optics further arranged with respect to a spatial light modulator such that the light received from the light source passes through the optics and illuminates the spatial light modulator The light illuminating the spatial light modulator is redirected back through the optics and is coupled into at least one waveguide through at least one in-coupling optical element. At least a portion of the coupled light is ejected from at least one waveguide by at least one out-coupling optical element and directed to the eye of the user.
An image sensor device includes a plurality of pixel cells arranged in a pixel array, a control circuit for controlling an exposure phase and a sampling phase of the image sensor device. Each of the plurality of pixel cells includes a photodiode, a storage diode, and a floating diffusion region. The control circuit is configured to activate the photodiode in a plurality of time windows to sense light reflected from a target as a result of a corresponding plurality of emitted light pulses, with a pre-determined delay time between each time window and a corresponding emitted light pulse. The photodiode can be activated using a plurality of bias voltage pulses or a plurality of global shutter signal pulses.
G01S 7/483 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group - Details of pulse systems
To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
An anti-reflective waveguide assembly comprising a waveguide substrate having a first index of refraction, a plurality of diffractive optical elements disposed upon a first surface of the waveguide and an anti-reflective coating disposed upon a second surface of the waveguide. The anti-reflective coating preferably increases absorption of light through a surface to which it is applied into the waveguide so that at least 97 percent of the light is transmitted. The anti-reflective coating is composed of four layers of material having different indices of refraction that the first index of refraction and an imaginary refractive index less than 1 x 10-3 but preferably less than 5 x 10-4.
Embodiments of optical scanners, optical projection systems, and methods of scanning optical waveguides and projecting images are described. The disclosed devices, systems and methods advantageously provide an improvement to the compactness, robustness, simplicity, and reliability of optical scanners and optical projection systems by implementing a thermally driven actuator for inducing oscillations of a cantilever within the optical scanners and optical projection systems. The stability and accuracy of optical scanners and optical projection systems are further enhanced using capacitive sensing, feedback, and phase correction techniques described herein.
Systems, devices, and methods for training a neural network and performing image interest point detection and description using the neural network. The neural network may include an interest point detector subnetwork and a descriptor subnetwork. An optical device may include at least one camera for capturing a first image and a second image. A first set of interest points and a first descriptor may be calculated using the neural network based on the first image, and a second set of interest points and a second descriptor may be calculated using the neural network based on the second image. A homography between the first image and the second image may be determined based on the first and second sets of interest points and the first and second descriptors. The optical device may adjust virtual image light being projected onto an eyepiece based on the homography.
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
G06V 10/774 - Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Methods and systems for meta-learning are described for automating learning of child tasks with a single neural network. The order in which tasks are learned by the neural network can affect performance of the network, and the meta-learning approach can use a task-level curriculum for multi-task training. The task-level curriculum can be learned by monitoring a trajectory of loss functions during training. The meta-learning approach can learn to adapt task loss balancing weights in the course of training to get improved performance on multiple tasks on real world datasets. Advantageously, learning to dynamically balance weights among different task losses can lead to superior performance over the use of static weights determined by expensive random searches or heuristics. Embodiments of the meta-learning approach can be used for computer vision tasks or natural language processing tasks, and the trained neural networks can be used by augmented or virtual reality devices.
Multiple cameras are coupled to a camera rig. Multiple diffractive optical elements (DOEs) are coupled to a DOE prop. Each camera is positioned in an eye box of a DOE and takes an image when the cameras are positioned at a first position, a second position and a third position relative to the DOEs. Three images taken by a each camera at each one of the first position, the second position and third position. The images are transmitted to a processor coupled to the camera rig. For each image, the processor identifies data pairs, each data pair including pixel coordinates of an intensity peak in the image and virtual light source produced by the diffractive optical element that corresponds to the intensity peak, and determines extrinsic parameters of the cameras using the identified data pairs for each image.
G06T 7/80 - Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
H04N 13/239 - Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
A method of fabricating a diffractive structure with varying diffractive element depth includes providing a shadow mask having a first region with a first aperture dimension to aperture periodicity ratio and a second region with a second aperture dimension to aperture periodicity ratio less than the first aperture dimension to aperture periodicity ratio. The method also includes positioning the shadow mask adjacent a substrate. The substrate comprises an etch mask corresponding to the diffractive structure. The method further includes exposing the substrate to an etchant, etching the substrate to form diffractive elements adjacent the first region having a first depth, and etching the substrate to form diffractive elements adjacent the second region having a second depth less than the first depth.
An example system is used to mix components and dispense a mixture for forming a thiol-ene polymer article. The system includes a first reservoir containing a first component of the thiol-ene polymer including a first polymerizable compound, and a second reservoir containing a second component of the thiol-ene polymer including a second polymerizable compound. The system also includes a mixing vessel having a mixing chamber, a delivery manifold providing a conduit for fluid from the first and second reservoirs to the mixing vessel, and a dispensing manifold providing a conduit for fluid from the mixing vessel. The system also includes a control module programmed to control the operation of the system.
Systems and methods for training a multitask network is disclosed. In one aspect, training the multitask network includes determining a gradient norm of a single-task loss adjusted by a task weight for each task, with respect to network weights of the multitask network, and a relative training rate for the task based on the single-task loss for the task. Subsequently, a gradient loss function, comprising a difference between (1) the determined gradient norm for each task and (2) a corresponding target gradient norm, can be determined. An updated task weight for the task can be determined and used in the next iteration of training the multitask network, using a gradient of the gradient loss function with respect to the task weight for the task.
35.
BROADBAND ADAPTIVE LENS ASSEMBLY FOR AUGMENTED REALITY DISPLAY
A display device comprises a waveguide configured to guide light in a lateral direction parallel to an output surface of the waveguide. The waveguide is further configured to outcouple the guided light through the output surface. The display device additionally comprises a broadband adaptive lens assembly configured to incouple and to diffract therethrough the outcoupled light from the waveguide. The broadband adaptive lens assembly comprises a first waveplate lens comprising a liquid crystal (LC) layer arranged such that the waveplate lens has birefringence (Dn) that varies in a radially outward direction from a central region of the first waveplate lens and configured to diffract the outcoupled light at a diffraction efficiency greater than 90% within a wavelength range including at least 450 nm to 630 nm. The broadband adaptive lens assembly is configured to be selectively switched between a plurality of states having different optical powers.
G02B 6/12 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type of the integrated circuit kind
A display device includes a waveguide assembly comprising a waveguide configured to outcouple light out of a major surface of the waveguide to form an image in the eyes of a user. An adaptive lens assembly has a major surface facing the output surface and a waveplate lens and a switchable waveplate assembly. The switchable waveplate assembly includes quarter-wave plates on opposing sides of a switchable liquid crystal layer, and electrodes on the quarter-wave plates in the volume between the quarter-wave plates. The electrodes can selectively establish an electric field and may serve as an alignment structure for molecules of the liquid crystal layer. Portions of the adaptive lens assembly may be manufactured by roll-to-roll processing in which a substrate roll is unwound, and alignment layers and liquid crystal layers are formed on the substrate as it moves towards a second roller, to be wound on that second roller.
G02F 1/295 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the position or the direction of light beams, i.e. deflection in an optical waveguide structure
Systems and methods for displaying a virtual reticle in an augmented or virtual reality environment by a wearable device are described. The environment can include real or virtual objects that may be interacted with by the user through a variety of poses, such as, e.g., head pose, eye pose or gaze, or body pose. The user may select objects by pointing the virtual reticle toward a target object by changing pose or gaze. The wearable device can recognize that an orientation of a user's head or eyes is outside of a range of acceptable or comfortable head or eye poses and accelerate the movement of the reticle away from a default position and toward a position in the direction of the user's head or eye movement, which can reduce the amount of movement by the user to align the reticle and target.
An example system for molding a photocurable material into a planar object includes a first mold structure having a first mold surface, a second mold structure having a second mold surface, and one or more protrusions disposed along at least one of the first mold surface or the second mold surface. During operation, the system is configured to position the first and second mold structures such that the first and second mold surfaces face each other with the one or more protrusions contacting the opposite mold surface, and a volume having a total thickness variation (TTV) of 500 nm or less is defined between the first and second mold surfaces. The system is further configured to receive the photocurable material in the volume, and direct radiation at the one or more wavelengths into the volume.
B29C 70/64 - Shaping composites, i.e. plastics material comprising reinforcements, fillers or preformed parts, e.g. inserts comprising fillers only the filler influencing the surface characteristics of the material, e.g. by concentrating near the surface or by incorporation into the surface by force
G02B 1/04 - Optical elements characterised by the material of which they are made; Optical coatings for optical elements made of organic materials, e.g. plastics
A method of presenting an audio signal to a user of a mixed reality environment is disclosed. According to examples of the method, an audio event associated with the mixed reality environment is detected. The audio event is associated with a first audio signal. A location of the user with respect to the mixed reality environment is determined. An acoustic region associated with the location of the user is identified. A first acoustic parameter associated with the first acoustic region is determined. A transfer function is determined using the first acoustic parameter. The transfer function is applied to the first audio signal to produce a second audio signal, which is then presented to the user.
An imprint lithography method of configuring an optical layer includes selecting one or more parameters of a nanolayer to be applied to a substrate for changing an effective refractive index of the substrate and imprinting the nanolayer on the substrate to change the effective refractive index of the substrate such that a relative amount of light transmittable through the substrate is changed by a selected amount.
G03F 7/00 - Photomechanical, e.g. photolithographic, production of textured or patterned surfaces, e.g. printed surfaces; Materials therefor, e.g. comprising photoresists; Apparatus specially adapted therefor
41.
AUGMENTED REALITY DISPLAY COMPRISING EYEPIECE HAVING A TRANSPARENT EMISSIVE DISPLAY
An augmented reality head mounted display system an eyepiece having a transparent emissive display. The eyepiece and transparent emissive display are positioned in an optical path of a user's eye in order to transmit light into the user's eye to form images. Due to the transparent nature of the display, the user can see an outside environment through the transparent emissive display. The transmissive emissive display comprising a plurality of emitters configured to emit light into the eye of the user. A first variable focus optical element is positioned between the transparent emissive display and the user's eye and is configured to modify emitted light to provide the appropriate amount of divergence for image information to be interpreted by the user as being on a particular depth plane. A second variable focus optical element is positioned between the transparent emissive display and the environment and is configured to offset effects of the first variable focus optical element on the view of the environment.
G02B 30/27 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer’s left and right eyes of the autostereoscopic type involving lenticular arrays
G02B 27/18 - Optical systems or apparatus not provided for by any of the groups , for optical projection, e.g. combination of mirror and condenser and objective
G09F 9/33 - Indicating arrangements for variable information in which the information is built-up on a support by selection or combination of individual elements in which the desired character or characters are formed by combining individual elements being semiconductor devices, e.g. diodes
G09G 5/377 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of individual graphic patterns using a bit-mapped memory - Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
42.
NEAR EYE 3D DISPLAY WITH SEPARATE PHASE AND AMPLITUDE MODULATORS
Augmented reality glasses include near eye displays the include sources of imagewise amplitude modulated light optical coupled to spatial phase modulators or active zone plate modulators and optically coupled to eye coupling optics. The sources of imagewise amplitude modulated light can include emissive 2D display panels or light sources coupled to imagewise amplitude modulators. The eye coupling optics can include volume holographic diffraction gratings.
An eyepiece for a head-mounted display includes one or more first waveguides arranged to receive light from a spatial light modulator at a first edge, guide at least some of the received light to a second edge opposite the first edge, and extract at least some of the light through a face of the one or more first waveguides between the first and second edges. The eyepiece also includes a second waveguide positioned to receive light exiting the one or more first waveguides at the second edge and guide the received light to one or more light absorbers.
G02B 26/00 - Optical devices or arrangements for the control of light using movable or deformable optical elements
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.
A head mounted display system can include a camera, at least one waveguide, at least one coupling optical element that is configured such that light is coupled into said waveguide and guided therein, and at least one out-coupling element. The at least one out-coupling element can be configured to couple light that is guided within said waveguide out of said waveguide and direct said light to said camera. The camera can be disposed in an optical path with respect to said at least one out-coupling optical element to receive at least a portion of the light that is coupled into said waveguide via the coupling element and guided therein and that is coupled out from said waveguide by said out-coupling coupling element such that images may be captured by said camera.
Disclosed is an approach for constructing a new frame using rendered content and non-rendered content from a previous perspective. Points of visible surfaces of a first set of objects from a first perspective are rendered. Both rendered content and non-rendered content from the first perspective are stored. The new frame from the second perspective is generated using the rendered content and the non-rendered content from the first perspective.
Disclosed herein are examples of a wearable display system capable of determining a user interface (UI) event with respect to a virtual UI device (e.g., a button) and a pointer (e.g., a finger or a stylus) using a neural network. The wearable display system can render a representation of the UI device onto an image of the pointer captured when the virtual UI device is shown to the user and the user uses the pointer to interact with the virtual UI device. The representation of the UI device can include concentric shapes (or shapes with similar or the same centers of gravity) of high contrast. The neural network can be trained using training images with representations of virtual UI devices and pointers.
Apparatus and methods for displaying an image by a rotating structure are provided. The rotating structure can comprise blades of a fan. The fan can be a cooling fan for an electronics device such as an augmented reality display. In some embodiments, the rotating structure comprises light sources that emit light to generate the image. The light sources can comprises light-field emitters. In other embodiments, the rotating structure is illuminated by an external (e.g., non-rotating) light source.
G09F 9/30 - Indicating arrangements for variable information in which the information is built-up on a support by selection or combination of individual elements in which the desired character or characters are formed by combining individual elements
49.
METHOD AND SYSTEM FOR INTEGRATION OF ELECTRONIC SENSORS WITH THERMAL COOLING SYSTEM
A system includes a computing device operatively coupled to one or more user interface components, and a thermal cooling system for cooling the computing device. The system further includes a proximity sensor positioned adjacent a thermal vent and configured to set a proximity flag in response to detecting an object in proximity to the thermal vent. The system further includes an orientation sensor coupled to the computing device and configured to set an orientation flag in response to detecting an orientation of the computing device being such that the thermal vent is facing downward. The system further includes a processor communicatively coupled to the proximity sensor, the orientation sensor, and the one or more user interface components. The processor is configured to provide an alert for output through the one or more user interface components in response to having the proximity flag set or the orientation flag set.
An imaging system includes a diffusing element configured to couple portions of a light beam back into a laser diode. The system includes a diode laser driven into a chaotic regime by a combination of a diffuser and a modulated drive current such that it emits light across a frequency spectrum having an envelope between 3 and 10 nanometers wide. The system further includes a diffusing element at least 0.1 mm to 0.5 mm away from the diode laser to couple portions of the light beam back into the laser diode. Another embodiment is directed to using the diffusing element to illuminate a flat panel display or a spatial light modulator.
Augmented reality devices and methods for computing a homography based on two images. One method may include receiving a first image based on a first camera pose and a second image based on a second camera pose, generating a first point cloud based on the first image and a second point cloud based on the second image, providing the first point cloud and the second point cloud to a neural network, and generating, by the neural network, the homography based on the first point cloud and the second point cloud. The neural network may be trained by generating a plurality of points, determining a 3D trajectory, sampling the 3D trajectory to obtain camera poses viewing the points, projecting the points onto 2D planes, comparing a generated homography using the projected points to the ground-truth homography and modifying the neural network based on the comparison.
A dynamically actuable diffractive optical element (DOE) includes a substrate and a diffraction grating disposed on a first region of a surface of the substrate. The DOE further includes a quantity of a fluid disposed on a second region of the surface of the substrate, a fluid displacer disposed adjacent the second region of the surface of the substrate, and a drive signal source configured to send an electric signal to the fluid displacer. The fluid displacer is configured to, upon receiving the electric signal in a first state, causing a portion of the quantity of the fluid to be displaced from the second region of the surface into grooves of the diffraction grating, and upon receiving the electric signal in a second state, causing the portion of the quantity of the fluid to retract from the grooves of the diffraction grating to the second region of the surface.
G01N 21/25 - Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
G02B 26/00 - Optical devices or arrangements for the control of light using movable or deformable optical elements
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
An augmented reality system includes at least one waveguide configured to receive and redirect light toward a user, and to allow ambient light to pass toward the user. A first adaptive lens assembly is positioned between the waveguide and the environment, a second adaptive lens assembly is positioned between the waveguide and the user, and at least one processor is operatively coupled to the first and second adaptive lens assemblies. Each lens assembly is selectively switchable between at least two different states in which the respective lens assembly is configured to impart at least two different optical powers to light passing therethrough, respectively. The processor is configured to cause the first and second adaptive lens assemblies to synchronously switch between different states in a maimer such that the first and second adaptive lens assemblies impart a substantially constant net optical power to ambient light from the environment.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
G02F 1/01 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour
54.
POWER SUPPLY ASSEMBLY WITH FAN ASSEMBLY FOR ELECTRONIC DEVICE
A fan assembly is disclosed. The fan assembly can include a first support frame. The fan assembly can comprise a shaft assembly having a first end coupled with the first support frame and a second end disposed away from the first end. A second support frame can be coupled with the first support frame and disposed at or over the second end of the shaft assembly. An impeller can have fan blades coupled with a hub, the hub being disposed over the shaft assembly for rotation between the first and second support frames about a longitudinal axis. Transverse loading on the shaft assembly can be controlled by the first and second support frames.
H05K 5/02 - Casings, cabinets or drawers for electric apparatus - Details
H01M 10/623 - Portable devices, e.g. mobile telephones, cameras or pacemakers
H01M 10/6563 - Gases with forced flow, e.g. by blowers
H01M 10/667 - Heat-exchange relationships between the cells and other systems, e.g. central heating systems or fuel cells the system being an electronic component, e.g. a CPU, an inverter or a capacitor
H01M 50/247 - Mountings; Secondary casings or frames; Racks, modules or packs; Suspension devices; Shock absorbers; Transport or carrying devices; Holders specially adapted for portable devices, e.g. mobile phones, computers, hand tools or pacemakers
H05K 7/20 - Modifications to facilitate cooling, ventilating, or heating
55.
MECHANICAL JOINT FOR USE IN FIBER OPTIC IMAGING SYSTEMS
Apparatus includes an optical fiber, an actuator, and a joint mechanically coupling the actuator to the optical fiber. The joint includes a neck extending along an axis. The optical fiber is threaded through an aperture extending along the axis through the neck. The optical fiber is attached to the joint at a surface of the neck facing the axis. The joint also includes a collar extending along the axis. The actuator is mechanically attached to the joint at an inner surface of the collar facing the axis. The joint also includes a flexural element extending radially from the neck to the collar. During operation, the joint couples a force from the actuator to the optical fiber to vary an orientation of a portion of the optical fiber extending from the neck with respect to the axis.
A61B 1/06 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
A61B 1/07 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements using light-conductive means, e.g. optical fibres
Systems and methods for eye tracking calibration in a wearable system are described. The wearable system can present three-dimensional (3D) virtual content and allow a user to interact with the 3D virtual content using eye gaze. During an eye tracking calibration, the wearable system can validate that a user is indeed looking at a calibration target while the eye tracking data is acquired. The validation may be performed based on data associated with the user's head pose and vestibulo-ocular reflex.
H04N 13/383 - Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
Disclosed herein are systems and methods for device authentication or pairing. In an aspect, a wearable display system comprises a display, an image capture device configured to capture images of a companion device, a computer-readable storage medium configured to store the images of the companion device, and a processor in communication with the image capture device and the storage medium. The processor can be programmed with executable instructions to receive a first image of a first optical pattern displayed by the companion device captured by the image capture device, wherein the first optical pattern is generated by the companion device based on first shared data, extract first data from the first optical pattern in the received first image, authenticate the companion device based on the first data extracted from the first optical pattern, and notify a user of the wearable display system that the companion device is authenticated.
G06Q 20/40 - Authorisation, e.g. identification of payer or payee, verification of customer or shop credentials; Review and approval of payers, e.g. check of credit lines or negative lists
G09G 5/377 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of individual graphic patterns using a bit-mapped memory - Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
Systems, apparatus, and methods for double-sided imprinting are provided. An example system includes first rollers for moving a first web including a first template having a first imprinting feature, second rollers for moving a second web including a second template having a second imprinting feature, dispensers for dispensing resist, a locating system for locating reference marks on the first and second webs for aligning the first and second templates, a light source for curing the resist, such that a cured first resist has a first imprinted feature corresponding to the first imprinting feature on one side of the substrate and a cured second resist has a second imprinted feature corresponding to the second imprinting feature on the other side of the substrate, and a moving system for feeding in the substrate between the first and second templates and unloading the double-imprinted substrate from the first and second webs.
B29C 33/42 - SHAPING OR JOINING OF PLASTICS; SHAPING OF MATERIAL IN A PLASTIC STATE, NOT OTHERWISE PROVIDED FOR; AFTER-TREATMENT OF THE SHAPED PRODUCTS, e.g. REPAIRING - Details thereof or accessories therefor characterised by the shape of the moulding surface, e.g. ribs or grooves
B29C 59/02 - Surface shaping, e.g. embossing; Apparatus therefor by mechanical means, e.g. pressing
B29C 59/16 - Surface shaping, e.g. embossing; Apparatus therefor by wave energy or particle radiation
G03F 7/00 - Photomechanical, e.g. photolithographic, production of textured or patterned surfaces, e.g. printed surfaces; Materials therefor, e.g. comprising photoresists; Apparatus specially adapted therefor
G03F 9/00 - Registration or positioning of originals, masks, frames, photographic sheets or textured or patterned surfaces, e.g. automatically
59.
KEYBOARDS FOR VIRTUAL, AUGMENTED, AND MIXED REALITY DISPLAY SYSTEMS
User interfaces for virtual reality, augmented reality, and mixed reality display systems are disclosed. The user interfaces may be virtual or physical keyboards. Techniques are described for displaying, configuring, and/or interacting with the user interfaces. In one example, a physical keyboard is provided in the physical environment of a user. An outward-facing imaging system images the environment and a hardware processor analyzes the image to recognize the keyboard. The hardware processor then determines a specification for the physical keyboard based on contextual information and dynamically configures functions of the physical keyboard based on the specification. The hardware processor then determines a rendering location of a virtual key label based on the specification and instructs a display system, which can present virtual content in the physical environment of the user, to render the virtual key label at the determined rendering location.
A virtual image generation system comprises a planar optical waveguide having opposing first and second faces, an in-coupling (IC) element configured for optically coupling a collimated light beam from an image projection assembly into the planar optical waveguide as an in-coupled light beam, a first orthogonal pupil expansion (OPE) element associated with the first face of the planar optical waveguide for splitting the in-coupled light beam into a first set of orthogonal light beamlets, a second orthogonal pupil expansion (OPE) element associated with the second face of the planar optical waveguide for splitting the in-coupled light beam into a second set of orthogonal light beamlets, and an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the first and second sets of orthogonal light beamlets into an array of out-coupled light beamlets that exit the planar optical waveguide.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
Systems and methods for matching content elements to surfaces in a spatially organized 3D environment. The method includes receiving content, identifying one or more elements in the content, determining one or more surfaces, matching the one or more elements to the one or more surfaces, and displaying the one or more elements as virtual content onto the one or more surfaces.
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06T 19/00 - Manipulating 3D models or images for computer graphics
H04N 13/10 - Processing, recording or transmission of stereoscopic or multi-view image signals
A light emitting user input device can include a touch sensitive portion configured to accept user input (e.g., from a user's thumb) and a light emitting portion configured to output a light pattern. The light pattern can be used to assist the user in interacting with the user input device. Examples include emulating a multi-degree-of-freedom controller, indicating scrolling or swiping actions, indicating presence of objects nearby the device, indicating receipt of notifications, assisting pairing the user input device with another device, or assisting calibrating the user input device. The light emitting user input device can be used to provide user input to a wearable device, such as, e.g., a head mounted display device.
Augmented reality headgear includes transparent displays that allow a user to simultaneously view the real world and virtual content positioned in the real world and further includes at least one source of coherent light and at least one sensor array for sensing, at a series of times, speckle patterns produced when the coherent light impinges environment surfaces. Circuitry is provided for sensing shifts in the speckle pattern and determining motion which caused the shift of the speckle pattern and adjusting the display of virtual objects displayed by the augmented reality headgear to compensate for the motion.
G02B 6/00 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings
G02B 26/00 - Optical devices or arrangements for the control of light using movable or deformable optical elements
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
Examples of wearable systems and methods can use multiple inputs (e.g., gesture, head pose, eye gaze, voice, and/or environmental factors (e.g., location)) to determine a command that should be executed and objects in the three-dimensional (3D) environment that should be operated on. The multiple inputs can also be used by the wearable system to permit a user to interact with text, such as, e.g., composing, selecting, or editing text.
In some embodiments, compositions and methods comprising reflective flowable materials, e.g., reflective liquids including reflective inks and/or liquid metals, are described. In some embodiments, a surface is contacted with a reflective flowable material, thereby forming a reflective layer on the surface. In some embodiments, the surface is a surface of a waveguide, for example a waveguide for a display device, and the flowable material coats surfaces of protrusions on the surface to form reflective diffractive optical elements. Some embodiments include a display device comprising a reflective layer of reflective flowable material.
G02B 1/10 - Optical coatings produced by application to, or surface treatment of, optical elements
G02B 6/10 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
A method is disclosed, the method comprising the steps of receiving, at a first time interval from a first sensor configured to output data indicative of a first position of an eye, first data; receiving, at a second time interval from a second sensor configured to output data indicative of a delta position of the eye, second data; determining, based on the first data, a first position of the eye; determining, based on the second data, a delta position of the eye; determining, using the first position of the eye and the delta position of the eye, a second absolute position of the eye; and in response to determining the second position of the eye, generating an output signal indicative of the second position of the eye.
Described herein are embodiments of fiber scanning systems and methods of scanning optical fibers. The disclosed systems and methods advantageously provide an improvement to the scanning range, the oscillation amplitude, and/or the maximum pointing angle for an optical fiber in a fiber scanning system by inducing a buckling of a portion of the optical fiber.
A61B 1/00 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
A61B 1/07 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements using light-conductive means, e.g. optical fibres
G02B 23/24 - Instruments for viewing the inside of hollow bodies, e.g. fibrescopes
G02B 23/26 - Instruments for viewing the inside of hollow bodies, e.g. fibrescopes using light guides
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
A head-worn sound reproduction device is provided in the form of left and right earphones, which can either be clipped to each ear or mounted on other headgear. The earphones deliver high fidelity audio to a user's eardrums from near-ear range, in a lightweight form factor that is fully "non-blocking" (allows coupling in and natural hearing of ambient sound). Each earphone has a woofer component that produces bass frequencies, and a tweeter component that produces treble frequencies. The woofer outputs the bass frequencies from a position close to the ear canal, while the tweeter outputs treble frequencies from a position that is either close to the ear canal or further away. In certain embodiments, the tweeter is significantly further from the ear canal than the woofer, leading to a more expansive perceived "sound stage", but still with a "pure" listening experience.
A method is disclosed, the method comprising the steps of receiving, from a first client application, first graphical data comprising a first node; receiving, from a second client application independent of the first client application, second graphical data comprising a second node; and generating a scenegraph, wherein the scenegraph describes a hierarchical relationship between the first node and the second node.
An augmented reality system includes left and right transparent eyepieces through which a user can view the real world and which serve to couple imagewise modulated light into the user's eyes in order to display virtual content components of a mixed reality experience. The system further includes left and right speakers and a facility for user spatial manipulation of virtual objects. The system produces spatialized audio that has a virtual sound source position fixed to one or more virtual objects that are spatially manipulated by the user. Thus the system provides more realistic visual and auditory presentation of virtual components of a mixed reality experience.
A system and method for operating a sensor which has at least two modes of operation. The sensor may be provided with a sequence of common operation steps which are included in both a first sequence of operation steps which define a first mode of operation and a second sequence of operation steps which define a second mode of operation. The sensor may also be provided with one or more dummy operation steps which relate to the difference between the first mode of operation and the second mode of operation. The sensor can be operated in the first mode of operation by causing it to execute at least the common operation steps and it can be operated in the second mode of operation by causing it to execute the common operation steps and at least one dummy operation step.
A waveguide display disposed in glasses includes a first pupil expander assembly operable to project a first image defined by a first field of view and a second pupil expander assembly disposed adjacent the first pupil expander assembly and operable to project a second image defined by a second field of view different from the first field of view.
In one aspect, an optical device comprises a plurality of waveguides formed over one another and having formed thereon respective diffraction gratings, wherein the respective diffraction gratings are configured to diffract visible light incident thereon into respective waveguides, such that visible light diffracted into the respective waveguides propagates therewithin. The respective diffraction gratings are configured to diffract the visible light into the respective waveguides within respective field of views (FOVs) with respect to layer normal directions of the respective waveguides. The respective FOVs are such that the plurality of waveguides are configured to diffract the visible light within a combined FOV that is continuous and greater than each of the respective FOVs.
Examples of light projector systems for directing input light from a light source to a spatial light modulator are provided. For example, an optical device is disclosed which includes a first surface having a diffractive optical element, a second surface normal to the first surface, and a third surface arranged at an angle to the second surface. The third surface may be a beam splitting surface that is reflective to light of a first state and transmissive to light of a second state. The diffractive optical element may receive an input beam made up of light having the first state, and convert the input beam into at least a first diffracted beam at a first diffraction angle such that the first diffracted beam is directed toward the third surface and is reflected by the third surface in a direction substantially parallel to the first surface.
G02F 1/29 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the position or the direction of light beams, i.e. deflection
An optical device can include a wedge-shaped light turning element having a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface, which is inclined with respect to the horizontal axis by a wedge angle. A light module, which includes a number of light emitters, can be configured to combine light from the emitters. A light input surface is between the first and the second surfaces and receives light emitted from the emitters. An end reflector is disposed on a side opposite the light input surface. The second surface may be inclined such that a height of the light input surface is less than a height of the side opposite the light input surface. The light coupled into the turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.
G02B 30/34 - Stereoscopes providing a stereoscopic pair of separated images corresponding to parallactically displaced views of the same object, e.g. 3D slide viewers
An eye tracking system includes a light scanning projector and an eyepiece optically coupled to the light scanning projector. The eye tracking system also includes an optical sensor and a processor coupled to the light scanning projector and the optical sensor.
A61B 3/113 - Objective types, i.e. instruments for examining the eyes independent of the patients perceptions or reactions for determining or recording eye movement
G02B 6/10 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
A fiber scanning projector includes a piezoelectric element and a scanning fiber mechanically coupled to the piezoelectric element. The fiber scanning projector also includes an optical assembly section operable to receive light from the scanning fiber. The optical assembly section includes a prism element, a collimating element coupled to the prism element at an interface, a quarter wave plate, and a polarizing beam splitter disposed at the interface.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
An optical device comprising may include a light turning element. The optical device can include a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface. The optical device may include a light module that includes a plurality of light emitters. The light module can be configured to combine light for the plurality of emitters. The optical device can further include a light input surface that is between the first and the second surfaces and is disposed with respect to the light module to receive light emitted from the plurality of emitters. The optical device may include an end reflector that is disposed on a side opposite the light input surface. The light coupled into the light turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.
G03B 21/00 - Projectors or projection-type viewers; Accessories therefor
F21V 9/00 - Elements for modifying spectral properties, polarisation or intensity of the light emitted, e.g. filters
G02F 1/00 - Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
G09G 3/24 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix using controlled light sources using incandescent filaments
79.
WEARABLE DISPLAY DEVICE UTILIZING A COMPOSITE FIELD OF VIEW
A wearable display device suitable for use in an augmented reality environment is disclosed. The wearable display device can include a projector configured to project light through diffractive optical elements that then distributed the light to multiple display regions. Each of the display regions can be arranged to project light out of the wearable display device towards an eye of a user. Since each of the display regions are positioned in different locations with respect to an eye of a user, the result is that each display region directs light in a different direction. In this way the apparent field of view for a user of the wearable display can be substantially increased.
An augmented reality (AR) device is described with a display system configured to adjust an apparent distance between a user of the AR device and virtual content presented by the AR device. The AR device includes a first tunable lens that changes shape in order to affect the position of the virtual content. Distortion of real-world content on account of the changes made to the first tunable lens is prevented by a second tunable lens that changes shape to stay substantially complementary to the optical configuration of the first tunable lens. In this way, the virtual content can be positioned at almost any distance relative to the user without degrading the view of the outside world or adding extensive bulk to the AR device. The augmented reality device can also include tunable lenses for expanding a field of view of the augmented reality device.
Examples of eye-imaging apparatus using diffractive optical elements are provided. For example, an optical device comprises a substrate having a proximal surface and a distal surface, a first coupling optical element disposed on one of the proximal and distal surfaces of the substrate, and a second coupling optical element disposed on one of the proximal and distal surfaces of the substrate and offset from the first coupling optical element. The first coupling optical element can be configured to deflect light at an angle to totally internally reflect (TIR) the light between the proximal and distal surfaces and toward the second coupling optical element, and the second coupling optical element can be configured to deflect at an angle out of the substrate. The eye-imaging apparatus can be used in a head-mounted display such as an augmented or virtual reality display.
A63F 13/20 - Input arrangements for video game devices
A63F 13/213 - Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
G01B 11/30 - Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces
82.
DEPTH BASED FOVEATED RENDERING FOR DISPLAY SYSTEMS
Methods and systems for depth-based foveated rendering in the display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of a display system based on detected sensor information. A fixation point is determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual objects to present is obtained, with the location information indicating three-dimensional positions of the virtual objects. Resolutions of at least one virtual object is adjusted based on a proximity of the at least one virtual object to the fixation point. The virtual objects are presented to a user by display system with the at least one virtual object being rendered according to the adjusted resolution.
Illuminations systems that separate different colors into laterally displaced beams may be used to direct different color image content into an eyepiece for displaying images in the eye. Such an eyepiece may be used, for example, for an augmented reality head mounted display. Illumination systems may be provided that utilize one or more waveguides to direct light from a light source towards a spatial light modulator. Light from the spatial light modulator may be directed towards an eyepiece. Some aspects of the invention provide for light of different colors to be outcoupled at different angles from the one or more waveguides and directed along different beam paths.
G02B 6/34 - Optical coupling means utilising prism or grating
G02B 6/10 - Light guides; Structural details of arrangements comprising light guides and other optical elements, e.g. couplings of the optical waveguide type
Systems and methods pertaining to iris recognition are disclosed. A system is disclosed comprising an iris imager configured to acquire an iris image, and a processor configured to receive a first plurality of iris images from the iris imager, generate a first plurality of iris codes corresponding to the first plurality of iris images, generate a distribution metric corresponding to an iris cell location, generate a first composite iris code using the distribution metric, and generate a first match value using the first composite iris code and a first stored iris code.
85.
MIXED REALITY SYSTEM WITH MULTI-SOURCE VIRTUAL CONTENT COMPOSITING AND METHOD OF GENERATING VIRTUAL CONTENT USING SAME
A computer implemented method for warping virtual content from two sources includes a first source generating first virtual content based on a first pose. The method also includes a second source generating second virtual content based on a second pose. The method further includes a compositor processing the first and second virtual content in a single pass. Processing the first and second virtual content includes generating warped first virtual content by warping the first virtual content based on a third pose, generating warped second virtual content by warping the second virtual content based on the third pose, and generating output content by compositing the warped first and second virtual content.
G06T 11/60 - Editing figures and text; Combining figures or text
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/0346 - Pointing devices displaced or positioned by the user; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
86.
MIXED REALITY SYSTEM WITH VIRTUAL CONTENT WARPING AND METHOD OF GENERATING VIRTUAL CONTENT USING SAME
A computer implemented method for warping virtual content includes generating warped virtual content by transforming source virtual content. The method also includes determining whether a memory location corresponding to an X, Y location of the warped virtual content in an output frame of reference is occupied by pre-existing virtual content. The method further includes storing the warped virtual content in the memory location if the memory location is not occupied. Moreover, the method includes comparing respective Z locations of the warped virtual content and the pre-existing virtual content to identify virtual content with a Z location closer to a viewing location if the memory location is occupied. The method also includes storing the warped virtual content in the memory location corresponding to the X, Y location if a Z location of warped virtual content is closer to the viewing location than a pre-existing Z location of pre-existing virtual content.
In some embodiments, a display device includes one or more waveguides having a vapor deposited light absorbing film on edges of the waveguide to mitigate ghost images. In some embodiments, the film is formed directly on the edge of the waveguide by a vapor deposition, such as an evaporative deposition process. In some embodiments, the light absorbing films may comprise carbon, for example carbon in the form of one or more aliotropes of carbon, such as fullerenes, or black silicon.
A fiber scanning system can have optimized performance by substantially matching the natural frequencies of the fiber scanning system's actuator and fiber optic scanning element. By matching the natural frequencies, the fiber scanning system can increase the maximum distance that the tip of the fiber optic scanning element may be driven relative to a resting position. Such an effect may be produced because matching the natural frequencies of the fiber scanner allows for larger amplitudes to be achieved. It should be noted that the natural frequency of the scanning system can be selected to avoid excitation frequencies that could destabilize the system. In this way, the system as a whole may act as a tuned mass damper or a tuned resonance structure, thereby improving scan performance while maintaining a stable scanning system.
A61B 1/00 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
G02B 6/35 - Optical coupling means having switching means
A computer implemented method for warping multi-field color virtual content for sequential projection includes obtaining first and second color fields having different first and second colors. The method also includes determining a first time for projection of a warped first color field. The method further includes determining a second time for projection of a warped second color field. Moreover, the method includes predicting a first pose at the first time and predicting a second pose at the second time. In addition, the method includes generating the warped first color field by warping the first color field based on the first pose. The method also includes generating the warped second color field by warping the second color field based on the second pose.
Disclosed is an improved approach for generated recordings from augmented reality systems from the perspective of a camera within the system. Instead of re-using rendered virtual content from the perspective of the user's eyes for AR recordings, additional virtual content is rendered from an additional perspective specifically for the AR recording. That additional virtual content is combined with image frames generated by a camera to form the AR recording.
G06T 15/00 - 3D [Three Dimensional] image rendering
G09G 5/377 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of individual graphic patterns using a bit-mapped memory - Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
Systems and methods for estimating a layout of a room are disclosed. The room layout can comprise the location of a floor, one or more walls, and a ceiling, in one aspect, a neural network can analyze an image of a portion of a room to determine the room layout. The neural network can comprise a convolutional neural network having an encoder sub-network, a decoder sub-network, and a side sub-network. The neural network can determine a three-dimensional room layout using two-dimensional ordered keypoints associated with a room type. The room layout can be used in applications such as augmented or mixed reality, robotics, autonomous indoor navigation, etc.
A virtual image generation system for use by an end user comprises memory, a display subsystem, an object selection device configured for receiving input from the end user and persistently selecting at least one object in response to the end user input, and a control subsystem configured for rendering a plurality of image frames of a three-dimensional scene, conveying the image frames to the display subsystem, generating audio data originating from the at least one selected object, and for storing the audio data within the memory.
A wearable augmented reality head-mounted display system can be configured to pass light from the world forward a wearer wearing the head-mounted system into an eye of the wearer. The head-mounted display system can include an optical display that is configured to output light to form an image. The system may include one or more waveguides that are disposed to receiving the light from the display. A variable power reflector can be disposed on the forward side of the one or more waveguides. The reflector can be configured to have an optical power that is adjustable upon application of an electrical signal.
Example display devices include a waveguide configured to propagate visible light under total internal reflection in a direction parallel to a major surface of the waveguide. The waveguide has formed thereon an outcoupling element configured to outcouple a portion of the visible light in a direction normal to the major surface of the waveguide. The example display devices additionally include a polarization-selective notch reflector disposed on a first side of the waveguide and configured to reflect visible light having a first polarization while transmitting the portion of the visible light having a second polarization. The example display devices further include a polarization-independent notch reflector disposed on a second side of the waveguide and configured to reflect visible light having the first polarization and the second polarization, where the polarization-independent notch reflector is configured to convert a polarization of visible light reflecting therefrom.
= An eyepiece unit with optical filters includes a set of waveguide layers including a first waveguide layer and a second waveguide layer. The first waveguide layer is disposed in a first lateral plane and includes a first incoupling diffractive element disposed at a first lateral position, a first waveguide, and a first outcoupling diffractive element. The second waveguide layer is disposed in a second lateral plane adjacent to the first lateral plane and includes a second incoupling diffractive element disposed at a second lateral position, a second waveguide, and a second outcoupling diffractive element. The eyepiece unit also includes a set of optical filters including a first optical filter positioned at the first lateral position and operable to attenuate light outside a first spectral band and a second optical filter positioned at the second lateral position and operable to attenuate light outside a second spectral band.
96.
METHOD AND SYSTEM FOR DISPLAY DEVICE WITH INTEGRATED POLARIZER
An eyepiece unit for projecting an image to an eye of a viewer includes an eyepiece having a world side and a viewer side opposite the world side. The eyepiece unit also includes a polarizer disposed adjacent the world side of the eyepiece. In an example, the polarizer comprises a wire grid polarizer. In some embodiments, the wire grid polarizer is operable to transmit p-polarized light and reflect s-polarized light. In some embodiments, the polarizer can include an absorptive polarizer.
An eyepiece unit with optical filters includes a set of waveguide layers including a first waveguide layer and a second waveguide layer. The first waveguide layer is disposed in a first lateral plane and includes a first incoupling diffractive element disposed at a first lateral position, a first waveguide, and a first outcoupling diffractive element. The second waveguide layer is disposed in a second lateral plane adjacent to the first lateral plane and includes a second incoupling diffractive element disposed at a second lateral position, a second waveguide, and a second outcoupling diffractive element. The eyepiece unit also includes a set of optical filters including a first optical filter positioned at the first lateral position and operable to attenuate light outside a first spectral band and a second optical filter positioned at the second lateral position and operable to attenuate light outside a second spectral band.
Antireflection coatings for metasurfaces are described herein. In some embodiments, the metasurtace may include a substrate, a plurality of nanostructures thereon, and an antireflection coating disposed over the nanostructures. The antireflection coating may be a transparent polymer, for example a photoresist layer, and may have a refractive index lower than the refractive index of the nanostructures and higher than the refractive index of the overlying medium (e.g., air). Advantageously, the antireflection coatings may reduce or eliminate ghost images in an augmented reality display in which the metasurtace is incorporated.
Metasurfaces provide compact optical elements in head-mounted display systems to, e.g., incouple light into or outcouple light out of a waveguide. The metasurfaces may be formed by a plurality of repeating unit cells, each unit cell comprising two sets or more of nanobeams elongated in crossing directions: one or more first nanobeams elongated in a first direction and a plurality of second nanobeams elongated in a second direction. As seen in a top-down view, the first direction may be along a y-axis, and the second direction may be along an x-axis. The unit cells may have a periodicity in the range of 10 nm to 1 µm, including 10 nm to 500 nm or 300 nm to 500 nm. Advantageously, the metasurfaces provide diffraction of light with high diffraction angles and high diffraction efficiencies over a broad range of incident angles and for incident light with circular polarization.
To enable shared user experiences using augmented reality systems, shared reference points must be provided to have consistent placement (position and orientation) of virtual objects. Furthermore, the position and orientation (pose) of the users must be determinable with respect to the same shared reference points. However, without highly sensitive and expensive global positioning system (GPS) devices, pose information can be difficult to determine to a reasonable level of accuracy. Therefore, what is provided is an alternative approach to determining pose information for augmented reality systems, which can be used to perform location based content acquisition and sharing. Further, what is provided is an alternative approach to determining pose information for augmented reality systems that uses information from already existing GPS devices.
patents
