Sensor data identifying an emergency vehicle approaching the autonomous vehicle may be received. A predicted trajectory for the emergency vehicle may be received. Whether the autonomous vehicle is impeding the emergency vehicle may be determined based on the predicted trajectory and map information identifying a road on which the autonomous vehicle is currently traveling. Based on a determination that the autonomous vehicle is impeding the emergency vehicle, the autonomous vehicle may be controlled in an autonomous driving mode in order to respond to the emergency vehicle.
To operate an autonomous vehicle, a rail agent is detected in a vicinity of the autonomous vehicle using a detection system. One or more tracks are determined on which the detected rail agent is possibly traveling, and possible paths for the rail agent are predicted based on the determined one or more tracks. One or more motion paths are determined for one or more probable paths from the possible paths, and a likelihood for each of the one or more probable paths is determined based on each motion plan. A path for the autonomous vehicle is then determined based on a most probable path associated with a highest likelihood for the rail agent, and the autonomous vehicle is operated using the determined path.
G05D 1/617 - Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D 1/693;arrangements for reacting to or preventing system or operator failure G05D 1/80)
G05D 1/228 - Command input arrangements located on-board unmanned vehicles
Aspects of the disclosure relate to generating scouting objectives in order to update map information used to control a fleet of vehicles in an autonomous driving mode. For instance, a notification from a vehicle of the fleet identifying a feature and a location of the feature may be received. A first bound for a scouting area may be identified based on the location of the feature. A second bound for the scouting area may be identified based on a lane closest to the feature. A scouting objective may be generated for the feature based on the first bound and the second bound.
Systems and methods for extrinsic calibration of vehicle-mounted sensors are provided. One example method involves obtaining first sensor data collected by a first sensor and a second sensor while a vehicle is aligned in a first yaw direction. The method also involves obtaining second sensor data collected by the first sensor and the second sensor while the vehicle is aligned in a second yaw direction. The method also involves determining, based on the first sensor data and the second sensor data, (i) first pitch and roll misalignments of the first sensor relative to the vehicle and (ii) second pitch and roll misalignments of the second sensor relative to the first sensor. The method also involves determining third pitch and roll misalignments of the second sensor relative to the vehicle based on (i) the first pitch and roll misalignments and (ii) the second pitch and roll misalignments.
G01C 25/00 - Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass
G01C 21/16 - Navigation; Navigational instruments not provided for in groups by using measurement of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
Described examples relate to an apparatus comprising one or more image sensors coupled to a vehicle and at least one processor. The at least one processor may be configured to capture, in a burst sequence using the one or more image sensors, multiple frames of an image of a scene, the multiple frames having respective, relative offsets of the image across the multiple frames and perform super-resolution computations using the captured, multiple frames of the image of the scene. The at least one processor may also be configured to accumulate, based on the super-resolution computations, color planes and combine, using the one or more processors, the accumulated color planes to create a super-resolution image of the scene.
The present disclosure relates to devices, lidar systems, and vehicles that include optical redirectors. An example lidar system includes a transmitter and a receiver. The transmitter includes at least one light-emitter device configured to transmit emission light into an environment. The receiver is configured to detect return light from the environment and includes a plurality of apertures, a plurality of photodetectors, and a plurality of optical redirectors. Each optical redirector is configured to optically couple a respective portion of return light from a respective aperture to at least one photodetector of the plurality of photodetectors. Each optical redirector also has a rotational orientation relative to other optical redirectors such that the redirection paths of optical redirectors that correspond to adjacent apertures are not coplanar with one another.
Example embodiments relate to techniques for adjusting vehicle behavior based on estimated unintentional lateral movements. A computing system may receive sensor data representing a truck's environment as the truck pulls a trailer and navigates above a threshold speed on a freeway or another type of road. The computing system may use the sensor data to detect another truck navigating in an adjacent lane and at a speed that indicates an increased likelihood of a pass maneuver occurring between the trucks. Responsive to determining the increased likelihood of the pass maneuver occurring between the vehicles, the computing system may estimate an unintentional lateral movement for the pass maneuver based on parameters that can include the truck's speed, the size of the other truck's trailer, and a wind condition of the environment. The computing system can subsequently control the truck based on estimated unintentional lateral movement for the pass maneuver.
Aspects of the disclosure relate to reducing the likelihood of injury to a passenger in a collision. In one example, a computing device may determine that an impact between a vehicle and an object external to the vehicle is imminent. The computing device may determine a protection range for a vehicle's airbag based on characteristics of the passenger, including the passenger's seating location within the vehicle. An airbag extension system may position an airbag package, including the vehicle's airbag, such that the vehicle's airbag is within the protection range of the passenger's seating location.
B60R 21/20 - Arrangements for storing inflatable members in their non-use or deflated condition; Arrangement or mounting of air bag modules or components
B60R 21/0134 - Electrical circuits for triggering safety arrangements in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over responsive to imminent contact with an obstacle
B60R 21/015 - Electrical circuits for triggering safety arrangements in case of vehicle accidents or impending vehicle accidents including means for detecting the presence or position of passengers, passenger seats or child seats, e.g. for disabling triggering
B60R 21/264 - Inflatable occupant restraints or confinements designed to inflate upon impact or impending impact, e.g. air bags characterised by the inflation fluid source or means to control inflation fluid flow using instantaneous generation of gas, e.g. pyrotechnic
12.
Handling sensor occlusions for autonomous vehicles
The technology relates to identifying sensor occlusions due to the limits of the ranges of a vehicle's sensors and using this information to maneuver the vehicle. As an example, the vehicle is maneuvered along a route that includes traveling on a first roadway and crossing over a lane of a second roadway. A trajectory is identified from the lane that will cross with the route during the crossing at a first point. A second point beyond a range of the vehicle's sensors is selected. The second point corresponds to a hypothetical vehicle moving towards the route along the lane. A distance between the first point and the second point is determined. An amount of time that it would take the hypothetical vehicle to travel the distance is determined and compared to a threshold amount of time. The vehicle is maneuvered based on the comparison to complete the crossing.
B60W 30/00 - Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
G01S 13/931 - Radar or analogous systems, specially adapted for specific applications for anti-collision purposes of land vehicles
G01S 17/00 - Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
G01S 17/931 - Lidar systems, specially adapted for specific applications for anti-collision purposes of land vehicles
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G06V 20/56 - Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
A vehicle has a plurality of control apparatuses, a user input, a geographic position component, an object detection apparatus, memory, and a display. A processor is also included and is programmed to receive the destination information, identify a route, and determine the current geographic location of the vehicle. The processor is also programmed to identify an object and object type based on object information received from the object detection apparatus and to determine at least one warning characteristic of the identified object based on at least one of: the object type, a detected proximity of the detected object to the vehicle, the location of the detected object relative to predetermined peripheral areas of the vehicle, the current geographic location of the vehicle, and the route. The processor is also configured to select and display on the display an object warning image based on the at least one warning characteristic.
G05D 1/02 - Control of position or course in two dimensions
B60T 7/22 - Brake-action initiating means for initiation not subject to will of driver or passenger initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle
B60W 50/14 - Means for informing the driver, warning the driver or prompting a driver intervention
G01C 21/36 - Input/output arrangements for on-board computers
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
G08G 1/09 - Arrangements for giving variable traffic instructions
G08G 1/0962 - Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
Aspects of the disclosure relate to arranging a pickup between a driverless vehicle and a passenger. For instance, dispatch instructions dispatching the vehicle to a predetermined pickup area in order to pick up the passenger are received by the vehicle which begins maneuvering to the predetermined pickup area. While doing so, the vehicle receives from the passenger's client computing device the device's location. An indication that the passenger is interested in a fly-by pickup is identified. The fly-by pickup allows the passenger to safely enter the vehicle at a location outside of the predetermined pickup area and prior to the one or more processors have maneuvered the vehicle to the predetermined pickup area. The vehicle determines that the fly-by pickup is appropriate based on at least the location of the client computing device and the indication, and based on the determination, maneuvers itself in order to attempt the fly-by pickup.
Aspects of the technology provide a method including receiving a request for a user to be picked up or dropped off by an autonomous vehicle, in which the request identifying a location, and determining a land parcel containing the identified location. The method also includes identifying a set of features that are within a selected distance from the identified location, filtering the set of identified features to obtain only curated features that are within the selected distance from the identified location, determining a distance between each curated feature and the identified location, and inferring an access point for the identified location based on the distances determined between each curated feature and the identified location. The inferred access point can then be provided to enable the autonomous vehicle to perform a pickup or drop-off.
Example embodiments relate to wiper devices for cleaning sensor housings. An example embodiment includes a wiper device for cleaning a surface of a rotatable sensor housing. The wiper device may have a driving arm having a first end and a second end. The first end of the driving arm is connected to a rotatable shaft. The wiper device may also have a wiper arm pivotally connected to the second end of the driving arm. The pivotal connection is located between a front region and a rear region of the wiper arm. Further, the wiper device may have a wiper blade coupled to the front region of the wiper arm. The wiper blade may be configured to wipe the surface of the rotatable sensor housing. Additionally, the wiper device may have a counterweight coupled to the rear region of the wiper arm.
B60S 1/60 - Cleaning windscreens, windows, or optical devices specially adapted for cleaning other parts or devices than front windows or windscreens for signalling devices, e.g. reflectors
One example system includes a first light detection and ranging (LIDAR) device that scans a first field-of-view defined by a first range of pointing directions associated with the first LIDAR device. The system also includes a second LIDAR device that scans a second FOV defined by a second range of pointing directions associated with the second LIDAR device. The second FOV at least partially overlaps the first FOV. The system also includes a first controller that adjusts a first pointing direction of the first LIDAR device. The system also includes a second controller that adjusts a second pointing direction of the second LIDAR device synchronously with the adjustment of the first pointing direction of the first LIDAR device.
Aspects of the disclosure relate to managing parking maneuvers for autonomous vehicles. For instance, a vehicle may be stopped at a parking location. A distance between the autonomous vehicle and a road edge at the parking location may be determined. Based on the distance, a plurality of points corresponding to parking locations at various distances from the road edge may be sampled. For each of the plurality of points, a trajectory for the autonomous vehicle may be determined. One of the determined trajectories may be selected. The autonomous vehicle closer to the road edge using the selected one of the determined trajectories.
A first driving solution for a vehicle along a portion of a route is determined based on an agent detected in the vehicle's environment following a first trajectory of a plurality of possible trajectories. A switching time is determined for the vehicle to deviate from the first driving solution for a situation in which the agent is following a second trajectory of the plurality of possible trajectories. The first driving solution is revised such that the vehicle will be able to switch from the revised first driving solution to another driving solution at the switching time in case if the detected agent is following the second trajectory.
Aspects of the disclosure relate to a system that includes a memory storing a queue for arranging tasks, a plurality of self-driving systems for controlling an autonomous vehicle, and one or more processors. The one or more processors may receive a non-passenger task request with a priority level of the non-passenger task request. When the non-passenger task request is accepted, the one or more processors may insert the task in the queue based on the priority level of the task request. Then, the one or more processors may provide instructions to one or more self-driving systems according to the non-passenger task request. Having received updates of the status of the autonomous vehicle, the one or more processors may determine that the task is completed based on the updates. After determining that the task is completed, the one or more processors may remove the task from the queue.
G07C 5/00 - Registering or indicating the working of vehicles
G07C 5/08 - Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle, or waiting time
The present disclosure relates to systems and methods relating to the fabrication of light guide elements. An example system includes an optical component configured to direct light emitted by a light source to illuminate a photoresist material at one or more desired angles so as to expose an angled structure in the photoresist material. The photoresist material overlays at least a portion of a first surface of a substrate. The optical component includes a container containing a light-coupling material that is selected based in part on the one or more desired angles. The system also includes a reflective surface arranged to reflect at least a first portion of the emitted light to illuminate the photoresist material at the one or more desired angles.
G03F 7/00 - Photomechanical, e.g. photolithographic, production of textured or patterned surfaces, e.g. printed surfaces; Materials therefor, e.g. comprising photoresists; Apparatus specially adapted therefor
G02B 6/42 - Coupling light guides with opto-electronic elements
22.
Vehicle Sensor Modules with External Audio Receivers
Example embodiments relate to vehicle sensor modules with external audio receivers. An example sensor module may include sensors and can be coupled to a vehicle's roof with a first microphone positioned proximate to the front of the sensor module. The sensor module can also include a second microphone extending into a first side of the sensor module such that the second microphone is configured to detect audio originating from an environment located relative to a first side of the vehicle and a third microphone extending into a second side of the sensor module such that the third microphone is configured to detect audio originating from the environment located relative to a second side of the vehicle, wherein the second side is opposite of the first side.
The subject matter of this specification can be implemented in, among other things, systems and methods of optical sensing that use destructive interference to suppress retro-reflected light during generation of sensing beams. Described, among other things, is a system to produce a transmitted (TX) beam and collect a received (RX) beam. The RX beam can include a reflected beam caused by interaction of the TX beam with an object, and a retro-reflected (RR) beam caused by interaction of the TX beam internal components of the system. The system is further to combine the RX beam with a phase-controlled beam to obtain a combined beam, control a phase of the phase-controlled beam to cause destructive interference of the phase-controlled beam and the RR beam, and determine, using the combined beam, one or more of the characteristics of the object.
G01S 17/34 - Systems determining position data of a target for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated using transmission of continuous, frequency-modulated waves while heterodyning the received signal, or a signal derived therefrom, with a locally-generated signal related to the contemporaneously transmitted signal
G01S 7/481 - Constructional features, e.g. arrangements of optical elements
Aspects of the disclosure provide for controlling an autonomous vehicle to enter a driveway. As an example, an instruction to pickup or drop off a passenger at a location may be received. It may be determined that the vehicle is arriving in a lane on an opposite side of a street as the location, the lane having a traffic direction opposite to a traffic direction of a lane adjacent to the location. A difficulty score to maneuver the vehicle to the lane adjacent to the location may be determined, and the difficulty score may be compared to a predetermined difficulty score. Based on the comparison, it may be determined to an available driveway on the same side of the street as the location. The vehicle may be controlled to enter the available driveway.
A sensor assembly contains at least first and second sensors configured to perform imaging from a portion of a vehicle, each sensor having a respective field of view (FOV) based on an imaging direction of the sensor, each sensor having an FOV origin. The sensor assembly is configured to be disposed at the portion of the vehicle such that (i) the first and second sensors' FOV origins are each outboard from the vehicle, (ii) the first and second sensors' imaging directions are each within 90 degrees in yaw of being parallel to the backward direction of the vehicle, (iii) the first sensor's imaging direction is closer in yaw than the second sensor's imaging direction to being parallel to the backward direction of the vehicle, and (iv) the first sensor's FOV origin is more outboard from the vehicle than the second sensor's FOV origin.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using slice-based dynamic neural networks. One of the methods includes receiving a new input for processing by a neural network that includes a first conditional neural network layer that has a set of shared parameters and a respective set of slice parameters for each of a plurality of slices. One or more slices to which the new input belongs are identified. The new input is processed to generate a network output, including: receiving a layer input to the first conditional neural network layer; and processing the layer input using the set of shared parameters, the respective one or more sets of slice parameters for the identified one or more slices, but not the respective sets of slice parameters for any other slices to which the new input does not belong.
B60W 40/00 - Estimation or calculation of driving parameters for road vehicle drive control systems not related to the control of a particular sub-unit
G06F 18/211 - Selection of the most significant subset of features
An example method involves detecting a sensor-testing trigger. Detecting the sensor-testing trigger may comprise determining that a vehicle is within a threshold distance to a target in an environment of the vehicle. The method also involves obtaining sensor data collected by a sensor of the vehicle after the detection of the sensor-testing trigger. The sensor data is indicative of a scan of a region of the environment that includes the target. The method also involves comparing the sensor data with previously-collected sensor data indicating detection of the target by one or more sensors during one or more previous scans of the environment. The method also involves generating performance metrics related to the sensor of the vehicle based on the comparison.
G01S 7/52 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group
G05D 1/02 - Control of position or course in two dimensions
28.
EXTERNAL INDICATION HANDLING FOR AUTONOMOUS DRIVING SYSTEMS
A method includes detecting, by a sensing system of an autonomous vehicle (AV) executing a trip from a first location to a second location, a signal to stop the AV, determining, in response to the signal, that the AV is to be stopped for a roadside inspection, and causing a vehicle control system of the AV to stop the AV at a third location to allow the roadside inspection to be performed.
Aspects of the disclosure relate to detecting and responding to malfunctioning traffic signals for a vehicle having an autonomous driving mode. For instance, information identifying a detected state of a traffic signal for an intersection. An anomaly for the traffic signal may be detected based on the detected state and prestored information about expected states of the traffic signal. The vehicle may be controlled in the autonomous driving mode based on the detected anomaly.
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
G06V 40/10 - Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
One example method of testing an electrical device comprises transmitting a data pattern to a memory device of the electrical device by a controller of the electrical device to provide a written data pattern to the memory device, wherein the data pattern replicates a resonant frequency of at least a portion of the electrical device, reading the written data pattern from the memory device with the controller, and comparing the written data pattern to the data pattern.
One example device comprises a plurality of emitters including at least a first emitter and a second emitter. The first emitter emits light that illuminates a first portion of a field-of-view (FOV) of the device. The second emitter emits light that illuminates a second portion of the FOV. The device also comprises a controller that obtains a scan of the FOV. The controller causes each emitter of the plurality of emitters to emit a respective light pulse during an emission time period associated with the scan. The controller causes the first emitter to emit a first-emitter light pulse at a first-emitter time offset from a start time of the emission time period. The controller causes the second emitter to emit a second-emitter light pulse at a second-emitter time offset from the start time of the emission time period.
G01S 7/481 - Constructional features, e.g. arrangements of optical elements
G01S 7/491 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group - Details of non-pulse systems
32.
LIDAR SYSTEMS WITH PLANAR MULTI-PIXEL SENSING ARRAYS
The subject matter of this specification can be implemented in, among other things, systems and methods of optical sensing for pixel multiplexing and polarized beam steering. Described, among other things, is a system that includes a light source configured to generate a light and one or more optical devices, integrated on a photonic integrated circuit, configured to impart modulation to the generated light and deliver the modulated light to an interface coupling device. The interface coupling device can include a plurality of interface couplers (ICs), each IC configured to scatter the modulated light in a direction that makes at least 5 degrees with an optical axis of lens configured to transmit the light scattered by each IC along a respective direction of a plurality of directions.
G01S 7/499 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group using polarisation effects
G01S 17/58 - Velocity or trajectory determination systems; Sense-of-movement determination systems
The present disclosure relates to devices, light detection and ranging (lidar) systems, and vehicles involving solid-state, single photon detectors. An example device includes a substrate defining a primary plane and a plurality of photodetector cells disposed along the primary plane. The plurality of photodetector cells includes at least one large-area cell and at least one small-area cell. The large-area cell has a first area and the small-area cell has a second area and the first area is greater than the second area. The device also includes read out circuitry coupled to the plurality of photodetector cells. The read out circuitry is configured to provide an output signal based on incident light detected by the plurality of photodetector cells.
H01L 31/0232 - Optical elements or arrangements associated with the device
H01L 31/107 - Devices sensitive to infrared, visible or ultraviolet radiation characterised by only one potential barrier or surface barrier the potential barrier working in avalanche mode, e.g. avalanche photodiode
34.
OBJECT IDENTIFICATION IN BIRD'S-EYE VIEW REFERENCE FRAME WITH EXPLICIT DEPTH ESTIMATION CO-TRAINING
The described aspects and implementations enable efficient detection and classification of objects with machine learning models that deploy a bird's-eye view representation and are trained using depth ground truth data. In one implementation, disclosed are system and techniques that include obtaining images, generating, using a first neural network (NN), feature vectors (FVs) and depth distributions pixels of images, wherein the first NN is trained using training images and a depth ground truth data for the training images. The techniques further include obtaining a feature tensor (FT) in view of the FVs and the depth distributions, and processing the obtained FTs, using a second NN, to identify one or more objects depicted in the images.
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
G06T 7/55 - Depth or shape recovery from multiple images
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
35.
SEMANTIC SEGMENTATION NEURAL NETWORK FOR POINT CLOUDS
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a semantic segmentation neural network for point clouds. One of the methods includes: obtaining a plurality of training points divided into a respective plurality of components; obtaining, for each of the respective plurality of components, data identifying a ground truth category for one or more labeled point; processing each training points using a semantic segmentation neural network to generate a semantic segmentation that includes a respective score for each of the plurality of categories; determining a gradient of a loss function that penalizes the semantic segmentation neural network for generating, for points in the component, non-zero scores for categories that are not the ground truth category for any labeled point in the component; and updating, using the gradient, the parameters of the semantic segmentation neural network.
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
36.
CARRIER EXTRACTION FROM SEMICONDUCTING WAVEGUIDES IN HIGH-POWER LIDAR APPLICATIONS
The subject matter of this specification can be implemented in, among other things, systems and methods of optical sensing that use carrier extraction from waveguides that can support propagation of high-power sensing beams. Described, among other things, is a system that includes one or more waveguides that include a semiconducting material with a temperature-dependent refractive index. The system further includes a plurality of extraction electrodes configured to extract from the waveguide(s), charge carriers generated by an electromagnetic wave propagating in the waveguide(s). The system further includes a heating electrode configured to cause a change of a temperature of the waveguide(s).
The technology relates to a seamless interface between an autonomous vehicle service provider and one or more ridesharing, ride-hailing or delivery partner companies, in order to provide timely, efficient rider or delivery services and support. A partner trip application programming interface (API) provides robust features to the ridesharing or ride-hailing partner companies and also supports delivery of meals, groceries, packages or other cargo. For instance, the API may be employed with delivery partners (e.g., food, package or bulk cargo delivery), and/or for concierge services (e.g., a hotel or medical provider or an e-commerce specialist that coordinates deliveries). The technology supports both ad hoc and scheduled trips. Agents for different partners may schedule trips to or from specific stores or other (geofenced) locations, and observe and manage trips for an entire enterprise. Permissions can be tailored for each agent.
A simulation may be used to determine a difference between progress of a manually-driven vehicle and progress of a simulated autonomous vehicle. The method includes retrieving log data collected for the manually-driven vehicle driving along a route, generating a plurality of path segments for a portion of the route. The plurality of path segments corresponds to points in a lane that the manually-driven vehicle traveled through on the portion of the route. The method also includes running, using a software of the autonomous vehicle, a simulation of the autonomous vehicle driving along the plurality of path segments, extracting metrics from the log data and the simulation, and determining the difference between a first progress of the manually-driven vehicle and a second progress of the simulated autonomous vehicle based on the metrics.
G01C 22/00 - Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers or using pedometers
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
The technology relates to detecting and responding to emergency vehicles. This may include using a plurality of microphones to detect a siren noise corresponding to an emergency vehicle and to estimate a bearing of the emergency vehicle. This estimated bearing is compared to map information to identify a portion of roadway on which the emergency vehicle is traveling. In addition, information identifying a set of objects in the vehicle's environment as well as characteristics of those objects is received from a perception system is used to determine whether one of the set of objects corresponds to the emergency vehicle. How to respond to the emergency vehicle is determined based on the estimated bearing and identified road segments and the determination of whether one of the set of objects corresponds to the emergency vehicle. This determined response is then used to control the vehicle in an autonomous driving mode.
G08G 1/0965 - Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages responding to signals from another vehicle, e.g. emergency vehicle
G06F 16/68 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
G06F 16/683 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
The technology involves operation of a self-driving truck or other cargo vehicle when it is being inspected at a weigh station. This may include determining whether a weigh station is open for inspection. Once at the weigh station, the vehicle may follow instructions of an inspection officer or autonomous inspection system. The vehicle may perform predefined actions or operations so that various vehicle systems and safety issues can be evaluated, such as the brakes, lights, tires, connections between the tractor and trailer, exposed fuel tanks, leaks, etc. A visual inspection may be performed to ensure the load is secured, vehicle and cargo documents meet certain criteria, and the carrier's safety record meets any requirements. In addition, the weigh station itself may be operated in a partly or fully autonomous mode when dealing with autonomous and manually driven vehicles.
G07C 5/08 - Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle, or waiting time
G01G 19/02 - Weighing apparatus or methods adapted for special purposes not provided for in groups for weighing wheeled or rolling bodies, e.g. vehicles
G05D 1/02 - Control of position or course in two dimensions
G07C 5/00 - Registering or indicating the working of vehicles
G07C 5/02 - Registering or indicating driving, working, idle, or waiting time only
Example embodiments relate to LIDAR systems with multi-faceted mirrors. An example embodiment includes a LIDAR system. The system includes a multi-faceted mirror that includes a plurality of reflective facets, which rotates about a first rotational axis. The system also includes a light emitter configured to emit a light signal toward one or more regions of a scene. Further, the system includes a light detector configured to detect a reflected light signal. In addition, the system includes an optical window positioned between the multi-faceted mirror and the one or more regions of the scene such that light reflected from one or more of the reflective facets is transmitted through the optical window. The optical window is positioned such that the optical window is non-perpendicular to the direction toward which the light emitted along the optical axis is directed for all angles of the multi-faceted mirror.
G01S 17/931 - Lidar systems, specially adapted for specific applications for anti-collision purposes of land vehicles
G01C 3/06 - Use of electric means to obtain final indication
G01S 7/48 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group
G01S 7/481 - Constructional features, e.g. arrangements of optical elements
G01S 17/10 - Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
G01S 17/42 - Simultaneous measurement of distance and other coordinates
G01S 17/89 - Lidar systems, specially adapted for specific applications for mapping or imaging
43.
Using Wheel Orientation To Determine Future Heading
The technology relates to determining a future heading of an object. In order to do so, sensor data, including information identifying a bounding box representing an object in a vehicle's environment and locations of sensor data points corresponding to the object, may be received. Based on dimensions of the bounding box, an area corresponding to a wheel of the object may be identified. An orientation of the wheel may then be estimated based on the sensor data points having locations within the area. The estimation may then be used to determine a future heading of the object.
B60W 50/00 - CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - Details of control systems for road vehicle drive control not related to the control of a particular sub-unit
B60W 50/14 - Means for informing the driver, warning the driver or prompting a driver intervention
G01S 13/931 - Radar or analogous systems, specially adapted for specific applications for anti-collision purposes of land vehicles
G01S 15/931 - Sonar systems specially adapted for specific applications for anti-collision purposes of land vehicles
G01S 17/931 - Lidar systems, specially adapted for specific applications for anti-collision purposes of land vehicles
G06V 10/25 - Determination of region of interest [ROI] or a volume of interest [VOI]
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
Described herein is a LIDAR device that may include a transmitter, first and second receivers, and a rotating platform. The transmitter may be configured to emit light having a vertical beam width. The first receiver may be configured to detect light at a first resolution while scanning the environment with a first FOV and the second receiver may be configured to detect light at a second resolution while scanning the environment with a second FOV. In this arrangement, the first resolution may be higher than the second resolution, the first FOV may be at least partially different from the second FOV, and the vertical beam width may encompass at least a vertical extent of the first and second FOVs. Further, the rotating platform may be configured to rotate about an axis such that the transmitter and first and second receivers each move based on the rotation.
Example embodiments relate to temporally modulated light emission and defect detection in light detection and ranging (lidar) devices and cameras. An example embodiment includes a method. The method includes detecting, by a first detector via an optical component, a background signal corresponding to a surrounding environment. The method also includes illuminating, by a first light source, a first portion of the optical component with a first light signal. Additionally, the method includes detecting, by the first detector when one or more defects are present in a body of the first portion of the optical component or on a surface of the first portion of the optical component, the first light signal. Further, the method includes determining, by a computing device, when one or more defects are present in the body of the first portion of the optical component or on the surface of the first portion of the optical component.
The technology relates to controlling a vehicle in an autonomous driving mode in accordance with behavior predictions for other road users in the vehicle's vicinity. In particular, the vehicle's onboard computing system may predict whether another road user will perform a “continuing” lane driving operation, such as going straight in a turn-only lane. Sensor data from detected/observed objects in the vehicle's nearby environment may be evaluated in view of one or more possible behaviors for different types of objects. In addition, roadway features, in particular whether lane segments are connected in a roadgraph, are also evaluated to determine probabilities of whether other road users may make an improper continuing lane driving operation. This is used to generate more accurate behavior predictions, which the vehicle can use to take alternative (e.g., corrective) driving actions.
G05D 1/02 - Control of position or course in two dimensions
B60W 60/00 - Drive control systems specially adapted for autonomous road vehicles
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
47.
DETECTING AND RESPONDING TO PROCESSIONS FOR AUTONOMOUS VEHICLES
The technology relates to detecting and responding to processions. For instance, sensor data identifying two or more objects in an environment of a vehicle may be received. The two or more objects may be determined to be disobeying a predetermined rule in a same way. Based on the determination that the two or more objects are disobeying a predetermined rule, that the two or more objects are involved in a procession may be determined. The vehicle may then be controlled autonomously in order to respond to the procession based on the determination that the two or more objects are involved in a procession.
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
48.
PHYSICS-INFORMED OPTIMIZATION FOR AUTONOMOUS DRIVING SYSTEMS
A method includes identifying map data comprising driving constraint data for a route of an autonomous vehicle (AV), the map data being of a road network associated with the route of the AV, the driving constraint data being based on physical vehicle data. The method further includes, while the AV is travelling the route, identifying current environmental sensing data for a portion of the route. The method further includes causing, based on the map data comprising the driving constraint data for the route and the current environmental sensing data associated with the portion of the route, the AV to travel the portion of the route.
Aspects of the disclosure provide for determining when to provide and providing secondary disengage alerts for a vehicle having autonomous and manual driving modes. For instance, while the vehicle is being controlled in the autonomous driving mode, user input is received at one or more user input devices of the vehicle. In response to receiving the user input, the vehicle may be transitioned from the autonomous driving mode to a manual driving mode and provide a primary disengage alert to an occupant of the vehicle regarding the transition. Whether to provide a secondary disengage alert may be determined based on at least circumstances of the user input. After the transition, the secondary disengage alert may be provided based on the determination.
Example embodiments relate to low-overhead, bidirectional error checking for a serial peripheral interface. An example device includes an integrated circuit. The device also includes a serial peripheral interface (SPI) with a Master In Slave Out (MISO) channel and a Master Out Slave In (MOSI) channel. The MOSI channel is configured to receive a write address, payload data, and a forward error-checking code usable to identify data corruption within the write address or the payload data. The integrated circuit is configured to calculate and provide a reverse error-checking code usable to identify data corruption within the write address or the payload data. Additionally, the integrated circuit is configured to compare the forward error-checking code to the reverse error-checking code. Further, the integrated circuit is configured to write, to the write address if the forward error-checking code matches the reverse error-checking code, the payload data.
An integrated hybrid rotary assembly is configured to provide power, torque and bi-directional communication to a rotatable sensor, such as a lidar, radar or optical sensor. A common ferrite core is shared by a motor, rotary transformer and radio frequency communication link. This hybrid configuration reduces cost, simplifies the manufacturing process, and can improve system reliability by employing a minimum number of parts. The assembly can be integrated with the sensor unit, which may be used in vehicles and other systems.
Example embodiments relate to replaceable, heated, and wipeable apertures for optical systems. An example embodiment includes an apparatus that includes an optical system (e.g., a camera) with a lens. The apparatus also includes a housing in which the lens is disposed. The housing has a sealing surface and a threated portion. The apparatus further includes a chamfered window that provides a protective aperture for the lens. The apparatus also includes a retainer ring that has a retaining portion and a threaded portion. The retaining portion contacts the chamfer of the window while the threaded portion of the retainer ring threads onto the threaded portion of the housing to hold the window against the sealing surface of the housing and form a watertight seal.
Aspects of the disclosure provide for controlling an autonomous vehicle. For instance, a first probability distribution may be generated for the vehicle at a first future point in time using a generative model for predicting expected behaviors of objects and a set of characteristics for the vehicle at an initial time expected to be perceived by an observer. Planning system software of the vehicle may be used to generate a trajectory for the vehicle to follow. A second probability distribution may be generated for a second future point in time using the generative model based on the trajectory and a set of characteristics for the vehicle at the first future point expected to be perceived by the observer. A surprise assessment may be generated by comparing the first probability distribution to the second probability distribution. The vehicle may be controlled based on the surprise assessment.
B60W 50/00 - CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - Details of control systems for road vehicle drive control not related to the control of a particular sub-unit
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G05D 1/02 - Control of position or course in two dimensions
Example implementations are provided for an arrangement of co-aligned rotating sensors. One example device includes a light detection and ranging (LIDAR) transmitter that emits light pulses toward a scene according to a pointing direction of the device. The device also includes a LIDAR receiver that detects reflections of the emitted light pulses reflecting from the scene. The device also includes an image sensor that captures an image of the scene based on at least external light originating from one or more external light sources. The device also includes a platform that supports the LIDAR transmitter, the LIDAR receiver, and the image sensor in a particular relative arrangement. The device also includes an actuator that rotates the platform about an axis to adjust the pointing direction of the device.
Aspects of the disclosure relate to controlling a vehicle having an autonomous driving mode. For instance, a current state of a traffic light may be determined. One of a plurality of yellow light durations may be selected based on the current state of the traffic light. When the traffic light will turn red may be predicted based on the selected one. The prediction may be used to control the vehicle in the autonomous driving mode.
A vehicle configured to operate in an autonomous mode can obtain sensor data from one or more sensors observing one or more aspects of an environment of the vehicle. At least one aspect of the environment of the vehicle that is not observed by the one or more sensors could be inferred based on the sensor data. The vehicle could be controlled in the autonomous mode based on the at least one inferred aspect of the environment of the vehicle.
B60W 40/00 - Estimation or calculation of driving parameters for road vehicle drive control systems not related to the control of a particular sub-unit
G01C 21/26 - Navigation; Navigational instruments not provided for in groups specially adapted for navigation in a road network
G01S 13/88 - Radar or analogous systems, specially adapted for specific applications
G01S 15/88 - Sonar systems specially adapted for specific applications
G01S 17/88 - Lidar systems, specially adapted for specific applications
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
58.
Approach for consolidating observed vehicle trajectories into a single representative trajectory
A method and apparatus is provided for controlling the operation of an autonomous vehicle. According to one aspect, the autonomous vehicle may track the trajectories of other vehicles on a road. Based on the other vehicle's trajectories, the autonomous vehicle may generate a pool of combined trajectories. Subsequently, the autonomous vehicle may select one of the combined trajectories as a representative trajectory. The representative trajectory may be used to change at least one of the speed or direction of the autonomous vehicle.
A laser diode firing circuit for a light detection and ranging device is disclosed. The firing circuit includes a laser diode coupled in series to a transistor, such that current through the laser diode is controlled by the transistor. The laser diode is configured to emit a pulse of light in response to current flowing through the laser diode. The firing circuit includes a capacitor that is configured to charge via a charging path that includes an inductor and to discharge via a discharge path that includes the laser diode. The transistor controlling current through the laser diode can be a Gallium nitride field effect transistor.
The technology relates to route planning and performing driving operations in autonomous vehicles, such as cargo trucks, articulating buses, as well as other vehicles. A detailed kinematic model of the vehicle in evaluated in conjunction with roadgraph and other information to determine whether a route or driving operation is feasible for the vehicle. This can include evaluating a hierarchical set of driving rules and whether current driving conditions impact any of the rules. Driving trajectories and cost can be evaluated when pre-planning a route for the vehicle to follow. This can include determining an ideal trajectory for the vehicle to take a particular driving action. Pre-planned routes may be shared with a fleet of vehicles, and can be modified based on information obtained by different vehicles of the fleet.
A system includes a dome, a wiper assembly, a position sensor and a control device. The wiper assembly includes a wiper blade configured to rotate around the dome. The position sensor may be configured to send a signal to a control device when a wiper blade passes the position sensor. The control device may include one or more processors configured to receive the signal from the position sensor and determine a location of the wiper blade relative to the dome based on the received signal.
G02B 27/00 - Optical systems or apparatus not provided for by any of the groups ,
H02P 7/06 - Arrangements for regulating or controlling the speed or torque of electric DC motors for regulating or controlling an individual dc dynamo-electric motor by varying field or armature current
63.
Pipeline Architecture for Road Sign Detection and Evaluation
The technology provides a sign detection and classification methodology. A unified pipeline approach incorporates generic sign detection with a robust parallel classification strategy. Sensor information such as camera imagery and lidar depth, intensity and height (elevation) information are applied to a sign detector module. This enables the system to detect the presence of a sign in a vehicle's external environment. A modular classification approach is applied to the detected sign. This includes selective application of one or more trained machine learning classifiers, as well as a text and symbol detector. Annotations help to tie the classification information together and to address any conflicts with different outputs from different classifiers. Identification of where the sign is in the vehicle's surrounding environment can provide contextual details. Identified signage can be associated with other objects in the vehicle's driving environment, which can be used to aid the vehicle in autonomous driving.
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
B60W 60/00 - Drive control systems specially adapted for autonomous road vehicles
Example embodiments relate to time-division multiple access scanning for crosstalk mitigation in light detection and ranging (lidar) devices. An example embodiment includes a method. The method includes emitting a first group of light signals into a surrounding environment. The first group of light signals corresponds to a first angular resolution. The method also includes detecting, during a first listening window, a first group of reflected light signals. Additionally, the method includes emitting a second group of light signals into the surrounding environment. The second group of light signals corresponds to a second angular resolution with respect to the surrounding environment. The second angular resolution is lower than the first angular resolution. Further, the method includes detecting a second group of reflected light signals from the surrounding environment. In addition, the method includes synthesizing, by a controller of the lidar device, a dataset usable to generate one or more point clouds.
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
B60W 60/00 - Drive control systems specially adapted for autonomous road vehicles
65.
Methods and Systems for using Interference to Detect Sensor Impairment
Example embodiments relate to methods and systems for using interference to detect sensor impairment. Radar or another type of sensor on a vehicle may receive radio-frequency (RF) signals propagating in the environment. These RF signals may originate from an external source and a computing device can be used to determine a distance and an angle to the source in order to identify a power level threshold that represents an expected power associated with the RF signals. The computing device may then perform a comparison between a power level of the RF signals and a power level threshold. Based on the comparison, the computing device may decrease a confidence assigned to the radar coupled to the vehicle and control the vehicle based on the decreased confidence assigned to the radar.
A rotary joint includes a shaft having a first end, a second end, and a cavity. The rotary joint includes a first waveguide section having a first proximal end and a first distal end. The first proximal end of the first waveguide section is positioned within the cavity and secured to the inner surface of the shaft. The rotary joint includes a second waveguide section that includes a second proximal end and a second distal end. The second proximal end of the second waveguide section is positioned within the cavity of the shaft and unsecured to the inner surface of the shaft to form a radial gap between an outer surface of the second proximal end and a laterally adjacent portion of the inner surface of the shaft. The shaft and the first waveguide section are configured to rotate about the rotational axis and relative to the second waveguide section.
Aspects of the present disclosure relate to a vehicle having one or more computing devices that may receive instructions to pick up a passenger at a location, determine when the vehicle is within a first distance of the location, provide a first notification that the vehicle is within the first distance, and stop the vehicle. When the vehicle is stopped, the computing device may initiate a countdown. When a client computing device associated with the passenger has not been authenticated, the computing devices may provide a second notification based on a first amount of time remaining in the countdown and a third notification indicating that the trip is cancelled based on a second amount of time remaining in the countdown less than the first amount of time. Once the third notification is provided, the computing devices move the vehicle from the where the vehicle is stopped without the passenger.
B60W 60/00 - Drive control systems specially adapted for autonomous road vehicles
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
E05F 15/70 - Power-operated mechanisms for wings with automatic actuation
E05F 15/76 - Power-operated mechanisms for wings with automatic actuation responsive to movement or presence of persons or objects responsive to devices carried by persons or objects, e.g. magnets or reflectors
B60R 25/24 - Means to switch the anti-theft system on or off using electronic identifiers containing a code not memorised by the user
G01C 21/20 - Instruments for performing navigational calculations
G07C 5/00 - Registering or indicating the working of vehicles
G01C 21/26 - Navigation; Navigational instruments not provided for in groups specially adapted for navigation in a road network
G01C 21/36 - Input/output arrangements for on-board computers
G06Q 10/1093 - Calendar-based scheduling for persons or groups
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
G08G 1/14 - Traffic control systems for road vehicles indicating individual free spaces in parking areas
The technology relates to an exterior sensor system for a vehicle configured to operate in an autonomous driving mode. The technology includes a close-in sensing (CIS) camera system to address blind spots around the vehicle. The CIS system is used to detect objects within a few meters of the vehicle. Based on object classification, the system is able to make real-time driving decisions. Classification is enhanced by employing cameras in conjunction with lidar sensors. The specific arrangement of multiple sensors in a single sensor housing is also important to object detection and classification. Thus, the positioning of the sensors and support components are selected to avoid occlusion and to otherwise prevent interference between the various sensor housing elements.
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G01S 17/86 - Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
B60W 60/00 - Drive control systems specially adapted for autonomous road vehicles
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
B60W 10/18 - Conjoint control of vehicle sub-units of different type or different function including control of braking systems
B60W 10/20 - Conjoint control of vehicle sub-units of different type or different function including control of steering systems
G01S 7/02 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group
G01S 7/481 - Constructional features, e.g. arrangements of optical elements
G01S 13/86 - Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
G01S 13/89 - Radar or analogous systems, specially adapted for specific applications for mapping or imaging
G01S 17/89 - Lidar systems, specially adapted for specific applications for mapping or imaging
G05D 1/02 - Control of position or course in two dimensions
H04N 23/74 - Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
70.
AUTONOMOUS VEHICLE SENSOR SECURITY, AUTHENTICATION AND SAFETY
A method includes obtaining, by a processing device, an impact analysis configuration related to an image sensor operation type for an autonomous vehicle (AV), receiving, by the processing device, image data from a sensing system including at least one image sensor of the AV, causing, by the processing device, fault detection to be performed based on the image data, causing, by the processing device, a fault notification to be generated using the impact analysis configuration, and sending, by the processing device to a data processing system of the AV, the fault notification to perform at least one action to address the fault notification. The fault notification includes a fault summary related to the image sensor operation type.
G06V 20/56 - Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
B60W 50/02 - Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
B60W 60/00 - Drive control systems specially adapted for autonomous road vehicles
G06V 10/75 - Image or video pattern matching; Proximity measures in feature spaces using context analysis; Selection of dictionaries
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
G06V 10/98 - Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing sensor data, e.g., laser sensor data, using neural networks. One of the methods includes obtaining a temporal sequence of multiple three-dimensional point clouds generated from sensor readings of an environment collected by one or more sensors within a given time period, each three-dimensional point cloud comprising a respective plurality of points in a first coordinate system; processing, using a feature extraction neural network, an input that comprises data derived from the temporal sequence of multiple three-dimensional point clouds to generate a feature embedding; receiving a query that specifies one time point within the given time period; and generating, from the feature embedding and conditioned on the query, one or more outputs that characterize one or more objects in the environment at the time point specified in the received query.
The present disclosure relates to systems and methods that facilitate light detection and ranging operations. An example method includes determining, for at least one light-emitter device of a plurality of light-emitter devices, a light pulse schedule. The plurality of light-emitter devices is operable to emit light along a plurality of emission vectors. The light pulse schedule is based on a respective emission vector of the at least one light-emitter device and a three-dimensional map of an external environment. The light pulse schedule includes at least one light pulse parameter and a listening window duration. The method also includes causing the at least one light-emitter device of the plurality of light-emitter devices to emit a light pulse according to the light pulse schedule. The light pulse interacts with an external environment.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predictability estimation using a behavior prediction model. One of the methods includes receiving a candidate future behavior to be performed by an agent in an environment after a current time point; receiving data characterizing a scene that includes the agent in the environment as of the current time point; processing a behavior prediction input generated from the data using a behavior prediction model, wherein the behavior prediction model is configured to receive the behavior prediction input and to process the behavior prediction input to generate a behavior prediction output that characterizes a set of predicted future behaviors for the agent after the current time point; and determining a predictability score for the candidate future behavior by comparing the candidate future behavior with the behavior prediction output.
B60W 50/00 - CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - Details of control systems for road vehicle drive control not related to the control of a particular sub-unit
74.
Modifying autonomous vehicle behavior prediction based on behavior prediction errors
A system includes a memory device and a processing device, operatively coupled to the memory device, to obtain a set of predicted behavior data including data indicative of one or more predicted object behaviors for one or more respective objects in an environment of an autonomous vehicle, obtain a set of observed behavior data including data indicative of one or more observed object behaviors of the one or more objects, determine, based on a comparison of the set of predicted behavior data and the set of observed behavior data, whether a set of incorrect object behavior predictions exists within the set of predicted behavior data, and upon determining that the set of incorrect object behavior predictions exists, initiate, based on at least a generated subset of the set of incorrect object behavior predictions, one or more operations to address the set of incorrect object behavior predictions.
One example device includes a rotor platform that rotates about an axis of rotation. The device also includes a rotor coil comprising a first plurality of conductive loops disposed along a planar mounting surface of the rotor platform. The device also includes a stator platform and a stator coil comprising a second plurality of conductive loops disposed along a planar mounting surface of the stator platform. The rotor coil and the stator coil are coaxially arranged about the axis of rotation. The stator coil remains within a first predetermined distance to the rotor coil in response to rotation of the rotor platform. The device also includes a magnetic core extending along the axis of rotation and through the stator coil. The magnetic core remains within a second predetermined distance to the stator coil in response to rotation of the rotor platform.
H02K 1/22 - Rotating parts of the magnetic circuit
H02K 3/28 - Layout of windings or of connections between windings
H02K 1/12 - Stationary parts of the magnetic circuit
H02K 11/20 - Structural association of dynamo-electric machines with electric components or with devices for shielding, monitoring or protection for measuring, monitoring, testing, protecting or switching
Aspects of the disclosure relate to determining a sign type of an unfamiliar sign. The system may include one or more processors. The one or more processors may be configured to receive an image and identify image data corresponding to a traffic sign in the image. The image data corresponding to the traffic sign may be input in a sign type model. The processors may determine that the sign type model was unable to identify a type of the traffic sign and determine one or more attributes of the traffic sign. The one or more attributes of the traffic sign may be compared to known attributes of other traffic signs and based on this comparison, a sign type of the traffic sign may be determined. The vehicle may be controlled in an autonomous driving mode based on the sign type of the traffic sign.
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G06T 7/90 - Determination of colour characteristics
G06V 20/20 - Scenes; Scene-specific elements in augmented reality scenes
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
77.
COMPUTING AGENT RESPONSE TIMES IN TRAFFIC SCENARIOS
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting agent response times. One of the methods includes continually updating, at each time step of a plurality of time steps, an accumulated measure of surprise for the agent due to the movements of another entity in the traffic environment. A distribution of previously predicted trajectories is obtained at a previous time step for the other entity in the environment. A measure of surprise is computed from the perspective of the agent. An accumulated measure of surprise is updated for the time step using the computed measure of surprise for the agent. If the accumulated measure of surprise crosses a threshold at a particular point in time, a predicted response time for the agent is generated based on the particular point in time that the accumulated measure of surprise crosses the threshold.
Example embodiments relate to identifying defects in optical detector systems based on extent of stray light. An example embodiment includes a method. The method includes capturing, using an optical detector system, an image of a scene that includes a bright object. The method also includes determining a location of the bright object within the image. Further, the method includes determining, based on the location of the bright object within the image, an extent of stray light from the bright object that is represented in the image. In addition, the method includes determining, by comparing the extent of stray light from the bright object that is represented in the image to a predetermined threshold extent of stray light, whether one or more defects are present within the optical detector system. The predetermined threshold extent of stray light corresponds to an expected extent of stray light.
The technology relates to keeping sensors of a perception system optically clear and free from condensation. A transparent film, such as Indium Tin Oxide (ITO), acts as a moisture sensor that covers the optical area of interest. When a measured value of the moisture sensor meets a certain threshold that indicates the presence of condensate, power is applied to the sensor, turning it into a heater. When the measured value no longer meets the threshold, power is removed and heating ceases. The ITO layer may be lithographically applied to a glass sensor cover or other window, with interleaved sections of material that are spaced closely to detect a minimum amount of condensate. This arrangement enables the system to be employed in sensor assemblies at various locations along a self-driving vehicle, and can be used with different types of sensors such as lidar sensors, cameras and other imaging devices.
H05B 3/84 - Heating arrangements specially adapted for transparent or reflecting areas, e.g. for demisting or de-icing windows, mirrors or vehicle windshields
H05B 3/34 - Heating elements having extended surface area substantially in a two-dimensional plane, e.g. plate-heater flexible, e.g. heating nets or webs
B60S 1/56 - Cleaning windscreens, windows, or optical devices specially adapted for cleaning other parts or devices than front windows or windscreens
B60H 1/00 - Heating, cooling or ventilating devices
B60S 1/02 - Cleaning windscreens, windows, or optical devices
Aspects of the present disclosure relate to a vehicle for maneuvering a passenger to a destination autonomously. The vehicle includes one or more computing devices and a set of user input buttons for communicating requests to stop the vehicle and to initiate a trip to the destination with the one or more computing devices. The set of user input buttons consisting essentially of a dual-purpose button and an emergency stopping button different from the dual-purpose button configured to stop the vehicle. The dual-purpose button has a first purpose for communicating a request to initiate the trip to the destination and a second purpose for communicating a request to pull the vehicle over and stop the vehicle. The vehicle has no steering wheel and no user inputs for the steering, acceleration, and deceleration of the vehicle other than the set of user input buttons.
G05D 1/02 - Control of position or course in two dimensions
B60K 28/02 - Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
82.
Light Detection and Ranging (LIDAR) Device Range Aliasing Resilience by Multiple Hypotheses
A computing system may operate a LIDAR device to emit light pulses in accordance with a time sequence including a time-varying dither. The system may then determine that the LIDAR detected return light pulses during corresponding detection periods for each of two or more emitted light pulses. Responsively, the system may determine that the detected return light pulses have (i) detection times relative to corresponding emission times of a plurality of first emitted light pulses that are indicative of a first set of ranges and (ii) detection times relative to corresponding emission times of a plurality of second emitted light pulses that are indicative of a second set of ranges. Given this, the system may select between using the first set of ranges as a basis for object detection and using the second set of ranges as a basis for object detection, and may then engage in object detection accordingly.
G01S 17/10 - Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
G01S 7/483 - RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - Details of systems according to groups , , of systems according to group - Details of pulse systems
G01S 17/89 - Lidar systems, specially adapted for specific applications for mapping or imaging
G01S 7/4865 - Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak
G01S 17/42 - Simultaneous measurement of distance and other coordinates
Methods and devices for actively modifying a field of view of an autonomous vehicle in view of constraints are disclosed. In one embodiment, an example method is disclosed that includes causing a sensor in an autonomous vehicle to sense information about an environment in a first field of view, where a portion of the environment is obscured in the first field of view. The example method further includes determining a desired field of view in which the portion of the environment is not obscured and, based on the desired field of view and a set of constraints for the vehicle, determining a second field of view in which the portion of the environment is less obscured than in the first field of view. The example method further includes modifying a position of the vehicle, thereby causing the sensor to sense information in the second field of view.
B60R 1/00 - Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
84.
Methods and Systems for using Remote Assistance to Maneuver an Autonomous Vehicle to a Location
Example embodiments relate to using remote assistance to maneuver an autonomous vehicle to a location. A computing device used by a remote operator may receive a request for assistance from a vehicle that indicates the vehicle is stopped at a first location with one or more navigation options for enabling the vehicle to navigate from the first location to a second location. At least one navigation option includes a maneuver technique that requires operator approval prior to execution. The computing device may then display a graphical user interface (GUI) that conveys the one or more navigation options. Based on detecting a selection of a particular navigation option, the computing device may transmit instructions to the vehicle to perform the particular navigation option. The vehicle may configured to navigate from the first location to the second location by performing the particular navigation option while monitoring for changes in the environment.
A method is provided that involves mounting a transmit block and a receive block in a LIDAR device to provide a relative position between the transmit block and the receive block. The method also involves locating a camera at a given position at which the camera can image light beams emitted by the transmit block and can image the receive block. The method also involves obtaining, using the camera, a first image indicative of light source positions of one or more light sources in the transmit block and a second image indicative of detector positions of one or more detectors in the receive block. The method also involves determining at least one offset based on the first image and the second image. The method also involves adjusting the relative position between the transmit block and the receive block based at least in part on the at least one offset.
Systems and methods are disclosed to identify a presence of a volumetric medium in an environment associated with a LIDAR system. In some implementations, the LIDAR system may emit a light pulse into the environment, receive a return light pulse corresponding to reflection of the emitted light pulse by a surface in the environment, and determine a pulse width of the received light pulse. The LIDAR system may compare the determined pulse width with a reference pulse width, and determine an amount of pulse elongation of the received light pulse. The LIDAR system may classify the surface as either an object to be avoided by a vehicle or as air particulates associated with the volumetric medium based, at least in part, on the determined amount of pulse elongation.
Assessing a likelihood of a person experiencing a fatigue event when the person tasked with monitoring a vehicle operating in an autonomous driving mode may include receiving a set of response times for a psychomotor vigilance test administered to the person. The test may include a plurality of trials which involve a person lifting a finger from a user input device. Whether the person passed or failed each trial of the set of trials may be determined. A model trained using data from prior psychomotor vigilance tests administered to the person may be identified for the person. Results of the determinations of whether the person passed or failed each trial of the set of trials may be input into the model in order to determine a value representative of a likelihood of a fatigue event. An intervention response may be initiated based on the value.
The present disclosure relates to multi-channel optical transmitter modules, lidar systems, and methods that involve micromirror devices. An example optical transmitter module includes at least one light-emitter device and a plurality of micromirror devices optically-coupled to the at least one light-emitter device. The at least one light-emitter device is configured to emit respective light beams toward an environment via the micromirror devices. The micromirror devices are configured to deflect the light beams. The optical transmitter module also includes a controller having at least one processor and a memory. The controller is configured to carry out operations. The operations include receiving information indicative of a retroreflector object in the environment. The operations include, based on the received information, causing at least one micromirror device of the plurality of micromirror devices to deflect at least one light beam so that the at least one light beam does not interact with the retroreflector object.
G02B 26/08 - Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system.
G06V 20/56 - Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
G07C 5/00 - Registering or indicating the working of vehicles
One example system comprises a light source configured to emit light. The system also comprises a waveguide configured to guide the emitted light from a first end of the waveguide toward a second end of the waveguide. The waveguide has an output surface between the first end and the second end. The system also comprises a plurality of mirrors including a first mirror and a second mirror. The first mirror reflects a first portion of the light toward the output surface. The second mirror reflects a second portion of the light toward the output surface. The first portion propagates out of the output surface toward a scene as a first transmitted light beam. The second portion propagates out of the output surface toward the scene as a second transmitted light beam.
Aspects of the disclosure provide for advanced trip planning for an autonomous vehicle service. For instance, an example method may include determining a potential pickup location for a user, determining a set of potential destination locations for a user, and determining a set of potential trips. For each potential trip a vehicle of a fleet of autonomous vehicles of the service may be assigned and trip information, including an estimated time of arrival for the assigned vehicle of the potential trip to reach the destination location of the potential trip, may be determined. The trip information for each potential trip may be provided for display to the user. Thereafter, confirmation information identifying one of the set of potential trips may be received, and the assigned vehicle for one first of the set of potential trips may be dispatched to pick up the user.
A camera module includes a housing with an opening and a portion that surrounds the opening, wherein the portion of the housing is transparent to near infrared (NIR) light. A fisheye lens is disposed within the opening such that a portion of the fisheye lens protrudes through the opening. An image sensor is disposed within the housing and optically coupled to the fisheye lens. The image sensor is sensitive to visible light and NIR light. A plurality of NIR light emitters is disposed within the housing. The NIR light emitters are configured to emit NIR light through the NIR-transparent portion of the housing. The NIR-transparent portion of the housing may include a light-diffusing structure, such as a pattern of microlenses formed on an inner surface of the NIR-transparent portion of the housing, to spread out the NIR light emitted by the NIR light emitters.
Aspects of the disclosure relate to controlling a vehicle in an autonomous driving mode using trajectories. For instance, a trajectory may be received by one or more first computing devices from one or more second computing devices. While the first computing devices are controlling the vehicle in the autonomous driving mode based on the trajectory, an error may be generated by second computing devices. Whether the error is a recoverable error may be determined, and if so, the second computing devices attempt to generate a new trajectory. When the second computing devices generate the new trajectory, the vehicle may be controlled by the first computing devices according to the new trajectory.
Aspects of the disclosure relate to detecting sidewalks adjacent to roads. In this regard, a set of potential sidewalk areas adjacent to one or more roads in a vehicle's vicinity may be determined based on map data. Topology data for the set of potential sidewalk areas may be generated based on sensor data received from a perception system of the vehicle. The set of potential sidewalks may be filtered to remove areas unlikely to include a sidewalk. The vehicle may be operated based on the filtered set of potential sidewalk areas, which may include taking precautionary measures when within a predetermined distance from any of the filtered set of potential sidewalks.
G05D 1/02 - Control of position or course in two dimensions
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
G06V 40/10 - Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
G06F 18/2113 - Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
G06F 18/214 - Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Example embodiments relate to vehicle sensor modules with external audio receivers. An example sensor module may include sensors and can be coupled to a vehicle's roof with a first microphone positioned proximate to the front of the sensor module. The sensor module can also include a second microphone extending into a first side of the sensor module such that the second microphone is configured to detect audio originating from an environment located relative to a first side of the vehicle and a third microphone extending into a second side of the sensor module such that the third microphone is configured to detect audio originating from the environment located relative to a second side of the vehicle, wherein the second side is opposite of the first side.
An autonomous vehicle includes sensor subsystem(s) that output a sensor signal. A perception subsystem (i) detects an agent in a vicinity of the autonomous vehicle and (ii) generates a motion signal that describes at least one of a past motion or a present motion of the agent. An intention prediction subsystem processes the sensor signal to generate an intention signal that describes at least one intended action of the agent. A behavior prediction subsystem processes the motion signal and the intention signal to generate a behavior prediction signal that describes at least one predicted behavior of the agent. A planner subsystem processes the behavior prediction signal to plan a driving decision for the autonomous vehicle.
The technology relates to enhancing the operation of autonomous vehicles. Extendible sensors are deployed based on detected or predicted conditions around a vehicle while operating in a self-driving mode. When not needed, the sensors are fully retracted into the vehicle to reduce drag and increase fuel economy. When the onboard system determines that there is a need for a deployable sensor, such as to enhance the field of view of the perception system, the sensor is extended in a predetermined manner. The deployment may depend on one or more operating conditions and/or particular driving scenarios. These and other sensors of the vehicle may be protected with a rugged housing, for instance to protect against damage from the elements. And in other situations, deployable foils may extend from the vehicle's chassis to increase drag and enhance braking. This may be helpful for large trucks in steep descent situations.
B60R 1/00 - Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
99.
MOTION PLANNING CONSTRAINTS FOR AUTONOMOUS VEHICLES
A method includes obtaining road patch type data associated with at least one road patch type, deriving, from the road patch type data, a set of road patch type parameters for the at least one road patch type, generating, based on the set of road patch type parameters, a set of risk metrics, each risk metric of the set of risk metrics corresponding to a respective road patch type parameter of the set of road patch type parameters, identifying, based at least in part on the set of risk metrics, a set of autonomous vehicle (AV) motion planning constraints selected for the at least one road patch type, and providing the set of AV motion planning constraints to update motion planning functionality performed by at least one component of an AV.
Aspects of the disclosure relate to providing information about a vehicle dispatched to pick up the user. In one example, a request for the vehicle to stop at a particular location is sent. In response, information identifying a current location of the vehicle is received. A map is generated. The map includes a first marker identifying the received location of the vehicle, a second marker identifying the particular location, and a shape defining an area around the second marker at which the vehicle may stop. The shape has an edge at least a minimum distance greater than zero from the second area. A route is displayed on the map between the first marker and the shape such that the route ends at the shape and does not reach the second marker. Progress of the vehicle towards the area along the route is displayed based on received updated location information for the vehicle.
patents
