DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 have been examined.
P = paragraph, e.g. p5 = paragraph 5.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being disclosed by Abari et al. 2019/0204425.
As per claims 1, 10 and 20, Abari discloses a method/apparatus/computer readable medium, comprising: sending identification information of a vehicle; receiving, based on the identification information, first motion track information of a mechanical arm for calibration of a first to-be-calibrated device in the vehicle (p’s 34, 53, ab; fig’s 1, 3, 6); and Abari discloses via fig 1:
PNG
media_image1.png
772
648
media_image1.png
Greyscale
controlling, based on the first motion track information, the mechanical arm to calibrate the first to-be-calibrated device (p’s 51, 50, 34, 36; fig’s 2, 5, 9).
Abari discloses via p34:
[0034] In particular embodiments, calibration facility 100 may be a facility dedicated to calibrating sensors of sensor array 144. Calibration facility 100 may include numerous calibration targets 102 that allow a vehicle to calibrate one or more of its sensors. As an example and not by way of limitation, calibration facility 100 may have a room or a track 106 with a variety of calibration targets 102 enabling AV 140 to simultaneously calibrate the sensors (e.g., optical cameras and LiDAR transceiver) of sensor array 144. In particular embodiments, AV 140 may be placed on a platform 104 for transporting AV 140 into one or more pre-determined positions relative to calibration targets 102. As an example and not by way of limitation, platform 104 may be configured to rotate, elevate, and/or move AV 140 along a path defined by a track 106, as well as laterally side to side, to position sensor array 144 at different vantage points to calibration targets 102. As an example and not by way of limitation, the path defined by track 106 may move to various positions within calibration facility 100 to position sensor array 144 in range of a particular set of calibration targets 102. At each pre-defined position, the particular set of calibration targets may be carefully laid out to facilitate a portion of the calibration routine when AV 140 is at a particular position, orientation, distance from each set of calibration targets 102. As described above, platform 104 may be configured to move AV 140 laterally, vertically, and angularly to properly and precisely orient sensor array 144 relative to the particular set of calibration targets 102. Calibration targets 102 may be placed throughout calibration facility 100 so that AV 140 may collect calibration data as it drives or is transported on platform 104 through calibration facility 100. For example, calibration targets 102 for camera calibration (e.g., checkerboard pattern) and LiDAR calibration (e.g., different 3D geometric shapes) may be held in place by rods hanging from the ceiling at different heights or orientation relative to track 106. In particular embodiments, calibration targets 102 may occupy a substantial portion of the space outside the path of movement defined by track 106.
As per claims 2 and 11, Abari discloses receiving, when completing calibration of the first to-be-calibrated device, second motion track information of the mechanical arm for calibration of a second to-be-calibrated device in the vehicle; and controlling, based on the second motion track information, the mechanical arm to calibrate the second to-be-calibrated device (p’s 47, 34, 53, ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p47:
[0047] If a confidence score (e.g., based on reprojection error) of sensors of sensor array 144 falls below a first threshold, a computing system of AV 140 may attempt to re-calibrate itself based on arbitrary surroundings, and may escalate to requesting a service vehicle 602 if the recalibration based on arbitrary surroundings does not produce sufficiently accurate results. AV 140 may not stop in response to determining a calibration issue, and instead automatically perform sensor recalibration based on the surroundings without interruption to driving. In particular embodiments, the confidence score may be based on detecting a point X in the environment of the AV as being at a pixel location A in an image from a first optical camera and at a pixel location B in an image from a second optical camera. Given a calibrated camera array, the two pixel coordinates (A and B) may be used to triangulate a 3-D coordinate of the point in the environment. Using the calibrated models, as described above, point X is projected back into the images from the first and second optical cameras, resulting in translated pixel coordinates A′ and B′, respectively. The reprojection error that is calculated for two optical cameras using the pixel coordinates from images obtained from the same scene may be approximated by the following equation:
As per claims 3 and 12, Abari discloses comprising controlling the vehicle to travel to a first work station, at which the first to-be-calibrated device is calibrated (ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via figure 5:
PNG
media_image2.png
966
584
media_image2.png
Greyscale
As per claims 4 and 13, Abari discloses sending first information about to-be-calibrated devices, wherein the to-be-calibrated devices comprise the first to-be-calibrated device, receiving traveling track information for traveling of the vehicle among work stations, wherein the work stations comprise the first work station; and further controlling, based on the traveling track information, the vehicle to travel to the first work station (fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein.
As per claims 5 and 14, Abari discloses wherein the first to-be-calibrated device comprises a camera, a head-up display, or a radar (p’s 1, 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab; fig’s 1, 3, 6) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p36:
[0036] FIG. 2A illustrates an example service facility with calibration targets. In other embodiments, service facility 200 may also provide other services for AV 140, such as for example services such as for example charging/refueling (depending on whether AV 140 is an electric or internal combustion vehicle) or maintenance of AV 140. Although this disclosure describes and illustrates a particular service facility configured to provide particular services for autonomous vehicles, this disclosure contemplates any suitable facility providing any suitable function for autonomous vehicles. Service facility 200 may be configured with a space equipped with a number of charging stations for charging multiple AVs 140. In particular embodiments, this space may include calibration targets 102 fixed in position through the use of rods. As an example and not by way of limitation, calibration targets 102 may be target for optical camera calibration (e.g., checkerboard pattern) and LiDAR calibration (e.g., different 3D geometric shapes) held in place by rods hanging from the ceiling at different heights or orientations throughout the space. extrinsic calibration, however, we need to have a few trihedral targets at very well-specified locations (for radars). In particular embodiments, IMUS may be calibrated by cross-checking the IMU of the sensor array with a well-calibrated reference IMU.
As per claims 6 and 15, Abari discloses handling an exception when the exception occurs in a process of calibrating the first to-be-calibrated device (p’s 47-48; fig’s 2, 5, 9; p’s 34, 53; ab; fig’s 1, 3, 6; p’s 51, 50, 36) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p’s 47-48:
[0047] If a confidence score (e.g., based on reprojection error) of sensors of sensor array 144 falls below a first threshold, a computing system of AV 140 may attempt to re-calibrate itself based on arbitrary surroundings, and may escalate to requesting a service vehicle 602 if the recalibration based on arbitrary surroundings does not produce sufficiently accurate results. AV 140 may not stop in response to determining a calibration issue, and instead automatically perform sensor recalibration based on the surroundings without interruption to driving. In particular embodiments, the confidence score may be based on detecting a point X in the environment of the AV as being at a pixel location A in an image from a first optical camera and at a pixel location B in an image from a second optical camera. Given a calibrated camera array, the two pixel coordinates (A and B) may be used to triangulate a 3-D coordinate of the point in the environment. Using the calibrated models, as described above, point X is projected back into the images from the first and second optical cameras, resulting in translated pixel coordinates A′ and B′, respectively. The reprojection error that is calculated for two optical cameras using the pixel coordinates from images obtained from the same scene may be approximated by the following equation:
[0048] A value of reprojection error larger than 1 may a sensor calibration issue. In some cases, a high value of the reprojection error may be an outlier caused by a high mismatch in pixel locations in the images from the first and second optical cameras (e.g., pixel location A does not correspond to the same point in the environment as pixel location B). In particular embodiments, a distribution of reprojection errors may be calculated a relatively large number of points in the environment. If only 10% of the matches between the different points in the environment and the corresponding point locations in the images of the first and second optical cameras are correct, then the value of the 10th percentile of reprojection errors should be less than 1.
As per claims 7 and 16, Abari discloses wherein the first to-be-calibrated device is a camera, and wherein handling the exception comprises: receiving, from the vehicle, an offset of a calibration board disposed at an end of the mechanical arm when image information from the camera does not comprise a complete image of the calibration board, wherein the image information is based on the mechanical arm being at a first position; and controlling, based on the offset, the mechanical arm to be adjusted from the first calibration position to a second calibration position (p’s 34, 53; ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p 51:
[0051] In particular embodiments, a computing system of service vehicle 602 may move the robotic arm in a prescribed path around sensor array 144 of AV 140. As an example and not by way of limitation, the robotic arm may move calibration target 102 in a spiral path about sensor array 144. In other examples, the robotic arm may move calibration target 102 in a zig-zag or randomized path in front of or around the sensor array. Although this disclosure describes moving a calibration target in particular types of paths, this disclosure contemplates moving a calibration target in any suitable path. The computing system of AV 140 may receive an indication that calibration target 102 of the service vehicle is ready for use and in a position where calibration target 102 may be detected by sensors (e.g., optical camera and LiDAR) of sensor array 144. In particular embodiments, AV 140 may initiate a sensor calibration routine to calibrate the sensors of sensor array 144 in response to receiving the data that calibration target 102 is ready for use. In particular embodiments, the computing system of AV 140 may receive data indicating that a robotic arm of service vehicle 602 holding calibration target 102 has moved to a calibration position relative to AV 140. The computing system of AV 140 may initiate and complete the calibration routine in response to receiving the data that calibration target 102 has moved into the calibration position. In particular embodiments, the computing system of AV 140 may send data informing a remote management system or a computing system associated with service vehicle 602 that calibration has been completed by AV 140. In other embodiments, AV's 140 calibration software may estimate the position of calibration target 102 whether calibration target 102 is held by a robotic arm on the service vehicle or service vehicle 602 (e.g., a drone).
As per claims 8 and 17, Abari discloses wherein the first to-be-calibrated device is a head-up display, wherein handling the exception comprises controlling the mechanical arm to be adjusted from a first calibration position that is comprised in the first motion track information to a second calibration position when image information from a camera that is disposed at an end of the mechanical arm does not comprise a complete image of the head-up display, and wherein the image information is based on the mechanical arm being at the first position (ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p51:
[0051] In particular embodiments, a computing system of service vehicle 602 may move the robotic arm in a prescribed path around sensor array 144 of AV 140. As an example and not by way of limitation, the robotic arm may move calibration target 102 in a spiral path about sensor array 144. In other examples, the robotic arm may move calibration target 102 in a zig-zag or randomized path in front of or around the sensor array. Although this disclosure describes moving a calibration target in particular types of paths, this disclosure contemplates moving a calibration target in any suitable path. The computing system of AV 140 may receive an indication that calibration target 102 of the service vehicle is ready for use and in a position where calibration target 102 may be detected by sensors (e.g., optical camera and LiDAR) of sensor array 144. In particular embodiments, AV 140 may initiate a sensor calibration routine to calibrate the sensors of sensor array 144 in response to receiving the data that calibration target 102 is ready for use. In particular embodiments, the computing system of AV 140 may receive data indicating that a robotic arm of service vehicle 602 holding calibration target 102 has moved to a calibration position relative to AV 140. The computing system of AV 140 may initiate and complete the calibration routine in response to receiving the data that calibration target 102 has moved into the calibration position. In particular embodiments, the computing system of AV 140 may send data informing a remote management system or a computing system associated with service vehicle 602 that calibration has been completed by AV 140. In other embodiments, AV's 140 calibration software may estimate the position of calibration target 102 whether calibration target 102 is held by a robotic arm on the service vehicle or service vehicle 602 (e.g., a drone).
As per claims 9 and 18, Abari discloses wherein before controlling the mechanical arm to calibrate the first to-be-calibrated device, the method further comprises: controlling the mechanical arm to move to a first initial position; obtaining a wheel arch measurement result of the vehicle; and controlling, based on the wheel arch measurement result, the mechanical arm to be adjusted from the first initial position to a second initial position (fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein.
As per claim 19, Abari discloses send, to the vehicle, request information requesting the identification information; and receive, from the vehicle and in response to the request information, the identification information (p’s 66, 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab; fig’s 1, 3, 6) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p66:
[0066] In particular embodiments, service vehicle 904 may include a network-addressable computing system that may control particular functionality, such as autonomous navigation or control of a calibration platform, as described above. Service vehicle 904 may be configured to receive data from one or more sensors from an autonomous vehicle 940 or its ride-service computing device 948, or one or more sensors of the calibration platform. Service vehicle 904 may be accessed by the other computing entities of the network environment either directly or via network 910. For example, transport management system 960 may communicate with service vehicle 904 via a network (e.g., Internet). The transportation management system 960 may provide instructions to the service vehicle 904, such as instructing the vehicle 904 or its driver where to drive to, when and how to prepare its calibration targets, the location and/or calibration needs of the autonomous vehicle 940, etc. In particular embodiments, the service vehicle 904 may be equipped with a processing unit (e.g., one or more CPUs and GPUs), memory, and storage. Service vehicle 904 may be equipped to perform a variety of computational and processing tasks, including processing the sensor data, extracting useful information, and operating accordingly. For example, based on images captured by its cameras and a machine-vision model, service vehicle 904 may determine a driving route to a location of vehicle 940 and control one or more functions of the calibration platform. In particular embodiments, the service vehicle 904 may communicate with the autonomous vehicle 940 over direct short-range wireless connection (e.g., WI-FI, Bluetooth, NFC) and/or over a network (e.g., the Internet or via the transportation management system 960). Communication between the service vehicle 904 and autonomous vehicle 940 may include, for example, relative positioning information between the vehicles, current configuration (e.g., position, orientation) of the calibration targets, calibration status, and any other information that would facilitate coordination of the calibration process.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Abari et al. 2019/0204425, and further in view of Lawrence et al. USPAP 2020/0141724.
As per claims 1, 10 and 20, Abari discloses a method/apparatus/computer readable medium, comprising: sending identification information of a vehicle; receiving, based on the identification information, first motion track information of a mechanical arm for calibration of a first to-be-calibrated device in the vehicle (p’s 34, 53, ab; fig’s 1, 3, 6); and Abari discloses via fig 1:
PNG
media_image1.png
772
648
media_image1.png
Greyscale
controlling, based on the first motion track information, the mechanical arm to calibrate the first to-be-calibrated device (p’s 51, 50, 34, 36; fig’s 2, 5, 9).
Abari discloses all the limitations of the invention, however, arguendo, if Abari is or might be interpreted such that it might not explicitly disclose controlling a mechanical arm, then Lawrence discloses controlling a mechanical arm (p’s 83-84; ab; p’s 37, 56, 81; fig’s 1, 6, 7, 10, 13, 14, 18). If this interpretation is taken, then it would have been obvious, before the effective filing date of the claimed invention, to modify Abari to include controlling a mechanical arm such as that taught by Lawrence in order to have the arms 189 extending between a moveable rail, such as rail 194, and a target, such as target 188. As such, the alignment and calibration system 20 may be used to position alternative targets about vehicle 22 (Lawrence, p84).
Lawrence discloses via figure 14:
PNG
media_image3.png
915
691
media_image3.png
Greyscale
Abari discloses via p34:
[0034] In particular embodiments, calibration facility 100 may be a facility dedicated to calibrating sensors of sensor array 144. Calibration facility 100 may include numerous calibration targets 102 that allow a vehicle to calibrate one or more of its sensors. As an example and not by way of limitation, calibration facility 100 may have a room or a track 106 with a variety of calibration targets 102 enabling AV 140 to simultaneously calibrate the sensors (e.g., optical cameras and LiDAR transceiver) of sensor array 144. In particular embodiments, AV 140 may be placed on a platform 104 for transporting AV 140 into one or more pre-determined positions relative to calibration targets 102. As an example and not by way of limitation, platform 104 may be configured to rotate, elevate, and/or move AV 140 along a path defined by a track 106, as well as laterally side to side, to position sensor array 144 at different vantage points to calibration targets 102. As an example and not by way of limitation, the path defined by track 106 may move to various positions within calibration facility 100 to position sensor array 144 in range of a particular set of calibration targets 102. At each pre-defined position, the particular set of calibration targets may be carefully laid out to facilitate a portion of the calibration routine when AV 140 is at a particular position, orientation, distance from each set of calibration targets 102. As described above, platform 104 may be configured to move AV 140 laterally, vertically, and angularly to properly and precisely orient sensor array 144 relative to the particular set of calibration targets 102. Calibration targets 102 may be placed throughout calibration facility 100 so that AV 140 may collect calibration data as it drives or is transported on platform 104 through calibration facility 100. For example, calibration targets 102 for camera calibration (e.g., checkerboard pattern) and LiDAR calibration (e.g., different 3D geometric shapes) may be held in place by rods hanging from the ceiling at different heights or orientation relative to track 106. In particular embodiments, calibration targets 102 may occupy a substantial portion of the space outside the path of movement defined by track 106.
As per claims 2 and 11, Abari discloses receiving, when completing calibration of the first to-be-calibrated device, second motion track information of the mechanical arm for calibration of a second to-be-calibrated device in the vehicle; and controlling, based on the second motion track information, the mechanical arm to calibrate the second to-be-calibrated device (p’s 47, 34, 53, ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p47:
[0047] If a confidence score (e.g., based on reprojection error) of sensors of sensor array 144 falls below a first threshold, a computing system of AV 140 may attempt to re-calibrate itself based on arbitrary surroundings, and may escalate to requesting a service vehicle 602 if the recalibration based on arbitrary surroundings does not produce sufficiently accurate results. AV 140 may not stop in response to determining a calibration issue, and instead automatically perform sensor recalibration based on the surroundings without interruption to driving. In particular embodiments, the confidence score may be based on detecting a point X in the environment of the AV as being at a pixel location A in an image from a first optical camera and at a pixel location B in an image from a second optical camera. Given a calibrated camera array, the two pixel coordinates (A and B) may be used to triangulate a 3-D coordinate of the point in the environment. Using the calibrated models, as described above, point X is projected back into the images from the first and second optical cameras, resulting in translated pixel coordinates A′ and B′, respectively. The reprojection error that is calculated for two optical cameras using the pixel coordinates from images obtained from the same scene may be approximated by the following equation:
As per claims 3 and 12, Abari discloses comprising controlling the vehicle to travel to a first work station, at which the first to-be-calibrated device is calibrated (ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via figure 5:
PNG
media_image2.png
966
584
media_image2.png
Greyscale
As per claims 4 and 13, Abari discloses sending first information about to-be-calibrated devices, wherein the to-be-calibrated devices comprise the first to-be-calibrated device, receiving traveling track information for traveling of the vehicle among work stations, wherein the work stations comprise the first work station; and further controlling, based on the traveling track information, the vehicle to travel to the first work station (fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein.
As per claims 5 and 14, Abari discloses wherein the first to-be-calibrated device comprises a camera, a head-up display, or a radar (p’s 1, 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab; fig’s 1, 3, 6) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p36:
[0036] FIG. 2A illustrates an example service facility with calibration targets. In other embodiments, service facility 200 may also provide other services for AV 140, such as for example services such as for example charging/refueling (depending on whether AV 140 is an electric or internal combustion vehicle) or maintenance of AV 140. Although this disclosure describes and illustrates a particular service facility configured to provide particular services for autonomous vehicles, this disclosure contemplates any suitable facility providing any suitable function for autonomous vehicles. Service facility 200 may be configured with a space equipped with a number of charging stations for charging multiple AVs 140. In particular embodiments, this space may include calibration targets 102 fixed in position through the use of rods. As an example and not by way of limitation, calibration targets 102 may be target for optical camera calibration (e.g., checkerboard pattern) and LiDAR calibration (e.g., different 3D geometric shapes) held in place by rods hanging from the ceiling at different heights or orientations throughout the space. extrinsic calibration, however, we need to have a few trihedral targets at very well-specified locations (for radars). In particular embodiments, IMUS may be calibrated by cross-checking the IMU of the sensor array with a well-calibrated reference IMU.
As per claims 6 and 15, Abari discloses handling an exception when the exception occurs in a process of calibrating the first to-be-calibrated device (p’s 47-48; fig’s 2, 5, 9; p’s 34, 53; ab; fig’s 1, 3, 6; p’s 51, 50, 36) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p’s 47-48:
[0047] If a confidence score (e.g., based on reprojection error) of sensors of sensor array 144 falls below a first threshold, a computing system of AV 140 may attempt to re-calibrate itself based on arbitrary surroundings, and may escalate to requesting a service vehicle 602 if the recalibration based on arbitrary surroundings does not produce sufficiently accurate results. AV 140 may not stop in response to determining a calibration issue, and instead automatically perform sensor recalibration based on the surroundings without interruption to driving. In particular embodiments, the confidence score may be based on detecting a point X in the environment of the AV as being at a pixel location A in an image from a first optical camera and at a pixel location B in an image from a second optical camera. Given a calibrated camera array, the two pixel coordinates (A and B) may be used to triangulate a 3-D coordinate of the point in the environment. Using the calibrated models, as described above, point X is projected back into the images from the first and second optical cameras, resulting in translated pixel coordinates A′ and B′, respectively. The reprojection error that is calculated for two optical cameras using the pixel coordinates from images obtained from the same scene may be approximated by the following equation:
[0048] A value of reprojection error larger than 1 may a sensor calibration issue. In some cases, a high value of the reprojection error may be an outlier caused by a high mismatch in pixel locations in the images from the first and second optical cameras (e.g., pixel location A does not correspond to the same point in the environment as pixel location B). In particular embodiments, a distribution of reprojection errors may be calculated a relatively large number of points in the environment. If only 10% of the matches between the different points in the environment and the corresponding point locations in the images of the first and second optical cameras are correct, then the value of the 10th percentile of reprojection errors should be less than 1.
As per claims 7 and 16, Abari discloses wherein the first to-be-calibrated device is a camera, and wherein handling the exception comprises: receiving, from the vehicle, an offset of a calibration board disposed at an end of the mechanical arm when image information from the camera does not comprise a complete image of the calibration board, wherein the image information is based on the mechanical arm being at a first position; and controlling, based on the offset, the mechanical arm to be adjusted from the first calibration position to a second calibration position (p’s 34, 53; ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p 51:
[0051] In particular embodiments, a computing system of service vehicle 602 may move the robotic arm in a prescribed path around sensor array 144 of AV 140. As an example and not by way of limitation, the robotic arm may move calibration target 102 in a spiral path about sensor array 144. In other examples, the robotic arm may move calibration target 102 in a zig-zag or randomized path in front of or around the sensor array. Although this disclosure describes moving a calibration target in particular types of paths, this disclosure contemplates moving a calibration target in any suitable path. The computing system of AV 140 may receive an indication that calibration target 102 of the service vehicle is ready for use and in a position where calibration target 102 may be detected by sensors (e.g., optical camera and LiDAR) of sensor array 144. In particular embodiments, AV 140 may initiate a sensor calibration routine to calibrate the sensors of sensor array 144 in response to receiving the data that calibration target 102 is ready for use. In particular embodiments, the computing system of AV 140 may receive data indicating that a robotic arm of service vehicle 602 holding calibration target 102 has moved to a calibration position relative to AV 140. The computing system of AV 140 may initiate and complete the calibration routine in response to receiving the data that calibration target 102 has moved into the calibration position. In particular embodiments, the computing system of AV 140 may send data informing a remote management system or a computing system associated with service vehicle 602 that calibration has been completed by AV 140. In other embodiments, AV's 140 calibration software may estimate the position of calibration target 102 whether calibration target 102 is held by a robotic arm on the service vehicle or service vehicle 602 (e.g., a drone).
As per claims 8 and 17, Abari discloses wherein the first to-be-calibrated device is a head-up display, wherein handling the exception comprises controlling the mechanical arm to be adjusted from a first calibration position that is comprised in the first motion track information to a second calibration position when image information from a camera that is disposed at an end of the mechanical arm does not comprise a complete image of the head-up display, and wherein the image information is based on the mechanical arm being at the first position (ab; fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p51:
[0051] In particular embodiments, a computing system of service vehicle 602 may move the robotic arm in a prescribed path around sensor array 144 of AV 140. As an example and not by way of limitation, the robotic arm may move calibration target 102 in a spiral path about sensor array 144. In other examples, the robotic arm may move calibration target 102 in a zig-zag or randomized path in front of or around the sensor array. Although this disclosure describes moving a calibration target in particular types of paths, this disclosure contemplates moving a calibration target in any suitable path. The computing system of AV 140 may receive an indication that calibration target 102 of the service vehicle is ready for use and in a position where calibration target 102 may be detected by sensors (e.g., optical camera and LiDAR) of sensor array 144. In particular embodiments, AV 140 may initiate a sensor calibration routine to calibrate the sensors of sensor array 144 in response to receiving the data that calibration target 102 is ready for use. In particular embodiments, the computing system of AV 140 may receive data indicating that a robotic arm of service vehicle 602 holding calibration target 102 has moved to a calibration position relative to AV 140. The computing system of AV 140 may initiate and complete the calibration routine in response to receiving the data that calibration target 102 has moved into the calibration position. In particular embodiments, the computing system of AV 140 may send data informing a remote management system or a computing system associated with service vehicle 602 that calibration has been completed by AV 140. In other embodiments, AV's 140 calibration software may estimate the position of calibration target 102 whether calibration target 102 is held by a robotic arm on the service vehicle or service vehicle 602 (e.g., a drone).
As per claims 9 and 18, Abari discloses wherein before controlling the mechanical arm to calibrate the first to-be-calibrated device, the method further comprises: controlling the mechanical arm to move to a first initial position; obtaining a wheel arch measurement result of the vehicle; and controlling, based on the wheel arch measurement result, the mechanical arm to be adjusted from the first initial position to a second initial position (fig’s 1, 3, 6; p’s 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein.
As per claim 19, Abari discloses send, to the vehicle, request information requesting the identification information; and receive, from the vehicle and in response to the request information, the identification information (p’s 66, 51, 50, 36; fig’s 2, 5, 9; p’s 34, 53; ab; fig’s 1, 3, 6) as per the discussion above and the rejection of corresponding parts of the claims above incorporated herein and further, Abari discloses via p66:
[0066] In particular embodiments, service vehicle 904 may include a network-addressable computing system that may control particular functionality, such as autonomous navigation or control of a calibration platform, as described above. Service vehicle 904 may be configured to receive data from one or more sensors from an autonomous vehicle 940 or its ride-service computing device 948, or one or more sensors of the calibration platform. Service vehicle 904 may be accessed by the other computing entities of the network environment either directly or via network 910. For example, transport management system 960 may communicate with service vehicle 904 via a network (e.g., Internet). The transportation management system 960 may provide instructions to the service vehicle 904, such as instructing the vehicle 904 or its driver where to drive to, when and how to prepare its calibration targets, the location and/or calibration needs of the autonomous vehicle 940, etc. In particular embodiments, the service vehicle 904 may be equipped with a processing unit (e.g., one or more CPUs and GPUs), memory, and storage. Service vehicle 904 may be equipped to perform a variety of computational and processing tasks, including processing the sensor data, extracting useful information, and operating accordingly. For example, based on images captured by its cameras and a machine-vision model, service vehicle 904 may determine a driving route to a location of vehicle 940 and control one or more functions of the calibration platform. In particular embodiments, the service vehicle 904 may communicate with the autonomous vehicle 940 over direct short-range wireless connection (e.g., WI-FI, Bluetooth, NFC) and/or over a network (e.g., the Internet or via the transportation management system 960). Communication between the service vehicle 904 and autonomous vehicle 940 may include, for example, relative positioning information between the vehicles, current configuration (e.g., position, orientation) of the calibration targets, calibration status, and any other information that would facilitate coordination of the calibration process.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Erdei et al. (U.S. patent application publication 2021/0375001) discloses
calibrating at least one sensor, in particular an environment sensor, of a vehicle, in which at least one calibration object is used that is located at a distance from the vehicle and is detectable by the sensor. In the method, the position of the calibration object with respect to the sensor of the vehicle is changed by moving the calibration object (2) or the vehicle. Following a change in position, the new position data are detected with the aid of the sensor, on the one hand, and with the aid of at least one external camera, on the other hand, and reconciled. A device for carrying out the method is also described.
Wiesenberg (U.S. patent application publication 2021/0215506) discloses
determining a test vehicle in motion is proximate to a transport-under-test in motion, wherein the test vehicle includes a calibration device, and the transport-under-test includes one or more sensors, transmitting calibration results from the calibration device to the one or more sensors, receiving a calibration result, via the calibration device, from the one or more sensors, determining, via the test vehicle, an error with the one or more sensors based on the calibration result and calibrating, via the transport-under-test, the one or more sensors based on the error.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEHRANG BADII whose telephone number is (571)272-6879. The examiner can normally be reached Monday-Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at 571-272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Behrang Badii/
Primary Examiner
Art Unit 3665
Prosecution Insights