DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Claims
Claims 1-20, as originally filed, are currently pending and have been considered below.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3 and 14-18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mu et al., Chinese Patent Publication No. CN109726692A, hereinafter, “Mu”.
As per claim 1, Mu discloses a three-dimensional (3D) object detection correction apparatus comprising:
a processor configured to detect a 3D object on image data and to correct an error of the detected 3D object based on a difference value of camera information of a vehicle (Mu, ¶0038, the high-definition camera is equipped with sensing sensors arranged to view or capture images of selected portions of the vehicle's environment; Mu, ¶0039, the sensors of the high-definition camera are configured to send corresponding sensing signals to the central processing unit when the vehicle encounters road features; Mu, ¶0045-0052, A preferred method for using a deep learning-based high-definition camera 3D object detection system includes the following steps: (1) Image acquisition: Obtain the original input image through a high-definition camera; (2) Feature extraction: The feature extraction unit uses the residual network feature extraction method to extract multi-scale feature maps from the input original image; (3) Orthogonal projection transformation: Orthogonal feature transformation is used to convert the feature maps of all extracted multi-scale feature maps into orthogonal bird's-eye view maps, forming residual network units; (4) Construct a multi-layer network: Combine these processed residual network units into a multi-layer network from top to bottom; (5) Localization and boundary estimation: Generate a confidence score for the projection of each object in each layer of the multilayer network onto the ground plane, while introducing position offset, dimension offset and direction vector; (6) Final localization: The non-maximum suppression algorithm is used to identify the peaks in the confidence map and generate discrete bounding boxes to improve the accuracy of object position and bounding box data; (7) Output data to the terminal); and
a storage device configured to store data and algorithms executable by the processor (Mu, ¶0010, The hardware module includes a power supply circuit, a communication circuit, a data format conversion circuit, a central processing unit, and a storage circuit. The high-definition camera is connected to the central processing unit via a circuit; Mu, ¶0042, the storage circuit is a memory, which is electrically connected to the central processing unit; Mu, ¶0061, The storage circuit is a memory that exchanges data with the CPU; Mu, ¶0062, The feature extraction unit obtains the original input image from the image acquisition unit and generates a multi-scale two-dimensional feature map. The image processing unit performs target recognition based on the three-dimensional target recognition algorithm of the high-definition camera. The result output unit is used to transmit the final information to the autonomous driving terminal).
As per claim 3, Mu discloses the 3D object detection correction apparatus of claim 1, wherein the processor is configured to correct a position or angle of the detected 3D object (Mu, ¶0050-0051, (5) Localization and boundary estimation: Generate a confidence score for the projection of each object in each layer of the multilayer network onto the ground plane, while introducing position offset, dimension offset and direction vector; (6) Final localization: The non-maximum suppression algorithm is used to identify the peaks in the confidence map and generate discrete bounding boxes to improve the accuracy of object position and bounding box data).
As per claim 14, Mu discloses the 3D object detection correction apparatus of claim 1, wherein the storage device is configured to store information of a first camera of a first vehicle type, a network learned based on image data of the first camera, and camera information of a host vehicle (Mu, ¶0008, a high-definition camera 3D target detection system based on deep learning, comprising a vehicle sensor device, a hardware module, and a data processing module; Mu, ¶0010, The hardware module includes a power supply circuit, a communication circuit, a data format conversion circuit, a central processing unit, and a storage circuit. The high-definition camera is connected to the central processing unit via a circuit; Mu, ¶0042, the storage circuit is a memory, which is electrically connected to the central processing unit; Mu, ¶0045-0052; Mu, ¶0053, the image acquisition unit of the deep learning-based high-definition camera 3D target detection system mainly acquires image information through a high-definition camera, the model training unit evaluates and corrects the image recognition model by introducing an autonomous driving dataset, the image processing unit performs target recognition according to the high-definition camera 3D target recognition algorithm, and the data module is used to transmit the final obtained information to the autonomous driving terminal; Mu, ¶0061, The storage circuit is a memory that exchanges data with the CPU).
As per claim 15, Mu discloses the 3D object detection correction apparatus of claim 1, wherein the processor is configured to: generate a 3D bounding box including the 3D object by detecting the 3D object; and correct a position and angle of the 3D object by rotating the 3D bounding box (Mu, ¶0021, In the formula: B(x, z) is the projection of the three-dimensional mapping onto the ground plane; Mu, ¶0027, In the formula: N(x,z) is a smooth function that represents the probability of the existence of a bounding box centered at (x, y0, z), where y0 is the distance from the HD camera to the ground plane, and δ is the scaling factor; Mu, ¶0028-0030. Relative position offset Δpos: Δpos(x,z) represents the relative deviation between the bounding box center coordinates (x, y0, z) and the actual position of the target, and Δdim (xi, yi, zi) is the center coordinate of object I; Mu, ¶0050, (5) Localization and boundary estimation: Generate a confidence score for the projection of each object in each layer of the multilayer network onto the ground plane, while introducing position offset, dimension offset and direction vector; Mu, ¶0051, (6) Final localization: The non-maximum suppression algorithm is used to identify the peaks in the confidence map and generate discrete bounding boxes to improve the accuracy of object position and bounding box data).
As per claim 16, Mu discloses a three-dimensional (3D) object detection correction method comprising:
detecting, by a processor, a 3D object based on image data (Mu, ¶0010, The hardware module includes a power supply circuit, a communication circuit, a data format conversion circuit, a central processing unit, and a storage circuit. The high-definition camera is connected to the central processing unit via a circuit; Mu, ¶0038, the high-definition camera is equipped with sensing sensors arranged to view or capture images of selected portions of the vehicle's environment; Mu, ¶0039, the sensors of the high-definition camera are configured to send corresponding sensing signals to the central processing unit when the vehicle encounters road features); and
correcting, by the processor, an error of the detected 3D object based on a difference value of camera information of a vehicle (Mu, ¶0038, the high-definition camera is equipped with sensing sensors arranged to view or capture images of selected portions of the vehicle's environment; Mu, ¶0039, the sensors of the high-definition camera are configured to send corresponding sensing signals to the central processing unit when the vehicle encounters road features; Mu, ¶0045-0052, A preferred method for using a deep learning-based high-definition camera 3D object detection system includes the following steps: (1) Image acquisition: Obtain the original input image through a high-definition camera; (2) Feature extraction: The feature extraction unit uses the residual network feature extraction method to extract multi-scale feature maps from the input original image; (3) Orthogonal projection transformation: Orthogonal feature transformation is used to convert the feature maps of all extracted multi-scale feature maps into orthogonal bird's-eye view maps, forming residual network units; (4) Construct a multi-layer network: Combine these processed residual network units into a multi-layer network from top to bottom; (5) Localization and boundary estimation: Generate a confidence score for the projection of each object in each layer of the multilayer network onto the ground plane, while introducing position offset, dimension offset and direction vector; (6) Final localization: The non-maximum suppression algorithm is used to identify the peaks in the confidence map and generate discrete bounding boxes to improve the accuracy of object position and bounding box data; (7) Output data to the terminal).
As per claim 17, Mu discloses the 3D object detection correction method of claim 16, wherein detecting the 3D object includes:
generating, by the processor, a first feature of image data of a first camera (Mu, ¶0012, The feature extraction unit obtains the original input image from the image acquisition unit and generates a multi-scale two-dimensional feature map);
generating, by the processor, a second feature using information of the first camera (Mu, ¶0013, The feature extraction unit generates a multi-scale two-dimensional feature map from the original input image, represented by a plane f(u,v)∈Rn, where Rn represents an n-dimensional space; where (u,v) are feature points on the plane of this two-dimensional feature map);
generating, by the processor, a third feature by combining the first feature and the second feature (Mu, ¶0014, The orthographic projection bird's-eye view transformation converts f(u,v)∈Rn into a three-dimensional feature map, denoted by s(x,y,z)∈Rn, where (x,y,z) are points in three-dimensional space; Mu, ¶0017, Average the bounding box of the image f(a,b)∈Rn projection, and assign each feature to the appropriate position in s(x,y,z)∈Rn; Mu, ¶0019, The generated 3D feature map g(x, y, z) is multiplied by a set of learned weight matrices M(y) and s(x, y, z), and then accumulated along the vertical axis to obtain an orthogonal feature map; Mu, ¶0021, In the formula: B(x, z) is the projection of the three-dimensional mapping onto the ground plane; Mu, ¶0012, The image processing unit performs target recognition based on the three-dimensional target recognition algorithm of the high-definition camera); and
detecting, by the processor, a 3D object by using the third feature (Mu, ¶0012, The image processing unit performs target recognition based on the three-dimensional target recognition algorithm).
As per claim 18, Mu discloses the 3D object detection correction method of claim 17, wherein: detecting the 3D object further includes generating, by the processor, a 3D bounding box including the 3D object by detecting the 3D object; and correcting of the error of the detected 3D object includes correcting, by the processor, a position and angle of the 3D object by rotating the 3D bounding box (Mu, ¶0021, In the formula: B(x, z) is the projection of the three-dimensional mapping onto the ground plane; Mu, ¶0027, In the formula: N(x,z) is a smooth function that represents the probability of the existence of a bounding box centered at (x,y0,z), where y0 is the distance from the HD camera to the ground plane, and δ is the scaling factor; Mu, ¶0028-0030. Relative position offset Δpos: Δpos(x,z) represents the relative deviation between the bounding box center coordinates (x, y0, z) and the actual position of the target, and Δdim (xi, yi, zi) is the center coordinate of object I; Mu, ¶0050, (5) Localization and boundary estimation: Generate a confidence score for the projection of each object in each layer of the multilayer network onto the ground plane, while introducing position offset, dimension offset and direction vector; Mu, ¶0051, (6) Final localization: The non-maximum suppression algorithm is used to identify the peaks in the confidence map and generate discrete bounding boxes to improve the accuracy of object position and bounding box data).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2, 4-13, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mu et al., Chinese Patent Publication No. CN109726692A, hereinafter, “Mu” as applied to claims 1 and 16 above, and further in view of Lai et al., Chinese Patent Publication No. CN114723820A, hereinafter, “Lai”.
As per claim 2, Mu discloses the 3D object detection correction apparatus of claim 1, wherein the camera information of the vehicle includes a camera position or a camera angle (Mu, ¶0043, the high-definition cameras are in multiple sets, respectively installed at the central axis of the rearview mirror, on the rear shell of the trunk, and on both sides of the vehicle body).
Mu does not explicitly disclose the following limitation as further recited however Lai discloses
for a particular vehicle type (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0076, the positions of the cameras installed on the first and second vehicle models are different. Therefore, the deviation between the installation positions of the cameras on the first and second vehicle models can be obtained based on the installation positions of the cameras on the first and second vehicle models).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Lai and Mu because they are in the same field of invention. One skilled in the art would have been motivated to include the installation of positions of cameras on different vehicle models or types as taught by Lai in the system of Mu in order to improve the training and performance of driver assistance systems (Lai, ¶0001-0002).
As per claim 4, Mu discloses the 3D object detection correction apparatus of claim 1, wherein the processor is configured to correct position and angle errors of the detected 3D object by using pose information of 6 degrees of freedom (DOF) in relation to a ground (Mu, ¶0050, (5) Localization and boundary estimation: Generate a confidence score for the projection of each object in each layer of the multilayer network onto the ground plane, while introducing position offset, dimension offset and direction vector; Mu, ¶0071, In the formula: B(x, z) is the projection of the three-dimensional mapping onto the ground plane).
Mu does not explicitly disclose the following limitations as further recited however Lai discloses
between a first camera of a first vehicle type and a second camera of a second type (Lai, Figure 6; Lai, ¶0099, In one embodiment, step 205 involves performing coordinate transformation on the pixel positions in the first image data based on the first and second extrinsic parameter matrices to obtain the second image data, as shown in Figure 4. This specifically includes the following steps; Lai, ¶0102, Step 406: Determine the coordinate transformation relationship based on the first position coordinates and the second position coordinates; Lai, ¶0103, Step 408: Based on the coordinate transformation relationship, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0104, refer to Figures 5 and 6. The extrinsic parameter matrix describes how real-world coordinate points are transferred to the camera coordinates through rotation and translation; Lai, ¶0106, in this embodiment, the first extrinsic parameter matrix is the extrinsic parameter matrix corresponding to the installation position of the camera on the first vehicle model, and the second extrinsic parameter matrix is the extrinsic parameter matrix corresponding to the installation position of the camera on the second vehicle model. Therefore, based on the difference between the first extrinsic parameter matrix and the second extrinsic parameter matrix, this embodiment can perform coordinate transformation on the pixel positions in the first image data to obtain the second image data).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Lai and Mu because they are in the same field of endeavor. One skilled in the art would have been motivated to include the rotation and translation of image pixel positions between vehicle types as taught by Lai in the system of Mu in order to improve training of an assisted driving system and detection information of traffic participants and the environment (Lai, ¶0071).
As per claim 5, Mu and Lai disclose the 3D object detection correction apparatus of claim 4, wherein the processor is configured to detect the 3D object using image data of the first camera of the first vehicle type and information of the first camera using an artificial neural network (Mu, ¶0045, using a deep learning-based high-definition camera 3D object detection system; Mu, ¶0063, this algorithm uses the convolutional feature extraction method through the feature extraction unit to generate a multi-scale two-dimensional feature map from the original input image).
As per claim 6, Mu and Lai disclose the 3D object detection correction apparatus of claim 5, wherein the processor is configured to implement:
a backbone for generating a first feature of the image data of the first camera (Mu, ¶0012, The feature extraction unit obtains the original input image from the image acquisition unit and generates a multi-scale two-dimensional feature map);
an embedding network for generating a second feature using the information of the first camera (Mu, ¶0013, The feature extraction unit generates a multi-scale two-dimensional feature map from the original input image, represented by a plane f(u,v)∈Rn, where Rn represents an n-dimensional space; where (u,v) are feature points on the plane of this two-dimensional feature map);
a combiner for generating a third feature by combining the first feature and the second feature; and a 3D head for detecting a 3D object by using the third feature (Mu, ¶0014, The orthographic projection bird's-eye view transformation converts f(u,v)∈Rn into a three-dimensional feature map, denoted by s(x,y,z)∈Rn, where (x,y,z) are points in three-dimensional space; Mu, ¶0017, Average the bounding box of the image f(a,b)∈Rn projection, and assign each feature to the appropriate position in s(x,y,z)∈Rn; Mu, ¶0019, The generated 3D feature map g(x, y, z) is multiplied by a set of learned weight matrices M(y) and s(x, y, z), and then accumulated along the vertical axis to obtain an orthogonal feature map; Mu, ¶0021, In the formula: B(x, z) is the projection of the three-dimensional mapping onto the ground plane; Mu, ¶0012, The image processing unit performs target recognition based on the three-dimensional target recognition algorithm of the high-definition camera).
As per claim 7, Mu and Lai discloses the 3D object detection correction apparatus of claim 5, wherein the information of the first camera includes one or both of camera intrinsic property information including at least one of a focal length of the first camera, a center point of the first camera, a distortion correction coefficient of the first camera, or a combination thereof, or extrinsic property information including at least one of position information of the first camera, angle information of the first camera, or a combination thereof (Mu, ¶0016, Where f is the focal length of the high-definition camera, and (ca, cb) is the origin [See first equation, page 2 original document]).
As per claim 8, Mu discloses the 3D object detection correction apparatus of claim 1, but does not explicitly disclose the following limitations as further recited however Lai discloses wherein the processor is configured to, in detecting a 3D object based on image data of a second camera of a second vehicle type using a network learned based on image data of a first camera of a first vehicle type and information of the first camera detect a 3D object using the information of the first camera of the first vehicle type and image data of the second camera, and correct an angle and position of the detected 3D object using information of the second camera of the second vehicle type (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0077, Step 202: Obtain the first extrinsic parameter matrix; the first extrinsic parameter matrix is the extrinsic parameter matrix of the camera on the first vehicle model; Lai, ¶0079, Step 203: Based on the deviation between the installation positions of the camera on the first vehicle model and the camera on the second vehicle model, the first extrinsic parameter matrix is recalibrated to obtain the second extrinsic parameter matrix; Lai, ¶0081, Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model; Lai, ¶0082, Step 205: Based on the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0117, The video analysis module 93 is used to identify target objects in the second image data and obtain the first identification information of the target objects; the target objects include lane lines, license plates, traffic signs and pedestrians).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Lai and Mu because they are in the same field of invention. One skilled in the art would have been motivated to include the object detection and position correction using camera installation positions on different vehicle models or types as taught by Lai in the system of Mu in order to improve the training and performance of driver assistance systems (Lai, ¶0001-0002).
As per claim 9, Mu and Lai disclose the 3D object detection correction apparatus of claim 8. Lai discloses wherein the processor is configured to correct the angle and position of the detected 3D object using a positional difference between the first camera and the second camera and an angular difference between the first camera and the second camera (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0077, Step 202: Obtain the first extrinsic parameter matrix; the first extrinsic parameter matrix is the extrinsic parameter matrix of the camera on the first vehicle model; Lai, ¶0079, Step 203: Based on the deviation between the installation positions of the camera on the first vehicle model and the camera on the second vehicle model, the first extrinsic parameter matrix is recalibrated to obtain the second extrinsic parameter matrix; Lai, ¶0081, Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model; Lai, ¶0082, Step 205: Based on the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0104, refer to Figures 5 and 6. The extrinsic parameter matrix describes how real-world coordinate points are transferred to the camera coordinates through rotation and translation). The motivation would be the same as above in claim 8.
As per claim 10, Mu and Lai disclose the 3D object detection correction apparatus of claim 9, wherein the processor is configured to calculate x and y information in which a center point of the 3D object detected based on the information of the first camera is projected as an image as a normal coordinate system value (Mu, ¶0027, In the formula: N(x, z) is a smooth function that represents the probability of the existence of a bounding box centered at (x, y0, z), where y0 is the distance from the HD camera to the ground plane, and δ is the scaling factor; Mu, ¶0028, Relative position offset Δpos; Mu, ¶0030, Δpos (x, z) represents the relative deviation between the bounding box center coordinates (x, y0, z) and the actual position of the target, and Δdim (xi, yi, zi) is the center coordinate of object i).
As per claim 11, Mu and Lai disclose the 3D object detection correction apparatus of claim 10, wherein the processor is configured to calculate the normal coordinate system value as a 3D value by reflecting z information on the normal coordinate system value (Mu, ¶0028, Relative position offset Δpos; Mu, ¶0030, Δpos (x, z) represents the relative deviation between the bounding box center coordinates (x, y0, z) and the actual position of the target, and Δdim (xi, yi, zi) is the center coordinate of object i).
As per claim 12, Mu and Lai disclose the 3D object detection correction apparatus of claim 11. Lai discloses wherein the processor is configured to correct the detected 3D object by reflecting a difference between the angle and position of the first camera and the angle and position of the second camera to the 3D value (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0077, Step 202: Obtain the first extrinsic parameter matrix; the first extrinsic parameter matrix is the extrinsic parameter matrix of the camera on the first vehicle model; Lai, ¶0079, Step 203: Based on the deviation between the installation positions of the camera on the first vehicle model and the camera on the second vehicle model, the first extrinsic parameter matrix is recalibrated to obtain the second extrinsic parameter matrix; Lai, ¶0081, Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model; Lai, ¶0082, Step 205: Based on the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0104, refer to Figures 5 and 6. The extrinsic parameter matrix describes how real-world coordinate points are transferred to the camera coordinates through rotation and translation). The motivation would be the same as above in claim 8.
As per claim 13, Mu discloses the 3D object detection correction apparatus of claim 1, but does not explicitly disclose the following limitations as further recited however Lai discloses wherein the processor is configured to:
detect a 3D object based on image data of a second camera of a second vehicle type based on a coordinate system of a first camera of a first vehicle type; and correct a position and angle of the 3D object by converting the coordinate system of the first camera into a coordinate system of the second camera (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0077, Step 202: Obtain the first extrinsic parameter matrix; the first extrinsic parameter matrix is the extrinsic parameter matrix of the camera on the first vehicle model; Lai, ¶0079, Step 203: Based on the deviation between the installation positions of the camera on the first vehicle model and the camera on the second vehicle model, the first extrinsic parameter matrix is recalibrated to obtain the second extrinsic parameter matrix; Lai, ¶0081, Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model; Lai, ¶0082, Step 205: Based on the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0104, refer to Figures 5 and 6. The extrinsic parameter matrix describes how real-world coordinate points are transferred to the camera coordinates through rotation and translation).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Lai and Mu because they are in the same field of invention. One skilled in the art would have been motivated to include the object detection and position correction using camera installation positions on different vehicle models or types as taught by Lai in the system of Mu in order to improve the training and performance of driver assistance systems (Lai, ¶0001-0002).
As per claim 19, Mu discloses the 3D object detection correction method of claim 16, but does not explicitly disclose the following limitations as further recited however Lai discloses wherein detecting the 3D object includes detecting, by the processor, a 3D object based on image data of a second camera of a second vehicle type using a network learned based on image data of a first camera of a first vehicle type and information of the first camera, and correcting the error of the detected 3D object includes correcting, by the processor, an angle and position of the detected 3D object using information of the second camera of the second vehicle type (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0077, Step 202: Obtain the first extrinsic parameter matrix; the first extrinsic parameter matrix is the extrinsic parameter matrix of the camera on the first vehicle model; Lai, ¶0079, Step 203: Based on the deviation between the installation positions of the camera on the first vehicle model and the camera on the second vehicle model, the first extrinsic parameter matrix is recalibrated to obtain the second extrinsic parameter matrix; Lai, ¶0081, Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model; Lai, ¶0082, Step 205: Based on the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0117, The video analysis module 93 is used to identify target objects in the second image data and obtain the first identification information of the target objects; the target objects include lane lines, license plates, traffic signs and pedestrians).
It would have been obvious to one skilled in the art before the effective filing date of the claimed invention to combine the teachings of Lai and Mu because they are in the same field of invention. One skilled in the art would have been motivated to include the object detection and position correction using camera installation positions on different vehicle models or types as taught by Lai in the system of Mu in order to improve the training and performance of driver assistance systems (Lai, ¶0001-0002).
As per claim 20, Mu and Lai disclose the 3D object detection correction method of claim 19. Lai discloses wherein correcting the angle and position of the detected 3D object includes correcting, by the processor, the angle and position of the detected 3D object using a positional difference between the first camera and the second camera and an angular difference between the first camera and the second camera (Lai, ¶0074, Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different; Lai, ¶0077, Step 202: Obtain the first extrinsic parameter matrix; the first extrinsic parameter matrix is the extrinsic parameter matrix of the camera on the first vehicle model; Lai, ¶0079, Step 203: Based on the deviation between the installation positions of the camera on the first vehicle model and the camera on the second vehicle model, the first extrinsic parameter matrix is recalibrated to obtain the second extrinsic parameter matrix; Lai, ¶0081, Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model; Lai, ¶0082, Step 205: Based on the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data; Lai, ¶0104, refer to Figures 5 and 6. The extrinsic parameter matrix describes how real-world coordinate points are transferred to the camera coordinates through rotation and translation). The motivation would be the same as above in claim 19.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRACY MANGIALASCHI whose telephone number is (571)270-5189. The examiner can normally be reached M-F, 9:30AM TO 6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vu Le can be reached at (571) 272-7332. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TRACY MANGIALASCHI/Examiner, Art Unit 2668