DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7-11, 15-17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chaudhuri (PGPUB: 20210201055 A1) in view of Purdy (US-PAT-NO: 12026956 B1), and further in view of ZHENG (CN 109764887 A ).
Regarding claims 1, 9, and 17. Chaudhuri teaches a method of generating three-dimensional map data, performed by a computing device including a processor, the method comprising:
collecting sensor data related to surrounding objects from a vehicle (see paragraph 36, a vehicle may be equipped with sensors that are configured to capture various types of sensor data that is representative of the vehicle's surrounding environment);
classifying the sensor data for each object (see paragraph 39, 2D sensor data may be well suited for tasks such as detecting and classifying objects in a vehicle's surrounding environment, but it is typically difficult to determine where objects are positioned within the real-world environment surrounding a vehicle based on 2D sensor data alone. On the other hand, 3D sensor data such as LIDAR data, RADAR data, or SONAR data may be well suited for tasks such as determining the position of objects in the real-world environment surrounding a vehicle, but it is typically difficult to detect and classify objects based on 3D sensor data alone, which is due in part to the fact that each individual capture of 3D sensor data provides a relatively sparse representation of a vehicle's surrounding environment that typically includes only partial views of objects in that surrounding environment. Thus, in order to leverage the relative strengths of these different types of sensor data, it is generally desirable to make use of multiple different types of sensor data when performing object detection and/or creating HD maps, such as 2D image data together with at least one type of 3D sensor data);
generating three-dimensional modeling data for each object based on the sensor data classified for each object (see paragraph 88, overlaying a 3D shape model for a given type of object onto a set of 3D data points identified as being associated with a detected object of the given type (where the detected object's associated set of 3D data points may be identified in any of the manners described herein));
generating position data for each object based on the sensor data classified for each object (see paragraph 42, applying a respective 3D label to 3D sensor data within the given sweep that is associated with each object of interest. In practice, a given 3D label applied by a curator may include an indication of a classification of a given object of interest and a 3D bounding box that circumscribes a respective portion of a given sweep that is associated with the given object of interest. A 3D bounding box may be defined in a variety of manners. For example, a 3D bounding box may be defined in terms of an x-, y-, and z-coordinate that indicates a centerpoint of the 3D bounding box, a set of width, length, and depth values for the 3D bounding box).
However, Chaudhuri does not expressly teach storing the sensor data classified for each object.
Purdy teaches that when an object is successfully classified into an object type in operation 406, and the contour generator 108 determines that an object depth model is available for the object type (e.g., stored locally or retrievable from a remote system) (408:Yes), then in operation 410 the contour generator 108 may use the object depth model to determine depth data for the object (see Col. 16, lines 20-25).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chaudhuri by Purdy to obtain when an object is successfully classified into an object type in operation 406, and the contour generator 108 determines that an object depth model is available for the object type (e.g., stored locally or retrievable from a remote system), in order to provide storing the sensor data classified for each object. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
However, the combination does not expressly teach generating three-dimensional map data for navigation based on the three-dimensional modeling data and the position data for each object.
ZHENG teaches that in operation 1430, the display device by performing map matching to representing the vehicle position on the map, the map matching execution processing position of the vehicle obtained by the location based sensor based on: 3D model (such as the 3D driving environment model) by a position in a lane of travel of positioning processing based on visual operation 1410 for identifying a vehicle travel lane obtained by the position of the object in operation 1232 estimating a depth value of an object based on the positioning processing, and obtained by a visual modeling processing obtained in the 3D. In this example, the 3D modeling operation 1234 performed by the following operation from the divided image generated by image segmentation operation 1230 extracting landmark (e.g., traffic signal lamp or sign) area, and performing 3D modeling based on the 3D depth value of the landmark area (see page 23, lines 23-33).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by ZHENG to obtain the display device by performing map matching to representing the vehicle position on the map, the map matching execution processing position of the vehicle obtained by the location based sensor based on: 3D model (such as the 3D driving environment model) by a position in a lane of travel of positioning processing based on visual operation 1410 for identifying a vehicle travel lane obtained by the position of the object in operation 1232 estimating a depth value of an object based on the positioning processing, and obtained by a visual modeling processing obtained in the 3D, in order to provide generating three-dimensional map data for navigation based on the three-dimensional modeling data and the position data for each object. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 2 and 10. The combination teaches the method of claim 9, wherein the sensor data are data detected by at least one of a camera, a lidar, a radar, or an ultrasonic sensor provided in the vehicle (see Chaudhuri paragraph 50, leverages both 2D sensor data (e.g., image data) and 3D sensor data (e.g., LIDAR data) captured by a collection vehicle while on a mission in a given real-world environment during a given window of time (which may at times be referred to as a “scene”)).
Regarding claims 3 and 11. The method of claim 9, wherein generating the three-dimensional modeling data for each object includes:
estimating a shape of the object by using the sensor data and generating the three-dimensional modeling data based on the estimated shape (see Purdy, Fig. 1, Col. 8 and 16, lines 51-67 and 11-16, when an object is classified into an object type (or a subtype, having particular attributes, etc.), the contour generator 108 may use the object classification to determine precise object boundaries, after which depth data based on image and/or other sources (e.g., lidar) can be received to correspond to the object boundaries. Additionally, certain object types or classifications may have associated depth models. For instance, after using image-based techniques to classify an object (e.g., a traffic cone, a football, a skateboard, etc.), the contour generator 108 may use the associated depth model to determine a three-dimensional representation of the object from which the object depth data may be determined. Thus, the bounding contours for a classified object may be generated as fully closed geometric shapes, even when the back side or other portions of the object are hidden from the sensors of the autonomous vehicle 102; the contour generator 108 may store an object depth model associated with a traffic cone object type, one or more vehicle object types, a fire hydrant object type, a box object type, and so on. Object depth models may include 3D representations and/or other depth data (e.g., size data including per-dimension maxima/minimum and distributions, size ratios, shape data, models, indicators of symmetry about one or more axes)).
Regarding claims 7 and 15. The method of claim 9, wherein generating the three-dimensional map data includes performing, for each object, (see Purdy, Col. 11, lines 28-34, the object depth model component 212 may be used alternatively, or in addition to, any of the other techniques described herein for determining depth data. As noted above, the object depth model component 212 may include a number of depth models (e.g., 3D representations and/or other depth data) associated with particular object types/classifications), a process of disposing an object model (see Co. 16, lines 3-8, the contour generator 108 may determine whether an object depth model is available based on the object type(s) determined in operation 406. As discussed above, an object depth model may include a 3D representation and/or any other depth data associated with an object type), determined by the three-dimensional modeling data (see Chaudhuri, paragraph 136, system 302 may incorporate a 3D shape model for the moving object into the 3D point cloud (e.g., in place of the 3D data points associated with a moving object)), at a position determined by the position data (see Chaudhuri, paragraph 53, when an object in the collection vehicle's surrounding environment was in motion, the captured 3D data points associated with the object are spread across multiple different positions within the surrounding environment, which makes it more difficult to effectively present such 3D data points in a time-aggregated manner. Second, combined with the fact that the collection vehicle itself is often in motion during its mission, an object in the collection vehicle's surrounding environment that was in motion makes it harder to determine the position of the object vis-à-vis the collection vehicle (and thus the perspective at which the collection vehicle was perceiving the object) from capture-to-capture, which further increases the complexity of labeling and presenting the 3D data points associated with the object in a time-aggregated manner).
Regarding claims 8, 16, and 20. The combination does not expressly teach the method of claim 9, wherein the computing device is implemented as a server or a cloud computing system.
The examiner is taking "Official Notice" that the limitation about wherein the computing device is implemented as a server or a cloud computing system is well known in the art.
Therefore, it would have been obvious to a person having ordinary skill in the art at the time the invention was made to have modified the combination so that “wherein the computing device is implemented as a server or a cloud computing system” would be available.
Allowable Subject Matter
Claims 4-6, 12-14, and 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN JIA whose telephone number is (571)270-5536. The examiner can normally be reached 9:00 am-7:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIN JIA/Primary Examiner, Art Unit 2663
Prosecution Insights