DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/13/2026 has been entered.
Response to Arguments
Applicant's arguments filed 8/13/2026 have been fully considered but they are not persuasive. Applicant argues that “Ge'351 does not describe nor suggest creating a mapping between the two-dimensional boundary and the three-dimensional point cloud by registering the image with the point cloud into a registered image and a registered point cloud”. Examiner would like to point out that the claim language of “by registering the image with the point cloud into a registered image and a registered point cloud” is nowhere to be found in the specification. The only place in the public specification that mentions the word registers is paragraph [0026] “The computing system 102 aligns, or registers, the images 112 with the LiDAR point cloud 116.” Therefore the examiner is interpreting that the images align/register with Lidar point cloud. Nowhere in the specification is shown that an image is registered neither a point cloud is registered from registering the image with a point cloud. It is considered indefinite and new matter wherein how a registered image and registered point cloud are created from a mapping by registering of an image of the scene using a camera being mapped to a three-dimensional point cloud from a LIDAR.
Arguments regarding are considered moot, new art is being used.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Newly amended subject matter states “ creating a mapping between the two-dimensional boundary and the three-dimensional point cloud by registering the image with the point cloud into a registered image and a registered point cloud;”. Paragraph [0026] of the current published application specification is the only paragraph that utilizes the word register. Said paragraph describes support for the claim language of creating a mapping between the two-dimensional boundary and the three-dimensional point cloud by registering the image with the point cloud. However, at no point in any part of the specification is there a description of turning the registering of the image with the point cloud into a registered image and a registered point cloud. This effectively creates a registered image in addition to the image captured by the camera and a registered point cloud in addition to the point cloud captured by the LIDAR.
The registered point cloud has a determined subset of points contained within the registered image. However as described above there is no description of a registered image or registered point cloud; only creating a mapping by registering “the image” with “the point cloud”. An image is not the same as a registered image and a point cloud is not the same as a registered point cloud.
A registered image and a registered point cloud are considered new matter.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As described above, a registered image and a registered point cloud are considered new matter. Claiming to determine a subset of points of an undisclosed registered point cloud contained within the two-dimensional boundary in an undisclosed registered image is considered indefinite.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Mahieu et al. (US 2025/0298135 A1) in view of Geng et al. (US 2025/0037302 A1).
With respect to claim 1, Mahieu’135 shows a method for automatically digitally annotating a three-dimensional object in a scene (paragraph [0015] annotations may identify static objects (three dimensional) within the driving environment), the method comprising:
capturing an image of the scene using a camera (paragraph [0023] two-dimensional images captured by image capturing devices (e.g., cameras));
capturing a point cloud representing the scene using a LiDAR system (paragraphs [0022] and [0025] lidar data captured, receive the lidar data as a single lidar point cloud), [ ];
[ ] ; [ ];
determining a subset of points of the registered point cloud contained within the two-dimensional boundary in the registered image (figure 2 step 204) [ ];
assigning a unique identifier to the subset of points contained within the two-dimensional boundary (paragraphs [0015], [0019], [0028]-[0029] and [0058] identifying lidar points that are associated with static objects and using such lidar points to annotate objects within two-dimensional images); and [ ].
Mahieu’135 does not specifically show shows show the point cloud being a three-dimensional point cloud; determining a two-dimensional boundary of the three-dimensional object contained in the image; creating a mapping between the two-dimensional boundary and the three-dimensional point cloud by registering the image with the point cloud into a registered image and a registered point cloud; updating the two-dimensional boundary into a three-dimensional boundary of the three-dimensional object, based on the subset of points, by defining the three-dimensional boundary by extrema of the subset of points, wherein an autonomy computing system of an autonomous vehicle is configured to operate the autonomous vehicle based on the three-dimensional boundary.
Geng’302 shows show the point cloud being a three-dimensional point cloud (paragraph [0040] figure 3, the 3D data 112);
determining a two-dimensional boundary (figure 2B 216 paragraph [0037]) of the three-dimensional object (210 paragraph [0036]) contained in the image (214);
creating a mapping between the two-dimensional boundary (216) and the three-dimensional point cloud by registering the image with the point cloud into a registered image and a registered point cloud (figure 6A points 504, from point cloud paragraph [0045], in plane 602 and two-dimensional boundary 216 is projected/mapped to the plane 602);
updating the two-dimensional boundary (216) into a three-dimensional boundary of the three-dimensional object (paragraphs [0052] and [0054] the plane 602 representing the 3D shape may be associated with a 3D location within the environment 204), based on the subset of points, by defining the three-dimensional boundary by extrema of the subset of points (Figure 4 extrema 410+416 paragraph [0043] minimum and maximum to comprise the 3D shape 402), wherein an autonomy computing system of an autonomous vehicle is configured to operate the autonomous vehicle based on the three-dimensional boundary (paragraph [0021], [0030] and [0059] ).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Mahieu’135 to include the point cloud is a three-dimensional point cloud; creating a mapping between the two-dimensional boundary and the three-dimensional point cloud; determining a two-dimensional boundary of the three-dimensional object contained in the image; updating the two-dimensional boundary into a three-dimensional boundary of the three-dimensional object, based on the subset of points, by defining the three-dimensional boundary by extrema of the subset of points, wherein an autonomy computing system of an autonomous vehicle is configured to operate the autonomous vehicle based on the three-dimensional boundary and the unique identifier method taught by Geng’302. The suggestion/motivation for doing so would have been to improve the vehicle system’s ability to determine how to proceed within the environment, such as by determining one or more paths for navigating within the environment (paragraphs [0059]).
With respect to Claim 2, the combination of Mahieu’135 and Geng’302, the method of Claim 1 further comprising determining the three- dimensional boundary of the three-dimensional object using extrema of the subset of points (in Geng’302: Figure 4 extrema 410+416 paragraph [0043] minimum and maximum to comprise the 3D shape 402).
With respect to Claim 3, the combination of Mahieu’135 and Geng’302 shows the method of claim 2 further comprising training a machine learning model using the three-dimensional boundary and the unique identifier (in Mahieu’135: paragraphs [0018] and [0037] machine learning models configured to perform various object detection functionality, such as object identification, classification, instance segmentation, semantic segmentation, object tracking, and the like, may be implemented using artificial neural networks and trained with model training data).
With respect to Claim 4, the combination of Mahieu’135 and Geng’302 shows the method of claim 2, wherein the three-dimensional boundary of the three-dimensional object includes a cuboid having six facets (in Mahieu’135: paragraphs [0027] and [0054] the object manager may have a total of six annotated images. Further, the object manager may project a lidar point into each of the six annotated image planes, figure 1 230).
With respect to Claim 5, the combination of Mahieu’135 and Geng’302 shows the method of claim 1, wherein the LiDAR system includes at least one of a laser source and a detector (in Mahieu’135: paragraph [0017] The sensor data, which may include image data, radar data, lidar data, time-of-flight data, etc., may be analyzed by the autonomous vehicle to detect and classify various objects within the operating environment ).
With respect to Claim 6, the combination of Mahieu’135 and Geng’302 shows the method of claim 1, wherein determining the two-dimensional boundary of the three-dimensional object includes determining an instance segmentation two- dimensional boundary (in Mahieu’135: paragraphs [0017]-[0018] perform semantic and/or instance segmentation of the objects).
With respect to Claim 7, the combination of Mahieu’135 and Geng’302 shows the method of claim 1, wherein the unique identifier corresponds to a category of objects (in Mahieu’135: paragraph [0091] object type).
With respect to Claim 8, the combination of Mahieu’135 and Geng’302 shows the method of claim 1, wherein points contained in the point cloud each include three-dimensional coordinates and an intensity value (in Mahieu’135: paragraphs [0098] intensity information (e.g., lidar information, radar information, and the like) and map may include a three-dimensional mesh of the environment, and paragraph [0134] lidar points to a global reference frame (e.g., global coordinate frame)).
With respect to Claims 9 and 17, rejection analogous to those presented for claim 1, are applicable.
With respect to Claims 10 and 18, rejection analogous to those presented for claim 2, are applicable.
With respect to Claims 11 and 19, rejection analogous to those presented for claim 4, are applicable.
With respect to Claims 12 and 20, rejection analogous to those presented for claim 5, are applicable.
With respect to Claim 13, rejection analogous to those presented for claim 6, are applicable.
With respect to Claim 14, rejection analogous to those presented for claim 7, are applicable.
With respect to Claim 15, rejection analogous to those presented for claim 8, are applicable.
With respect to Claim 16, rejection analogous to those presented for claim 3, are applicable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Goel (US 12475725 B2): column 2, lines 51-58 3D point clouds generated based on 2D image data and associated per-pixel depth estimates can be provided as input trained 3D networks and object detection algorithms designed to process point clouds output by lidar and/or radar systems. Thus, autonomous vehicles may leverage 3D object detection networks and algorithms more efficiently based on image data, without the need to capture and process lidar and/or radar data as well.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IRIANA CRUZ whose telephone number is (571)270-3246. The examiner can normally be reached 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M. Sarpong can be reached at (571) 270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IRIANA CRUZ/ Primary Examiner, Art Unit 2681
Prosecution Insights