Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 2, 4, 7-10, 12, 15, 16, 18 and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Emmons et al. (US 20230057509).
With respect to claim 1, Emmons et al. teach obtaining image data generated using a first camera (Fig. 1A, ref label 102A-F) having a first field of view (Fig. 1B ref label 122, image information3);
generating transformed image data by applying a transformation to the image data based at least on extrinsic camera data corresponding to the first camera, the transformation converting the first field of view to simulate a second field of view of a second camera (Fig. 3A and 3B; para [0026], receive the feature maps and transform the information into an output vector space. For example, the output vector space may be associated with a virtual camera at a first height.);
computing, using a machine learning model, output data representative of one or more predictions (para [0093], three dimensional graphics of objects (e.g., vehicles, pedestrians, optionally performing actions based on the signals or information described herein) may be rendered based on positional information, velocity information, signal information, and so on, determined by the machine learning model.); and
transmitting the output data to cause a vehicle to perform one or more operations based at least on the one or more predictions(para [0093], In some embodiments, the information (e.g., the outputs described herein) determined by the machine learning model described herein may be presented in a display of the vehicle. For example, the information may be used to inform autonomous driving (e.g., used by a planning and/or navigation engine).
With respect to claim 2, Emmons et al. teach that the first camera is a physical camera (para [0016]) and the second camera is a virtual camera (Fig. 3).
With respect to claim 4, Emmons et al. teach that the extrinsic camera data includes at least one of positional data, pose data, location data, or orientation data associated with the first camera (para [0054], roll, pitch, and/or yaw).
With respect to claim 7, Emmons et al. teach that the transformed label data includes an annotation selected from the group consisting of middle lane, right lane, left lane, split left, split right, merge from left, and merge from right. (para [0060], Example signals may include lane assignment, whether a vehicle has its door open, a particular lane in which a vehicle is located, whether a vehicle is cutting into the autonomous vehicle's lane, and so on).
With respect to claim 8, Emmons et al. teach identifying a region of interest (ROI) based at least on the transformed image data (para [0060], The output may represent information associated with objects, such as location (e.g., position with a virtual camera space)).
Claim 9 is rejected as same reason as claim 1 above.
Claim 10 is rejected as same reason as claim 2 above.
Claim 12 is rejected as same reason as claim 4 above.
With respect to claim 15, Emmons et al. teach a control system for an autonomous or semi-autonomous machine (abstract).
Claim 16 is rejected as same reason as claim 1 above.
Claim 18 is rejected as same reason as claim 2 above.
Claim 20 is rejected as same reason as claim 15 above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
A. Claim 3, 11 and 19 are rejected under 35 USC 103 as being unpatentable over Emmons et al. (US 2023/0057509) in view of Jin et al. (US 2020/0334861).
Emmons et al. teach all the limitations of claim 1 as applied above from which claim 3 respectively depend.
Emmons et al. do not teach expressly that the generating the transformed image data further includes associating one or more two-dimensional (2D) locations included in the image data with one or more 2D locations included in the transformed image data based on at least one of a transformation matrix or a lookup table.
Jin et al. teach the generating the transformed image data further includes associating one or more two-dimensional (2D) locations included in the image data with one or more 2D locations included in the transformed image data based on at least one of a transformation matrix(para [0087], determine the offset compensation transformation matrix which may be observed by both the virtual camera and the real camera. This is the offset compensation transformation matrix between the real camera and the virtual camera).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to transform image using transformation matrix in the method of Emmons et al.
The suggestion/motivation for doing so would have been that to formulate calibration of real camera a mathematically solvable problem.
Therefore, it would have been obvious to combine Jin et al. with Emmons et al. to obtain the invention as specified in claim 3.
Claim 11 is rejected as same reason as claim 3 above.
Claim 19 is rejected as same reason as claim 3 above.
B. Claim 5 and 13 are rejected under 35 USC 103 as being unpatentable over Emmons et al. (US 2023/0057509) in view of Nelson et al. (US 2010/0182400).
Emmons et al. teach all the limitations of claim 4 as applied above from which claim 5 respectively depend.
Emmons et al. do not teach expressly that the positional data includes three-dimensional (3D) positional coordinates referenced from a location within the vehicle.
Nelson et al. teach the positional data includes three-dimensional (3D) positional coordinates referenced from a location within the vehicle (Fig. 4A).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to transform image using 3D) positional coordinates of physical camera in the method of Emmons et al.
The suggestion/motivation for doing so would have been that to use critical baseline to transform an image view point.
Therefore, it would have been obvious to combine Nelson et al. with Emmons et al. to obtain the invention as specified in claim 5.
Claim 13 is rejected as same reason as claim 5 above.
C. Claim 6, 14 and 17 are rejected under 35 USC 103 as being unpatentable over Emmons et al. (US 2023/0057509) in view of Elluswamy et al. (US 10997461).
Emmons et al. teach all the limitations of claim 4 as applied above from which claim 5 respectively depend.
Emmons et al. do not teach expressly that the machine learning model is trained using transformed training image data and corresponding transformed label data, the transformed label data including a set of 2D locations included in the transformed training image data defining a ground truth vehicle path associated with the transformed training image data.
Elluswamy et al. teach the machine learning model is trained using transformed training image data and corresponding transformed label data, the transformed label data including a set of 2D locations included in the transformed training image data defining a ground truth vehicle path associated with the transformed training image data. (col. 2 lines 32-37, For example, a ground truth is determined based on a group of time series elements and is associated with a single element from the group. As one example, a series of images for a time period, such as 30 seconds, is used to determine the actual path of a vehicle lane line over the time period the vehicle travels; col. 9 lines 55-59, For example, a time series of elements made up of sensor and odometry data is collected from a vehicle and used to automatically create training data. In various embodiments, the process of FIG. 3 is used to automatically label training data with corresponding ground truths; col. 18 lines 5-7, By training the machine learning model using a ground truth determined using image and related data of a time series).
At the time of effective filing, it would have been obvious to a person of ordinary skill in the art to train machine learning model using ground truth data in the method of Emmons et al.
The suggestion/motivation for doing so would have been that to make machine learning model with high accuracy, reliable and directly measurable performance.
Therefore, it would have been obvious to combine Elluswamy et al. with Emmons et al. to obtain the invention as specified in claim 6.
Claim 14 is rejected as same reason as claim 6 above.
Claim 17 is rejected as same reason as claim 6 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Randolph Chu whose telephone number is 571-270-1145. The examiner can normally be reached on Monday to Thursday from 7:30 am - 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached on (571) 272-7778.
The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/RANDOLPH I CHU/
Primary Examiner, Art Unit 2667
Prosecution Insights