DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. There are a total of 20 claims and claims 1-20 are pending.
Response to Amendment
Applicant's argument, filed on August 17, 2026 has been entered and carefully considered. Claims 1, 3 and 9-14 are amended. Claims 1-20 are pending.
Response to Arguments
Applicant's arguments filed on 08/17/2026 remarks have been fully considered but are moot in view of the new ground(s) of rejection which is deemed appropriate to address all of the needs at this time.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims1-4,6-12, 14-17, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Sypitkowski et al. (US 10,166,921 B2) in view of Bopp et al. (US 2024/0291954 A1).
Regarding claim 1, Sypitkowski discloses One or more processors comprising processing circuitry to: determine, using frames of image data that represent overlapping views of an environment surrounding an ego-machine ([col. 5, lines 16-21] image data from each of the cameras 14, 16, 18, 20 is stitched together and then used to perform optimized color matching. FIG. 4A shows the overhead view of the vehicle -an image that is generated by the controller 22 and displayed to the driver of the vehicle" figs. 4A-4B-), a reference frame and a target frame([col. 6, lines 5-7]- the controller designates one camera as a "Master" (at step 52). In this example, the rear view camera 14 is designated as the "Master", fig. 5; col. 6, lines 8-11 :"This method matches the brightness and color for the other cameras 16, 18, 2.Q_to the brightness and color of the Master camera 14 by comparing overlapping areas in each region of interest (e.g., areas 40, 42, 44, 46 of FIGS. 4A-4E); generate a ground projection of the reference frame based at least on projecting at least a portion of the reference frame onto a ground plane([col. 5, lines 23-27]- The field of view of each respective camera includes a portion of the ground. The image data from each respective camera is processed by the controller 22 to generate a simulated overhead view of at least a portion of the ground within the field of view of the camera. This ground portion is referred to herein as a "region of interest" for each respective camera."; col. 8, lines 37-45:"The AFAIR module can create multiple concatenated camera views it is possible to generate a 3D perspective and a top-view perspective concurrently. When this occurs, a top perspective view is generated and buffered outside the viewable range and used to perform the camera matching in the exact same manner as in the standard top view matching); compute a reference property of at least a portion of the ground projection of the reference frame([col. 6, lines 11-34]- Cmavgrgb is the average RGB color value from the overlapping region of the master camera" and equation); and transfer the reference property of at least the portion of the ground projection of the reference frame to the target frame([col. 6, lines 11-50]- equations (1) and (2) where gain factors are determined using "Cmavgrgb"; col. 7, lines 1-6:"After appropriate gain factors are determined for each color component for each side camera, the controller 22 applies the gain factors to the video input streams from the side cameras 16, 18 thereby adjusting the RGB color levels of the images captured by the side cameras 16, 18 to match the colors of the rear camera 14 image (at step 60)).
However, Sypitkowski does not explicitly disclose a ground plane of a three-dimensional (3D) bowl model of the environment.
In an analogous art, Bopp discloses a ground plane of a three-dimensional (3D) bowl model of the environment([004]- surround view as well as electronic mirror replacement systems, the camera images are either displayed directly in so-called “single views” or transformed in accordance with special processes, in order to display fused views. These views are, e.g., a so-called “3D bowl” (“bowl view” or “360 camera”) or “top view” (“bird's-eye view” or “plan view”), in which images or textures from the surround view cameras are merged or seamlessly amalgamated (stitching). As a general rule, the images or textures of the surround view cameras have overlapping areas or overlapping regions, in particular in the bowl view, in which the textures from the cameras are projected into a spatial plane in order to visualize a virtual 3D view of the surroundings which represents the entire surroundings around the car). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide the technique of Bopp to the modified system of Sypitkowski a method for generating a stereoscopic or holographic view for a generic camera system, such as in conjunction with an autostereoscopic display element which allows improved visibility or less visibility restrictions to be achieved more simply and in a more cost-effective way, as a result of which the user-friendliness and safety are improved [Bopp, 0009].
Regarding claim 2, Sypitkowski discloses wherein the processing circuitry is further to select one of the frames of image data as the reference frame based at least on one or more of: a direction of ego-motion of the ego-machine, an active viewport into the environment, or a gaze of an operator of the ego-machine([col. 6, lines 4-7)]- discloses to select the rear camera among the cameras of the surround view system as the master camera, but other cameras could also be selected).
Regarding claim 3, Sypitkowski discloses wherein the processing circuitry is further to generate the ground projection of the reference frame based at least on projecting the reference frame onto the 3D bowl model to generate a projected 3D bowl representation, and using a portion of the projected 3D bowl representation corresponding to the ground plane of the environment as the ground projection of the reference frame([col. 8, lines 37-45]- The AFAIR module can create multiple concatenated camera views it is possible to generate a 3D perspective and a top-view perspective concurrently).
Regarding claim 4, Sypitkowski discloses wherein the reference and target frames represent different views of the environment in a same time slice, and the processing circuitry is further to transfer the reference property of at least the portion of the ground projection of the reference frame to the target frame based at least on a determination that an overlapping region between the ground projection of the reference frame and a second ground projection of the target frame has less than or equal to a threshold number of pixels that belong to a detected object([col. 8, lines 6-16]- ln most situations when two adjacent cameras are looking at the ground plane, it may be appropriate to assume that the two camera video images are very similar. However. there are situations when one camera sees objects that the other camera does not. This can lead to inappropriate gain calculations. A solution for this type of scenario is to either limit).
Regarding claim 6, Sypitkowski discloses wherein the processing circuitry is further to compute the reference property of at least the portion of the ground projection of the reference frame based at least on computing the reference property from a majority cluster of the ground projection of the reference frame, and to transfer the reference property from the majority cluster of the ground projection of the reference frame to the target frame based at least on a determination that an overlapping region between the ground projection of the reference frame and a second([col. 8, lines 6-16]- ln most situations when two adjacent cameras are looking at the ground plane, it may be appropriate to assume that the two camera video images are very similar. However. there are situations when one camera sees objects that the other camera does not. This can lead to inappropriate gain calculations. A solution for this type of scenario is to either limit).
Regarding claim 7, Sypitkowski discloses wherein the processing circuitry is further to generate a modified target frame based at least on transferring the reference property of at least the portion of the ground projection of the reference frame to the target frame, stitch at least the reference frame and the modified target frame into a stitched image, and cause presentation of a visualization based at least on the stitched image([col. 4, lines 62-67]- The controller 22 is also configured to process the image data received from the cameras 14, 16, 18, 20 and stitch them together to create a simulated overhead view of the vehicle 12 in its surroundings (a surround view) as shown in the left-side image of FIG. 3. The two images of FIG. 3 can be shown on the display simultaneously"; col. 5, line 65 - col 6, line 2:"FIG. 5 illustrates an example of one such color-matching routine that adjusts for individual color component errors in overlapping portions 40, 42, 44, 46 where video images from two cameras are combined to create the simulated overhead surround image).
Regarding claim 8, Sypitkowski discloses wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing real-time streaming; a system for presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system for performing digital twin operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for generating synthetic data; or a system implemented at least partially using cloud computing resources([col. 3, lines 19-21]- FIG. 6 is display image generated and displayed by the system of FIG. 1 showing a front 3D virtual perspective view of a SUV vehicle."; col. 8, lines 17-23 fig. 6) for a system for presenting a virtual reality content).
Regarding claim 9, the claim is interpreted and rejected for the same reason as set forth in claim 1. Hence; all limitations for claim 9 have been met in claim 1.
Regarding claim 10, the claim is interpreted and rejected for the same reason as set forth in claim 2.
Regarding claim 11, the claim is interpreted and rejected for the same reason as set forth in claim 3.
Regarding claim 12, the claim is interpreted and rejected for the same reason as set forth in claim 4.
Regarding claim 14, the claim is interpreted and rejected for the same reason as set forth in claim 6.
Regarding claim 15, the claim is interpreted and rejected for the same reason as set forth in claim 8.
Regarding claim 16, the claim is interpreted and rejected for the same reason as set forth in claim 1. Hence; all limitations for claim 16 have been met in claim 1.
Regarding claim 17, the claim is interpreted and rejected for the same reason as set forth in claim 4.
Regarding claim 19, the claim is interpreted and rejected for the same reason as set forth in claim 6.
Regarding claim 20, the claim is interpreted and rejected for the same reason as set forth in claim 8.
Allowable Subject Matter
Claims 5, 13 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 5, The one or more processors of claim 1, wherein the reference frame represents at least a portion of the environment from a preceding time slice, the target frame represents at least a portion of the environment from a subsequent time slice, and the processing circuitry is further to transfer the reference property of at least the portion of the ground projection of the reference frame based at least on a determination that an overlapping region between a second ground projection of a candidate reference frame from the subsequent time slice and a third ground projection of the target frame includes more than a threshold number or percentage of pixels that correspond to a detected object.
Citation of Pertinent Prior Art
The prior art are made of record and not relied upon but considered pertinent to applicant’s disclosure:
1. Ibrahim et al., US 2015/0302561 A1, discloses Method a method and system for performing geometric calibration for a surround view camera solution.
2. Hua et. al., US 2021/0209737 A1, discloses method for error handling in a geometric correction engine (GCE) is provided that includes receiving configuration parameters by the GCE.
3. Dabral et al., US 2021/0027522 A1, discloses a technique for surround view.
4. Hartmann et al., US 2020/0406897 A1, discloses method for classifying a roadway condition on the basis of image data from a camera.
5. HARIYANI et al., US 2023/0166659 A1, discloses technique for rendering an under-vehicle view including obtaining a first location of a vehicle, the vehicle having a set of cameras disposed about the vehicle, capturing a set of images; storing images of the set of images in a memory.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MD NAZMUL HAQUE whose telephone number is (571)272-5328. The examiner can normally be reached IFW.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Czekaj can be reached at 5712727327. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MD N HAQUE/Primary Examiner, Art Unit 2487
