DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/14/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
The examiner would also like to address the Applicant’s Remarks on page 11. The Applicant stated “Claim 3 recites "wherein the generating of the local map includes determining to omit, from the image data that is projected onto the corresponding 3D points on the ground plane, one or more color values of one or more pixels of the image data that do not belong to a segmented navigable space." The Office Action cites Kim as establishing replacement of vehicle pixels with the road color. Office Action at p. 15. However, this does not involve "image data that is projected onto [] corresponding 3D points on the groundplane" and therefore cannot establish "determining to omit, from the image data that is projected onto the corresponding 3D points on the ground plane, one or more color values of one or more pixels of the image data that do not belong to a segmented navigable space." Applicant has been unable to locate a corresponding disclosure elsewhere in the cited references”.
Through the broadest reasonable interpretation, claim 3 is removing colors in an image that does not belong on a road. Similarly, Kim is painting over unwanted vehicles on a road, thus “omitting” objects on a navigable space. As stated in the mapping below, the examiner is using the concept of omitting a part of an image that doesn’t belong on the road, and thus this reference reads on the limitation.
Information Disclosure Statement
The IDS dated 5/15/2026, 6/24/2026, 7/16/2026, 8/5/2026 has been considered and placed in the application file.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“The method of claim 1, wherein the method is performed by at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational Al operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
or a system implemented at least partially using cloud computing resources” in claim 10;
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 2 and 12 are rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, had possession of the claimed invention.
Claim 2 recites “wherein the generating of the local map includes orienting the ground projection in a direction onto a 3D representation of the ground plane corresponding to a direction of ego-motion of the ego-machine”. Specification paragraph [00216] states “In some embodiments, the local map generator 2325 uses the ego-motion data 2340 to identify a direction of ego-motion and construct a local map of the environment, the ground, and/or the navigable space (e.g., the road), where the local map faces in the direction of ego-motion and may include a length that depends on the speed of ego-motion”. While the spec provides support for the local map facing the direction of the ego-motion, it does not provide support for orienting the ground in a direction onto a 3d representation of the ground plane. If the Applicant believes there is sufficient support elsewhere in the specification, they are encouraged to point it out.
Claim 12 corresponds to claim 2 and thus is rejected for similar reasons of lack of written description.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 2 and 12 is/are rejected under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim 2 recites “wherein the generating of the local map includes orienting the ground projection in a direction onto a 3D representation of the ground plane corresponding to a direction of ego-motion of the ego-machine”. It is unclear what the Applicant is stating when claiming “generating of the local map includes orienting the ground projection in a direction onto a 3D representation of the ground plane”, as through BRI, the claim is stating a direction of the ground projection is pasted onto the ground plane. For examination purposes, the examiner will interpret this claim as wherein the generating of the local map includes orienting the ground projection local map in a direction corresponding to a direction of ego-motion of the ego-machine.
Claim 12 corresponds to claim 2 and thus is rejected for similar reasons of indefiniteness.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 5-10, 11-12, 15-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Hariyani (US 20230166659 A1).
Regarding claim 1, Hariyani discloses a method comprising: generating a local map representing a ground projection of image data (Hariyani, paragraph [0020], "The calibrated camera system produces a surround view synthesis function which receives input video streams from the four fish-eye cameras and creates a composite 3D surround view 132"), onto corresponding three-dimensional (3D) points on a ground plane, the image data generated using one or more cameras of an ego-machine (Hariyani, paragraph [0024], "Images, such as the stitched output image, may be overlaid, for example, by a graphics processing unit (GPU) or image processor, onto the 3D bowl mesh 200 and a set of virtual viewpoints, or virtual cameras, may be defined, along with mappings from the cameras used to create the stitched output image and the virtual viewpoints").
updating a composite representation of the ground plane based at least on the local map (Hariyani, paragraph [0042], "At block 718, a view under the vehicle is rendered based on the stored images and set of motion data"),
and virtually reconstructing an area of the ground plane under the ego-machine based at least on retrieving a corresponding portion of the composite representation of the ground plane (Hariyani, paragraph [0030], "In some cases, the 3D bowl mesh 200 may be defined relative to the under-vehicle mesh 504A and/or region underneath the vehicle 502A.").
Regarding claim 2, Hariyani discloses the method of claim 1, wherein the generating of the local map includes orienting the ground projection in a direction onto a 3D representation of the ground plane corresponding to a direction of ego-motion of the ego-machine (Hariyani, paragraph [0028], "In FIG. 4, a vehicle 402A at time t.sub.0 having a camera pointed in the direction of travel, here forward, is able to image a region 404 ahead of the vehicle 402A, including reference region 406").
Regarding claim 5, Hariyani discloses the method of claim 1, wherein the generating of the local map includes texturing the local map with color values of pixels of the image data that project onto the corresponding 3D points on the ground plane (Hariyani, paragraph [0043], "Block 812 may perform color corrections operations (such as conversion from Bayer format to YUV420 format, color tone mapping, noise filter, gamma correction, etc.) if required, using known or later developed image processing methods").
Regarding claim 6, Hariyani discloses the method of claim 1, wherein the updating of the composite representation of the ground plane comprises merging the local map corresponding to a current time slice into a composite set of local maps corresponding to one or more previous time slices (Hariyani, paragraph [0042], "At block 718, a view under the vehicle is rendered based on the stored images and set of motion data. For example, the motion parameters may be applied to the vertices of the temporal under-vehicle mesh, weights may be applied to the one or more cameras of the vehicle. The weights may be based on the motion parameters and an angle associated with each camera of the one or more cameras", motion data indicates time slices).
Regarding claim 7, Hariyani discloses the method of claim 1, wherein the updating of the composite representation of the ground plane limits the composite representation of the ground plane to representing local maps generated during a designated number of time slices (Hariyani, paragraph [0042], " For example, the motion data may be used to determine a set of images stored in the temporal buffer at a time when the region associated with the under-vehicle mesh was not obscured by the vehicle, and the set of images associated with the determined time may be retrieved", as shown in this example, the image data is limited by the times when the vehicle mesh was obscured by the vehicle).
Regarding claim 8, Hariyani discloses the method of claim 1, wherein the retrieving of the corresponding portion of the composite representation of the ground plane texturizes the area under the ego-machine based at least on assigning a color value to at least one cell of one or more cells in a 3D grid in the area under the ego-machine, the color value being retrieved from a corresponding pixel of the composite representation of the ground plane (Hariyani, paragraph [0043], "This example system 800 includes multiple cameras, such as cameras 800-808 that are placed around the periphery of the vehicle and coupled to a capture block 810. Block 812 may perform color corrections operations (such as conversion from Bayer format to YUV420 format, color tone mapping, noise filter, gamma correction, etc.) if required, using known or later developed image processing methods", as the under-vehicle representation is produced from the surround view video streams, the under-vehicle representation will naturally be color coded).
Regarding claim 9, Hariyani discloses the method of claim 1, wherein the generating of the local map comprises populating the image data that is projected onto the corresponding 3D points on the ground plane using a camera that is selected from the one or more cameras based at least on the camera being orientated corresponding to a direction of ego-motion of the ego-machine (Hariyani, paragraph [0041], "The cameras may be arranged such that the cameras are able to view an area around the vehicle. For example, a vehicle may have a front facing, rear facing, right facing, and left facing cameras").
Regarding claim 10, Hariyani discloses the The method of claim 1, wherein the method is performed by at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational Al operations;
a system for generating synthetic data (Hariyani, paragraph [0021], " This synthesis creates the stitched output image using the mapping encoded in the geometric LUT.").
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center;
or a system implemented at least partially using cloud computing resources.
Regarding claim 11, Hariyani discloses one or more processors (Hariyani, paragraph [0054], "As illustrated in FIG. 9, device 900 includes a processing element such as processor 905 that contains one or more hardware processors, where each hardware processor may have a single or multiple processor cores") comprising:
one or more circuits (Hariyani, paragraph [0054], "Although not illustrated in FIG. 9, the processing elements that make up processor 905 may also include one or more other types of hardware processing components, such as graphics processing units (GPUs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and/or digital signal processors (DSPs)") to:
merge, into a composite representation of a ground plane representing portions of the ground plane observed by one or more sensors of an ego-machine, a local map representing a portion of the ground plane observed by the one or more sensors during a first time slice (Hariyani, paragraph [0042], "At block 718, a view under the vehicle is rendered based on the stored images and set of motion data. For example, the motion parameters may be applied to the vertices of the temporal under-vehicle mesh, weights may be applied to the one or more cameras of the vehicle. The weights may be based on the motion parameters and an angle associated with each camera of the one or more cameras"),
And virtually reconstruct an area of the ground plane under the ego-machine during the first time slice based at least on retrieving a corresponding portion of the composite representation of the ground plane (Hariyani, paragraph [0030], "In some cases, the 3D bowl mesh 200 may be defined relative to the under-vehicle mesh 504A and/or region underneath the vehicle 502A.").
Claims 12 and 15 corresponds to claims 2 and 10 respectively, additionally reciting the one or more processors of claim 11 (Hariyani, paragraph [0054], "As illustrated in FIG. 9, device 900 includes a processing element such as processor 905 that contains one or more hardware processors, where each hardware processor may have a single or multiple processor cores"),
One or more circuits (Hariyani, paragraph [0054], "Although not illustrated in FIG. 9, the processing elements that make up processor 905 may also include one or more other types of hardware processing components, such as graphics processing units (GPUs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and/or digital signal processors (DSPs)"). Thus, they are rejected for the same reasons of anticipation as claims 2 and 10 respectively.
Claims 16-20 corresponds to claims 1, 7-10 respectively, additionally reciting a system (Hariyani, paragraph [0057], “ For example, the computing device 900 may include a general-purpose operating system which is utilized for normal operations”) comprising: one or more processors (Hariyani, paragraph [0054], "As illustrated in FIG. 9, device 900 includes a processing element such as processor 905 that contains one or more hardware processors, where each hardware processor may have a single or multiple processor cores"). Thus, they are rejected for the same reasons of anticipation as claims 1, 7-10 respectively.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Hariyani (US 20230166659 A1) in view of Kim (US 20140300623 A1).
Regarding claim 3, Hariyani disclose the method of claim 1.
Hariyani does not teach “wherein the generating of the local map includes determining to omit, from the image data that is projected onto the corresponding 3D points on the ground plane, one or more color values of one or more pixels of the image data that do not belong to a segmented navigable space”.
However, Kim discloses wherein the generating of the local map includes determining to omit, from the local map, one or more color values of one or more pixels of the image data that do not belong to a segmented navigable space (Kim, paragraph [0080], “That is, as illustrated in FIG. 7, when the object information extracted from the numerical map is a road, the control unit 30 extracts the lane components from the road area in the photomap and removes unwanted vehicles from the road by applying the road color to the road area except the lane components, so that the resultant photomap from which the vehicles are removed is displayed as the moving body is moving in block S170.”, this technique removes the color of the unwanted objects and filling in their positions with the color of where they reside. It should be noted the examiner is using the concept of removing unwanted objects from an image in their modification and is not limited to the specific technique of removing unwanted vehicles on a road.).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to remove undesired objects from Hariyani’s stitched image, as taught by Kim.
The suggestion/motivation for doing so would have been to improve the visibility of the display, allowing the driver to concentrate on what they need to.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hariyani in view of Kim to obtain the invention as specified in claim 3.
Claim 13 corresponds to claim 3, additionally reciting the one or more processors of claim 11, the one or more circuits (Hariyani, paragraph [0054], "As illustrated in FIG. 9, device 900 includes a processing element such as processor 905 that contains one or more hardware processors, where each hardware processor may have a single or multiple processor cores"). Thus, they are rejected for the same reasons of obviousness as claim 3.
Claim(s) 4 are rejected under 35 U.S.C. 103 as being unpatentable over Hariyani (US 20230166659 A1) in view of Chen (US 11225193 B2).
Regarding claim 4, Hariyani discloses the method of claim 1.
Hariyani does not teach “wherein the generating of the local map includes sizing a dimension of the image data, that is projected onto a 3D representation of the ground plane, based at least on a speed of the ego-machine”.
However, Chen teaches wherein the generating of the local map includes sizing a dimension of the ground projection based at least on a speed of the ego-machine (Chen, Col. 6, Line 51-55, " Through the surround view system 1, a synthetic scene can be generated taking into consideration both the vehicle turning angle and the vehicle speed, and thus the viewport transformation can be optimized to enlarge the area where drivers expect to see clearly").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify Hariyani’s camera’s zoom based on the vehicle speed and angle, as taught by Chen.
The suggestion/motivation for doing so would have been to allow the driver to have a better view of where they are turning and heading to, resulting in better safety and caution.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Hariyani in view of Chen to obtain the invention as specified in claim 4.
Claim 14 corresponds to claim 4, additionally reciting the one or more processors of claim 11, the one or more circuits (Hariyani, paragraph [0054], "As illustrated in FIG. 9, device 900 includes a processing element such as processor 905 that contains one or more hardware processors, where each hardware processor may have a single or multiple processor cores"). Thus, they are rejected for the same reasons of obviousness as claim 4.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WAYNE ZHANG whose telephone number is (571) 272-0245. The examiner can normally be reached Monday-Friday 10:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WAYNE ZHANG/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights