DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/20/2026 has been entered.
Status of Claims
Claims 1-8, 10-14 and 16-22 are pending. Claims 1-8, 10-14, 16, 17 and 19 are amended. Claims 21 and 22 are new. Claims 9 and 15 are cancelled.
Response to Amendment
In light of Applicant’s amendment of the claims, the rejections of record with respect to the claims under 35 U.S.C. 101 for reciting an abstract idea have been withdrawn.
Claim Objections
Claim 22 is objected to for minor typographical error. Specifically, the claim is missing a period “.” at the end.
Response to Arguments
Applicant's arguments filed on April 20, 2026 with respect to rejection of claims under 35 U.S.C. 103 has been fully considered; but they are not found persuasive. Specifically, in page 18 of its reply, Applicant argues in second paragraph that Nayak does not teach classification of a driving surface for a ‘point’. Examiner respectfully disagrees. Nayak discloses in, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”. Applicant continues to argue in third paragraph, that Nayak does not teach occlusion associated with images. Examiner respectfully disagrees. Nayak teaches in, col. 9, lines 60-61: “obstructing objects (e.g., another vehicle, a barrier, a cone”. Therefore, Applicant’s arguments are not found persuasive.
Applicant’s amendment of independent Claims 1, 10 and 19, which has altered the scope of the claims of the instant application, has necessitated the new ground(s) of rejection presented in this office action with respect to claims of the instant application. Accordingly, in response to Applicant’s arguments that are merely directed to the amended portion of the claims, new analyses have been presented below, which make Applicant’s arguments moot.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Amended Claims 1-8 and new claim 22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre- AlA), first paragraph, as failing to comply with the written description requirement. The claims contain subject matter which were not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AlA the inventor(s), at the time the application was filed, had possession of the claimed invention. Specifically, independent claim 1 recites “training data that is generated, at least, by: determining, using one or more second machine learning models”, and dependent claim 22 recites “determining a location of a second machine that obtained the image data within the environment”. Applicant’s original specification lacks these disclosures. Therefore, these amended and new claim limitations constitute introduction of “new matter’ in the claims. It is suggested that Applicant amend the claims to match the limitation with the specification, in order to overcome the rejection.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
inventor of carrying out his invention.
Claims 10, 11, 12, 16, 17 and 19 are rejected under 35 U.S.C. 112(b) because the claim language contains intended use which does not have patentable weight. Specifically, the claims recite ‘one or more processors to’ and ‘one or more processors are further to’ are interpreted as intended use and not seen as actual functions. Please amend the claims to include the term ‘configured’ as in ‘one or more processors configured to’ and ‘one or more processors are further configured to’ in order to overcome the rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5-8 and 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Gummadi et al. (US 2021/0012120 A1) in view of Eyjolfsdottir et al. (US 2022/0019852 A1) and in further view of Nayak et al. (US 11,551,548 B1).
Regarding claim 1, Gummadi teaches, A method comprising: causing a machine to navigate within an environment (Gummadi, ¶0003: “The method of the present teachings for estimating free space based on image data and point cloud data, the free space used for navigating an autonomous vehicle”) based at least on one or more first machine learning models processing sensor data (Gummadi, ¶0002: “applies a machine learning model to camera images of a navigation area”) obtained using one or more sensors of the machine, (Gummadi, ¶0005: “receiving camera image data from at least one camera into the autonomous vehicle”) wherein the one or more first machine learning models are trained using training data (Gummadi, ¶0022: “classifications 215 can be based on the training received by a machine learning model”). However, Gummadi does not explicitly teach, training data that is generated, at least, by: determining, using one or more second machine learning models and based at least on image data representative of an image, a first classification corresponding to an object that is depicted at a portion of the image; determining, based at least on map data representing a map associated with an environment, a point within the environment that should be depicted within the portion of the image; determining, based at least on the map data, that the map indicates a second classification of driving surface for the point within the environment; determining, based at least on the first classification being different from the second classification of the driving surface, that the object at least partially occludes the driving surface at the portion of the image; and generating the training data to include the image data and first data indicating that the object at least partially occludes the driving surface at the portion of the image.
In an analogous field of endeavor, Eyjolfsdottir teaches, training data that is generated, at least, by: (Eyjolfsdottir, ¶0156: “After the scene is simulated… used as a training label for the synthetic image”; Applicant’s specification, ¶0202: “The training data may be generated by the vehicles, and/or may be generated in a simulation”) determining, using one or more second machine learning models and based at least on image data representative of an image, a first classification corresponding to an object that is depicted at a portion of the image; (Eyjolfsdottir, ¶0019: “training data generation can include: determining a set of images for an object, wherein the set of images includes RGB images and depth images; generating an object mesh using the set of images; determining object components (e.g., object face, object part”). Eyjolfsdottir further teaches, and generating the training data to include the image data (Eyjolfsdottir, ¶0139: “generate training data for training the one or more object detectors in S800. The training data is preferably one or more synthetic or augmented images”) and first data indicating that the object at least partially occludes the driving surface at the portion of the image. (Eyjolfsdottir, ¶0139: “the training label (e.g., training target) can include: an object keypoint associated with the portion of the object (mesh) depicted in the image”; ¶0168: “the deployment environment is a physical navigation space (e.g., road”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi using the teachings of Eyjolfsdottir to introduce generating labeled training data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of train a machine learning model for image object classification. Therefore, it would have been obvious to combine the analogous arts Gummadi and Eyjolfsdottir to obtain the above-described limitations in claim 1. However, the combination of Gummadi and Eyjolfsdottir does not explicitly teach, determining, based at least on map data representing a map associated with an environment, a point within the environment that should be depicted within the portion of the image; determining, based at least on the map data, that the map indicates a second classification of driving surface for the point within the environment; determining, based at least on the first classification being different from the second classification of the driving surface, that the object at least partially occludes the driving surface at the portion of the image.
In another analogous field of endeavor, Nayak teaches, determining, based at least on map data representing a map associated with an environment, (Nayak, col. 9, lines 36-38: “The map data may include one or more data points indicating attributes (e.g., geographical attributes) associated with the location”) a point within the environment that should be depicted within the portion of the image; (Nayak, col. 9, lines 50-53: “the sensor data may indicate an existence of a traffic barrier within the location of the WWD event; whereas, the map data does not include a datapoint that defines the traffic barrier within the location”) determining, based at least on the map data, that the map indicates a second classification of driving surface for the point within the environment; (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”) determining, based at least on the first classification being different from the second classification of the driving surface, that the object at least partially occludes the driving surface at the portion of the image; (Nayak, col. 9, lines 48-51: “determine a difference of one or more objects as indicated by the map data and the sensor data. For example, the sensor data may indicate an existence of a traffic barrier within the location”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir using the teachings of Nayak to introduce a comparison of sensor data and map data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically detecting obstacles on the road for safe autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 1.
Regarding claim 2, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 1, wherein the training data is further generated by determining, based at least on the first classification including one or more object classifications, (Gummadi, ¶0002: “the cell is occupied and object classifications of objects that could occupy the cells with cells in the navigation area based on sensor data”) that the first classification is different from the second classification of the driving surface. (Gummadi, ¶0021: “classifying 1263 each of the second transformed points that represents a non-obstructed space”).
Regarding claim 3, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 1, wherein the training data is further generated by determining, based at least on the first classification not including one or more object classifications associated with driving surfaces, (Gummadi, ¶0021: “classifying 1263 each of the second transformed points that represents a non-obstructed space”) that the first classification is different from the second classification of the driving surface. (Gummadi, ¶0018: “detection of drivable surfaces, where the drivable surfaces can include, but are not limited to, road”).
Regarding claim 5, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 1, wherein the generating the first data comprises generating the first data representing a label associated with the portion of the image, (Gummadi, ¶0006: “The semantic segmentation output point (XRGB, YRGB) can optionally include values including 0=non-drivable, 1=road, 2=sidewalk, 3=terrain, 4=lane marking, >0=drivable, 0=obstructed”) the label indicating one of: the driving surface is occluded by a dynamic object at the portion of the image; (Nayak, col. 21, lines 44-45: “The sensor data may indicate geographical attributes and/or dynamic attributes”) or the driving surface is occluded by a static object at the portion of the image. (Nayak, col. 9, lines 60-61: “obstructing objects (e.g., another vehicle, a barrier, a cone”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teachings of Nayak to introduce detecting dynamic or static object. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically detecting moving or still objects of an environment for autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 5.
Regarding claim 6, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 1, wherein the training data is further generated by: determining, using the one or more second machine learning models and based at least on the image data, (Gummadi, ¶0003: “classifying the image data based on a machine learning model forming point classifications”) a third classification corresponding to a second portion of the image; (Gummadi, ¶0006: “classifying each of the second transformed points that represents a non-obstructed space and an obstructed space within a pre-selected area surrounding the autonomous vehicle”) determining, based at least on the map data, that the map indicates a fourth classification of the driving surface for a second point within the environment that corresponds to the second portion of the image; (Nayak, col. 9, lines 50-53: “the sensor data may indicate an existence of a traffic barrier within the location of the WWD event; whereas, the map data does not include a datapoint that defines the traffic barrier within the location”) determining, based at least on the third classification and the fourth classification, whether the driving surface is occluded at the second portion of the image; (Nayak, col. 15, lines 40-43: “In the second scenario 400B, a road work 413 having a double-lane closure is impacting the second portion, thereby forcing the vehicles 401 and 403 to drive through the lane 409C”) and generating the training data to further include second data indicating whether the driving surface is occluded at the second portion of the image. (Nayak, col. 15, lines 36-37: “a second subsequent portion of the road segment 409 including a road work”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teachings of Nayak to introduce detection of occluding objects in different portions of an image. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically asses which portion of the road are occluded. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 6.
Regarding claim 7, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 1, wherein the training data is further generated by: determining, based at least on the map data, a first distance associated with the point within the environment; (Nayak, col. 8, lines 37-41: “detects that a relative distance between the front of the vehicle and another object (e.g., a vehicle, a barrier, etc.) is less than a threshold distance (e.g., the relative distance becomes less than 4.2 meters) at the location”) and determining, based at least on point cloud data, a second distance associated with the point within the environment, (Gummadi, ¶0031: “LIDAR 420 can provide data on the range or distance to surfaces around autonomous vehicle 121”) wherein the determining that the object at least partially occludes the driving surface at the portion of the image is further based at least on the first distance and the second distance. (Nayak, col. 8, lines 35-39: “detects a change with respect to one or more road objects at the location as indicated by map data; (5) detects that a relative distance between the front of the vehicle and another object (e.g., a vehicle, a barrier”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teachings of Nayak to introduce detecting a change between map data and Lidar data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically operating an autonomous vehicle to avoid collisions with obstructions. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 7.
Regarding claim 8, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 7, wherein the training data is further generated by: determining whether the second distance is within a threshold distance to the first distance, wherein the determining that the object at least partially occludes the driving surface at the portion of the image is further based at least on whether the first distance is within the threshold distance to the second distance. (Nayak, col. 8, lines 35-40: “detects a change with respect to one or more road objects at the location as indicated by map data; (5) detects that a relative distance between the front of the vehicle and another object (e.g., a vehicle, a barrier, etc.) is less than a threshold distance (e.g., the relative distance becomes less than 4.2 meters”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teachings of Nayak to introduce determining whether a distance threshold is met. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of identifying an occluding object between an autonomous vehicle and map object. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 8.
Regarding claim 19, Gummadi teaches, One or more processors comprising processing circuitry to: (Gummadi, ¶0005: “the system can include, but is not limited to including, a pre-processor”) cause a machine to navigate within an environment (Gummadi, ¶0003: “estimating free space based on image data and point cloud data, the free space used for navigating an autonomous vehicle”) based at least on one or more first machine learning models processing sensor data (Gummadi, ¶0002: “applies a machine learning model to camera images of a navigation area”) obtained using one or more sensors of the machine, (Gummadi, ¶0005: “receiving camera image data from at least one camera into the autonomous vehicle”) wherein the one or more first machine learning models are trained using training data (Gummadi, ¶0022: “classifications 215 can be based on the training received by a machine learning model”). However, Gummadi does not explicitly teach, training data that is generated, at least, by: determining, based at least on image data representative of an image, a first classification corresponding to a traffic object that is depicted at a portion of the image; determining, based at least on map data representing a map associated with an environment, a point within the environment that should be depicted within the portion of the image; determining, based at least on the map data, that the map indicates a second classification of the traffic object for the point within the environment; determining, based at least on the first classification corresponding to the second classification of the traffic object, that the traffic object is depicted at the portion of the image; and generating the training data to include the image data and first data indicating that the traffic object is depicted at the portion of the image.
In an analogous field of endeavor, Eyjolfsdottir teaches, training data that is generated, at least, by: (Eyjolfsdottir, ¶0156: “After the scene is simulated… used as a training label for the synthetic image”; Applicant’s specification, ¶0202: “The training data may be generated by the vehicles, and/or may be generated in a simulation”) determining, based at least on image data representative of an image, a first classification corresponding to a traffic object that is depicted at a portion of the image; (Eyjolfsdottir, ¶0019: “training data generation can include: determining a set of images for an object, wherein the set of images includes RGB images and depth images; generating an object mesh using the set of images; determining object components (e.g., object face, object part”). Eyjolfsdottir further teaches, and generating the training data to include the image data and first data indicating that the traffic object is depicted at the portion of the image.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi using the teachings of Eyjolfsdottir to introduce generating labeled training data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of train a machine learning model for image object classification. Therefore, it would have been obvious to combine the analogous arts Gummadi and Eyjolfsdottir to obtain the above-described limitations in claim 19. However, the combination of Gummadi and Eyjolfsdottir does not explicitly teach, determining, based at least on map data representing a map associated with an environment, a point within the environment that should be depicted within the portion of the image; determining, based at least on the map data, that the map indicates a second classification of the traffic object for the point within the environment; determining, based at least on the first classification corresponding to the second classification of the traffic object, that the traffic object is depicted at the portion of the image.
In another analogous field of endeavor, Nayak teaches, determining, based at least on map data representing a map associated with an environment, (Nayak, col. 9, lines 36-38: “The map data may include one or more data points indicating attributes (e.g., geographical attributes) associated with the location”) a point within the environment that should be depicted within the portion of the image; (Nayak, col. 9, lines 50-53: “the sensor data may indicate an existence of a traffic barrier within the location of the WWD event; whereas, the map data does not include a datapoint that defines the traffic barrier within the location”) determining, based at least on the map data, that the map indicates a second classification of the traffic object for the point within the environment; (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”) determining, based at least on the first classification corresponding to the second classification of the traffic object, that the traffic object is depicted at the portion of the image. (Nayak, col. 9, lines 48-51: “determine a difference of one or more objects as indicated by the map data and the sensor data. For example, the sensor data may indicate an existence of a traffic barrier within the location”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir using the teachings of Nayak to introduce a comparison of sensor data and map data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically detecting obstacles on the road for safe autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 19.
Regarding claim 20, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The one or more processors of claim 19, wherein the one or more processor are comprised in at least one of: a control system for an autonomous or semi-autonomous machine (Gummadi, ¶0005: “The system of the present teachings for estimating free space based on image data and point cloud data, where the free space can be used for navigating an autonomous vehicle”).
Regarding claim 21, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The one or more processors of claim 19, wherein the training data is further generated by determining, based at least on the first classification including one or more object classifications associated with the traffic object, (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”; Applicant’s specification, ¶0026: “the traffic objects (e.g., road boundary, road markings, etc.”) that the first classification corresponds to the second classification of the traffic object. (Nayak, col. 16, lines 56-58: “calculation module 303 may identify a correct portion of a road (as indicated in map data”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teaching of Nayak to introduce corresponding lane markings between sensor and map data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of operating a vehicle within the confirmed lane markings for autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the invention in claim 21.
Claims 10-12, 14 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Gummadi et al. (US 2021/0012120 A1) in view of Nayak et al. (US 11,551,548 B1).
Regarding claim 10, Gummadi teaches, A system comprising: one or more processors (Gummadi, ¶0005: “the system can include, but is not limited to including, a pre-processor”) to: determine, based at least on image data representative of an image, a first classification corresponding to an object that is depicted at a portion of the image; (Gummadi, ¶0003: “an image coordinate system associated with the image data and classifying each of the first transformed points that represents an obstructed space and the non-obstructed space”). However, Gummadi does not explicitly teach, determine, based at least on map data representing a map associated with an environment, a point within the environment that should be depicted within the portion of the image; determine, based at least on the map data, that the map indicates a second classification of a traffic object for the point within the environment; determine, based at least on the first classification being different from the second classification of the traffic object, that the object at least partially occludes the traffic object at the portion of the image; and cause, based at least on the object at least partially occluding the traffic object at the portion of the image, a machine to navigate within the environment.
In an analogous field of endeavor, Nayak teaches, determine, based at least on map data representing a map associated with an environment, (Nayak, col. 9, lines 36-38: “The map data may include one or more data points indicating attributes (e.g., geographical attributes) associated with the location”) a point within the environment that should be depicted within the portion of the image; (Nayak, col. 9, lines 36-38: “The map data may include one or more data points indicating attributes (e.g., geographical attributes) associated with the location”) determine, based at least on the map data, that the map indicates a second classification of a traffic object for the point within the environment; (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”; Applicant’s specification, ¶0026: “the traffic objects (e.g., road boundary, road markings, etc.”) determine, based at least on the first classification being different from the second classification of the traffic object, that the object at least partially occludes the traffic object at the portion of the image; (Nayak, col. 9, lines 48-51: “determine a difference of one or more objects as indicated by the map data and the sensor data. For example, the sensor data may indicate an existence of a traffic barrier within the location”) and cause, based at least on the object at least partially occluding the traffic object at the portion of the image, a machine to navigate within the environment. (Nayak, col. 10, lines 18-20: “assessment platform 123 causes the vehicle 105 to rely more on sensor data rather than map data as the vehicle 105 navigates through the location”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi using the teaching of Nayak to introduce map data of an environment representing lane markings. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of autonomously operating a vehicle within the lane markings based on the map data and sensor data. Therefore, it would have been obvious to combine the analogous arts Gummadi and Nayak to obtain the invention in claim 10.
Regarding claim 11, Gummadi in view of Nayak teaches, The system of claim 10, wherein the determination of whether the traffic object is occluded at the portion of the image comprises: one or more processors are further to determine, based at least on the first classification including one or more object classifications, (Nayak, col. 9, lines 50-53: “the sensor data may indicate an existence of a traffic barrier within the location of the WWD event; whereas, the map data does not include a datapoint that defines the traffic barrier within the location”) that the first classification is different from the second classification of the traffic object. (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”; Applicant’s specification, ¶0026: “the traffic objects (e.g., road boundary, road markings, etc.”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Nayak using the teaching of Nayak to introduce a comparison of sensor data and map data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically detecting obstacles on the road that aren’t present in the map for safe autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi and Nayak to obtain the invention in claim 11.
Regarding claim 12, Gummadi in view of Nayak teaches, The system of claim 10, wherein the determination of whether the traffic object is occluded at the portion of the image comprises: one or more processors are further to determine, based at least on the first classification not including one or more object classifications associated with traffic objects, (Nayak, col. 9, lines 50-53: “the sensor data may indicate an existence of a traffic barrier within the location of the WWD event; whereas, the map data does not include a datapoint that defines the traffic barrier within the location”) that the first classification is different from the second classification of the traffic object. (Nayak, col. 9, lines 48-51: “determine a difference of one or more objects as indicated by the map data and the sensor data. For example, the sensor data may indicate an existence of a traffic barrier within the location”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Nayak using the teaching of Nayak to introduce a comparison of sensor data and map data. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically detecting road barriers based on map and sensor data for safe autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi and Nayak to obtain the invention in claim 12.
Regarding claim 14, Gummadi in view of Nayak teaches, The system of claim 10, wherein further comprising generating first data that represents a label associated with the portion of the image, the label indicating one of: the traffic object is not occluded at the portion of the image; (Gummadi, ¶0006: “The semantic segmentation output point (XRGB, YRGB) can optionally include values including 0=non-drivable, 1=road, 2=sidewalk, 3=terrain, 4=lane marking, >0=drivable, 0=obstructed”) the traffic object is occluded by a dynamic object at the portion of the image; (Nayak, col. 21, lines 44-45: “The sensor data may indicate geographical attributes and/or dynamic attributes”) or the driving surface is occluded by a static object at the portion of the image. (Nayak, col. 9, lines 60-61: “obstructing objects (e.g., another vehicle, a barrier, a cone”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Nayak using the additional teachings of Nayak to introduce detecting dynamic or static object. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically detecting moving or still objects of an environment for autonomous driving. Therefore, it would have been obvious to combine the analogous arts Gummadi and Nayak to obtain the invention in claim 14.
Regarding claim 16, Gummadi in view of Nayak teaches, The system of claim 10, wherein the one or more processors are further to: determine, based at least on the map data, a first distance associated with the point within the environment; (Nayak, col. 8, lines 37-41: “detects that a relative distance between the front of the vehicle and another object (e.g., a vehicle, a barrier, etc.) is less than a threshold distance (e.g., the relative distance becomes less than 4.2 meters) at the location”) and determine, based at least on point cloud data, a second distance associated with the point within the environment, (Gummadi, ¶0031: “LIDAR 420 can provide data on the range or distance to surfaces around autonomous vehicle 121”) wherein the determination of whether that the object at least partially occludes the traffic object is occluded at the portion of the image is further based at least on the first distance and the second distance. (Nayak, col. 8, lines 35-39: “detects a change with respect to one or more road objects at the location as indicated by map data; (5) detects that a relative distance between the front of the vehicle and another object (e.g., a vehicle, a barrier”).
Regarding claim 17, Gummadi in view of Nayak teaches, The system of claim 16, wherein the one or more processors are further to: generate a first determination on whether the traffic object is occluded at the portion of the image based at least on the first classification being different from the second classification; (Nayak, col. 9, lines 48-49: “determine a difference of one or more objects as indicated by the map data and the sensor data”) and generate a second determination that the object at least partially occludes the traffic object at the portion of the image based at least on the first distance and the second distance, (Nayak, col. 8, lines 35-39: “detects a change with respect to one or more road objects at the location as indicated by map data; (5) detects that a relative distance between the front of the vehicle and another object (e.g., a vehicle, a barrier”) wherein the determination of whether the traffic object is occluded at the portion of the image is based at least on the first determination and the second determination. (Nayak, col. 19, lines 17-22: “identifying a correct portion of a road (as indicated in map data); (2) determining whether a route from the current position of the vehicle 105 to the correct portion does not interfere with any obstructing objects (e.g., another vehicle, a barrier, a cone, etc.); and (3) if such route exists, causing the vehicle 105 to move to the correct portion”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Nayak using the additional teachings of Nayak to introduce detecting an occluding object in the environment. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of automatically operating an autonomous vehicle to avoid collisions with obstructions. Therefore, it would have been obvious to combine the analogous arts Gummadi and Nayak to obtain the invention in claim 17.
Regarding claim 18, Gummadi in view Nayak teaches, The system of claim 10, wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine (Gummadi, ¶0005: “The system of the present teachings for estimating free space based on image data and point cloud data, where the free space can be used for navigating an autonomous vehicle”).
Claims 4 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Gummadi et al. (US 2021/0012120 A1), in view of Eyjolfsdottir et al. (US 2022/0019852 A1), in further view of in view of Nayak et al. (US 11,551,548 B1) and still in further view of Smolyanskiy et al. (US 2021/0150230 A1).
Regarding claim 4, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The method of claim 1, wherein the determining that the point within the environment should be depicted within the portion of the image (Nayak, col. 9, lines 57-58: “identifying a correct portion of a road (as indicated in map data)”) comprises: obtaining the map data associated with the environment, the map data representing (Nayak, col. 7, lines 61-62: “provide content or data (e.g., including geographic data, parametric representations of mapped features”) at least the second classification (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teachings of Nayak to introduce map data representing environmental features. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of detecting environmental features for operating an autonomous vehicle. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the above-described limitations in claim 4. However, the combination of Gummadi, Eyjolfsdottir and Nayak does not explicitly teach, a three-dimensional location for the point within the environment; and projecting the three-dimensional location to a two-dimensional location associated with the portion of the image.
In an analogous field of endeavor, Smolyanskiy teaches, a three-dimensional location for the point (Smolyanskiy, ¶0062: “identify 3D locations of objects in the world space corresponding to each pixel”) within the environment; (Smolyanskiy, ¶0062: “location in a 3D representation of the environment (e.g., a 3D map or some other world space) and projecting the three-dimensional location to a two-dimensional location associated with the portion of the image. (Smolyanskiy, ¶0007: “projecting a LiDAR point cloud into one or more height maps in a top-down view) and/or images of the 3D space (e.g., by unprojecting an image into world space and projecting into a top-down view”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the teachings of Smolyanskiy to introduce projecting a 3D map into a 2D view. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of autonomous navigation within an environment based on the top down 2D map of the road. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir, Nayak and Smolyanskiy to obtain the invention in claim 4.
Regarding claim 22, Gummadi in view of Eyjolfsdottir and in further view of Nayak teaches, The one or more processors of claim 19, wherein the determination that the point within the environment should be depicted at the portion of the image (Nayak, col. 9, lines 57-58: “identifying a correct portion of a road (as indicated in map data)”) comprises: obtaining the map data associated with the environment, the map data representing (Nayak, col. 7, lines 61-62: “provide content or data (e.g., including geographic data, parametric representations of mapped features”) at least the second classification and a three-dimensional location for the point within the environment; (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”) determining a location of a second machine that obtained the image data within the environment; (Nayak, col. 8, lines 50-52: “one or more detection entities 113 proximate to the location of the vehicle 105 may observe the vehicle 105, one or more vehicles proximate to the vehicle”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the additional teachings of Nayak to introduce map data representing environmental features. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of detecting environmental features for operating an autonomous vehicle. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir and Nayak to obtain the above-described limitations in claim 22. However, the combination of Gummadi, Eyjolfsdottir and Nayak does not explicitly teach, projecting, based at least on the location of the second machine, the three-dimensional location to a two-dimensional location associated with the portion of the image
In an analogous field of endeavor, Smolyanskiy teaches, projecting, based at least on the location of the second machine, the three-dimensional location to a two-dimensional location associated with the portion of the image (Smolyanskiy, ¶0007: “projecting a LiDAR point cloud into one or more height maps in a top-down view) and/or images of the 3D space (e.g., by unprojecting an image into world space and projecting into a top-down view”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Eyjolfsdottir and in further view of Nayak using the teachings of Smolyanskiy to introduce projecting a 3D map into a 2D view. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of autonomous navigation within an environment based on the top down 2D map of the road. Therefore, it would have been obvious to combine the analogous arts Gummadi, Eyjolfsdottir, Nayak and Smolyanskiy to obtain the invention in claim 22.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Gummadi et al. (US 2021/0012120 A1), in view of Nayak et al. (US 11,551,548 B1) and in further view of Smolyanskiy et al. (US 2021/0150230 A1).
Regarding claim 13, Gummadi in view of Nayak teaches, The system of claim 10, wherein the determination that the point within the environment should be depicted at the portion of the image (Nayak, col. 9, lines 57-58: “identifying a correct portion of a road (as indicated in map data)”) comprises: obtaining the map data associated with the environment, the map data representing (Nayak, col. 7, lines 61-62: “provide content or data (e.g., including geographic data, parametric representations of mapped features”) at least the second classification and a three-dimensional location for the point within the environment; (Nayak, col. 10, lines 60-61: “one or more data points indicating the one or more road lane markings in map data”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Nayak using the additional teachings of Nayak to introduce map data representing environmental features. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of detecting environmental features for operating an autonomous vehicle. Therefore, it would have been obvious to combine the analogous arts Gummadi and Nayak to obtain the above-described limitations in claim 13. However, the combination of Gummadi and Nayak does not explicitly teach, projecting the three-dimensional location to a two-dimensional location associated with the portion of the image.
In an analogous field of endeavor, Smolyanskiy teaches, projecting the three-dimensional location to a two-dimensional location associated with the portion of the image. (Smolyanskiy, ¶0007: “projecting a LiDAR point cloud into one or more height maps in a top-down view) and/or images of the 3D space (e.g., by unprojecting an image into world space and projecting into a top-down view”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Gummadi in view of Nayak using the teachings of Smolyanskiy to introduce projecting a 3D map into a 2D view. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of autonomous navigation within an environment based on the top down 2D map of the road. Therefore, it would have been obvious to combine the analogous arts Gummadi, Nayak and Smolyanskiy to obtain the invention in claim 13.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHRAZUL ISLAM whose telephone number is (571)270-0489. The examiner can normally be reached Monday-Friday: 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Saini Amandeep can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEHRAZUL ISLAM/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights