DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Foreign Priority is acknowledged from German application DE10 2023 202 244.3 with a filing date of 03/13/2023.
Information Disclosure Statement
The information disclosure statements (“IDS”) filed on 01/30/2024 and 04/29/2024 were reviewed and the listed references were noted.
Drawings
The 6-page drawings have been considered and placed on record in the file.
Claim Objections
Claim 3 is objected to because of the following informalities: the claim recites “…an evaluation including a comparison, of the heights based on…” in which it is assumed that a typographical error has occurred with the inclusion of the comma. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 and 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Agia et al. (US 20230267615 A1) in view of Pham (US 20160140414 A1) and Kedarisetti et al. (US 20200327313 A1).
Regarding Claim 1, Agia teaches "A method for a three-dimensional road area segmentation for a vehicle, comprising the following steps: obtaining input data, the input data including multiple elements representing a scene of an environment of the vehicle"; (Agia, Paras. 3 and 44, teaches using cameras mounted on the vehicle to capture images of an environment surrounding the vehicle wherein detection and ranging sensors and LiDAR sensors can scan the environment surrounding the vehicle and generate 3D point clouds that represent each scan of the environment and wherein prediction may be performed for one or more pixels of an image that is generated based on the point clouds, i.e., obtain input data including multiple elements representing a scene of an environment of the vehicle being the captured sensor data and images with pixels);
"the input data indicating heights of the scene"; (Agia, Paras. 11 and 76, teaches the BEV feature map defines three elements for each pixel in the image including height, intensity, and density and wherein the refined elevation map contains estimated height values for each of the pixels in the refined labelled image, i.e., input data indicates heights of the scene);
"the input data resulting at least partially from a sensor detection of the environment"; (Agia, Para. 3, teaches using cameras mounted on the vehicle to capture images of an environment surrounding the vehicle wherein detection and ranging sensors and LiDAR sensors can scan the environment surrounding the vehicle and generate 3D point clouds that represent each scan of the environment, i.e., input data resulting from a sensor detection of the environment);
"and carrying out the three-dimensional road area segmentation based on the input data"; (Agia, Paras. 6 and 7, teaches performing semantic segmentation of a sequence of point clouds to reduce time and resources required to detect road surfaces in the 3D point clouds and classify the detected objects wherein a sequence of point clouds are received in which each point cloud represents a 3D scan of an environment at a different point in time and generating the road surface segmentation map, i.e., carry out 3D road area segmentation based on the input data);
"the three-dimensional road area segmentation including: classifying the elements based on the heights into different classes including at least a road class and an elevated surface class to provide a classification result for each of the elements"; (Agia, FIG. 6 and Paras. 98-100, and 105, teaches each pixel in the labelled image is associated with a class label from a set of class labels including road surface class labels such as the road and sidewalk wherein an elevation map contains a set of refined elevation data and estimated height values for each of the pixels in the labelled image wherein the labelled image is smoothed using the BEV feature map which includes average height, i.e., classify the elements or pixels based on heights into different classes including the road and the sidewalk being the elevate surface to provide a classification result for each of the elements or pixels).
However, Agia does not explicitly teach "determining a density level for multiple of the elements based on the classification results, the density levels being based on a density of the different classes; and identifying at least a road surface and an elevated surface in the scene by forming the surfaces based on the density levels".
In an analogous field of endeavor, Pham teaches "determining a density level for multiple of the elements based on the classification results, the density levels being based on a density of the different classes"; (Pham, Para. 44 and Claim 1, teaches an image including objects of a plurality of classes classified according to a predetermined rule wherein the first calculator divides the image into a plurality of regions including a plurality of pixels and calculates density of each object class captured in the region and a second calculator to calculate likelihood of the each object class captured in each of the regions from the density of objects captured in each of the regions and a generator to generate density data assigned to a position corresponding to each of the regions in the image assigned with the density of the object class having at least higher likelihood than the lowest likelihood calculated for object classes captured in the region, i.e., determine density level for the pixels or elements based on classification wherein the density levels are based on a density of different classes).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia by including the determination of density for multiple elements based on classification and a density of different classes taught by Pham. One of ordinary skill in the art would be motivated to combine the references since it accurately calculates density (Pham, Para. 4, teaches the motivation of combination to be to accurately calculate density of each object class in an image with a plurality of classes of objects).
However, the combination of references of Agia in view of Pham does not explicitly teach "and identifying at least a road surface and an elevated surface in the scene by forming the surfaces based on the density levels".
In an analogous field of endeavor, Kedarisetti teaches "and identifying at least a road surface and an elevated surface in the scene by forming the surfaces based on the density levels"; (Kedarisetti, Para. 240, teaches the paths of targets of different classes may be tracked through the lifetime of the target in the scene wherein a high density of human target trajectories corresponding to locations and direction of the path may confirm the path of the first human target corresponds to the location of a sidewalk and a high density of vehicle trajectories along the path of the vehicle and aligned with the path of the vehicle verifies the existence of the road in which target trajectory information may be obtained directly from the metadata provided to the global metrics/scene segmentation module, i.e., identify a road surface and an elevated surface being the sidewalk in the scene by forming the surfaces based on density levels).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia and Pham by including the identification of a road surface and elevated surface by forming the surfaces based on density levels taught by Kedarisetti. One of ordinary skill in the art would be motivated to combine the references since it improves plurality video playback (Kedarisetti, Para. 251, teaches the motivation of combination to be to improve playing back of a plurality of video feeds).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Regarding Claim 8, the combination of references of Agia in view of Pham and Kedarisetti teaches "The method of claim 1, wherein a control of the vehicle is initiated based on the identified surfaces, the control including autonomously driving the vehicle"; (Agia, Para. 6, teaches an automated driving system which autonomously controls operation of the vehicle based on information obtained from the different types of sensors and may use the road surface semantic segmentation results to perform vehicle localization and path planning, i.e., control of vehicle initiated based on identified surfaces and control including autonomously driving the vehicle).
Claim 9 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Agia, Pham, and Kedarisetti references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of the Agia in view of Pham and Kedarisetti references discloses a computer readable storage medium (for example, see Agia, Paragraph 71).
Claim 10 recites an apparatus with elements corresponding to the steps recited in Claim 1. Therefore, the recited elements of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the Agia in view of Pham and Kedarisetti references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of the Agia in view of Pham and Kedarisetti references discloses a processor and memory (for example, see Agia, Paragraph 29).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Agia in view of Pham, Kedarisetti, and Zhang et al. (US 20240387050 A1).
Regarding Claim 2, the combination of references of Agia in view of Pham and Kedarisetti teaches "The method of claim 1, wherein the elements are arranged in an at least two-dimensional arrangement"; (Agia, Abstract, teaches a labelled BEV image wherein each pixel in the labelled BEV image is associated with a class label, i.e., the element or pixels are arranged in a 2D arrangement being the BEV image).
However, the combination of references of Agia in view of Pham and Kedarisetti does not explicitly teach "the density levels being determined based on the density of the different classes according to the two-dimensional arrangement by determining a density for a position in the two-dimensional arrangement based on a percentage of elements belonging to the same class in a vicinity of the position".
In an analogous field of endeavor, Zhang teaches "the density levels being determined based on the density of the different classes according to the two-dimensional arrangement by determining a density for a position in the two-dimensional arrangement based on a percentage of elements belonging to the same class in a vicinity of the position"; (Zhang, Paras. 197-198 and 212, teaches obtaining each respective individual feature score using the corresponding one or more objects for a respective morphological class is performed by determining a number of objects within a specified area within the first image such as proportion, density, or area estimation, that fall within the respective morphological class and wherein the first image is segmented into ROIs identified as corresponding to a first morphological class, a second morphological class etc., i.e., density levels determined based on density of different classes according to the 2D arrangement being the image by determining a density for a position based on a percentage or proportion of elements belonging to the same class in a vicinity of the position).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, and Kedarisetti by including the density levels being determined based on the density of different classes by determining a density for a position based on a percentage of elements belonging to the same class in a vicinity of the position taught by Zhang. One of ordinary skill in the art would be motivated to combine the references since it improves predictive utility (Zhang, Para. 316, teaches the motivation of combination to be to improve predictive utility and recognize various conditions being the different classes).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Agia in view of Pham, Kedarisetti, Kang et al. (US 20170223235 A1), and Gorritxategi et al. (US 20160069856 A1).
Regarding Claim 3, the combination of references of Agia in view of Pham and Kedarisetti does not explicitly teach "The method of claim 1, wherein the classification is carried out based on an evaluation including a comparison, of the heights based on at least one threshold, the threshold being dynamically determined based on a standard deviation of the heights”.
In an analogous field of endeavor, Kang teaches "The method of claim 1, wherein the classification is carried out based on an evaluation including a comparison, of the heights based on at least one threshold"; (Kang, Para. 32, teaches determining whether to classify the object as being planar due to the measured height being below the threshold or classify the object as being a 3D object due to the measured height being at or above the threshold, i.e., classification based on a comparison of the object heights and a threshold).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, and Kedarisetti by including the classification based on heights compared to a threshold taught by Kang. One of ordinary skill in the art would be motivated to combine the references since it compensates for noise (Kang, Para. 32, teaches the motivation of combination to be to compensate for noise).
However, the combination of references of Agia in view of Pham, Kedarisetti, and Kang does not explicitly teach "the threshold being dynamically determined based on a standard deviation of the heights".
In an analogous field of endeavor, Gorritxategi teaches "the threshold being dynamically determined based on a standard deviation of the heights"; (Gorritxategi, Para. 92, teaches binarising the image with a dynamic threshold based on the standard deviation of luminance in different zones of the image, i.e., threshold is dynamically determined based on a standard deviation).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, Kedarisetti, and Kang wherein the threshold is based on heights for classification by including the threshold being dynamically determined based on a standard deviation taught by Gorritxategi. One of ordinary skill in the art would be motivated to combine the references since it improves detection sensitivity (Gorritxategi, Para. 88, teaches the motivation of combination to be to improve detection sensitivity).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Agia in view of Pham, Kedarisetti, Ferrari et al. (US 20190379847 A1), and Bern et al. (US 6384826 B1).
Regarding Claim 4, the combination of references of Agia in view of Pham and Kedarisetti teaches "The method of claim 1, wherein the three-dimensional road area segmentation includes carrying out a density-based multiple surface aggregation"; (Agia, Paras. 7-8 and 52, teaches receiving a sequence of point clouds representing a three-dimensional scan of an environment at a different point in time which are processed to generate a densified, aggregated point cloud wherein the road surface segmentation map is generated in which each pixel is associated with a class label from the set of class labels wherein each data point of the point cloud is a reflection from an object in the environment such as roadways, intersections, sidewalks, and crosswalks and wherein each pixel has a defined density, i.e., 3D road segmentation includes density-based multiple surface aggregation being the densified aggregated point cloud in an environment comprising multiple surfaces).
However, the combination of references of Agia in view of Pham and Kedarisetti does not explicitly teach "the density-based multiple surface aggregation including identifying core cells from the elements having a density level greater than a predefined threshold and border cells from the elements having a density level smaller than the threshold, the surfaces being formed by evaluating the identified cells using a Breath-First search strategy”.
In an analogous field of endeavor, Ferrari teaches "the density-based multiple surface aggregation including identifying core cells from the elements having a density level greater than a predefined threshold and border cells from the elements having a density level smaller than the threshold"; (Ferrari, FIG. 5 and Para. 39, teaches the controller may be configured to aggregate the obscured or inoperative pixels into one or more pixel groups wherein the image is divided into six pixel groups in which pixel group 142B exceeds above the predetermined threshold density value and the surrounding pixel groups 142A and 142C-F fall below the predetermined threshold density value, i.e., aggregation includes identify a core cell being the 142B pixel group as having a density greater than the threshold and border cells being the other pixel groups surrounding the core group as having a density level smaller than the threshold).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, and Kedarisetti wherein aggregation is density-based multiple surface aggregation by including the identification of cells having a density greater than a threshold and cells having a density smaller than a threshold taught by Ferrari. One of ordinary skill in the art would be motivated to combine the references since it improves monitoring sensor performance.
However, the combination of references of Agia in view of Pham, Kedarisetti, and Ferrari does not explicitly teach "the surfaces being formed by evaluating the identified cells using a Breath-First search strategy".
In an analogous field of endeavor, Bern teaches "the surfaces being formed by evaluating the identified cells using a Breath-First search strategy"; (Bern, Col. 7 lines 30-44, teaches the remaining trains form a quilted surface in which the manifold extraction step computes the outside of this quilted surface by a breadth-first search on triangles similar to the breadth-first search that oriented the triangles, i.e., forming the surface by evaluating the cells or triangles using a breadth-first search strategy).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, Kedarisetti, and Ferrari by including the surfaces being formed using breadth first search taught by Bern. One of ordinary skill in the art would be motivated to combine the references since it improves efficiency (Bern, Col. 6 lines 40-65, teaches the motivation of combination to be to improve efficiency).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Agia in view of Pham, Kedarisetti, Ferrari, Bern, and Gray et al. (US 20200324786 A1).
Regarding Claim 5, the combination of references of Agia in view of Pham, Kedarisetti, Ferrari, and Bern does not explicitly teach "The method of claim 4, wherein the three-dimensional road area segmentation includes fitting a planar surface to each of the formed surfaces to determine a lateral road profile".
In an analogous field of endeavor, Gray teaches "The method of claim 4, wherein the three-dimensional road area segmentation includes fitting a planar surface to each of the formed surfaces to determine a lateral road profile"; (Gray, Para. 45, teaches calculating a road profile using the sensor data by means of semantically locating the road features using segmentation methods based on fitting a plane to the road features, i.e., the road segmentation includes fitting a planar surface to the formed surfaces to determine a road profile).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, Kedarisetti, Ferrari, and Bern by including the segmentation including planar surface fitting to surfaces to determine a road profile taught by Gray. One of ordinary skill in the art would be motivated to combine the references since it improves vehicle operation (Gray, Para. 3, teaches the motivation of combination to be to improve the operation of the vehicles).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Agia in view of Pham, Kedarisetti, and Spangenberg et al. (US 20250231299 A1).
Regarding Claim 6, the combination of references of Agia in view of Pham and Kedarisetti does not explicitly teach "The method of claim 1, wherein, before carrying out the three-dimensional road area segmentation, a filtering procedure is carried out to filter the elements depending on their flatness and/or depth range and/or height relative to an input planar surface of the input data and position".
In an analogous field of endeavor, Spangenberg teaches "The method of claim 1, wherein, before carrying out the three-dimensional road area segmentation, a filtering procedure is carried out to filter the elements depending on their flatness and/or depth range and/or height relative to an input planar surface of the input data and position"; (Spangenberg, Paras. 21-27, teaches filtering the crop height measurements with a smoothing filter in which ground-surface heights in the crop height measurements are removed using a vertical threshold to remove undulating ground surface heights and/or removing false peaks under a horizontal threshold in a lengthwise scan size of samples prior to a segmentation module configured to automatically segment the crop height measurements into a plurality of mutually separate plot profiles corresponding to respective mutually separate plots of the crop along a direction of travel, i.e., filter the elements depending on height relative to an input planar surface being the ground and position being the lengthwise scan size of samples before carrying out segmentation).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, and Kedarisetti by including the filtering based on heights relative to an input surface and position prior to segmentation taught by Spangenberg. One of ordinary skill in the art would be motivated to combine the references since it improves geolocation measurements (Spangenberg, Para. 96, teaches the motivation of combination to be to improve the geolocation measurements).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Agia in view of Pham, Kedarisetti, and You et al. (US 10466715 B2).
Regarding Claim 7, the combination of references of Agia in view of Pham and Kedarisetti does not explicitly teach "The method of claim 1, wherein the input data includes a depth map and at least one surface estimation of at least one surface of the scene, heights of each of the elements being obtained from the depth map including from the at least one surface estimation".
In an analogous field of endeavor, You teaches "The method of claim 1, wherein the input data includes a depth map and at least one surface estimation of at least one surface of the scene, heights of each of the elements being obtained from the depth map including from the at least one surface estimation"; (You, Claim 1, teaches an image transform unit generating a depth map using depth information of an object in a front image of a road on which the vehicle travels, i.e., input data includes a depth map, wherein a height map is generated of the front image by transforming the generated depth map and a map analysis unit estimates a surface of the road based on the generated height map, i.e., input data includes at least one surface estimation of at least one surface of the scene and the input data includes heights of the elements being obtained from the depth map and the surface estimation).
It would have been obvious to one having ordinary skill in the art before the effective filing date to modify the invention of Agia, Pham, and Kedarisetti by including the input data including a depth map and a surface estimation of a surface in the scene in which heights are obtained from the depth map and from the surface taught by You. One of ordinary skill in the art would be motivated to combine the references since it improves driving control ability (You, Col. 1 lines 31-41, teaches the motivation of combination to be to improve the driving control ability of the driver when a vehicle passes through a narrow road).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW STEVEN BUDISALICH whose telephone number is (703)756-5568. The examiner can normally be reached Monday - Friday 8:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW S BUDISALICH/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662