DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a final office action on the merits. Claims 1-5, 8-15, 18-19, and 21-25 are currently pending and are addressed below.
The examiner notes that the fundamentals of the rejection are based on the broadest reasonable interpretation of the claim language. Applicant is kindly invited to consider the reference as a whole. References are to be interpreted as by one of ordinary skill in the art rather than as by a novice. See MPEP 2141. Therefore, the relevant inquiry when interpreting a reference is not what the reference expressly discloses on its face but what the reference would teach or suggest to one of ordinary skill in the art.
Response to Arguments
Applicant’s arguments with respect to the rejection of claims 1-20 under 35 U.S.C 101 have been fully considered and are persuasive. As discussed in the interview dated 1/20/26, applicant has amended the claims to include “a vehicle controller configured to autonomously control movement of the vehicle based on the generated map of the roadway,” which the examiner agreed cannot be practically performed in the human mind. In light of this, the rejection of claims 1-20 under 35 U.S.C 101 has been withdrawn.
Applicant’s arguments with respect to the 35 U.S.C 112(f) interpretation of claims 1-10 and 20 have been fully considered and are persuasive. The claims have been amended to replace “control module” with “controller” with proper support in the specification. As such, the 35 U.S.C 112(f) interpretation of claims 1-10 and 20 has been withdrawn.
Applicant’s arguments with respect to the 35 U.S.C 103 rejection of claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 8-15, 18-19, and 21-25 are rejected under 35 U.S.C. 103 as being unpatentable over Jia Bin et al. (US20230192094A1), hereinafter referred to as Bin, in view of Yuhang He et al. (“Using Edit Distance and Junction Feature to Detect and Recognize Arrow Road Marking”), hereinafter referred to as He, in further view of Youcheng Zhang et al. (“Deep Learning in Lane Marking Detection”), hereinafter referred to as Zhang in even further view of Rodridues Jose Felix et al. (US20230221139A1), hereinafter referred to as Felix.
Regarding claim 1, Bin discloses: a system for creating a map of a roadway (see at least Bin, ¶¶ [0003]-[0004], [0017]-[0018] which discloses a system for accurately modeling a map of a roadway through a grid approach), the system comprising:
a plurality of vehicle systems for a plurality of vehicles on the roadway, each vehicle system including one or more sensors configured to capture vehicle data and data relating to a roadway (see at least Bin, ¶¶ [0022], [0024]-[0026], [0033]-[0036] which discloses a system controller able to receive input from one or more sensors of a plurality of vehicles serving as external sources, in the form of crowdsourcing, this means each vehicle system including one or more sensors configured to capture vehicle data and data relating to a roadway)
a system controller in communication with the vehicle system (see at least Bin, ¶¶ [0022], [0024]-[0026], [0033]-[0036] which discloses a system controller able to receive input from one or more sensors of a plurality of vehicles serving as external sources, this means a system controller in communication with the vehicle system) the system controller configured to:
receive the captured vehicle data and data relating to the roadway from the one or more sensors of the vehicle systems (see at least Bin, ¶¶ [0022], [0024]-[0026], [0033]-[0036] which discloses a system controller able to receive input from one or more sensors of a plurality of vehicles serving as external sources, in the form of crowdsourcing, this means each vehicle system including one or more sensors configured to capture vehicle data and data relating to a roadway, this means receive the captured vehicle data and data relating to the roadway from the one or more sensors of the vehicle systems)
create a multi-layer probability density bitmap including a plurality of pixels representing a plurality of lane lines of the roadway based on the captured vehicle data and data relating to the roadway from the one or more sensors of the vehicle systems (see at least Bin, ¶¶ [0022]-[0025], which disclose receiving a multi-layer grid-based road model representative of attributes and elements based on the captured vehicle data related to the roadway from the one or more sensors of the vehicle systems (i.e. cell category, lane number, marker type, pavement marking, and lane type) of the roadway from various sources (i.e., nearby vehicles, infrastructure, the internet, etc.) including multiple frames of discernment (FOD) to define and describe possible states for a specific attribute; [0048]-[0049] discloses the lane marker type that includes a plurality of lane lines)
extract a plurality of line components of the plurality of lane lines (see at least Bin, ¶¶ [0011], [0024]-[0025]; Fig.4 which discloses extracting a plurality of line components of the plurality of lane lines to determine lane boundary cells; Fig.5A-B discloses an example of lane sets that an input processing module of a grid-based road model with multiple layers can use to shift input evidence for determining mass values for other layer hypotheses; [0066]-[0067] discloses input data extracted from a plurality of sensors indicating lane markers and semantic information of the lanes)
generate a map of the roadway including the at least one line (see at least Bin, Fig.5-6; ¶¶ [0018], [0032], [0041] which discloses generating a map of the roadway including the at least one line)
transmit the generated map of the roadway to the vehicle system of at least one vehicle of the plurality of vehicles (see at least Bin, ¶¶ [0041]-[0043], which discloses transmitting the generated map of the roadway to the vehicle system of at least one vehicle of the plurality of vehicles)
wherein the vehicle system of the at least one vehicle includes a display configured to display the generated map of the roadway and a vehicle controller configured to autonomously control movement of the vehicle based on the generated map of the roadway (see at least Bin, ¶¶ [0019]-[0022], [0041]-[0043], Fig.7, Item 712, which discloses, providing the generated map of the roadway as input to an autonomous driving system, or assisted-driving system, this means wherein the vehicle system of the at least one vehicle includes a display configured to display the generated map of the roadway and a vehicle controller configured to autonomously control movement of the vehicle based on the generated map of the roadway)
Bin is silent on, however, in the same field of endeavor, He teaches: create at least one line based on the plurality of line points (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), which a junction has one location and two branches that follow along the boundaries around the location, the system functionally identifies and separates the two line branches from the intersection, this means that in response to detecting the junction in the line component, split the line component into two or more subcomponents, this means that at least one line is created based on a plurality of line points)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include create at least one line based on the plurality of line points as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, in the same field of endeavor Zhang discloses:
generate a plurality of line points for each of the subcomponents using a regression model (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to include generate a plurality of line points for each of the subcomponents using a regression model as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents. Further modified Bin is silent on, however, in the same field of endeavor Felix discloses:
detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines (see at least Felix ¶¶ [0026], [0046], [0057] which discloses node detection that includes identifying locations where roadways connect, this means detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines)
in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component, determine an angle between each pair of the plurality of vectors, and cluster one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents (see at least Felix, ¶¶ [0020], [0046], [0063], which discloses the process of detecting an intersection and removing a junction point associated with a continuous direction in order to identify a plurality of separate vectors indicative of separate directions, rather than a one-way road, the junction point is the dividing line between the two directions of travel, this means that in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component; [0057]-[0058] discloses determining a first heading, or angle between a pair of vectors on a roadway and determining a first and new heading based on a change in the angular/heading threshold value, this means clustering one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents representative of directionality)
It would have been obvious to a person of ordinary skill in the art to change further modified Bin to include detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines and in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component, determine an angle between each pair of the plurality of vectors, and cluster one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents as taught by Felix. Incorporating the teachings would allow for an improvement to the base invention of Bin that processes a skeletonized roadmap based on the identified nodes to be indicative of crowdsourced intersection data and directionality of travel that improves the relativity in navigation systems of roadmaps.
Regarding claim 2, Bin discloses: the system of claim 1, wherein the system controller is configured to:
identify connected components of the plurality of lane lines in the bitmap (see at least Bin, ¶¶ [0061], [0067] which discloses assigning probabilities and labels to neighboring cells by identifying connected regions corresponding to individual lane lines, this means identifying connected components of the plurality of lane lines in the bitmap)
categorize each connected component as a line component of the plurality of line components (see at least Bin, ¶¶ [0058], [0066]-[0067] which discloses mapping functions assigning cells to categories (lane boundary, lane center, etc.) defining continuous regions alone lane-marker polylines, this means categorizing each connected component as a line component of the plurality of line components)
Regarding claim 3, Bin discloses: the system of claim 2, wherein the system controller is configured to implement a plurality of masks to extract the plurality of line components (see at least Bin, ¶¶ [0003]-[0004], [0025]-[0026] which discloses using multiple mass extraction formulas to generate respective layer hypotheses associated with belief and plausibility values per cell that functions as a probabilistic layer corresponding to a roadway feature, this means implementing a plurality of masks to extract the plurality of line components)
Regarding claim 4, Bin is silent on, however, in the same field of endeavor, He teaches: the system of claim 1, wherein the system controller is configured to:
skeletonize each line component of the plurality of line components to generate a skeletonized image (see at least He, pg.2318, III. L-Junction String Formation, which discloses extracting the markings’ contours with LSD which reduces the regions to subpixel centerline contours, this means that each line component of the plurality of line components is skeletonized to generate an image)
detect the junction based on the skeletonized image (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components based on the skeletonized image)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include skeletonize each line component of the plurality of line components to generate a skeletonized image and detect the junction based on the skeletonized image as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Regarding claim 5, Bin is silent on, however, in the same field of endeavor, He teaches: the system of claim 4, wherein the system controller is configured to:
scan, with a kernel, the skeletonized image (see at least He, pg.2318, III. L-Junction String Formation, which discloses analyzing local neighborhood of the skeletonized image to determine where multiple line segments intersect, specifically scanning the binary image with a local window to detect linear structures, this means scanning, with a kernel, the skeletonized image)
determine a value associated with the kernel (see at least He, pg.2318, III. L-Junction String Formation, which discloses the LSD operation of computing an edge strength in each local window to decide if a detected region is classified as a line segment, this means determining a value associated with the kernel)
compare the value to a threshold to detect the junction in the line component (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components based on the skeletonized image by comparing angle and proximity thresholds to detect a junction)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include scan, with a kernel, the skeletonized image, determine a value associated with the kernel, and compare the value to a threshold to detect the junction in the line component as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Regrading claim 8, modified Bin is silent on, however, Zhang teaches: the system of claim 1, wherein the regression model includes a B-spline regression model (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to include the system of claim 1, wherein the regression model includes a B-spline regression model as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents.
Regarding claim 9, Bin is silent on, however, in the same field of endeavor, He teaches: the system of claim 1, wherein the system controller is configured to:
determine a length of each of the subcomponents (see at least He, III. L-Junction String Formation, which discloses determining a length of each of the subcomponents and comparing it against a threshold)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include determine a length of each of the subcomponents as taught by He. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, Zhang teaches: generate the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to generate the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents.
Regarding claim 10, Bin is silent on, however, in the same field of endeavor, He teaches: the system of claim 1, wherein the system controller is configured to: in response to not detecting the junction in the line component (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components)
create a line based on the plurality of line points for the line component (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), which a junction has one location and two branches that follow along the boundaries around the location, the system functionally identifies and separates the two line branches from the intersection, this means that in response to detecting the junction in the line component, split the line component into two or more subcomponents, this means that at least one line is created based on a plurality of line points)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include in response to not detecting the junction in the line component and create a line based on the plurality of line points for the line component as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, in the same field of endeavor Zhang discloses: generate a plurality of line points for the line component using the regression model (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to include generate a plurality of line points for the line component using the regression mode as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents. Regarding claim 11, Bin discloses: a method for creating a map of a roadway (see at least Bin, ¶¶ [0003]-[0004], [0017]-[0018] a system for accurately modeling a map of a roadway through a grid approach), comprising:
receiving, from one or more sensors of a plurality of vehicle systems for a plurality of vehicles on the roadway, captured vehicle data and data relating to the roadway (see at least Bin, ¶¶ [0022], [0024]-[0026], [0033]-[0036] which discloses a system controller able to receive input from one or more sensors of a plurality of vehicles serving as external sources, in the form of crowdsourcing, this means each vehicle system including one or more sensors configured to capture vehicle data and data relating to a roadway)
creating a multi-layer probability density bitmap including a plurality of pixels representing a plurality of lane lines of the roadway sensed by one or more sensors of a plurality of vehicles (see at least Bin, ¶¶ [0022]-[0025], which disclose receiving a multi-layer grid-based road model representative of attributes and elements (i.e. cell category, lane number, marker type, pavement marking, and lane type) of the roadway from various sources (i.e., nearby vehicles, infrastructure, the internet, etc.) including multiple frames of discernment (FOD) to define and describe possible states for a specific attribute; [0048]-[0049] discloses the lane marker type that includes a plurality of lane lines)
extracting a plurality of line components of the plurality of lane lines (see at least Bin, ¶¶ [0011], [0024]-[0025]; Fig.4 which discloses extracting a plurality of line components of the plurality of lane lines to determine lane boundary cells; Fig.5A-B discloses an example of lane sets that an input processing module of a grid-based road model with multiple layers can use to shift input evidence for determining mass values for other layer hypotheses; [0066]-[0067] discloses input data extracted from a plurality of sensors indicating lane markers and semantic information of the lanes)
generating a map of the roadway including the at least one line (see at least Bin, Fig.5-6; ¶¶ [0018], [0032], [0041] which discloses generating a map of the roadway including the at least one line)
transmitting the generated map of the roadway to the vehicle system of at least one vehicle of the plurality of vehicles (see at least Bin, ¶¶ [0041]-[0043], which discloses transmitting the generated map of the roadway to the vehicle system of at least one vehicle of the plurality of vehicles)
displaying the generated map of the roadway and autonomously controlling movement of the vehicle based on the generated map of the roadway (see at least Bin, ¶¶ [0019]-[0022], [0041]-[0043], Fig.7, Item 712, which discloses, providing the generated map of the roadway as input to an autonomous driving system, or assisted-driving system, this means wherein the vehicle system of the at least one vehicle includes a display configured to display the generated map of the roadway and a vehicle controller configured to autonomously control movement of the vehicle based on the generated map of the roadway)
Bin is silent on, however, in the same field of endeavor, He teaches: creating at least one line based on the plurality of line points (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), which a junction has one location and two branches that follow along the boundaries around the location, the system functionally identifies and separates the two line branches from the intersection, this means that in response to detecting the junction in the line component, split the line component into two or more subcomponents, this means that at least one line is created based on a plurality of line points)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include d creating at least one line based on the plurality of line points as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, in the same field of endeavor Zhang discloses: generating a plurality of line points for each of the subcomponents using a regression model (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to include generating a plurality of line points for each of the subcomponents using a regression model as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents.
Further modified Bin is silent on, however, in the same field of endeavor Felix discloses: detecting whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines (see at least Felix ¶¶ [0026], [0046], [0057] which discloses node detection that includes identifying locations where roadways connect, this means detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines)
in response to detecting the junction in the line component removing a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component, determining an angle between each pair of the plurality of vectors, and clustering one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents (see at least Felix, ¶¶ [0020], [0046], [0063], which discloses the process of detecting an intersection and removing a junction point associated with a continuous direction in order to identify a plurality of separate vectors indicative of separate directions, rather than a one-way road, the junction point is the dividing line between the two directions of travel, this means that in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component; [0057]-[0058] discloses determining a first heading, or angle between a pair of vectors on a roadway and determining a first and new heading based on a change in the angular/heading threshold value, this means clustering one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents representative of directionality)
It would have been obvious to a person of ordinary skill in the art to change further modified Bin to include detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines and in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component, determine an angle between each pair of the plurality of vectors, and cluster one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents as taught by Felix. Incorporating the teachings would allow for an improvement to the base invention of Bin that processes a skeletonized roadmap based on the identified nodes to be indicative of crowdsourced intersection data and directionality of travel that improves the relativity in navigation systems of roadmaps.
Regarding claim 12, Bin discloses: the method of claim 11, further comprising:
identifying connected components of the plurality of lane lines in the bitmap (see at least Bin, ¶¶ [0061], [0067] which discloses assigning probabilities and labels to neighboring cells by identifying connected regions corresponding to individual lane lines, this means identifying connected components of the plurality of lane lines in the bitmap)
categorizing each connected component as a line component of the plurality of line components (see at least Bin, ¶¶ [0058], [0066]-[0067] which discloses mapping functions assigning cells to categories (lane boundary, lane center, etc.) defining continuous regions alone lane-marker polylines, this means categorizing each connected component as a line component of the plurality of line components)
Regarding claim 13, Bin discloses: the method of claim 12, wherein extracting the plurality of line components of the plurality of lane lines includes implementing a plurality of masks to extract the plurality of line components (see at least Bin, ¶¶ [0003]-[0004], [0025]-[0026] which discloses using multiple mass extraction formulas to generate respective layer hypotheses associated with belief and plausibility values per cell that functions as a probabilistic layer corresponding to a roadway feature, this means implementing a plurality of masks to extract the plurality of line components)
Regarding claim 14, Bin is silent on, however, in the same field of endeavor, He teaches:
the method of claim 12, wherein the method further includes:
skeletonizing each line component of the plurality of line components to generate a skeletonized image (see at least He, pg.2318, III. L-Junction String Formation, which discloses extracting the markings’ contours with LSD which reduces the regions to subpixel centerline contours, this means that each line component of the plurality of line components is skeletonized to generate an image)
detecting whether the junction exists in the line component includes detecting the junction based on the skeletonized image (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components based on the skeletonized image)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include skeletonizing each line component of the plurality of line components to generate a skeletonized image and detecting the junction based on the skeletonized image as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Regarding claim 15, Bin is silent on, however, in the same field of endeavor, He teaches: the method of claim 14, wherein detecting the junction based on the skeletonized image includes:
scanning, with a kernel, the skeletonized image (see at least He, pg.2318, III. L-Junction String Formation, which discloses analyzing local neighborhood of the skeletonized image to determine where multiple line segments intersect, specifically scanning the binary image with a local window to detect linear structures, this means scanning, with a kernel, the skeletonized image)
calculating a value associated with the kernel (see at least He, pg.2318, III. L-Junction String Formation, which discloses the LSD operation of computing an edge strength in each local window to decide if a detected region is classified as a line segment, this means determining a value associated with the kernel)
comparing the value to a threshold to detect the junction in the line component (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components based on the skeletonized image by comparing angle and proximity thresholds to detect a junction)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include scanning, with a kernel, the skeletonized image, calculating a value associated with the kernel, and comparing the value to a threshold to detect the junction in the line component as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Regrading claim 18, Bin is silent on, however, in the same field of endeavor, He teaches: the method of claim 11, wherein generating the plurality of line points for each of the subcomponents includes:
determining a length of each of the subcomponents (see at least He, III. L-Junction String Formation, which discloses determining a length of each of the subcomponents and comparing it against a threshold)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include determining a length of each of the subcomponents as taught by He. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, Zhang teaches: generating the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to generating the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents.
Regarding claim 19, modified Bin is silent on, however, Zhang teaches: the method of claim 11, wherein the regression model includes a B-spline regression model (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to include the system of claim 1, wherein the regression model includes a B-spline regression model as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents.
Regarding claim 21, Bin discloses: a system for creating a map of a roadway (see at least Bin, ¶¶ [0003]-[0004], [0017]-[0018] which discloses a system for accurately modeling a map of a roadway through a grid approach), the system comprising:
a system controller in communication with a plurality of vehicle systems for a plurality of vehicles on the roadway, the system controller configured to (see at least Bin, ¶¶ [0022], [0024]-[0026], [0033]-[0036] which discloses a system controller able to receive input from one or more sensors of a plurality of vehicles serving as external sources, in the form of crowdsourcing, this means each vehicle system including one or more sensors configured to capture vehicle data and data relating to a roadway)
receive the captured vehicle data and data relating to the roadway from the one or more sensors of the vehicle systems (see at least Bin, ¶¶ [0022], [0024]-[0026], [0033]-[0036] which discloses a system controller able to receive input from one or more sensors of a plurality of vehicles serving as external sources, in the form of crowdsourcing, this means each vehicle system including one or more sensors configured to capture vehicle data and data relating to a roadway, this means receive the captured vehicle data and data relating to the roadway from the one or more sensors of the vehicle systems)
create a multi-layer probability density bitmap including a plurality of pixels representing a plurality of lane lines of the roadway based on the captured vehicle data and data relating to the roadway from the one or more sensors of the vehicle systems (see at least Bin, ¶¶ [0022]-[0025], which disclose receiving a multi-layer grid-based road model representative of attributes and elements based on the captured vehicle data related to the roadway from the one or more sensors of the vehicle systems (i.e. cell category, lane number, marker type, pavement marking, and lane type) of the roadway from various sources (i.e., nearby vehicles, infrastructure, the internet, etc.) including multiple frames of discernment (FOD) to define and describe possible states for a specific attribute; [0048]-[0049] discloses the lane marker type that includes a plurality of lane lines)
extract a plurality of line components of the plurality of lane lines (see at least Bin, ¶¶ [0011], [0024]-[0025]; Fig.4 which discloses extracting a plurality of line components of the plurality of lane lines to determine lane boundary cells; Fig.5A-B discloses an example of lane sets that an input processing module of a grid-based road model with multiple layers can use to shift input evidence for determining mass values for other layer hypotheses; [0066]-[0067] discloses input data extracted from a plurality of sensors indicating lane markers and semantic information of the lanes)
generate a map of the roadway including the at least one line (see at least Bin, Fig.5-6; ¶¶ [0018], [0032], [0041] which discloses generating a map of the roadway including the at least one line)
transmit the generated map of the roadway to the vehicle system of at least one vehicle of the plurality of vehicles (see at least Bin, ¶¶ [0041]-[0043], which discloses transmitting the generated map of the roadway to the vehicle system of at least one vehicle of the plurality of vehicles)
wherein the vehicle system of the at least one vehicle includes a display configured to display the generated map of the roadway and a vehicle controller configured to autonomously control movement of the vehicle based on the generated map of the roadway (see at least Bin, ¶¶ [0019]-[0022], [0041]-[0043], Fig.7, Item 712, which discloses, providing the generated map of the roadway as input to an autonomous driving system, or assisted-driving system, this means wherein the vehicle system of the at least one vehicle includes a display configured to display the generated map of the roadway and a vehicle controller configured to autonomously control movement of the vehicle based on the generated map of the roadway)
Bin is silent on, however, in the same field of endeavor, He teaches: create at least one line based on the plurality of line points (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), which a junction has one location and two branches that follow along the boundaries around the location, the system functionally identifies and separates the two line branches from the intersection, this means that in response to detecting the junction in the line component, split the line component into two or more subcomponents, this means that at least one line is created based on a plurality of line points)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include create at least one line based on the plurality of line points as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, in the same field of endeavor Zhang discloses:
generate a plurality of line points for each of the subcomponents using a regression model (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to include generate a plurality of line points for each of the subcomponents using a regression model as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents. Further modified Bin is silent on, however, in the same field of endeavor Felix discloses: detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines (see at least Felix ¶¶ [0026], [0046], [0057] which discloses node detection that includes identifying locations where roadways connect, this means detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines)
in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component, determine an angle between each pair of the plurality of vectors, and cluster one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents (see at least Felix, ¶¶ [0020], [0046], [0063], which discloses the process of detecting an intersection and removing a junction point associated with a continuous direction in order to identify a plurality of separate vectors indicative of separate directions, rather than a one-way road, the junction point is the dividing line between the two directions of travel, this means that in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component; [0057]-[0058] discloses determining a first heading, or angle between a pair of vectors on a roadway and determining a first and new heading based on a change in the angular/heading threshold value, this means clustering one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents representative of directionality)
It would have been obvious to a person of ordinary skill in the art to change further modified Bin to include detect whether a junction exists in a line component of the plurality of line components, the junction representing an intersection of two or more of the lane lines and in response to detecting the junction in the line component remove a junction point representing the detected junction in the line component to create a plurality of separate vectors representing the line component, determine an angle between each pair of the plurality of vectors, and cluster one or more pairs of the plurality of vectors of the line component into a subcomponent based on the determined angles and a defined angular threshold to split the line component into two or more subcomponents as taught by Felix. Incorporating the teachings would allow for an improvement to the base invention of Bin that processes a skeletonized roadmap based on the identified nodes to be indicative of crowdsourced intersection data and directionality of travel that improves the relativity in navigation systems of roadmaps.
Regarding claim 22, Bin is silent on, however, in the same field of endeavor, He teaches: the system of claim 21, wherein the system controller is configured to:
skeletonize each line component of the plurality of line components to generate a skeletonized image (see at least He, pg.2318, III. L-Junction String Formation, which discloses extracting the markings’ contours with LSD which reduces the regions to subpixel centerline contours, this means that each line component of the plurality of line components is skeletonized to generate an image)
detect the junction based on the skeletonized image (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components based on the skeletonized image)
Regarding claim 23, in is silent on, however, in the same field of endeavor, He teaches: the system of claim 22,wherein the system controller is configured to:
scan, with a kernel, the skeletonized image (see at least He, pg.2318, III. L-Junction String Formation, which discloses analyzing local neighborhood of the skeletonized image to determine where multiple line segments intersect, specifically scanning the binary image with a local window to detect linear structures, this means scanning, with a kernel, the skeletonized image)
determine a value associated with the kernel (see at least He, pg.2318, III. L-Junction String Formation, which discloses the LSD operation of computing an edge strength in each local window to decide if a detected region is classified as a line segment, this means determining a value associated with the kernel)
compare the value to a threshold to detect the junction in the line component (see at least He, pg. 2318, III. L-Junction String Formation which discloses extracting all contours with LSD (Line Segment Detector) and detecting intersections between line components (i.e., a junction), this means it is detected whether a junction exists in a line component of the plurality of line components based on the skeletonized image by comparing angle and proximity thresholds to detect a junction)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include scan, with a kernel, the skeletonized image, determine a value associated with the kernel, and compare the value to a threshold to detect the junction in the line component as taught by He. The examiner would like to note that the disclosure of Bin analyzes where polylines intersect with each other, or with cell regions, which inherently performs the required detection of intersections, but it is not as explicitly stated as in He. However, the examiner believes that the mapping functions computing which cells belong to each category suggests that the system is capable of handling the detection of junctions. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Regarding claim 24, Bin is silent on, however, in the same field of endeavor, He teaches: the system of claim 21, wherein the system controller is configured to:
determine a length of each of the subcomponents (see at least He, III. L-Junction String Formation, which discloses determining a length of each of the subcomponents and comparing it against a threshold)
It would have been obvious to a person of ordinary skill in the art to modify Bin to include determine a length of each of the subcomponents as taught by He. Incorporating the teachings of He would allow for an improvement to the base invention of Bin to more accurately transform the detected input data and generate a map of the roadway.
Modified Bin is silent on, however, Zhang teaches: generate the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent (see at least Zhang, pg. 5985-5986, B. Deep Architecture Focusing on Lane Marking Classification, which discloses the generation of a plurality of line components for each of the subcomponents; pg.5987, A. Effective Data Processing, which discloses various regression models which discloses the method of fitting a line or curve)
It would have been obvious to a person of ordinary skill in the art to further change modified Bin to generate the plurality of line points for each of the subcomponents using the regression model based on the determined length for the subcomponent as taught by Zhang. The examiner would like to note that the disclosure of modified Bin within the introduction (see He, Introduction) acknowledges an optimization process where the system adjusts geometrical parameters through finding parameters that best fit observed data, but it is not explicitly mentioned. Incorporating the teachings of Zhang would allow for an improvement to the base invention of modified Bin to generate smoother continuous line points in order to generate a map according to the subcomponents.
Regarding claim 25, further modified Bin in silent on, however, in the same field of endeavor, Felix teaches: the system of claim 21, wherein the system controller is configured to, in response to not detecting the junction in the line component, generate a plurality of line points for the line component using the regression model and create a line based on the plurality of line points for the line component (see at least Felix, ¶¶ [0046], [0065]-[0073], which discloses herein the system controller is configured to, in response to not detecting the junction in the line component, generate a plurality of line points for the line component using the regression model and create a line based on the plurality of line points for the line component)
It would have been obvious to a person of ordinary skill in the art to change further modified Bin to include the system of claim 21, wherein the system controller is configured to, in response to not detecting the junction in the line component, generate a plurality of line points for the line component using the regression model and create a line based on the plurality of line points for the line component. Incorporating the teachings would allow for an improvement to the base invention of Bin that processes a skeletonized roadmap based on the identified nodes to be indicative of crowdsourced intersection data and directionality of travel that improves the relativity in navigation systems of roadmaps.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIRSTEN JADE M SANTOS whose telephone number is (571)272-7442. The examiner can normally be reached Monday: 8:00 am - 4:00 pm, 6:00-8:00 pm (+ with flex).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rachid Bendidi can be reached at (571) 272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KIRSTEN JADE M SANTOS/Examiner, Art Unit 3664
/RACHID BENDIDI/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights