DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 03/18/2026 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed 03/18/2026 have been fully considered but are found to be unpersuasive at this time.
The teaching of Adams discloses a consideration for vehicle operations within an operating environment (i.e. consisting of but not limited to traffic areas/structures such as roundabouts). A roundabout contains a drivable regions, sidewalks (i.e. non drivable), and traffic control signals, as disclosed by Adams, for the purpose of mapping, localization, and collision avoidance.
Adams teaches wherein [0012] Images in the sequence of images may be semantically segmented to associate pixels of an image with a label indicating the associated classification, e.g., drivable region, car, pedestrian, sidewalk, traffic control signal, and so forth… The machine-learned model may be further trained to extract associated features from the regions of the image associated with the object, such that the automatically detected features can be further input into a localization model using the map and/or an estimated position of the vehicle.
As such, a person of ordinary skill in the art would recognize that the system of Adams is configured to use localization and sensor techniques (i.e. LIDAR, dead-reckoning) to detect vehicles, traffic, and repeated “objects” (i.e. lane markers, signs, curbs/sidewalks) in order to navigate an environment which includes known traffic patterns/signals/objects (i.e. roundabout).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 6 and 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adams (US-20210182596-A1), Shi (US-20220114724-A), and Lee (US-20200166943-A1) in view of Ichinokawa (US-20200168180-A1).
1. (Currently Amended) Adams (US-20210182596-A1) discloses
A computer-implemented method [0094] computer-readable media storing instructions executable by the one or more processors for analyzing **** an environment of a vehicle [0009] This disclosure relates to determining a location of a vehicle (e.g., an autonomous vehicle) in an environment using locations of objects having a repeated object classification, such as lane markings, detected by the vehicle as the vehicle traverses the environment,
the method comprising:
generating at least one initial feature map [0013] … when creating the map, multiple observations of the features may be collected and combined in image space... by applying a feature encoder module [0010-13] SemSeg localization component to detect features (e.g., a corner, side, center, etc.) of the objects in the images. Further, the associated features of the objects may be associated with a map of an environment …the SemSeg localization component may detect a feature of the object in the semantically segmented image of a trained neural network [0086] …The machine-learned model used to segment the image in some cases may be a neural network, … ResNet… to an input image [0021] …the image 108 may be input into a neural network trained to semantically segment images, … may be received from the neural network, which may be used to determine features of labeled objects in the semantically segmented image.,
****;
applying the classificator module of the trained neural network to the at least one initial feature map [0047] the perception component 422 can include functionality to perform object detection, segmentation, and/or classification. …indicates a presence of an entity that is proximate to the vehicle 402 and/or a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, stoplight, stop sign, lane marker, unknown, etc.)… [0048] …maps 424 can be used in connection with the localization component 420, the perception component 422, the SemSeg localization component 428, or the planning component 434 to determine a location of the vehicle 402, identify objects in an environment…,
wherein an output of the classificator module represents a road region [0047] … a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, stoplight, stop sign, lane marker, unknown, etc.) in the input image;
****,
****; and
****.
Adams lacks distinctly disclosing the following underlined limitations:
…analyzing a roundabout in an environment of a vehicle,
…wherein the input image depicts the roundabout;
…wherein, prior to applying a classificator module of the trained neural network…, and prior to… the feature encoder module reduces a spatial dimension of the input image to generate the at least one initial feature map;
… applying the radius estimation module …
…. wherein an output of the radius estimation module depends on an inner radius of the roundabout and an outer radius of the roundabout; and
…determining at least one entry point and at least one exit point of the roundabout depending on the output of the classificator module and depending on the output of the radius estimation module.
Shi US-20220114724-A1 discloses in a similar invention field of endeavor, a consideration for image processing model generations and methods “…wherein, prior to applying a classificator module of the trained neural network…, and prior to [[another process (i.e. marking layer)]]… the feature encoder module reduces a spatial dimension of the input image to generate the at least one initial feature map”;
(Shi [0013] In some embodiments, the initial image processing model further includes a neural network.)
(Shi [claim.23] 23. The method according to claim 2, wherein the initial image processing model further includes a global average pooling layer; and before the classification layer and the marking layer are called, the method further comprises: inputting a plurality of groups of two-dimensional feature maps of the training sample lesion image into the global average pooling layer; and calling the global average pooling layer to perform global average pooling on the plurality of groups of two-dimensional feature maps, so as to obtain a plurality of groups of one-dimensional feature maps corresponding to the plurality of groups of two-dimensional feature maps.)
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include prior to applying a module of the trained neural network the feature encoder module reduces a spatial dimension of the input image to generate the at least one initial feature map with a reasonable expectation for success, as taught by Shi, for the benefit of reducing sample size/dimensions in simplifying a control system operation based upon sensed and pre-processed data.
Regarding the limitation(s); “…analyzing a roundabout in an environment of a vehicle … wherein the input image depicts the roundabout …applying a radius estimation module … wherein an output of the radius estimation module depends on an inner radius of the roundabout and an outer radius of the roundabout”, Lee (US-20200166943-A1) discloses in a similar invention field of endeavor, a consideration for wherein an input image depicts a recognized roundabout [0105, 0134] … the roundabout recognizer 210 … the camera may recognize the roundabout 1… Roundabout 1 is recognized by extracting an image …acquired by the camera. a radius estimation module dependent upon an outer radius of a roundabout [0106] … outermost boundary B may be formed using average perimeter information on the average perimeter obtained by averaging the circumference R of general roundabouts, or the structure of roundabout 1. Here, the average perimeter information may be information obtained through simulation and algorithm. and an inner radius [0108] … detects the traffic island 50, which is a circular structure formed at the center of the roundabout 1 by the sensor 120, and the outer lane 60 in the roundabout 1, respectively, and calculates the respective curvatures. From each curvature, calculate the radius of the traffic island 50 and the radius of the outer lane 60 to calculate the average value of the radius, and the average perimeter is calculated using the average value to form the outermost boundary B having the average perimeter as the length of the arc.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include analyzing a roundabout in an environment of a vehicle wherein an output of the radius estimation module depends on an inner radius of the roundabout and an outer radius of the roundabout with a reasonable expectation for success, as taught by Lee, for the benefit of providing measurements necessary for orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
However, while Lee discloses analyzing a roundabout in an environment of a vehicle wherein an output of the radius estimation module depends on an inner radius of the roundabout and an outer radius of the roundabout discussed above, Lee is silent as to distinctly disclosing “…determining at least one entry point and at least one exit point of the roundabout”;
Regarding the limitation underlined above; Ichinokawa (US-20200168180-A1) discloses in a similar invention field of endeavor, a consideration for measuring features of a roundabout once determined [0063] …an outer diameter of the roundabout RC1… [0083] …whether or not outer diameters OD2 and OD3 of the circular road lanes L10A and L10B around the central islands CI2 and CI3 of the roundabout RC2… [0041] …The information on the road includes, for example, at least one of a road situation (a road shape or a curvature), information indicating a lane or a traveling direction in which the vehicle M can travel, information on a form of a roundabout and a road at an entrance or exit of the roundabout (for example, a road connected to the roundabout)…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include determining at least one entry point and at least one exit point of the roundabout with a reasonable expectation for success, as taught by the combination of Lee and Ichinokawa, for the benefit of providing information regarding travel direction of roads connected to a roundabout, so as to ensure safety and control of movement into and out of road conditions (e.g. roundabouts).
2. Adams (US-20210182596-A1) discloses
The computer-implemented method according to claim 1,
wherein the output of the classificator module comprises a segmented image [0047] the perception component 422 can include functionality to perform object detection, segmentation, and/or classification…
wherein the road region comprises image points of the segmented image [0012] Images in the sequence of images may be semantically segmented to associate pixels of an image with a label indicating the associated classification, e.g., drivable region, car, pedestrian, sidewalk, traffic control signal, and so forth…, which are assigned to a road class [0047] classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, stoplight, stop sign, lane marker, unknown, etc.)… by applying the classificator module to the at least one initial feature map [0048] …maps 424 can be used in connection with the localization component 420, the perception component 422, the SemSeg localization component 428, or the planning component 434 to determine a location of the vehicle 402, identify objects in an environment.
3. Adams (US-20210182596-A1) discloses
The computer-implemented method according to claim 2,
**** a mask image,
wherein the mask image defines **** a feature [0021] … segment an image and extract features associated with particular classifications, and/or multiple steps may be used (e.g., output a first network trained to segment an image and provide masks of relevant portions of the image is then used to determine associated features).
Adams lacks disctincly disclosing the following underlined limitations:
…wherein the output of the radius estimation module comprises a mask image,
…wherein the mask image defines a ring with an inner ring radius approximating the inner radius of the roundabout and with an outer ring radius approximating the outer radius of the roundabout.
Regarding the limitation(s); the limitation(s) are similar in scope to those disclosed in the system of claim(s) 1 and are therefore rejected under the same premise, for more information please see the rejection in re claim(s) 1.
6. Adams (US-20210182596-A1) discloses
The computer-implemented method according to claim 3,
further comprising:
applying a road regression module [0047] the perception component 422 can include functionality to perform object detection, segmentation, and/or classification. …indicates a presence of an entity that is proximate to the vehicle 402 and/or a classification of the entity as an entity type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, stoplight, stop sign, lane marker, unknown, etc.)… of the trained neural network [0086] …The machine-learned model used to segment the image in some cases may be a neural network, … ResNet… to a combination of the segmented image and the masked image [0021-0022] … the segmented image 130 (and/or the image 108) may be masked by setting pixel values...,
wherein an output of the road regression module comprises a road points feature map [0013] … when creating the map, multiple observations of the features may be collected and combined in image space... [0042] the image 324 and/or the semantically segmented image 330 may be used to further train the machine-learned model to recognize repeating objects and/or features of said objects in an environment, determine distances between said features or objects, update a map, and so forth.; and
****.
Adams lacks disctincly disclosing the following underlined limitations:
… determining the at least one entry point and the at least one exit point depending on the road points feature map
Regarding the limitation; “…determining the at least one entry point and the at least one exit point”, Ichinokawa (US-20200168180-A1) discloses in a similar invention field of endeavor, a consideration for measuring features of a roundabout once determined [0063] …an outer diameter of the roundabout RC1… [0083] …whether or not outer diameters OD2 and OD3 of the circular road lanes L10A and L10B around the central islands CI2 and CI3 of the roundabout RC2… [0041] …The information on the road includes, for example, at least one of a road situation (a road shape or a curvature), information indicating a lane or a traveling direction in which the vehicle M can travel, information on a form of a roundabout and a road at an entrance or exit of the roundabout (for example, a road connected to the roundabout)…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include determining at least one entry point and at least one exit point with a reasonable expectation for success, as taught by Ichinokawa, for the benefit of providing information regarding travel direction of roads connected to a roundabout, so as to ensure safety and control of movement into and out of road conditions (e.g. roundabouts).
13. Adams (US-20210182596-A1) discloses
Computer implemented A computer-implemented method for planning a path for a vehicle [0054] the planning component 434 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route can be a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 434 can generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 434 can determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints,
the method comprising:
carrying out a computer-implemented method [0094] computer-readable media storing instructions executable by the one or more processors for analyzing **** an environment of the vehicle according to claim 1; and
planning the path for the vehicle ****.
Adams lacks distinctly disclosing the following underlined limitations:
for analyzing a roundabout in an environment of the vehicle according to claim 1; and
planning the path for the vehicle depending on the at least one entry point and at least one exit point of the roundabout.
Regarding the limitation(s); “…analyzing a roundabout in an environment”, Lee (US-20200166943-A1) discloses in a similar invention field of endeavor, a consideration for wherein an input image depicts a recognized roundabout [0105, 0134] … the roundabout recognizer 210 … the camera may recognize the roundabout 1… Roundabout 1 is recognized by extracting an image …acquired by the camera. a radius estimation module dependent upon an outer radius of a roundabout [0106] … outermost boundary B may be formed using average perimeter information on the average perimeter obtained by averaging the circumference R of general roundabouts, or the structure of roundabout 1. Here, the average perimeter information may be information obtained through simulation and algorithm. and an inner radius [0108] … detects the traffic island 50, which is a circular structure formed at the center of the roundabout 1 by the sensor 120, and the outer lane 60 in the roundabout 1, respectively, and calculates the respective curvatures. From each curvature, calculate the radius of the traffic island 50 and the radius of the outer lane 60 to calculate the average value of the radius, and the average perimeter is calculated using the average value to form the outermost boundary B having the average perimeter as the length of the arc.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include analyzing a roundabout in an environment with a reasonable expectation for success, as taught by Lee, for the benefit of providing measurements necessary for orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
Regarding the limitation; “…an entry point or to an exit point”, Ichinokawa (US-20200168180-A1) discloses in a similar invention field of endeavor, a consideration for measuring features of a roundabout once determined [0063] …an outer diameter of the roundabout RC1… [0083] …whether or not outer diameters OD2 and OD3 of the circular road lanes L10A and L10B around the central islands CI2 and CI3 of the roundabout RC2… [0041] …The information on the road includes, for example, at least one of a road situation (a road shape or a curvature), information indicating a lane or a traveling direction in which the vehicle M can travel, information on a form of a roundabout and a road at an entrance or exit of the roundabout (for example, a road connected to the roundabout)…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include determining at least one entry point and at least one exit point of the roundabout with a reasonable expectation for success, as taught by the combination of Lee and Ichinokawa, for the benefit of providing information regarding travel direction of roads connected to a roundabout, so as to ensure safety and control of movement into and out of road conditions (e.g. roundabouts).
14. (Currently Amended) Regarding the limitation;
the limitation is similar in scope to those disclosed in the system of claim(s) 1 and are therefore rejected under the same premise, for more information please see the rejection in re claim(s) 1.
15. Adams (US-20210182596-A1) discloses
A non-transitory [0074] Memory 418 and 442 are examples of non-transitory computer-readable media computer readable medium containing program instructions for causing a processor to perform a computer-implemented method according to claim 1 [0094] computer-readable media storing instructions executable by the one or more processors.
Claim(s) 4-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adams (US-20210182596-A1), Shi (US-20220114724-A), Lee (US-20200166943-A1) and Ichinokawa (US-20200168180-A1), as applied to claim 3 above and further in view of Wang (US-20200092463-A1) and Yuan (US-20220004808-A1).
4. Adams (US-20210182596-A1) discloses The computer-implemented method according to claim 3,
wherein **** to the at least one initial feature map [0013] … when creating the map, multiple observations of the features may be collected and combined in image space... comprises applying a radius regression sub-module [0071] …any type of machine learning can be used consistent with this disclosure. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g…
****, and
wherein an output of the **** regression sub-module [0071] comprises a **** feature map [0013] … when creating the map, multiple observations of the features may be collected and combined in image space...;
wherein the **** feature map [0013] **** comprising **** for each image point of the input image that the respective image point [0012] Images in the sequence of images may be semantically segmented to associate pixels of an image with a label indicating the associated classification…
****;
wherein the **** feature map [0013] **** comprising a regression value [0071] …machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based algorithms for the inner radius of the roundabout for each image point of the input image [0012] Images in the sequence of images may be semantically segmented to associate pixels of an image with a label indicating the associated classification, e.g., drivable region, car, pedestrian, sidewalk, traffic control signal, and so forth…; and
wherein the **** feature map [0013] comprises **** a regression value [0071] **** for each image point of the input image [0012] Images in the sequence of images may be semantically segmented to associate pixels of an image with a label indicating the associated classification, e.g., drivable region, car, pedestrian, sidewalk, traffic control signal, and so forth….
Adams lacks disctincly disclosing the following underlined limitations:
… applying the radius estimation module… of the radius estimation module to the at least one initial feature map… radius… corresponds to a center of the roundabout… for the outer radius of the roundabout…
… feature map comprises a first/second/third channel
… probability for each image point of the input image
Regarding the limitation; “…applying the radius estimation module… of the radius estimation module to the at least one initial feature map… radius… corresponds to a center of the roundabout… for the outer radius of the roundabout…”, Lee (US-20200166943-A1) discloses in a similar invention field of endeavor, a consideration for wherein an input image depicts a recognized roundabout [0105, 0134] … the roundabout recognizer 210 … the camera may recognize the roundabout 1… Roundabout 1 is recognized by extracting an image …acquired by the camera. a radius estimation module dependent upon an outer radius of a roundabout [0106] … outermost boundary B may be formed using average perimeter information on the average perimeter obtained by averaging the circumference R of general roundabouts, or the structure of roundabout 1. Here, the average perimeter information may be information obtained through simulation and algorithm. and an inner radius [0108] … detects the traffic island 50, which is a circular structure formed at the center of the roundabout 1 by the sensor 120, and the outer lane 60 in the roundabout 1, respectively, and calculates the respective curvatures. From each curvature, calculate the radius of the traffic island 50 and the radius of the outer lane 60 to calculate the average value of the radius, and the average perimeter is calculated using the average value to form the outermost boundary B having the average perimeter as the length of the arc.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include analyzing a roundabout in an environment of a vehicle wherein an output of the radius estimation module corresponds to an inner radius of the roundabout and an outer radius of a roundabout with a reasonable expectation for success, as taught by Lee, for the benefit of providing measurements necessary for orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
Regarding the limitation; “…feature map comprises a first/second/third channel”, Wang (US-20200092463-A1) discloses in a similar invention field of endeavor, a consideration for object detection [0103] …first through fourth detection modules 316a-d are used to detect objects of different sizes on the first through fourth channel feature maps 314a-d.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include feature map comprising a first/second/third channel with a reasonable expectation for success, as taught by Wang, for the benefit of providing [0015] splitting the enhanced feature map into channel feature maps of different resolutions.
Regarding the limitation; “…a probability for each image point of the input image”, Yuan (US-20220004808-A1) discloses in a similar invention field of endeavor, a consideration for a method and apparatus for image segmentations wherein [0226] determining a category confidence according to the category probability feature map, comprises: determining a maximum probability of each pixel in the category probability feature map; determining a category confidence based on an average of the maximum probability of each pixel … enter a next feature extraction module for feature extraction based on the category confidence
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a probability for each image point of the input image with a reasonable expectation for success, as taught by Yuan, for the benefit of providing ideal/individually segmentation effects for images with different characteristics according to shape, size, type of an object and providing confidence in object recognition.
However, while Yuan discloses a probability for each image point of the input image for feature extraction discussed above, Yuan is silent as to distinctly disclosing “…a probability for each image point of the input image that the respective image point corresponds to a center of the roundabout”;
Regarding the limitation underlined above; Lee (US-20200166943-A1) discloses in a similar invention field of endeavor, a consideration for [0108] …the ROI setting unit 220 detects the traffic island 50, which is a circular structure formed at the center of the roundabout 1 by the sensor 120, and the outer lane 60 in the roundabout 1, respectively, and calculates the respective curvatures…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a probability for each image point of the input image that the respective image point corresponds to a center of the roundabout with a reasonable expectation for success, as taught by the combination of Yuan and Lee, for the benefit of providing a confidence level associated with detected objects such that objects may be classified for effectively orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
5. Adams (US-20210182596-A1) discloses The computer-implemented method according to claim 4,
wherein a **** feature map is determined by a masking sub-module [0021] … segment an image and extract features associated with particular classifications, and/or multiple steps may be used (e.g., output a first network trained to segment an image and provide masks of relevant portions of the image is then used to determine associated features) ****;
**** the masking sub-module [0021] as the regression value [0071] …any type of machine learning can be used consistent with this disclosure. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g… ****; and
****.
Adams lacks disctincly disclosing the following underlined limitations:
… a global maximum
… the first/second/third channel
… radius estimation module… wherein the inner ring radius… for the inner radius of the roundabout corresponding to the global maximum… wherein the outer ring radius is determined by the masking sub-module as the regression value for the outer radius of the roundabout corresponding to the global maximum
Regarding the limitation; “…a global maximum”, Yuan (US-20220004808-A1) discloses in a similar invention field of endeavor, a consideration for a method and apparatus for image segmentations wherein [0226] determining a category confidence according to the category probability feature map, comprises: determining a maximum probability of each pixel in the category probability feature map; determining a category confidence based on an average of the maximum probability of each pixel … enter a next feature extraction module for feature extraction based on the category confidence… [0336] According to the category probability feature map, the probability (corresponding to the category confidence above) of each category is determined, which a larger probability means greater accuracy of predicted category ID map, and thus more suitable receptive field.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a global maximum with a reasonable expectation for success, as taught by Yuan, for the benefit of providing ideal/individually segmentation effects for images with different characteristics according to shape, size, type of an object and providing confidence in object recognition wherein a larger probability means greater accuracy … thus more suitable receptive field.
Regarding the limitation; “…a first/second/third channel”, Wang (US-20200092463-A1) discloses in a similar invention field of endeavor, a consideration for object detection [0103] …first through fourth detection modules 316a-d are used to detect objects of different sizes on the first through fourth channel feature maps 314a-d.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a first/second/third channel with a reasonable expectation for success, as taught by Wang, for the benefit of providing [0015] splitting the enhanced feature map into channel feature maps of different resolutions.
Regarding the limitation; “…radius estimation module… wherein the inner ring radius… for the inner radius of the roundabout corresponding to the global maximum … wherein the outer ring radius is determined …”, Lee (US-20200166943-A1) discloses in a similar invention field of endeavor, a consideration for wherein an input image depicts a recognized roundabout [0105, 0134] … the roundabout recognizer 210 … the camera may recognize the roundabout 1… Roundabout 1 is recognized by extracting an image …acquired by the camera. a radius estimation module dependent upon an outer radius of a roundabout [0106] … outermost boundary B may be formed using average perimeter information on the average perimeter obtained by averaging the circumference R of general roundabouts, or the structure of roundabout 1. Here, the average perimeter information may be information obtained through simulation and algorithm. and an inner radius [0108] … detects the traffic island 50, which is a circular structure formed at the center of the roundabout 1 by the sensor 120, and the outer lane 60 in the roundabout 1, respectively, and calculates the respective curvatures. From each curvature, calculate the radius of the traffic island 50 and the radius of the outer lane 60 to calculate the average value of the radius, and the average perimeter is calculated using the average value to form the outermost boundary B having the average perimeter as the length of the arc.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include radius estimation module determining an inner and outer radius with a reasonable expectation for success, as taught by Lee, for the benefit of providing measurements necessary for orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
However, while Lee discloses radius estimation module discussed above, Lee is silent as to distinctly disclosing “…for the inner radius of the roundabout corresponding to the global maximum … wherein the outer ring radius is determined by the masking sub-module as the regression value for the outer radius of the roundabout corresponding to the global maximum”;
Regarding the limitation underlined above; Yuan (US-20220004808-A1) discloses in a similar invention field of endeavor, a consideration for a method and apparatus for image segmentations wherein [0226] determining a category confidence according to the category probability feature map, comprises: determining a maximum probability of each pixel in the category probability feature map; determining a category confidence based on an average of the maximum probability of each pixel … enter a next feature extraction module for feature extraction based on the category confidence… [0336] According to the category probability feature map, the probability (corresponding to the category confidence above) of each category is determined, which a larger probability means greater accuracy of predicted category ID map, and thus more suitable receptive field.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include an inner radius of the roundabout corresponding to the global maximum and an outer radius of the roundabout corresponding to the global maximum with a reasonable expectation for success, as taught by the combination of Lee and Yuan, for the benefit of providing ideal/individually segmentation effects for images with different characteristics according to shape, size, type of an object and providing confidence in object recognition wherein a larger probability means greater accuracy … thus more suitable receptive field; beneficial in orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
Claim(s) 7-8 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adams (US-20210182596-A1), Shi (US-20220114724-A), Lee (US-20200166943-A1) and Ichinokawa (US-20200168180-A1), as applied to claim 6 above and further in view of Wang (US-20200092463-A1) and Yuan (US-20220004808-A1).
7. Adams (US-20210182596-A1) discloses The computer-implemented method according to claim 6,
wherein the road points feature map comprises a **** input image **** [0010-13] SemSeg localization component to detect features (e.g., a corner, side, center, etc.) of the objects in the images. Further, the associated features of the objects may be associated with a map of an environment.
Adams lacks distinctly disclosing the following underlined limitations:
wherein …:
… a first/second/third channel
… a probability for each image point of the input image that the respective image point corresponds to
… an entry point or to an exit point
Regarding the limitation; “…a first/second/third channel”, Wang (US-20200092463-A1) discloses in a similar invention field of endeavor, a consideration for object detection [0103] …first through fourth detection modules 316a-d are used to detect objects of different sizes on the first through fourth channel feature maps 314a-d.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a first/second/third channel with a reasonable expectation for success, as taught by Wang, for the benefit of providing [0015] splitting the enhanced feature map into channel feature maps of different resolutions.
Regarding the limitation; “…a probability for each image point of the input image”, Yuan (US-20220004808-A1) discloses in a similar invention field of endeavor, a consideration for a method and apparatus for image segmentations wherein [0226] determining a category confidence according to the category probability feature map, comprises: determining a maximum probability of each pixel in the category probability feature map; determining a category confidence based on an average of the maximum probability of each pixel … enter a next feature extraction module for feature extraction based on the category confidence
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a probability for each image point of the input image with a reasonable expectation for success, as taught by Yuan, for the benefit of providing ideal/individually segmentation effects for images with different characteristics according to shape, size, type of an object and providing confidence in object recognition.
Regarding the limitation; “…an entry point or to an exit point”, Ichinokawa (US-20200168180-A1) discloses in a similar invention field of endeavor, a consideration for measuring features of a roundabout once determined [0063] …an outer diameter of the roundabout RC1… [0083] …whether or not outer diameters OD2 and OD3 of the circular road lanes L10A and L10B around the central islands CI2 and CI3 of the roundabout RC2… [0041] …The information on the road includes, for example, at least one of a road situation (a road shape or a curvature), information indicating a lane or a traveling direction in which the vehicle M can travel, information on a form of a roundabout and a road at an entrance or exit of the roundabout (for example, a road connected to the roundabout)…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include determining at least one entry point and at least one exit point of the roundabout with a reasonable expectation for success, as taught by the combination of Lee and Ichinokawa, for the benefit of providing information regarding travel direction of roads connected to a roundabout, so as to ensure safety and control of movement into and out of road conditions (e.g. roundabouts).
8. Regarding the limitation; the limitation is similar in scope to those disclosed in the method of claim(s) 7 and are therefore rejected under the same premise, for more information please see the rejection in re claim(s) 7.
10. Regarding the limitation; the limitation is similar in scope to those disclosed in the method of claim(s) 8 and are therefore rejected under the same premise, for more information please see the rejection in re claim(s) 8.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adams (US-20210182596-A1), Shi (US-20220114724-A), Lee (US-20200166943-A1), Ichinokawa (US-20200168180-A1), Wang (US-20200092463-A1), and Yuan (US-20220004808-A1), as applied to claim 7 above and further in view of Mandai (US-20200242372-A1).
9. Adams (US-20210182596-A1) discloses Computer-implemented method according to claim 7, further comprising:
determining … by …the trained neural network;
….
Adams lacks distinctly disclosing the following underlined limitations:
… determining at least one local maximum
… of the first channel of the road points feature map by a point extraction module of the trained neural network;
… determining the at least one entry point and the at least one exit point is determined by the point extraction module depending on the at least one local maximum
Regarding the limitation; “…a global maximum”, Yuan (US-20220004808-A1) discloses in a similar invention field of endeavor, a consideration for a method and apparatus for image segmentations wherein [0226] determining a category confidence according to the category probability feature map, comprises: determining a maximum probability of each pixel in the category probability feature map; determining a category confidence based on an average of the maximum probability of each pixel … enter a next feature extraction module for feature extraction based on the category confidence… [0336] According to the category probability feature map, the probability (corresponding to the category confidence above) of each category is determined, which a larger probability means greater accuracy of predicted category ID map, and thus more suitable receptive field.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a global maximum with a reasonable expectation for success, as taught by Yuan, for the benefit of providing ideal/individually segmentation effects for images with different characteristics according to shape, size, type of an object and providing confidence in object recognition wherein a larger probability means greater accuracy … thus more suitable receptive field.
Regarding the limitation; “…a first/second/third channel”, Wang (US-20200092463-A1) discloses in a similar invention field of endeavor, a consideration for object detection [0103] …first through fourth detection modules 316a-d are used to detect objects of different sizes on the first through fourth channel feature maps 314a-d.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include feature map comprising a first/second/third channel with a reasonable expectation for success, as taught by Wang, for the benefit of providing [0015] splitting the enhanced feature map into channel feature maps of different resolutions.
Regarding the limitation; “…a point extraction module”, Mandai (US-20200242372-A1) discloses in a similar invention field of endeavor, a consideration for [0007] An arithmetic apparatus according to a first aspect of the present invention includes: a sensor information acquisition unit that acquires sensor information from a sensor which is mounted in a vehicle and collects information about surroundings of the vehicle as the sensor information; a feature point extraction unit that extracts feature points of an object by using the sensor information;
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include a point extraction module with a reasonable expectation for success, as taught by Mandai, for the benefit of providing data for system detection of objects/features within an operational zone in which a system functions.
Regarding the limitation; “…an entry point or to an exit point”, Ichinokawa (US-20200168180-A1) discloses in a similar invention field of endeavor, a consideration for measuring features of a roundabout once determined [0063] …an outer diameter of the roundabout RC1… [0083] …whether or not outer diameters OD2 and OD3 of the circular road lanes L10A and L10B around the central islands CI2 and CI3 of the roundabout RC2… [0041] …The information on the road includes, for example, at least one of a road situation (a road shape or a curvature), information indicating a lane or a traveling direction in which the vehicle M can travel, information on a form of a roundabout and a road at an entrance or exit of the roundabout (for example, a road connected to the roundabout)…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include determining at least one entry point and at least one exit point of the roundabout with a reasonable expectation for success, as taught by the combination of Lee and Ichinokawa, for the benefit of providing information regarding travel direction of roads connected to a roundabout, so as to ensure safety and control of movement into and out of road conditions (e.g. roundabouts).
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adams (US-20210182596-A1), Shi (US-20220114724-A), Lee (US-20200166943-A1) and Ichinokawa (US-20200168180-A1), as applied to claim 1 above and further in view of Costa (US-20230072552-A1), and Zhang (US-20200209860-A1).
11. Adams (US-20210182596-A1) discloses The computer-implemented method according to claim 1,
further comprising:
**** depending on the output of the classificator module [0047-0048] ****;
determining a source point **** and a destination point **** of the trained neural network depending on a predefined initial source point and a predefined initial destination point for the vehicle [0054] the planning component 434 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For the purpose of this discussion, a route can be a sequence of waypoints for travelling between two locations. As non-limiting examples, waypoints include streets, intersections, global positioning system (GPS) coordinates, etc. Further, the planning component 434 can generate an instruction for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 434 can determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints.
Adams lacks disctincly disclosing the following underlined limitations:
… determining at least two entry points and at least two exit points of the roundabout… determining a source point of the at least two entry points…
… and depending on the output of the radius estimation module;
… of the at least two exit points by a recurrent neural network module
Regarding the limitation; “…determining at least two entry points and at least two exit points of the roundabout”, Costa (US-20230072552-A1) discloses in a similar invention field of endeavor, a consideration for [claim.1] … the number of roads leading to the roundabout, the difference between the number of entrances to the roundabout and the number of exits from the roundabout, the number of lanes present in the roundabout, and the outside diameter of the roundabout.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include determining at least two entry points and at least two exit points of the roundabout with a reasonable expectation for success, as taught by Costa, for the benefit of providing vehicle recognition/navigation systems informational data on all entrances/exits of a traffic zone, to include one or more roads leading in or out of the traffic zone.
Regarding the limitation; “…the radius estimation module”, Lee (US-20200166943-A1) discloses in a similar invention field of endeavor, a consideration for wherein an input image depicts a recognized roundabout [0105, 0134] … the roundabout recognizer 210 … the camera may recognize the roundabout 1… Roundabout 1 is recognized by extracting an image …acquired by the camera. a radius estimation module dependent upon an outer radius of a roundabout [0106] … outermost boundary B may be formed using average perimeter information on the average perimeter obtained by averaging the circumference R of general roundabouts, or the structure of roundabout 1. Here, the average perimeter information may be information obtained through simulation and algorithm. and an inner radius [0108] … detects the traffic island 50, which is a circular structure formed at the center of the roundabout 1 by the sensor 120, and the outer lane 60 in the roundabout 1, respectively, and calculates the respective curvatures. From each curvature, calculate the radius of the traffic island 50 and the radius of the outer lane 60 to calculate the average value of the radius, and the average perimeter is calculated using the average value to form the outermost boundary B having the average perimeter as the length of the arc.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include radius estimation module with a reasonable expectation for success, as taught by Lee, for the benefit of providing measurements necessary for orienting a vehicle within a driving environment, so as to ensure safety and precision of movement dependent upon road conditions (e.g. roundabouts).
Regarding the limitation; “…recurrent neural network module”, Zhang (US-20200209860-A1) discloses in a similar invention field of endeavor, a consideration for [0027] ... The disclosed system 15 extracts data indicative of traffic context along a roadway 20 and shoulders 25 proximate the roadway by combining information from many different sensors systems within the vehicle 10, the other vehicles 12, 14 and static structures. The combined information from sensors within the traffic environment are used to build a feature map of an image region corresponding to the target vehicles 12, 14, and is combined in a Recurrent Neural Network (RNN) to predict possible maneuvers and paths of the target vehicles 12, 14 at future time steps [t0, t1, t2, . . . , tn) as indicated at 12′, 14′.
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include recurrent neural network module with a reasonable expectation for success, as taught by Zhang, for the benefit of providing environmental traffic data to predict possible maneuvers and paths of the target vehicles at future time steps.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Adams (US-20210182596-A1), Shi (US-20220114724-A), Lee (US-20200166943-A1) and Ichinokawa (US-20200168180-A1), as applied to claim 1 above and further in view of Sibiryakov (US-20160104047-A1).
12. Adams (US-20210182596-A1) lacks The computer-implemented method according to claim 1,
further comprising converting two or more camera images to a common top view image in order to generate the input image.
Regarding the limitation; Sibiryakov (US-20160104047-A1) discloses in a similar invention field of endeavor, a consideration for [claim.1] An image recognition system for a vehicle comprising: at least two camera units, each camera unit configured to record an image of a road in the vicinity of the vehicle and to provide image data representing the respective image of the road; a first image processor configured to combine the image data provided by the at least two camera units into a first top-view image, wherein the first top-view image is aligned to a road image plane; a first feature extractor configured to…
It would have been obvious to one of ordinary skill in the art before the time the instant application was effectively filed to adapt the modified system of Adams to include comprising converting two or more camera images to a common top view image in order to generate the input image with a reasonable expectation for success, as taught by Sibiryakov, for the benefit of providing an image recognition system which uses a plurality of images of such monocular cameras to provide information and data necessary to perform complex driver assistance tasks.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW JOHN MOSCOLA whose telephone number is (571)272-6944.
The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached on (571) 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.J.M./Examiner, Art Unit 3663
/ABBY J FLYNN/Supervisory Patent Examiner, Art Unit 3663
Prosecution Insights