DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/22/2026 has been entered.
Status of Claims
This action is in reply to the RCE filed on 05/22/2026.
Claims 1-8, 11-20, 23-24, 28-29, 34-56 are currently pending and have been examined.
Claims 1, 11, 12, 40, 44, 47, and 51 are currently amended.
Claims 1-8, 11-20, 23-24, 28-29, 34-56 are currently rejected.
This action is made NON-FINAL.
Response to Arguments
Applicant’s arguments filed 05/22/2026 have been fully considered but they are not persuasive.
Applicant argues that Zou teaching combining multiple top view images into a single top view image is not the same as “aggregating the plurality of top view images as claimed”. The rejections below are a 103 and Zou is not mapped to teaching all the limitations which are taught by the other references.
Applicant argues that Liang does not teach the claimed limitations because the top view image is generated after merging the multiple images together. While the examiner believes this would be considered obvious under the “changes in sequence of adding ingredients” rationale of MPEP 2144.04(IV)(C), the examiner is using a new reference to map to this limitation in order to advance prosecution.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 12, 14-15, 17-20, 23-24, 29, 34-36, 38, 40-43, 45-50, and 52-54 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (from IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter.
Regarding claim 1:
Zou teaches:
A system for automatically mapping a road segment (techniques for road feature detection using a vehicle camera system [abstract]), the system comprising:
at least one processor (processing system 700 has one or more central processing units (processors) 21a, 21b, 21c, etc. (collectively or generically referred to as processor(s) 21 and/or as processing device(s)) [0052]) programmed to:
receive, from at least one camera mounted on a vehicle (The cameras 130 capture images external to the vehicle 100 [0024]), a plurality of images acquired as the vehicle traversed the road segment (Each of the cameras 130 has a field-of-view (FOV) 131a, 131b, 131c, 131d (collectively referred to herein as “FOV 131”). The FOV is the area observable by a camera. For example, the camera 130a has an FOV 131a, the camera 131b has an FOV 131b, the camera 130c has an FOV 131c, and the camera 131d has an FOV 131d. The captured images can be the entire FOV for the camera or can be a portion of the FOV of the camera. [0024]);
convert each of the plurality of images to a corresponding top view image to provide a plurality of top view images (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026]);
aggregate the plurality of top view images (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026])… to provide an aggregated top view image of the road segment (generates a top view of the road based at least in part on the image [0027]);
analyze the aggregated top view image to identify at least one road feature associated with the road segment (detects lane boundaries of a lane of the road based at least in part on the top view of the road, and detects a road feature within the lane boundaries of the lane of the road using machine learning and/or computer vision techniques [0027]);
automatically annotate the at least one road feature relative to the aggregated top view image (fig. 3, bounding boxes showing detected lane markers; Once the lane boundaries of the lane are detected, the road feature detection engine 216 uses the lane boundaries to detect road features within the lane boundaries of the lane of the road using machine learning and/or computer vision techniques. The road feature detection engine 216 searches within the top view, as defined by the lane boundaries, to detect road features. The road feature detection engine 216 can determine a type of road feature (e.g., a straight arrow, a left-turn arrow, etc.) as well as a location of the road feature (e.g., arrow ahead, bicycle lane to the left, etc.) [0034]; The road features can be predefined in a database of road features (e.g., road feature database 218). Examples of road features include a speed limit indicator, a bicycle lane indicator, a railroad indicator, a school zone indicator, and a direction indicator (e.g., left-turn arrow, straight arrow, right-turn arrow, straight and left-turn arrow, straight and right-turn arrow, etc.), and the like. The road feature database 218 can be updated when road features are detected, and the road feature database 218 can be accessible by other vehicles, such as from a cloud computing environment over a network or from the vehicle 100 directly (e.g., using direct short-range communications (DSCR)). This enables crowd-sourcing of road features. [0035]); and
output to at least one memory the aggregated top view image including the annotated at least one road feature (The road feature database 218 can be updated when road features are detected, and the road feature database 218 can be accessible by other vehicles, such as from a cloud computing environment over a network or from the vehicle 100 directly [0035]; Graphics processing unit 37 is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display [0055]).
Zou does not explicitly teach, however Coulter teaches:
aggregate the plurality of top view images based on points correlated across the plurality of top view images (Feature matching generally utilizes a variety of information to compare image feature characteristics between image sets to identify feature pairs that meet specified matching criteria [0006]), the points corresponding to a portion of an object (Feature selection may be based upon the characteristics of regions, edges, contours, line intersections, corners, etc. [0006]), by determining a relative alignment for the plurality of top view images based on the correlated points (the image frames are each aligned by matching three control points or more between the image frame and the reference image [0022]), to provide an aggregated top view image of the road segment (images from multiple sensor station positions may be combined to create mosaics of WAIV across large areas [0063]; see also figs. 2, 3, 6, 7, and 8);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou to include the teachings as taught by Coulter with a reasonable expectation of success. Zou and Coulter both teach processing and combining images from cameras. Coulter teaches the benefit of the “non-traditional WAIV approach can exploit basic geometric principles to enable rapid and precise alignment of images using simple techniques [Coulter, 0068]”.
Regarding claim 2:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one camera has an optical axis projecting away from the vehicle (see at least fig. 1, cameras 130a - 130d showing cameras pointing away from vehicle.).
Regarding claim 3:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein each of the plurality of top view images is generated based on a simulated viewpoint that is elevated relative to an actual elevation of the at least one camera (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 4:
Zou in view of Coulter teaches all the limitations of claim 3, upon which this claim is dependent.
Zou further teaches:
wherein the simulated viewpoint is elevated by at least ten meters relative to the actual elevation of the camera (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]; examiner notes that the exact simulated heigh of the birds eye view would come down to routine optimization and be an obvious design choice.).
Regarding claim 5:
Zou in view of Coulter teaches all the limitations of claim 3, upon which this claim is dependent.
Zou further teaches:
wherein the simulated viewpoint is elevated by between ten meters and twenty meters relative to the actual elevation of the camera (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]; examiner notes that the exact simulated heigh of the birds eye view would come down to routine optimization and be an obvious design choice.).
Regarding claim 6:
Zou in view of Coulter teaches all the limitations of claim 3, upon which this claim is dependent.
Zou further teaches:
wherein an optical axis associated with the simulated viewpoint is normal to a road surface associated with the road segment (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 7:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein each of the plurality of top view images is generated by warping an image captured by the at least one camera from a viewpoint of the at least one camera (the top view generation engine 212 uses fisheye camera imaging techniques to generate the top view from an image captured with a fisheye camera (i.e., a camera having a fisheye lens). When using a fisheye camera, the top view generation engine 212 can be calibrated to compensate for radial distortion caused by the fisheye lens. [0032]) to a simulated camera viewpoint elevated relative to the at least one camera and directed along a line normal to a surface of the road segment (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 8:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one camera includes at least one of a forward-facing camera relative to the vehicle (fig. 1, camera 130a), a side-facing camera relative to the vehicle (fig. 1, cameras 130 b and 130c), or a rearward-facing camera relative to the vehicle (fig. 1, cameras 130d).
Regarding claim 12:
Zou in view of Coulter in view of Liang teaches all the limitations of claim 11, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images includes determining positions of each of the plurality of points relative to the road segment (The road feature detection engine 216 can determine a type of road feature (e.g., a straight arrow, a left-turn arrow, etc.) as well as a location of the road feature (e.g., arrow ahead, bicycle lane to the left, etc.). [0034]).
Regarding claim 14:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images includes an image segmentation process in which objects represented in the plurality of top view images are identified (a surround camera system of a vehicle to detect, track, and classify close range road features reliability and in real-time [0019]) and classified (a feature extraction to extract road features from the top view using a neural network, and performing a classification of the road feature using the neural network [0005]).
Regarding claim 15:
Zou in view of Coulter teaches all the limitations of claim 14, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images includes omitting from the aggregated top view image pixels from one or more of the plurality of top view images determined, via the image segmentation process (The top view generation engine 212 generates a top view of the road based at least in part on the image. That is, the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]), to be representative of at least a portion of a vehicle (see at least fig. 3 showing portion of vehicle in 302 which is not present in transformed view 304.).
Regarding claim 17:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein a first top view image and a second top view image among the plurality of top view images at least partially overlap in an overlap region (see at least fig. 1 showing images 130a-d showing the corners overlapping between the 4 views.) and wherein aggregation of the plurality of top view images includes incorporating into the aggregated top view image at least some of the pixels from the first top view image that reside in the overlap region and at least some of the pixels from the second top view image that reside in the overlap region (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026]).
Regarding claim 18:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein a first top view image (fig. 1, 131a), a second top view image (fig. 1, 131b), and a third top view image (fig. 1, 131c) among the plurality of top view images at least partially overlap in an overlap region and wherein aggregation of the plurality of top view images includes incorporating into the aggregated top view image at least some of the pixels from the first top view image that reside in the overlap region, at least some of the pixels from the second top view image that reside in the overlap region, and at least some of the pixels from the third top view image that reside in the overlap region (According to aspects of the present disclosure, although four cameras 130a-130d are shown, different numbers of cameras (e.g., 2 cameras, 3 cameras, 5 cameras, 8 cameras, 9 cameras, etc.) can be implemented [0025]; examiner notes that the birds eye view creation of Zou would inherently have an overlap region of 3 images if more cameras were implemented on the vehicles such as in the corners which would overlap the adjacent two images.).
Regarding claim 19:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the automatic annotation of the at least one road feature is performed by a trained neural network (detecting the road feature within the lane boundaries further includes performing a feature extraction to extract road features from the top view using a neural network, and performing a classification of the road feature using the neural network [0009])
Regarding claim 20:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one road feature includes at least one of a road surface, a lane marking, or a road edge (performing a classification of the road feature using the neural network. In some examples, the lane boundaries are defined by a lane marker, a road shoulder, or a curb [0005]).
Regarding claim 23:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one road feature includes a drivable path (detecting, by the processing device, lane boundaries of a lane of the road based at least in part on the top view of the road [0004]).
Regarding claim 24:
Zou in view of Coulter teaches all the limitations of claim 23, upon which this claim is dependent.
Zou further teaches:
wherein the drivable path is associated with at least one of a merge lane (examiner is interpreting this limitation in the alternative.), an exit lane (examiner is interpreting this limitation in the alternative.), an intersection (determine a type of road feature (e.g., a straight arrow, a left-turn arrow, etc.) [0034]), or a crossing road (a railroad indicator [0005]).
Regarding claim 29:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one road feature includes at least one of a traffic light (traffic direction control indicators [0019]), a pole (examiner is interpreting this limitation in the alternative.), a traffic sign (the road feature is one of a speed limit indicator, a bicycle lane indicator, a railroad indicator, a school zone indicator, and a direction indicator [0005]), a tree (examiner is interpreting this limitation in the alternative.), or a building (examiner is interpreting this limitation in the alternative.).
Regarding claim 34:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one processor is further programmed to convert the aggregated top view image to a series of frame view images each including a representation of at least a portion of the at least one road feature (the disclosure provide for road feature detection using machine learning to address computational inefficiency and accuracy issues in existing road feature detection. More particularly, the embodiments described herein detect road features by generating a top view of a road based on an image from a camera associated with a vehicle on the road, detect lane boundaries of a lane of the road based on the top view of the road, and detect (e.g., using deep learning) a road feature within the lane boundaries of the lane of the road. These aspects of the disclosure constitute technical features that yield the technical effect of reducing overall computational load, power consumption, hardware costs, and time [0021]), and wherein annotations of the least one road feature represented in the aggregated top view image are translated to each of the series of frame view images (the present techniques use a surround camera system of a vehicle to detect, track, and classify close range road features reliability and in real-time. Road features include lane marks, traffic direction control indicators, curbs, shoulders, and the like that are located on or about the road surface. To detect road features, the present techniques implement a deep learning network to enable multiple road feature detection and classification in parallel as one step and in real-time. In some examples, the road features can be fused with other in-vehicle sensors/data (e.g., long range sensors, other cameras, LIDAR sensors, maps, etc.) to improve detection and classification accuracy and robustness. In additional examples, the road features can be used for self-mapping and crowdsourcing to generate and/or update a road feature database [0019]).
Regarding claim 35:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one processor is further programmed to generate at least one navigational map based on the aggregated top view image stored to the at least one memory (the road features can be used for self-mapping and crowdsourcing to generate and/or update a road feature database [0019]).
Regarding claim 36:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the at least one processor is further programmed to overlay the aggregated top view image with a drivable path (Feature extraction takes as an input the boundary image 312, which represents the result of the image processing depicted in FIG. 3 as performed by the top view generation engine 212 and the lane boundaries detection engine 214 [0040]) generated based on trajectories collected from a plurality of vehicles during earlier traversals of the road segment (crowdsourcing to generate and/or update a road feature database [0019]).
Regarding claim 38:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the plurality of images are acquired by cameras included on a plurality of different vehicles as each of the plurality of different vehicles traversed the road segment (crowdsourcing to generate and/or update a road feature database [0019]).
Regarding claim 40:
Zou teaches:
A non-transitory computer-readable medium storing instructions executable by at least one processor to perform a method for automatically mapping a road segment (The system also includes a memory including computer readable instructions and a processing device for executing the computer readable instructions for performing a method [0006]), the method comprising:
receiving, from at least one camera mounted on a vehicle (The cameras 130 capture images external to the vehicle 100 [0024]), a plurality of images acquired as the vehicle traversed the road segment (Each of the cameras 130 has a field-of-view (FOV) 131a, 131b, 131c, 131d (collectively referred to herein as “FOV 131”). The FOV is the area observable by a camera. For example, the camera 130a has an FOV 131a, the camera 131b has an FOV 131b, the camera 130c has an FOV 131c, and the camera 131d has an FOV 131d. The captured images can be the entire FOV for the camera or can be a portion of the FOV of the camera. [0024]);
converting each of the plurality of images to a corresponding top view image to provide a plurality of top view images (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026]);
aggregate the plurality of top view images (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026])… to provide an aggregated top view image of the road segment (generates a top view of the road based at least in part on the image [0027]);
analyzing the aggregated top view image to identify at least one road feature associated with the road segment (detects lane boundaries of a lane of the road based at least in part on the top view of the road, and detects a road feature within the lane boundaries of the lane of the road using machine learning and/or computer vision techniques [0027]);
automatically annotating the at least one road feature relative to the aggregated top view image (Once the lane boundaries of the lane are detected, the road feature detection engine 216 uses the lane boundaries to detect road features within the lane boundaries of the lane of the road using machine learning and/or computer vision techniques. The road feature detection engine 216 searches within the top view, as defined by the lane boundaries, to detect road features. The road feature detection engine 216 can determine a type of road feature (e.g., a straight arrow, a left-turn arrow, etc.) as well as a location of the road feature (e.g., arrow ahead, bicycle lane to the left, etc.) [0034]; The road features can be predefined in a database of road features (e.g., road feature database 218). Examples of road features include a speed limit indicator, a bicycle lane indicator, a railroad indicator, a school zone indicator, and a direction indicator (e.g., left-turn arrow, straight arrow, right-turn arrow, straight and left-turn arrow, straight and right-turn arrow, etc.), and the like. The road feature database 218 can be updated when road features are detected, and the road feature database 218 can be accessible by other vehicles, such as from a cloud computing environment over a network or from the vehicle 100 directly (e.g., using direct short-range communications (DSCR)). This enables crowd-sourcing of road features. [0035]); and
outputting to at least one memory the aggregated top view image including the annotated at least one road feature (The road feature database 218 can be updated when road features are detected, and the road feature database 218 can be accessible by other vehicles, such as from a cloud computing environment over a network or from the vehicle 100 directly [0035]; Graphics processing unit 37 is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display [0055]).
Zou does not explicitly teach, however Coulter teaches:
aggregate the plurality of top view images based on points correlated across the plurality of top view images (Feature matching generally utilizes a variety of information to compare image feature characteristics between image sets to identify feature pairs that meet specified matching criteria [0006]), the points corresponding to a portion of an object (Feature selection may be based upon the characteristics of regions, edges, contours, line intersections, corners, etc. [0006]), by determining a relative alignment for the plurality of top view images based on the correlated points (the image frames are each aligned by matching three control points or more between the image frame and the reference image [0022]), to provide an aggregated top view image of the road segment (images from multiple sensor station positions may be combined to create mosaics of WAIV across large areas [0063]; see also figs. 2, 3, 6, 7, and 8);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou to include the teachings as taught by Coulter with a reasonable expectation of success. Zou and Coulter both teach processing and combining images from cameras. Coulter teaches the benefit of the “non-traditional WAIV approach can exploit basic geometric principles to enable rapid and precise alignment of images using simple techniques [Coulter, 0068]”.
Regarding claim 41:
Zou in view of Coulter teaches all the limitations of claim 40, upon which this claim is dependent.
Zou further teaches:
wherein the at least one camera has an optical axis projecting away from the vehicle (see at least fig. 1, cameras 130a - 130d showing cameras pointing away from vehicle.).
Regarding claim 42:
Zou in view of Coulter teaches all the limitations of claim 40, upon which this claim is dependent.
Zou further teaches:
wherein each of the plurality of top view images is generated based on a simulated viewpoint that is elevated relative to an actual elevation of the at least one camera (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 43:
Zou in view of Coulter teaches all the limitations of claim 40, upon which this claim is dependent.
Zou further teaches:
wherein each of the plurality of top view images is generated by warping an image captured by the at least one camera from a viewpoint of the at least one camera (the top view generation engine 212 uses fisheye camera imaging techniques to generate the top view from an image captured with a fisheye camera (i.e., a camera having a fisheye lens). When using a fisheye camera, the top view generation engine 212 can be calibrated to compensate for radial distortion caused by the fisheye lens. [0032]) to a simulated camera viewpoint elevated relative to the at least one camera and directed along a line normal to a surface of the road segment (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 45:
Zou in view of Coulter teaches all the limitations of claim 40, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images includes an image segmentation process in which objects represented in the plurality of top view images are identified (a surround camera system of a vehicle to detect, track, and classify close range road features reliability and in real-time [0019]) and classified (a feature extraction to extract road features from the top view using a neural network, and performing a classification of the road feature using the neural network [0005]).
Regarding claim 46:
Zou in view of Coulter teaches all the limitations of claim 40, upon which this claim is dependent.
Zou further teaches:
wherein the automatic annotation of the at least one road feature is performed by a trained neural network (detecting the road feature within the lane boundaries further includes performing a feature extraction to extract road features from the top view using a neural network, and performing a classification of the road feature using the neural network [0009]).
Regarding claim 47:
Zou teaches:
A method for automatically mapping a road segment (The system also includes a memory including computer readable instructions and a processing device for executing the computer readable instructions for performing a method [0006]), the method comprising:
receiving, from at least one camera mounted on a vehicle (The cameras 130 capture images external to the vehicle 100 [0024]), a plurality of images acquired as the vehicle traversed the road segment (Each of the cameras 130 has a field-of-view (FOV) 131a, 131b, 131c, 131d (collectively referred to herein as “FOV 131”). The FOV is the area observable by a camera. For example, the camera 130a has an FOV 131a, the camera 131b has an FOV 131b, the camera 130c has an FOV 131c, and the camera 131d has an FOV 131d. The captured images can be the entire FOV for the camera or can be a portion of the FOV of the camera. [0024]);
converting each of the plurality of images to a corresponding top view image to provide a plurality of top view images (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026]);
aggregate the plurality of top view images (the captured images from the cameras 130 can be combined to form a top view or “bird's eye” view that provides a surround view around the vehicle 100 [0026])… to provide an aggregated top view image of the road segment (generates a top view of the road based at least in part on the image [0027]);
analyzing the aggregated top view image to identify at least one road feature associated with the road segment (detects lane boundaries of a lane of the road based at least in part on the top view of the road, and detects a road feature within the lane boundaries of the lane of the road using machine learning and/or computer vision techniques [0027]);
automatically annotating the at least one road feature relative to the aggregated top view image (Once the lane boundaries of the lane are detected, the road feature detection engine 216 uses the lane boundaries to detect road features within the lane boundaries of the lane of the road using machine learning and/or computer vision techniques. The road feature detection engine 216 searches within the top view, as defined by the lane boundaries, to detect road features. The road feature detection engine 216 can determine a type of road feature (e.g., a straight arrow, a left-turn arrow, etc.) as well as a location of the road feature (e.g., arrow ahead, bicycle lane to the left, etc.) [0034]; The road features can be predefined in a database of road features (e.g., road feature database 218). Examples of road features include a speed limit indicator, a bicycle lane indicator, a railroad indicator, a school zone indicator, and a direction indicator (e.g., left-turn arrow, straight arrow, right-turn arrow, straight and left-turn arrow, straight and right-turn arrow, etc.), and the like. The road feature database 218 can be updated when road features are detected, and the road feature database 218 can be accessible by other vehicles, such as from a cloud computing environment over a network or from the vehicle 100 directly (e.g., using direct short-range communications (DSCR)). This enables crowd-sourcing of road features. [0035]); and
outputting to at least one memory the aggregated top view image including the annotated at least one road feature (The road feature database 218 can be updated when road features are detected, and the road feature database 218 can be accessible by other vehicles, such as from a cloud computing environment over a network or from the vehicle 100 directly [0035]; Graphics processing unit 37 is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display [0055]).
Zou does not explicitly teach, however Coulter teaches:
aggregate the plurality of top view images based on points correlated across the plurality of top view images (Feature matching generally utilizes a variety of information to compare image feature characteristics between image sets to identify feature pairs that meet specified matching criteria [0006]), the points corresponding to a portion of an object (Feature selection may be based upon the characteristics of regions, edges, contours, line intersections, corners, etc. [0006]), by determining a relative alignment for the plurality of top view images based on the correlated points (the image frames are each aligned by matching three control points or more between the image frame and the reference image [0022]), to provide an aggregated top view image of the road segment (images from multiple sensor station positions may be combined to create mosaics of WAIV across large areas [0063]; see also figs. 2, 3, 6, 7, and 8);
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou to include the teachings as taught by Coulter with a reasonable expectation of success. Zou and Coulter both teach processing and combining images from cameras. Coulter teaches the benefit of the “non-traditional WAIV approach can exploit basic geometric principles to enable rapid and precise alignment of images using simple techniques [Coulter, 0068]”.
Regarding claim 48:
Zou in view of Coulter teaches all the limitations of claim 47, upon which this claim is dependent.
Zou further teaches:
wherein the at least one camera has an optical axis projecting away from the vehicle (see at least fig. 1, cameras 130a - 130d showing cameras pointing away from vehicle.).
Regarding claim 49:
Zou in view of Coulter teaches all the limitations of claim 47, upon which this claim is dependent.
Zou further teaches:
wherein each of the plurality of top view images is generated based on a simulated viewpoint that is elevated relative to an actual elevation of the at least one camera (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 50:
Zou in view of Coulter teaches all the limitations of claim 47, upon which this claim is dependent.
Zou further teaches:
wherein each of the plurality of top view images is generated by warping an image captured by the at least one camera from a viewpoint of the at least one camera (the top view generation engine 212 uses fisheye camera imaging techniques to generate the top view from an image captured with a fisheye camera (i.e., a camera having a fisheye lens). When using a fisheye camera, the top view generation engine 212 can be calibrated to compensate for radial distortion caused by the fisheye lens. [0032]) to a simulated camera viewpoint elevated relative to the at least one camera and directed along a line normal to a surface of the road segment (the top view generation engine 212 uses the image to generate a top view of the road as if the point of view of the camera was directly above the road looking down at the road. An example of a top view (e.g., top-down view 304) is depicted in FIG. 3. [0031]).
Regarding claim 52:
Zou in view of Coulter teaches all the limitations of claim 47, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images includes an image segmentation process in which objects represented in the plurality of top view images are identified (a surround camera system of a vehicle to detect, track, and classify close range road features reliability and in real-time [0019]) and classified (a feature extraction to extract road features from the top view using a neural network, and performing a classification of the road feature using the neural network [0005]).
Regarding claim 53:
Zou in view of Coulter teaches all the limitations of claim 47, upon which this claim is dependent.
Zou further teaches:
wherein the automatic annotation of the at least one road feature is performed by a trained neural network (detecting the road feature within the lane boundaries further includes performing a feature extraction to extract road features from the top view using a neural network, and performing a classification of the road feature using the neural network [0009]).
Regarding claim 54:
Zou in view of Coulter teach all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein the points are located along a modeling structure of the road segment (fig. 3, showing modeling structure from lane markers of road surface.).
Claim(s) 11, 13, 44, 51 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (from IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter in further view of Liang et. al. (US 2020/0314333), herein Liang (from IDS).
Regarding claim 11:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images (generates a top view of the road based at least in part on the image [0027]) further includes:
Zou in view of Coulter does not explicitly teach, however Liang teaches:
determining a relative alignment for the plurality of top view images based on tracked ego motion of the vehicle (Other sensors, such as inertial measurement units, gyroscopes, wheel encoders, sonar sensors, motion sensors to perform odometry calculations with respect to nearby moving exterior objects, and exterior facing cameras (e.g., to perform computer vision processing) can provide further contextual information for generation of a more accurate three-dimensional map. [0078]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou and Coulter to include the teachings as taught by Liang with a reasonable expectation of success. Zou and Liang both teach processing images from cameras mounted on vehicles and creating a top down view from them. Liang teaches the benefit of “a novel method where the seam line is not fixed, but instead dynamically varies for each overlap region based on the real-world objects present in that region. This novel method significantly improves the visibility and clarity of objects in the overlap regions in the reconstructed 360 views [Liang, 0116]”.
Regarding claim 13:
Zou in view of Coulter teaches all the limitations of claim 12, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Liang teaches:
wherein the positions of each of the plurality of points (the like and inanimate objects and attributes thereof such as other vehicles (e.g., current vehicle state or activity (parked or in motion or level of automation currently employed), occupant or operator identity, vehicle type (truck, car, etc.), vehicle spatial location, etc.), curbs (topography and spatial location), potholes (size and spatial location), lane division markers (type or color and spatial locations), signage (type or color and spatial locations such as speed limit signs, yield signs, stop signs, and other restrictive or warning signs), traffic signals (e.g., red, yellow, blue, green, etc.), buildings (spatial locations), walls (height and spatial locations), barricades (height and spatial location), and the like [0074]) are determined using structure from motion calculations (Other sensors, such as inertial measurement units, gyroscopes, wheel encoders, sonar sensors, motion sensors to perform odometry calculations with respect to nearby moving exterior objects, and exterior facing cameras (e.g., to perform computer vision processing) can provide further contextual information for generation of a more accurate three-dimensional map. [0078]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou and Coulter to include the teachings as taught by Liang with a reasonable expectation of success. Zou and Liang both teach processing images from cameras mounted on vehicles and creating a top down view from them. Liang teaches the benefit of “a novel method where the seam line is not fixed, but instead dynamically varies for each overlap region based on the real-world objects present in that region. This novel method significantly improves the visibility and clarity of objects in the overlap regions in the reconstructed 360 views [Liang, 0116]”.
Regarding claim 44:
Zou in view of Coulter teaches all the limitations of claim 40, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images (generates a top view of the road based at least in part on the image [0027]) includes:
Zou in view of Coulter does not explicitly teach, however Liang teaches:
determining a relative alignment for the plurality of top view images based on tracked ego motion of the vehicle (Other sensors, such as inertial measurement units, gyroscopes, wheel encoders, sonar sensors, motion sensors to perform odometry calculations with respect to nearby moving exterior objects, and exterior facing cameras (e.g., to perform computer vision processing) can provide further contextual information for generation of a more accurate three-dimensional map. [0078]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou and Coulter to include the teachings as taught by Liang with a reasonable expectation of success. Zou and Liang both teach processing images from cameras mounted on vehicles and creating a top down view from them. Liang teaches the benefit of “a novel method where the seam line is not fixed, but instead dynamically varies for each overlap region based on the real-world objects present in that region. This novel method significantly improves the visibility and clarity of objects in the overlap regions in the reconstructed 360 views [Liang, 0116]”.
Regarding claim 51:
Zou in view of Coulter teaches all the limitations of claim 47, upon which this claim is dependent.
Zou further teaches:
wherein aggregation of the plurality of top view images (generates a top view of the road based at least in part on the image [0027]) includes:
Zou in view of Coulter does not explicitly teach, however Liang teaches:
determining a relative alignment for the plurality of top view images based on the correlated feature points (the images produced by cameras 802 and 804 are combined to avoid errors that may otherwise be produced when combining images from cameras having different vantages [0118]) and based on tracked ego motion of the vehicle (Other sensors, such as inertial measurement units, gyroscopes, wheel encoders, sonar sensors, motion sensors to perform odometry calculations with respect to nearby moving exterior objects, and exterior facing cameras (e.g., to perform computer vision processing) can provide further contextual information for generation of a more accurate three-dimensional map. [0078]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou and Coulter to include the teachings as taught by Liang with a reasonable expectation of success. Zou and Liang both teach processing images from cameras mounted on vehicles and creating a top down view from them. Liang teaches the benefit of “a novel method where the seam line is not fixed, but instead dynamically varies for each overlap region based on the real-world objects present in that region. This novel method significantly improves the visibility and clarity of objects in the overlap regions in the reconstructed 360 views [Liang, 0116]”.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (From IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter in further view of Stojanovic et. al. (US 2019/0050648), herein Stojanovic.
Regarding claim 16:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Stojanovic teaches:
wherein aggregation of the plurality of top view images includes omitting from the aggregated top view image pixels from one or more of the plurality of top view images determined to be representative of at least a portion of a moving object (at block 1012, dynamic objects are vetoed from the terrestrial-view semantic images. The vetoed dynamic objects are removed from the terrestrial-view semantic images. That is, dynamic objects, such as, but not limited to, vehicles, pedestrians, and the like may be vetoed and/or removed from the terrestrial-view semantic images. The removal of dynamic objects from the drive time semantic images is performed because such dynamic objects will not be included in the received semantic map. As such, the performance of the image registration discussed herein is improved by vetoing dynamic objects. [0098]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou in view of Coulter to include the teachings as taught by Stojanovic with a reasonable expectation of success. Zou and Stojanovic both teach processing images from cameras mounted on vehicles and creating a top down view from them. Stojanovic teaches the benefit of “The removal of dynamic objects from the drive time semantic images is performed because such dynamic objects will not be included in the received semantic map. As such, the performance of the image registration discussed herein is improved by vetoing dynamic objects [Stojanovic, 0098]”.
Claim(s) 28 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (From IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter in further view of Kang et. al. (US 2019/0095722), herein Kang.
Regarding claim 28:
Zou in view of Coulter teaches all the limitations of claim 1, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Kang teaches:
wherein the at least one road feature includes a virtual lane marking connecting two or more discontinuous lane markings (the driving lane identifying apparatus generates virtual lines, for example, virtual lines 1510 of FIG. 15, by fitting, in a segmentation image, the boundary lines that demarcate a left boundary and a right boundary of the lane. The driving lane identifying apparatus may generate the virtual lines, for example, the virtual lines 1510, by fitting the boundary lines in the segmentation image using a spline or a polyline based on a local gradient or a local threshold [0092]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou in view of Coulter to include the teachings as taught by Kang with a reasonable expectation of success. Zou and Kang both teach processing images from cameras mounted on vehicles. Kang teaches the benefit of “a method of identifying a driving lane, including extracting, from an input image, a left lane boundary line and a right lane boundary line of a driving lane of a vehicle, generating a segmentation image by segmenting the input image into objects included in the input image based on a semantic unit, generating a multi-virtual lane by fitting, in the segmentation image, the left lane boundary line and the right lane boundary line on a left side and a right side at equidistance intervals, determining a number of lanes of the multi-virtual lane based on whether the multi-virtual lane corresponds to a road component in the segmentation image, and identifying the driving lane by determining a relative location of the driving lane on a road on which the vehicle may be traveling based on the determined number of the lanes of the multi-virtual lane [Kang, 0022]”.
Claim(s) 37 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (From IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter in further view of Dorum et. al. (US 2010/0082248), herein Dorum.
Regarding claim 37:
Zou in view of Coulter teaches all the limitations of claim 36, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Dorum teaches:
wherein the drivable path is represented as a 3D spline (n blocks 1502 and 1504, the 3D B-spline routine 1500 associates the altitude B-spline data to the road link map database such that previous knowledge of the road attributes may be added to the splines. This allows for intelligent processing of the altitude B-splines, such as knowing if the elevation data is at an intersection or on a ramp (where crossing elevations are allowed to differ) [0133]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou in view of Coulter to include the teachings as taught by Dorum with a reasonable expectation of success. Zou and Dorum both teach processing images from cameras mounted on vehicles and creating a top down view from them. Dorum teaches the benefit of “link chains, optimized 2D B-splines, and height data are used to create 3D splines. The height data is preferably obtained from GPS/IMU traces collected as vehicles travel on roads represented by the road segments. The height data is corrected at crossing nodes to account for GPS height inaccuracies. The link chains are fitted to create an altitude B-spline using the corrected height data. The 2D B-spline and the altitude B-spline are merged to obtain a 3D B-spline. Like the 2D B-spline, knots not needed to preserve the position, curvature, slope and/or heading are removed from the 3D B-spline to minimize storage requirements [Dorum, 0010]”.
Claim(s) 39 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (From IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter in further view of Zhang et. al. (US 2013/0293717), herein Zhang (from IDS).
Regarding claim 39:
Zou in view of Coulter teaches all the limitations of claim 38, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Zhang teaches:
wherein the plurality of images are aligned based on collected ego motion associated with each of the different vehicles (the vehicle motion compensation process looks at the image points 72 and 74 in consecutive image frames where twice or more of the number of the image points 72 and 74 in the two or more frames are available for lane geometry analysis to align the image points 72 and 74 from one image frame to the next image frame based on the motion of the vehicle 10 [0030]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou in view of Coulter to include the teachings as taught by Zhang with a reasonable expectation of success. Zou and Zhang both teach processing images from cameras mounted on vehicles and creating a top down view from them. Zhang teaches the benefit of “Vehicle motion compensation can be used to enhance the identification of the lanes lines 50 and 52 in the image 32 at box 82 [Zhang, 0030]”.
Claim(s) 55-56 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zou et. al. (US 2017/0300763), herein Zou (from IDS) in view of Coulter et. al. (US 2020/0226352), herein Coulter in further view of Porter et. al. (US 2020/0098130), herein Porter.
Regarding claim 55:
Zou in view of Coulter teach all the limitations of claim 1, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Porter teaches:
wherein aggregating the plurality of top view images includes determining orientation (“an orientation”) and relative spacing (“a spatial position”) of the plurality of top view images based on the correlated points (In step 42, the system performs an image orientation phase. The image orientation step determines a spatial position and an orientation of each camera relative to each other. For example, the system selects matching key points in each image pair by using a feature detector algorithm, such as, for example, KAZE. Those skilled in the art would understand that other methods for selecting matching key points or other feature detector algorithms can be used. FIG. 6 is an illustration showing an example of how key points are matched between image pairs. [0035]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou in view of Coulter to include the teachings as taught by Porter with a reasonable expectation of success. All the references are in the same field on endeavor of aggregating images. Porter also teaches the benefit of “ground surface condition detection and extraction from digital images. The digital images can include, but are not limited to, aerial imagery, satellite imagery, ground-based imagery, imagery taken from unmanned aerial vehicles (UAVs), mobile device imagery, etc. The disclosed system can perform a high resolution scan and generate an orthomosaic and a digital surface model from the scans. The system can then perform damage detection and a geometric extraction. Finally, the system can generate a damage report. [Porter, 0005]”.
Regarding claim 56:
Zou in view of Coulter teach all the limitations of claim 1, upon which this claim is dependent.
Zou in view of Coulter does not explicitly teach, however Porter teaches:
wherein the points represent a 3D position associated with a surface of the object (Generating the digital surface model determines a point's 3D location when it is seen by a multiplicity of images [0038]).
It would have been obvious to one of ordinary skill in the art at the time of the effective filing date of the claimed invention to have modified Zou in view of Coulter to include the teachings as taught by Porter with a reasonable expectation of success. All the references are in the same field on endeavor of aggregating images. Porter also teaches the benefit of “ground surface condition detection and extraction from digital images. The digital images can include, but are not limited to, aerial imagery, satellite imagery, ground-based imagery, imagery taken from unmanned aerial vehicles (UAVs), mobile device imagery, etc. The disclosed system can perform a high resolution scan and generate an orthomosaic and a digital surface model from the scans. The system can then perform damage detection and a geometric extraction. Finally, the system can generate a damage report. [Porter, 0005]”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bai (US 2020/0232800) discloses A system for correcting HD map images is provided. The system is configured to: receive, from a plurality of vehicles, ground view image data of a map anomaly hotspot captured by the vehicles; convert the ground view image data to bird view image data using homography; remove outliers from the bird view image data; apply an error reduction algorithm; stitch together a sequence of consecutive images from the image data after error reduction to produce a high-definition image, wherein each of the consecutive images has been generated through converting ground view image data to bird view image data using homography, removing outliers from the bird view image data, and applying an error reduction algorithm; compare the stitched sequence of consecutive images to a high-definition image of the hotspot to determine revisions to the high-definition image; and provide revised high-definition image data for use in navigating near the map anomaly hotspot.
Rao (US 11,620,730) discloses A method for combining multiple images is disclosed herein. A target mapping matrix is determined based on a first image and a second image. The target mapping matrix is associated with a target correspondence between the first image and the second image. The first image and the second image are combined into a combined image based on the first target mapping matrix. The combined image is output by implementing the disclosed method.
Sadasue (US 2021/0058598) discloses An information processing apparatus includes an acquisition unit configured to acquire a plurality of captured images of a traveling surface where a movable apparatus travels, each of the captured images including distance information in a depth direction transverse to the traveling surface, the plurality of captured images having been captured using a plurality of stereo image capture devices, and an image processing unit configured to stitch together the plurality of images of the traveling surface captured by the plurality of stereo image capture devices by identifying partially overlapping portions of one or more pairs of the images captured by respective stereo image capture devices which are adjacent in a width direction of the traveling surface.
Murukeshan (US 2020/0219273) discloses A method of measuring the surface roughness of a component using an optical system comprising a tunable laser light source and a camera system. Positioning the component to be measured upon a mount in front of the optical system. Capturing a first image of the component at a first location at a first wavelength λ.sub.1, and then capturing a second image of the component at the first location at a second wavelength λ.sub.2. Determining the Speckle Statistical Correlation (SSC) coefficient of the first and second images. Plotting the SSC coefficient for the combined first and second images. Calculating the roughness parameters R.sub.a and R.sub.q from the SSC coefficient plot. Plotting a roughness map for the imaged surface from the calculated roughness parameters R.sub.a and R.sub.q. Moving the optical system to a new location and repeating steps (b) to (f) at the new location, and repeating these steps until a desired area of the component has been imaged. Stitching the roughness maps for each location to form an overall roughness map for the desired area of the component.
Elhage (US 2018/0330472) discloses systems and methods for image capturing technologies and, more particularly, to a slice scan imaging system and respective processes to achieve high quality images. The method can be implemented in a computing device, which includes: capturing multiple lines of an image in a single slice; capturing multiple slices; stitching together the multiple slices by aligning common features of the images of a previous slice with a successive slice; and blending together the stitched together multiple slices.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Scott R Jagolinzer whose telephone number is (571)272-4180. The examiner can normally be reached M-Th 8AM - 4PM Eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christian Chace can be reached at (571)272-4190. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Scott R. Jagolinzer
Examiner
Art Unit 3665
/S.R.J./Examiner, Art Unit 3665 /CHRISTIAN CHACE/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights