DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed on October 7, 2025 has been entered. Claims 1–8, 10–14, 16–17, and 19–23 are currently pending. Claims 9, 15, and 18 have been canceled. Claims 1, 8, 10, 16, and 19 have been amended. Claims 21–23 are newly added. Claim 16 has been amended to incorporate the subject matter of canceled claim 15.
Response to Arguments
Applicant’s arguments filed October 7, 2025 have been fully considered and are persuasive with respect to the previous rejections under 35 U.S.C. § 103, as explained below.
Applicant’s arguments, see pages 9–10 of the Remarks, state that amended independent claims 1, 10, and 19 now require that the broken lane-line geometry comprise lane-line geometry that is incorrect relative to ground-truth lane-line geometry. Applicant argues that Tanaka corrects road-region pixels extracted from an aerial image by reference to road-map information, but does not identify erroneous lane-line geometry by comparison with ground-truth lane-line geometry. Applicant further argues that Iizuka does not cure this deficiency.
The Examiner agrees that the portions of Tanaka and Iizuka cited in the previous Office Action do not expressly teach the newly added limitation requiring lane-line geometry that is incorrect relative to ground-truth lane-line geometry. Accordingly, the previous grounds of rejection, as they relied upon Tanaka and Iizuka, have been withdrawn.
However, upon further consideration, a new ground(s) of rejection is made in view of Zhang et al (McGAN). The newly cited Zhang reference is distinct from Zhang et al., “Least Squares Relativistic Generative Adversarial Network for Perceptual Super-Resolution Imaging” (2020), identified as “Zhang S” in this Office Action. Zhang S remains applied only for the loss-function and feature-map limitations of claims 4, 6–7, 13, and 16.
Applicant’s substantive amendments to independent claims 1, 10, and 19 added the ground-truth-relative geometry limitation. Applicant’s amendment to claim 8 added the location-probe-data limitation, and Applicant newly presented claims 21–23. These amendments directly necessitated the newly cited Zhang, and Jiao references and the updated grounds of rejection. The remaining dependent claims incorporate the limitations of their amended base claims and are therefore treated as amended for purposes of finality. Accordingly, the new grounds of rejection were necessitated by Applicant’s amendments, and this action is properly made final in accordance with MPEP § 706.07(a).
Based on these facts, this action is made FINAL.
Claim Objections
Claim 16 is objected to because of the following informalities: “-- and target representation’s feature maps --”, should be changed to “-- and a target representation’s feature maps --”. Appropriate correction is required.
Claim Rejections - 35 USC § 112(d)
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 23 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 1 already requires processing the representation through an inpainting model comprising a generator network, wherein the generator network generates inpainting of the masked area with corrected lane-line geometry in a corrected representation. Claim 23 merely restates that the corrected lane-line geometry in the corrected representation is generated by the inpainting model and therefore does not specify a further limitation of claim 1.
Applicant may cancel claim 23, amend claim 23 to provide a further limitation, rewrite the claim in independent form, or present a sufficient showing that claim 23 complies with the statutory requirements.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 5, 10-11, 14, 19-20 and 22-23 are rejected rejected under 35 U.S.C. §103 as being unpatentable over Tanaka (Tanaka et al, US 2021/0312176 A1, 2021), in view of Zhang (Zhang et al, "Road Topology Refinement via a Multi-Conditional Generative Adversarial Network", 2019), further in view of Iizuka (Iizuka et al ,“Globally and Locally Consistent Image Completion”, provided by Applicant’s IDS).
Regarding claim 1, Tanaka teaches an apparatus comprising at least one processor and at least one memory including computer program code ( Tanaka > [0030]: processor stored on memory ), the at least one memory and computer program code configured to, with the processor, cause the apparatus to at least:
receive a representation of lane line geometry for one or more roads of a road network;
( Tanaka > [0035-0036]: a segmented binary image represents a road network, with road regions and non-road regions represented in different colors. )
divide the representation into sub-areas of the representation;
( Tanaka > [0037], [0039]: the road region representation is divided into partial road regions (i.e., sub-areas). )
identify a detected sub-area within the representation including both existing lane line geometry and broken lane line geometry,
( Tanaka > [0037], [0039]: the road region representation is divided into partial road regions (i.e., sub-areas); [0046]: partial road regions with holes or breaks are processed for correction. )
wherein the broken lane line geometry comprises lane line geometry having a gap between lane line segments;
( [0035], [0046], [0048], [Figs. 3 and 5]: Tanaka teaches extracted road-region geometry 311/501 containing a breaking place 311a/501a and holes 311b/501b that prevent obtaining a correct lane network. The breaking place constitutes a gap separating portions of the road/ lane-network geometry. Tanaka further teaches filling the break across the corresponding road section and illustrates corrected road-region geometry 511 in which the breaking place and holes have been removed. )
generate a masked area over the detected sub-area within the representation including both existing lane line geometry and the broken lane line geometry;
( Tanaka > [0039-0041]: a partial road region is represented by an image with pixels distinguishing the partial road region from the other pixels in the image; see Figure 4, where a masked area identifies broken lane line geometry in a partial road region as white pixels and existing lane line geometry as colored pixels; [0048]: a binary image represents road regions before correction. )
process the masked area through an inpainting model, wherein the masked area is processed for inpainting of the masked area with corrected lane line geometry in a corrected representation;
( Tanaka > [0046] and [0048]: the pixels corresponding to the partial road region are corrected to close holes or fill breaks so that missing pixels correspond to the partial road region. )
replace the masked area of the detected sub-area with the corrected representation of the inpainting model; and
( Tanaka > [0046] and [0048]: missing pixels in the partial road region are corrected and replaced with pixels so as to create a binary image containing corrections in the partial road region. )
update a map database to include the corrected lane line geometry in place of the detected sub-area including the broken lane line geometry based on the corrected representation.
( Tanaka > [0030] and [0049]: the corrected road regions are written to storage, where the road region information is stored. )
Tanaka does not expressly disclose that the broken lane-line geometry is defined relative to ground-truth lane-line geometry. Zhang, however, teaches this feature:
wherein the broken lane line geometry comprises lane line geometry that is incorrect relative to ground truth lane line geometry;
( [Section 3.1], [Figure 2], [Section 4.2 and Figure 3]: Zhang teaches an initial road-network map containing disconnected or incomplete road-network geometry and a reference map expressly identified as ground truth. Zhang explains that the initial road network loses road structures, thereby producing gaps and fractured topology compared with the ground-truth reference map. Thus, Zhang’s disconnected or incomplete road-network geometry constitutes broken lane-line geometry that is incorrect relative to the corresponding ground-truth lane-line geometry. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to apply Zhang’s ground-truth-based topology-refinement technique to Tanaka’s road-region correction apparatus because both references address correcting gaps and disconnected geometry in image-derived road networks, and Zhang teaches that its technique produces a more complete and correct road-network representation. The modification would have predictably improved the completeness and accuracy of Tanaka’s corrected map geometry, amounting to the use of a known technique to improve a similar system in the same way.
However, Tanaka [as modified by Zhang] does not specifically teach where Iizuka teaches wherein the inpainting model comprises a generator network, wherein the masked area is processed through the generator network which includes dilated convolution layers.
( Iizuka > Figure 2 and Section 3.1 > para 2: image completion (inpainting) network uses a series of dilated convolution layers to generate a completed (corrected) image from an input image mask. )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanaka [as modified by Zhang]using Iizuka’s teachings by incorporating a generator network with dilated convolution layers for inpainting the broken lane lines of Tanaka [as modified by Zhang], in order for more context to be captured while using less computational power.
Regarding claim 2, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1, wherein the apparatus is further caused to:
process the corrected representation through a discriminator of the inpainting model, wherein the discriminator discerns a quality score for the corrected representation representing accuracy of the corrected lane line geometry.
( Iizuka > Section 3.3 > paras 1 and 4: a discriminator network takes an inpainted image as input, and determines a probability (score) that the image accurately represents a real image. )
Regarding claim 5, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1, wherein the generator network processes the representation using four dilated convolution layers including dilation factors from two to nine.
( Iizuka > Section 3.1 and Table 2: generator for image completion uses four dilated convolution layers with increasing dilation factors from 2 to 16. )
While Tanaka [as modified by Zhang and Iizuka] does not explicitly disclose wherein the generator uses eight dilated convolution layers including dilation factors from two to nine, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement such. Iizuka discusses including more dilated convolution layers to their model in order to accurately fill larger missing areas (Iizuka > Section 4.8 > para 1). Therefore, it would have been obvious to include more dilated convolution layers with a range of dilation factors, in order to achieve the predictable result of capturing information from a larger area throughout the dilated convolution layers.
Regarding claims 10-11, 14, and 19-20, the rationale provided in the rejection of claims 1-2 and 5 is incorporated herein. Further, the apparatus of claims 1-2 and 5 corresponds to the computer program product of claims 10-11 and 14 (Tanaka > [0029]: memory with computer program executed by processor), as well as the method of claims 19-20, and performs the steps disclosed herein.
Regarding claim 22, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1, wherein the lane line geometry that is incorrect relative to ground truth lane line geometry comprises missing elements of the lane line geometry that define a path of a road.
( Zhang > Section 3.1, Figure 2, and Section 4.2 and Figure 3: Zhang teaches that the initial road-network representation loses road structures and consequently contains gaps, disconnected portions, and fractured topology relative to a ground-truth road-network representation. The lost road structures constitute missing road-geometry elements that otherwise define a continuous road path. Tanaka > paragraphs [0046] and [0048] and Figures 3 and 5 further teach breaking places and holes that interrupt the road-network geometry and are filled to restore the road path. )
Regarding claim 23, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1, wherein the corrected lane line geometry in the corrected representation comprises lane line geometry generated by the inpainting model.
( [Zhang, Section 3.1 and Figure 2], [Zhang, Section 4.2 and Figure 3], [Iizuka, Sections 3.2 and 3.4 and Figure 2]: Zhang teaches that the McGAN generator receives an original image and an initial road-network map containing disconnected or incomplete geometry and generates a refined road-network map. Zhang further demonstrates that the generated road network is more complete and correct than the initial network. Iizuka teaches an image-completion network that receives an image and a binary completion-region mask and outputs a completed image, with the completion network function C(x,M) generating the completed portions within the masked region. Therefore, in the combined apparatus, the corrected lane-line geometry of the corrected representation is generated by the inpainting model. )
Claims 3 and 12 are rejected under 35 U.S.C. §103 as being unpatentable over Tanaka [as modified by Zhang and Iizuka] in view of Das (Das et al, DE 102019131971 A1, the attached English language translation is used hereinafter as the Official English language translation of this DE document).
Regarding claim 3, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 2, wherein causing the apparatus to update the map database to include the corrected lane line geometry in place of the detected sub-area including the broken lane line geometry based on the corrected representation comprises causing the apparatus to:
update the map database to include the corrected lane line geometry in place of the detected sub-area including the broken lane line geometry based on the corrected representation
( Tanaka > [0030] and [0049]: the corrected road regions are written to storage, where the road region information is stored. )
However, Tanaka [as modified by Zhang and Iizuka] does not specifically teach where Das teaches wherein a database is updated in response to the quality score for a corrected representation satisfying a predetermined value.
( Das > [0062-0064]: an output image from a GAN is input into a discriminator, which assigns it a quality score based. The image is only accepted by the discriminator if the quality score is above a predetermined threshold. )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanaka [as modified by Zhang and Iizuka] using the teachings of Das by incorporating only updating the map database of Tanaka [as modified by Zhang and Iizuka] when the quality score of the corrected image satisfies a threshold, in order to ensure the corrected lane line geometries are of an acceptable quality.
Regarding claim 12, the rationale provided in the rejection of claim 3 is incorporated herein. Further, the apparatus of claim 3 corresponds to the computer program product of claim 12 (Tanaka > [0029]: memory with computer program executed by processor), and performs the steps disclosed herein.
Claims 4, 6-7, 13, and 16 are rejected under 35 U.S.C. §103 as being unpatentable over Tanaka [as modified by Zhang and Iizuka] in view of Zhang S (Zhang et al, “Least Squares Relativistic Generative Adversarial Network for Perceptual Super-Resolution Imaging,” 2020).
Regarding claim 4, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 2,
However, Tanaka [as modified by Zhang and Iizuka] does not specifically teach where Zhang S teaches:
wherein causing the apparatus to process the representation through the inpainting model further comprises causing the apparatus to apply a Relativistic Least Square Generative Adversarial Network loss function to train the inpainting model to increase the quality score of the corrected representation.
( Zhang S > Section III-D: a least squares relativistic adversarial loss is used to train a generative adversarial network to generate more realistic images. )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanaka [as modified by Zhang and Iizuka] using Zhang S’s teachings by incorporating a relativistic least square generative adversarial network loss function to the inpainting model of Tanaka [as modified by Zhang and Iizuka], in order to train the generator to produce higher-quality images.
Regarding claim 6, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1,
However, Tanaka [as modified by Zhang and Iizuka] does not specifically teach where Zhang S teaches:
wherein the apparatus is further caused to train the inpainting model using a perceptual loss function.
( Zhang S > Section III > para 1: perceptual loss assembly is used to train a generator; Section III > para 2: a combination of loss functions is used for greater perceptual quality. )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Tanaka [as modified by Zhang and Iizuka] using Zhang S’s teachings by incorporating a perceptual loss function in the training of the inpainting model of Tanaka [as modified by Zhang and Iizuka], in order to train the generator to generate more realistic images.
Regarding claim 7, Tanaka [as modified by Zhang, Iizuka and Zhang S] teaches the apparatus of claim 6, wherein the perceptual loss function is based on a difference between a generated representation's feature maps of a convolutional neural network and target representation's feature maps of the convolutional neural network.
( Zhang S > Section III-B: perceptual loss combination comprises a feature loss, which is based on a difference between the feature representation of the generated images and the feature representation of an original (target) image. )
Regarding claims 13 and 16, the rationale provided in the rejection of claims 4 and 6-7 is incorporated herein. Further, the apparatus of claims 4 and 6-7 corresponds to the computer program product of claims 13 and 16 (Tanaka > [0029]: memory with computer program executed by processor), and performs the steps disclosed herein.
Claims 8 and 17 are rejected under 35 U.S.C. §103 as being unpatentable over Tanaka [as modified by Zhang and Iizuka] in view of Muthalagu (Muthalagu et al, (“Lane detection technique based on perspective transformation and histogram analysis for self-driving cars,” 2020).
Regarding claim 8, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1, wherein causing the apparatus to identify the detected sub-area within the representation including broken lane line geometry comprises causing the apparatus to:
process the representation of lane line geometry for one or more roads of a network using a detection model, wherein the detection model identifies missing lane line geometry and incorrect lane line geometry as broken lane line geometry.
( Tanaka > [0046-0047]: through association with an aerial image, corrections are made to holes or breaks in a partial road region while avoiding erroneous corrections made to partial road regions without holes. )
However, Tanaka [as modified by Zhang and Iizuka] does not specifically teach where Muthalagu teaches:
wherein the representation of lane line geometry is generated based, at least in part, on probe data point analysis to identify lane line geometry from probe data histograms.
( Muthalagu > Section 4.1 and Figure 19: road lanes are determined by analyzing the peaks in histograms extracted from images taken by a traveling vehicle (i.e., probe data; see [0060] of Applicant’s published specification. )
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify further modify Tanaka (as modified by Iizuka) using Muthalagu’s teachings by incorporating histogram analysis to generate the representation of lane line geometry used in the inpainting method of Tanaka (as modified by Iizuka), in order to simply the detection of lane lines in probe data utilizing clear peaks in image histograms (see Muthalagu > Figure 19).
Regarding claim 17, the rationale provided in the rejection of claim 8 is incorporated herein. Further, the apparatus of claim 8 corresponds to the computer program product of claim 17 (Tanaka > [0029]: memory with computer program executed by processor), and performs the steps disclosed herein.
Claim 21 is rejected under 35 U.S.C. §103 as being unpatentable over Tanaka [as modified by Zhang and Iizuka] in view of Jiao (Jiao et al, US 2021/0123748 A1, 2021).
Regarding claim 21, Tanaka [as modified by Zhang and Iizuka] teaches the apparatus of claim 1,
However, Tanaka [as modified by Zhang and Iizuka] does not specifically teach where Jiao teaches:
wherein the lane line geometry that is incorrect relative to ground truth lane line geometry comprises lane line geometry that includes an incorrect turn maneuver.
( Jiao > [0038], [0050], [0058-0059], [0063], and [Figure 3C]: Jiao teaches that road-network map data includes turn information specifying whether turns between road segments are permitted and that the stored turn information may differ from the actual physical road network. Jiao preserves the geometric shape of an actual vehicle route when determining its path of travel. At intersection point 307, the stored map information incorrectly requires a right turn from B Street onto F Street, whereas the actual vehicle route continues from B Street onto C Street. Jiao identifies this map-defined turn as defective and removes the erroneous turn information. Accordingly, Jiao teaches road-network geometry containing an incorrect turn maneuver relative to the actual vehicle path. )
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the road-network geometry correction system of Tanaka [as modified by Zhang and Iizuka] to identify an incorrect turn maneuver as taught by Jiao. Tanaka [as modified by Zhang and Iizuka] seek to identify and correct defective road-network geometry, while Jiao teaches that erroneous turn information is a known road-network defect that causes a stored route to differ from an actual drivable route. Applying Jiao’s known GPS-route comparison technique to the road-network correction system would have predictably enabled the system to identify and correct turn-related geometry errors, thereby preventing generation of an incorrect turn and improving the accuracy and routability of the corrected map. This modification would have used Jiao’s technique for its established purpose without changing the principle of operation of the combined system.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEN KUDO whose telephone number is (571)272-4498. The examiner can normally be reached M-F 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEN KUDO
Examiner
Art Unit 2671
/KEN KUDO/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights