DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Receipt is acknowledged of claim amendments with associated arguments/remarks, received February 12, 2023. Claims 1-12 are pending in which claims 1, 3, 7, 9 were amended.
Response to Arguments
Applicant’s arguments, see Remarks, pg 5, filed 02/12/2026, with respect to the rejections of claim 3, 9 under 35 USC § 112(b) has been fully considered and, in light of the associated amendment, is persuasive. Examiner notes support and definition for “normal or abnormal data” is identified in specification ¶ [0039]-[0041] and Fig 2A, 2B, 3A, 3B. Therefore, the rejection has been withdrawn. However, the scope of the claim limitation changed based on the amendment and upon review from an updated search, the claim is rejected under 35 U.S.C. § 103 over Kim et al (US 10,373,023) in view of Chao et al (US 2022/0156580), as discussed below.
Applicant’s arguments, see pg , filed 02/12/2026, with respect to the rejections of claim 1-12 under 35 USC has been fully considered but is not persuasive.
Applicant first argues “Kim fails to disclose or render obvious anything about pairing virtual images with real images, much less generating a golden pair based on respective levels of similarity between the real image and virtual images among a plurality of virtual images generated based on the virtual image, in the manner recited in claim 1.” (Remarks – 02/12/2026, pg 6-7). Applicant appears to argue the terminology of “transforming” a real image into a virtual image is distinct from “generating” a virtual image from a real image and that no pairing of a real image and virtual image is performed based on similarity to create an equivalent “golden pair.”
Respectfully, the examiner is not persuaded.
The claim limitation is to “generate, based on respective levels of similarity between the real image and virtual images among the plurality of virtual images, a golden pair that pairs the real image with a virtual image among the plurality of virtual images,”
Kim et al (US 10,373,023) is cited for the equivalent teaching and states “the first transformer 130 may transform at least one first input image, which is one of real images on a real world, into at least one first output image, whose one or more characteristics are same as or similar to those of virtual images on a virtual world.” (col 9 ln 29-33, cited for the claim 1 prior step “generate a plurality of virtual images based on a real image”, see Non-Final Rejection – 11/12/2025). Kim et al further states at least one virtual image, referenced as the at least one first output image, is then selected as input to the second transformer 150, referenced as the at least one second input image (col 9 ln 33-37) .
Kim et al explicitly states the second transformer 150 will use the inputted virtual image, based on the real image, to “generate an image with transformed characteristics” (col 9 ln 59-66) and “may perform transformations of each direction, a direction of from the real images to images whose characteristics are same as or similar to those of the virtual images, or a direction of from the virtual images to images whose characteristics are same as or similar to those of the real images” (col 9 ln 67-col 10 ln 3).
The applicant recites broad claim language and does not claim a specific machine model to generate the virtual images based on the real image and does not claim specificity regarding “respective levels of similarity” to generate a “golden pair.” A broadest reasonable interpretation of the claim limitation is a real image matched with any of the virtual images may be identified as a golden pair. The claim limitation “respective levels of similarity” may have any interpretation (ranging from very similar to not similar). The interpretation of the prior art is the selected second input image, which is one of the virtual images with characteristics that are the same or similar to those of the real image is equivalent to the applicant’s claimed “golden pair”. Kim et al refers to the real image as a ground truth “GT” image to the corresponding generated virtual image as a paired image “set” and may be used for training (col 10 ln 38-40).
Respectfully, the examiner is not persuaded.
Applicant second argues the prior art “fails to disclose or render obvious anything about performing domain adaptation training on a real image and a virtual image that is paired with the real image in a golden pair [generated based on respective levels of similarity between the real image and virtual images among a plurality of virtual images generated based on the real image], in the manner now recited in claim 1. (Remarks – 02/12/2026, pg 7-8). Applicant’s argument is rooted in the prior art merely teaches a difference between a transformed image and an original image for updating a transformer and does not teach generating the virtual image or golden pair.
Respectfully, the examiner is not persuaded.
The claim limitation is to “perform domain adaptation training on the real image and the virtual image that is paired with the real image in the golden pair.”
As discussed above, the second transformer generates a virtual image with substantial similarity to the real image. Kim et al refers to a virtual image paired with its corresponding real image, described as a corresponding ground truth “GT” image to the generated virtual image, as a paired image “set” (col 10 ln 35-40), where the transformers 130, 150 of the cycle GAN (Fig 3) to transform the virtual image based on the real image are trained using a forward process 101 using at least the real image (defined as the first image) and the virtual image (defined as the first image) (col 10 ln 59-63, col 11 ln 19-20). With performing the training of the transformers a transformer loss is determined and is applied to update the given transformer (col 11 ln 19-42). Transforming the real image to generate the virtual image is understood to include domain adaptation, as reiterated by Kim et al with stating “The present disclosure has an effect of alleviating a problem of procuring the sets of the training images in the non-RGB format by transforming the sets of the training images in the RGB format into those in the non-RGB format with a cycle GAN (Generative Adversarial Network) capable of being applied to domain adaptation.” (col 16 ln 15-20).
The applicant argues the prior art teaches a difference between an original image and transformed image is derived and used for updating the transformer (Remarks – 02/12/2026, pg 8). The applicant has not provided remarks as to how the claim language recites distinct teachings from the prior art. As discussed, the prior art teaches the transformers are trained using the paired image set of the virtual image and ground truth real image to transform (perform domain adaptation) a real image to a virtual image. Applicant’s claim language broadly claims to perform domain adaptation training using the virtual image and its corresponding real image and does not claim details regarding the model used to perform the training or specific step details to distinguish it from the prior art.
Respectfully, the examiner is not persuaded.
Applicant relies upon the aforementioned arguments for independent claim 7 (Remarks – 02/12/2026, pg 9), which recites identical claim limitation steps to claim 1 and is claimed in parallel (claim 1 is the apparatus while claim 1 is the method). Respectfully, the examiner is not persuaded for reasons given above.
Applicant relies upon the aforementioned arguments for dependent claims 2-6 of independent claim 1 and claims 8-12 of independent claim 7 (Remarks – 02/12/2026, pg 9). Respectfully, the examiner is not persuaded for reasons given above.
All arguments were addressed.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5, 7, 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kim et al (US 10373023, cited in Non-Final Rejection 11/12/2025).
Regarding Claim 1, Kim et al teach an apparatus for enhancing training data (learning device 100; Fig 1 and col 9 ln 3-26), the apparatus comprising:
a memory storing a program (memory 115 storing instructions; Fig 1 and col 9 ln 19-20); and
a processor coupled to the memory and configured to execute the program (processor 120 coupled to memory 115 and executes instructions stored on memory 115; Fig 1 and col 9 ln 19-21), wherein the processor is configured to:
generate a plurality of virtual images based on a real image (transforming real images to virtual images using the GAN first transformer 130; Fig 1, 3 and col 9 ln 29-33),
generate, based on respective levels of similarity between the real image and virtual images among the plurality of virtual images, a golden pair that pairs the real image with a virtual image among the plurality of virtual images (the virtual image that is identified to have characteristics that are the same or substantially similar to the real images (thereby interpreted as equivalent to identifying the golden pair) is input to a second transformer 150; Fig 1, 3 and col 9 ln 33-37), and
perform domain adaptation training (training images (real image and paired virtual image) as applicable for domain adaptation training using the GAN; col 16 pg 15-20) on the real image and the virtual image that is paired with the real image in the golden pair (the GAN first transformer 130 is trained using a pair of the real image (first image) and virtual image (second image) in the forward process 101 and a loss is determined with the first discriminator 140 used for training the first transformer 130; Fig 1, 3 and col 10 ln 59-col 11 ln 3, col 11 ln 19-24, 39-49).
Regarding Claim 5, Kim et al the apparatus of claim 1 (as described above), wherein the processor is further configured to perform domain adversarial training (training images described as applicable for domain adaptation using the GAN based on a first discriminator 140 used to train the first transformer 130 based on the pair of the real image (first image) and virtual image (second image) in the forward process 101; Fig 1, 3 and col 10 ln 59-col 11 ln 3, col 11 ln 19-24, 39-49, col 16 pg 15-20).
Regarding Claim 7, Kim et al teach a method of enhancing training data (method of generating training data using learning device cycle GAN; Fig 1 and col 9 ln 3-26), the method comprising: steps identical to claim 1 (as described above).
Regarding Claim 11, Kim et al teach a method of claim 7 (as described above), wherein further limitations are taught identical to claim 5 (as described above).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al (US 10373023) in view of Kobayashi et al (US 2007/0258658, cited in Non-Final Rejection 11/12/2025).
Regarding Claim 2, Kim et al the apparatus of claim 1 (as described above), wherein the processor (processor 120 executes instructions stored on memory 115; Fig 1 and col 9 ln 19-21) is configured to generate the plurality of virtual images based on the real image (transforming real images to virtual images using the GAN first transformer 130; Fig 1, 3 and col 9 ln 29-33).
Kim et al does not teach to generate the plurality of virtual images based on the real image by changing at least one of a distance, a pitch, or a yaw, with respect to the real image.
Kobayashi et al is analogous art pertinent to the technological problem addressed in this application and teaches to generate the plurality of virtual images based on the real image by changing at least one of a distance, a pitch, or a yaw, with respect to the real image (the virtual image generation unit 105 generates a virtual image based on a real image that is rotated with a given pitch angle and yaw angle by the image rotation unit 106a, 106b; Fig 1 and ¶ [0096]-[0097]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kim et al with Kobayashi et al including to generate the plurality of virtual images based on the real image by changing at least one of a distance, a pitch, or a yaw, with respect to the real image. By generating virtual images with variations of the position and orientation of a real image, the training data may provide improved virtual image system training thereby improving the inference state of viewing data in augmented or virtual reality, as recognized by Kobayashi et al (¶ [0008]).
Regarding Claim 8, Kim et al teach a method of claim 7 (as described above), wherein further limitations are taught identical to claim 2 (as described above).
Claims 3, 9 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al (US 10373023) in view of Chao et al (US 2022/0156580).
Regarding Claim 3, Kim et al the apparatus of claim 1 (as described above), including the processor (processor 120 executes instructions stored on memory 115; Fig 1 and col 9 ln 19-21).
Kim et al does not teach to label each image as i) a real image or a virtual image and ii) normal data or abnormal data.
Chao et al is analogous art pertinent to the technological problem addressed in this application and teaches to label each image as i) a real image or a virtual image and ii) normal data or abnormal data (image data is generated from a real image by a generator 20 with the synthetic (virtual) images being either normal or abnormal which the discriminator 30 is used to evaluate and identify (equivalent to label) the data as real or synthetic (virtual), normal or abnormal; Fig 1 and ¶ [0019]-[0021], [0041]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kim et al with Chao et al including to label each image as i) a real image or a virtual image and ii) normal data or abnormal data.
Regarding Claim 9, Kim et al teach a method of claim 7 (as described above), wherein further limitations are taught identical to claim 3 (as described above).
Claims 4, 10 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al (US 10373023) in view of Hu et al (DASGIL: Domain Adaptation for Semantic and Geometric-Aware Image-Based Localization, cited in Non-Final Rejection 11/12/2025).
Regarding Claim 4, Kim et al the apparatus of claim 1 (as described above), including the processor (processor 120 executes instructions stored on memory 115; Fig 1 and col 9 ln 19-21).
Kim et al does not teach to: compute a level of similarity between the real image and each of the virtual images among the plurality of virtual images generated based on the real image, and pair the real image with a virtual image having a highest level of similarity to the real image, thereby generating the golden pair that pairs the real image with the virtual image.
Hu et al is analogous art pertinent to the technological problem addressed in this application and teaches to: compute a level of similarity between the real image and each of the virtual images among the plurality of virtual images generated based on the real image (for the multi-scale feature extracted from the query image, a similarity or least distance analysis is performed against a database set of images to determine a most similar match; Fig 6 and IV.C. Image Retrieval for Localization), and pair the real image with a virtual image having a highest level of similarity to the real image, thereby generating the golden pair that pairs the real image with the virtual image (a retrieval result pairs a query image with the database image that represents the maximum similarity to the query image; Fig 6 and IV.C. Image Retrieval for Localization).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kim et al with Hu et al including to: compute a level of similarity between the real image and each of the virtual images among the plurality of virtual images generated based on the real image, and pair the real image with a virtual image having a highest level of similarity to the real image, thereby generating the golden pair that pairs the real image with the virtual image. By performing a similarity analysis between the images, the image-based localization is efficient in matching images and allows for higher-level tasks such as monocular depth prediction and semantic segmentation, and by improving domain adaptation techniques high-quality ground truth data is used to generate synthetic images for training algorithms to obtain depth and sematic segmentation maps for real-world images that are highly accurate and obtained with computing efficiency, as recognized by Hu et al (I. Introduction).
Regarding Claim 10, Kim et al teach a method of claim 7 (as described above), wherein further limitations are taught identical to claim 4 (as described above).
Claims 6, 12 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al (US 10373023) in view of Zhang et al (A Curriculum Domain Adaptation Approach to the Semantic Segmentation of Urban Scenes) and Hu et al (DASGIL: Domain Adaptation for Semantic and Geometric-Aware Image-Based Localization, cited in Non-Final Rejection 11/12/2025).
Regarding Claim 6, Kim et al the apparatus of claim 5 (as described above), including the processor (processor 120 executes instructions stored on memory 115; Fig 1 and col 9 ln 19-21), and
cause, using a first classifier, the triplet loss-performed images to be classified into normal images and abnormal images (a first discriminator 140 determines whether the input image is a primary virtual image or a secondary virtual image based on whether the image characteristics are transformed (abnormal) or untransformed (normal) from the original real-image; Fig 1, 3 and col 10 ln 8-17), and
cause, using a second classifier, the triplet loss-performed images be indistinguishable between the virtual image and the real image (a second discriminator 160 determines whether the input image fed thereinto is one of primary real (without transformation from a virtual image) or secondary real (with transformation from a virtual image); Fig 1, 3 and col 10 ln 17-25).
Kim et al does not teach to: convert the real image and the plurality of virtual images into vectors and to perform triplet loss on the golden pair of the real image and the virtual image that is paired with the real image, among the vector-converted images.
Zhang et al is analogous art pertinent to the technological problem addressed in this application and teaches to convert the real image and the plurality of virtual images into vectors (the real-world and virtual images are ended using one-hot vector encoding; Fig 1 and 3.1 Approach Preliminaries ¶ 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kim et al with Zhang et al including to convert the real image and the plurality of virtual images into vectors. By converting the real image and virtual image to vector representation the model analyze the necessary properties to determine differences between two domains, thereby improving computing efficiency while maintaining accuracy, as recognized by Zhang et al (3. Approach).
Hu et al is analogous art pertinent to the technological problem addressed in this application and teaches to perform triplet loss on the golden pair of the real image and the virtual image that is paired with the real image, among the vector-converted images (a triplet loss is performed on the positive pairs between the real-world image and virtual images with same place and the negative pairs between the real-world image and virtual images with different places; Fig 1, 5 and IV.B.1. Multi-Scale Fusion Triplet Loss).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Kim et al with Hu et al including to perform triplet loss on the golden pair of the real image and the virtual image that is paired with the real image, among the vector- converted images. By using a triplet loss learning approach to determine differences between the synthetic and real-world images, a multi-scale approach for feature discrimination is analyzed thereby improving the training data generation based on domain adaptation data, as recognized by Hu et al (I. Introduction ¶ 3-4).
Regarding Claim 12, Kim et al teach a method of claim 11 (as described above), wherein further limitations are taught identical to claim 6 (as described above).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Guizilini et al (US 2022/0156971, cited in Non-Final Rejection 11/12/2025) teach the generation of virtual images from a real-world image used for machine learning of depth mapping and segmentation with the virtual images generated with modification to the vector representation of object positioning.
Bousmalis et al (Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks, cited in Non-Final Rejection 11/12/2025) teach the use of a GAN to produce virtual images with variations to the backgrounds while maintaining the foreground.
Sibille et al (US 2023/0360366) teach a classification framework for labeling synthetic image as fake normal or real normal based on real images.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHLEEN M BROUGHTON whose telephone number is (571)270-7380. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN M BROUGHTON/Primary Examiner, Art Unit 2661