Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
This application claims the priority benefit under 35 U.S.C. § 119(e) of U.S. Provisional Application No. 63/606,851, filed on December 6, 2023.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/09/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claim 20 is objected to because of the following informalities: Considering that claim 20 is a non-transitory computer-readable medium it should be dependent upon claim 19 and not claim 1 as it is mentioned in the claims. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 10, 11,19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (US 20230026811 A1) referred to as Zhou hereinafter and further in view of Li et al. (US 20220004803 A1) referred to as Li hereinafter.
Regarding claim 1, Zhou teaches A method of performing image dehazing, the method comprising: (“a method for removing haze from remote sensing images” Zhou, para. [0005])
obtaining an input image; (inquiry module 107, fig. 1) estimating a transmission map by providing the input image to a dehazing transformer model, wherein the dehazing transformer model is trained by performing a training process on a network comprising the dehazing transformer model; (“The process includes generating one or more hazy input images with at least four spectral channels and one or more target images with the at least four spectral channels. The one or more hazy input images correspond to the one or more target images, respectively. The process further includes training a dehazing deep learning model using the one or more hazy input images and the one or more target images. The process additionally includes providing the dehazing deep learning model for haze removal processing.” Zhou, para. [0007])
and generating an output image based on the transmission map, wherein an amount of haze included in the output image is less than an amount of haze included in the input image. (“a hazy image disclosed herein can be an image with a degree of haze being greater than a predetermined hazy threshold. For example, a hazy image can be an image with an average dark-channel value being equal to or greater than a first dark-channel threshold. A haze-free image disclosed herein can be an image with a degree of haze being less than a predetermined haze-free threshold.” Zhou, para. [0035])
However, Zhou fails to teach a cyclic generative adversarial network (GAN)
Li teaches cyclic generative adversarial network (GAN) (“There is shown a network model training environment 1102 comprising hardware and software to define and configure, such as through conditioning, a GANs-based student model 1104 having a student generator 1106 (generator G.sub.S).” Li, para. [0082]) and (“In a CycleGANS framework, for example, a second student model and a corresponding pre-trained teacher model are also trained simultaneously (though not shown) to translate an image from the second domain space to the first domain space (e.g. from horses to zebras).” Li, para. [0037])
Zhou and Li are combinable because they are from the same filed of endeavor, image processing.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou in light of Li’s GAN. One would have been motivated to do so because along with the success of GANs in modeling high dimensional data, image-to-image translation tasks are dominated by GANs nowadays due to GANs' superiority in generating images of high fidelity and extendibility on different data domains. (Li, para. [0030])
Regarding claim 2, Zhou teaches wherein the training process is performed using a plurality of unpaired samples and a plurality of paired samples, wherein the plurality of paired samples comprises a plurality of real hazy images paired with a plurality of real clean images, (“by performing similar operations to the plurality of matched Sentinel-2 image pairs, training data generator 105 may generate 50,000 matched patch pairs. Training data generator 105 may filter the 50,000 matched patch pairs to generate training dataset 207 with 50,000 hazy input images and 50,000 corresponding target images.” Zhou, para. [0064])
Zhou fails to teach and wherein a training loss for the training process is determined based on at least one from among a cyclic loss, a paired loss, a GAN loss, a density loss, and a depth loss.
Li teaches and wherein a training loss for the training process is determined based on at least one from among a cyclic loss, a paired loss, a GAN loss, a density loss, and a depth loss. (“in CycleGAN [31], the original loss is weighted among two GAN losses and one cycle consistency loss.” Li, para. [0042])
Zhou and Li are combinable because they are from the same filed of endeavor, image processing.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou in light of Li’s GAN loss. One would have been motivated to do so because along with the success of GANs in modeling high dimensional data, image-to-image translation tasks are dominated by GANs nowadays due to GANs' superiority in generating images of high fidelity and extendibility on different data domains. (Li, para. [0030])
Regarding claim 10, refer to the explanation of claim 1.
Regarding claim 11, refer to the explanation of claim 2.
Regarding claim 19, Zhou teaches A non-transitory computer-readable medium storing instructions which, when executed by at least one processor of a device (“The non-transitory computer-readable storage medium is configured to store instructions which, in response to an execution by a processor, cause the processor to perform a process”
Regarding rest of claim 19, refer to the explanation of claim 1.
Regarding claim 20, refer to the explanation of claim 2.
Claim(s) 3-6, 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Zhou and Li as mentioned above and further in view of Kopf et al. (US 20100066732 A1) referred to as Kopf hereinafter.
Regarding claim 3, Zhou fails to teach wherein the cyclic GAN comprises a transformer-based dehazing network and a transformer-based rehazing network, and wherein the dehazing transformer model is included in the dehazing network.
Li teaches cyclic GAN (“There is shown a network model training environment 1102 comprising hardware and software to define and configure, such as through conditioning, a GANs-based student model 1104 having a student generator 1106 (generator G.sub.S).” Li, para. [0082]) and (“In a CycleGANS framework, for example, a second student model and a corresponding pre-trained teacher model are also trained simultaneously (though not shown) to translate an image from the second domain space to the first domain space (e.g. from horses to zebras).” Li, para. [0037])
However, the combination of Zhou and Li fails to teach wherein the GAN comprises a transformer-based dehazing network and a transformer-based rehazing network, and wherein the dehazing transformer model is included in the dehazing network.
Kopf teaches a transformer-based dehazing network and a transformer-based rehazing network, and wherein the dehazing transformer model is included in the dehazing network. (“With changes to the view frustum, the depth of objects in the modified image may differ from those of the original image or the objects may be new to the modified image. Because haze affects images based at least partially on the distance between the camera and the objects of the image, haze may be added and/or adjusted when producing the modified image from the image with a novel image view synthesis. Dehazing and re-hazing may be performed on-the-fly. For example, haze may first be removed from the image. After the novel image view has been synthesized, haze may be added back into the image based on the depth of objects.” Kopf, para. [0065])
Zhou, Li, and Kopf are combinable because they are from the same filed of endeavor, image processing.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou and Li in light of Kopf’s rehazing network. One would have been motivated to do so because it can leverage the availability of increasingly accurate 3D reference models to adjust an image. (Kopf, para. [0019])
Regarding claim 4, Zhou teaches wherein the training process comprises: obtaining a real hazy image from among the plurality of real hazy images; providing the real hazy image to the dehazing network to obtain a first synthetic clean image; (“Only image pairs that satisfy a dark-channel value condition can be used to generate hazy input images and target images in the training dataset. The training dataset may include numerous hazy input images and numerous corresponding target images for training the dehazing deep learning model disclosed herein.” Zhou, para. [0030])
Kopf teaches and providing the first synthetic clean image to the rehazing network to obtain a first synthetic hazy image. (“Because haze affects images based at least partially on the distance between the camera and the objects of the image, haze may be added and/or adjusted when producing the modified image from the image with a novel image view synthesis. Dehazing and re-hazing may be performed on-the-fly.” Kopf, para. [0065])
Zhou, Li, and Kopf are combinable because they are from the same filed of endeavor, image processing.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou and Li in light of Kopf’s rehazing network. One would have been motivated to do so because it can leverage the availability of increasingly accurate 3D reference models to adjust an image. (Kopf, para. [0019])
Regarding claim 5, Zhou fails to teach wherein the cyclic loss is determined based on a difference between the first synthetic hazy image and the real hazy image, wherein the paired loss is determined based on a difference between the first synthetic clean image and a real clean image corresponding to the real hazy image from among the plurality of real clean images, and wherein the GAN loss is determined based on the first synthetic hazy image and the real hazy image.
Li teaches wherein the cyclic loss is determined based on a difference between the first synthetic hazy image and the real hazy image, wherein the paired loss is determined based on a difference between the first synthetic clean image and a real clean image corresponding to the real hazy image from among the plurality of real clean images, and wherein the GAN loss is determined based on the first synthetic hazy image and the real hazy image. (“Semantic Relation Activation Matrix. FIG. 2 shows Semantic Relation Activation Matrix determination block 215 comprising operations as described herein. Tung & Mori [29] demonstrated interestingly distinct activation patterns among image instances of different classes versus image instances of the same class. However, on the image-to-image translation tasks, less information is contained in instances' correlation as they are typically from the same class (e.g. horses, oranges). Our hypothesis is that similarity and dissimilarity might likewise present in the feature encoding of different semantic pixels, which is also more informative on the image-to-image translation tasks. A distillation loss can be introduced to penalize the difference between a teacher and a student's encoded similarity.” Li, para. [0044]) and fig. 1.
Regarding claim 6, the combination of Zhou and Li fails to teach wherein the training process further comprises: providing the real clean image to the rehazing network to obtain a second synthetic hazy image; and providing the second synthetic hazy image to the dehazing network to obtain a second synthetic clean image
Kopf teaches wherein the training process further comprises: providing the real clean image to the rehazing network to obtain a second synthetic hazy image; and providing the second synthetic hazy image to the dehazing network to obtain a second synthetic clean image. (“visual data 404 from image 102 may be applied to reference model 104 to produce an image-augmented model. By way of example, augmented portions 406 are identified in block diagram 400A. As shown, augmented portions 406 extend the image leftward and rightward into regions (delineated by the short dashed lines) of reference model 104 that are not visible in image 102. The extended image 102 is output as modified image 108.” Kopf, para. [0042])
Zhou, Li, and Kopf are combinable because they are from the same filed of endeavor, image processing.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhou and Li in light of Kopf’s rehazing network. One would have been motivated to do so because it can leverage the availability of increasingly accurate 3D reference models to adjust an image. (Kopf, para. [0019])
Regarding claim 12, refer to the explanation of claim 3.
Regarding claim 13, refer to the explanation of claim 4.
Regarding claim 14, refer to the explanation of claim 5.
Regarding claim 15, refer to the explanation of claim 6.
Allowable Subject Matter
Claims 7-9, 16-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PARDIS SOHRABY whose telephone number is (571)270-0809. The examiner can normally be reached Monday - Friday 9 am till 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PARDIS SOHRABY/ Examiner, Art Unit 2664
/JENNIFER MEHMOOD/ Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights