DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 22 October 2024 is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7 and 9-16 are rejected under 35 U.S.C. 103 as being unpatentable over Meredith et al. (“Evaluating Rotation-Equivariant Deep Learning Models for On-Orbit Cloud Segmentation”) in view of Luo et al. (CN 117058059 A).
Regarding claim 1, Meredith et al. disclose an image processing system (Page 11, under the section Future Works & Applications, it is stated that a laptop was used as the image processing system.), comprising:
a memory configured to store computer-executable instructions (As mentioned above, a laptop was used for the described models, where laptops comprise memory that is configured to store instructions.); and
one or more processors configured to execute the instructions (As mentioned above, a laptop was used for the described models, where laptops comprise processors that execute the instructions.) to:
collect an aligned pair of an input optical image of a scene and another image of the scene (Page 1, under the section Introduction: “Second, we train it on data from four different combinations of bands: visible-spectrum (VIS), long-wave infrared (LWIR), and short-wave infrared (SWIR); VIS and LWIR; VIS and SWIR; and VIS only.” VIS is an input optical image of a scene, and LWIR is another image of the scene.);
generate a multimodal image from the aligned pair of the input optical image and the other image (Pages 2-3, under the section Dataset, Metrics, & Training and including Table 1, it is explained that the models are trained on images from the same dataset of the Landsat 8, and where Figure 8 on page 6 shows an input to the U-Net being a generated multimodal image from RGB [VIS] and LWIR.);
submit the multimodal image to a multimodal rotation-equivariant neural network to generate an estimate of an improved optical image of the scene, wherein the multimodal rotation-equivariant neural network is configured such that a rotation of an input image to the neural network causes a corresponding rotation of an output image of the multimodal rotation-equivariant neural network (Page 1, under the section Abstract: "Cloud detection in satellite imagery is key for autonomously taking and downlinking cloud-free images of a target region" and pages 5 and 6, under section C8-Equivariant U-Net in conjunction with Figures 7 and 8: "The C8-equivariant U-Net is very similar to the U-Net, but with nearly all operations replaced with C8-equivariant equivalents [...]. The input convolution step involves [...] a convolution that “lifts” the input from an image on R2 to a feature map on R2 ⋊ C8. The output convolution step involves “orientation pooling”, which projects a feature map on R2 ⋊ C8 back to R2 by taking the pixel-wise maximum or average across the orientation dimension, followed by the same activation block used in the U-Net"; where the "C8-equivariant U-Net" corresponds to said neural network; where Figure 7 and 8 illustrate that the multimodal "input image" is submitted to said neural network; and, where "cloud-free images" corresponds to said improved optical images); and
output the estimated improved optical image (Page 1, under the section Abstract: "Cloud detection in satellite imagery is key for autonomously taking and downlinking cloud-free images of a target region".).
Meredith et al. fail to specifically teach that the other image is a radar image.
Luo et al. disclose an image processing system, comprising:
a memory configured to store computer-executable instructions (See page 22 of the provided document, lines 28-33: “The invention claims a gradual restoration frame cloud removing device fused by optical remote sensing image and SAR image, comprising a processor and a memory; said memory is used for storing the computer program; the processor is connected with the memory for executing the computer program stored in the memory so that the gradual repairing frame cloud removing device fused by the optical remote sensing image and the SAR image executes the gradual repairing frame cloud removing method fused by the optical remote sensing image and the SAR image.”); and
one or more processors configured to execute the instructions (See page 22 of the provided document, lines 28-33: “The invention claims a gradual restoration frame cloud removing device fused by optical remote sensing image and SAR image, comprising a processor and a memory; said memory is used for storing the computer program; the processor is connected with the memory for executing the computer program stored in the memory so that the gradual repairing frame cloud removing device fused by the optical remote sensing image and the SAR image executes the gradual repairing frame cloud removing method fused by the optical remote sensing image and the SAR image.”) to:
collect an aligned pair of an input optical image of a scene and a radar image of the scene (See Figures 1 and 2, and page 20 of the provided document, the corresponding portions for Figures 1 and 2, which explain that an aligned pair of the optical image and the radar image are collected and used for the cloud removal method.).
Thus, Meredith et al. and Luo et al. each disclose an image processing system for cloud removal. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the radar images (SAR) of Luo et al. could have been substituted for the LWIR images of Meredith et al. because both provide high penetrability to the cloud layer. Furthermore, a person of ordinary skill in the art would have been able to carry out the substitution. Finally, the substitution achieves the predictable result of providing secondary images to the optical images for the purpose of cloud removal.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to substitute the radar images (SAR) of Luo et al. for the LWIR images of Meredith et al. according to known methods to yield the predictable result of providing an image processing system for cloud removal.
Regarding claim 2, Meredith et al. and Luo et al. disclose the system of claim 1, wherein the input optical image comprises a first proportion of pixels corresponding to clouds and the estimated improved optical image comprises a second proportion of pixels corresponding to clouds, and wherein the second proportion is less than the first proportion (Clearly, an improved optical image by means of an application of a cloud mask results in an image having a comparable less proportion of clouds than the input image, and thus the estimated improved optical image comprises a second proportion of pixels corresponding to clouds that is less than a first proportion of pixels corresponding to clouds in the input optical image.).
Regarding claim 3, Meredith et al. and Luo et al. disclose the system of claim 1, wherein the one or more processors are configured to generate the estimated improved optical image as a cloud-free optical image (Meredith et al.: Page 1, under the section Abstract: "Cloud detection in satellite imagery is key for autonomously taking and downlinking cloud-free images of a target region".).
Regarding claim 4, Meredith et al. and Luo et al. disclose the system of claim 1, wherein the multimodal rotation-equivariant neural network comprises:
a first layer configured to perform a lifting convolution that transforms a multimodal image to a feature map defined on a symmetry group (Meredith et al.: Pages 5 and 6, under section C8-Equivariant U-Net, and Figure 7(a): input convolution.);
a plurality of intermediate layers configured to perform group convolutions on a plurality of feature maps defined on the symmetry group (Meredith et al.: Pages 5 and 6, under section C8-Equivariant U-Net, and Figure 7(b): C8-equivariant group convolution.); and
a pooling layer configured to perform pooling along a rotation dimension of a penultimate feature map to yield the network output that is rotation-equivariant (Meredith et al.: Pages 5 and 6, under section C8-Equivariant U-Net, and Figure 7(c): orientation pooling.).
Regarding claim 5, Meredith et al. and Luo et al. disclose the system of claim 4, wherein the symmetry group comprises all compositions of translations and rotations by 90 degrees about any center of rotation in a square two-dimensional image grid (Meredith et al.: Page 2, under the section Deep Learning, last paragraph of the section: “…the group of rotations by integer multiples of 45°…” where 90° is an integer multiple of 45 [multiplied by 2].).
Regarding claim 6, Meredith et al. and Luo et al. disclose the system of claim 4, wherein each intermediate layer of the plurality of intermediate layers maps a unique feature map of the plurality of feature maps to other feature maps of the plurality of feature maps (Meredith et al.: Pages 5 and 6, under section C8-Equivariant U-Net, and Figure 7(b): C8-equivariant group convolution.).
Regarding claim 7, Meredith et al. and Luo et al. disclose the system of claim 4, wherein each intermediate layer of a plurality of second layers has multiple input and multiple output channels (Meredith et al.: Pages 5 and 6, under section C8-Equivariant U-Net, and Figure 7(b): C8-equivariant group convolution.).
Regarding claim 9, this claim is rejected under the same rationale as claim 1.
Regarding claim 10, this claim is rejected under the same rationale as claim 2.
Regarding claim 11, this claim is rejected under the same rationale as claim 3.
Regarding claim 12, this claim is rejected under the same rationale as claim 4.
Regarding claim 13, this claim is rejected under the same rationale as claim 5.
Regarding claim 14, this claim is rejected under the same rationale as claim 6.
Regarding claim 15, this claim is rejected under the same rationale as claim 7.
Regarding claim 16, this claim is rejected under the same rationale as claim 1, where the claimed non-transitory computer readable medium of claim 16 is the memory of claim 1.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Meredith et al. (“Evaluating Rotation-Equivariant Deep Learning Models for On-Orbit Cloud Segmentation”) in view of Luo et al. (CN 117058059 A) and further in view of Lee et al. (US 2021/0012541).
Regarding claim 8, Meredith et al. and Luo et al. disclose the system of claim 4.
Meredith et al. and Luo et al. fail to explicitly teach wherein the multimodal rotation-equivariant neural network is trained to minimize a mean absolute error loss computed between the network output and a cloudless optical image based on ground truth data in a training dataset.
Lee et al. disclose wherein a neural network is trained to minimize a mean absolute error loss computed between a network output and data based on ground truth data in a training dataset (Paragraph [0057]).
Therefore, it would have been obvious to “one of ordinary skill” in the art before the effective filing date of the claimed invention to use the mean absolute error loss minimization teachings of Lee et al. to the network output and the cloudless optical image taught by the combination of Meredith et al. and Luo et al. The motivation to combine would have been in order to make use of the known benefit of a loss function, which is to minimize loss and improve the accuracy of the output of the neural network.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEPHEN G SHERMAN whose telephone number is (571)272-2941. The examiner can normally be reached Monday - Friday, 8:00am - 4pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, AMR AWAD can be reached at (571)272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEPHEN G SHERMAN/Primary Examiner, Art Unit 2621
14 July 2026
Prosecution Insights