DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-6, 9-12, 14-17, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gudavalli (US 20230419655 A1) and in view of Abady (NPL; 06-24-2026-IDS; Manipulation and generation of synthetic satellite images using deep learning models).
Regarding to claim 1, Gudavalli discloses a computer-implemented method ([0032]: produce satellite images that are indistinguishable from the original satellite images; [0036]: the image blending system 210 blends the GAN-generated satellite image 208 with the original, pristine satellite image; Fig. 3; [0037]: a process 300 for generating a manipulated image based on a corresponding semantic segmentation image; [0044]: the processor generates a blended satellite image; Fig. 6; [0049]: the machine 600 includes processors 604, memory 606, and input/output I/O components 602; the processors; Fig. 6; [0050]: the memory 606 includes a main memory 614, a static memory 616, and a storage unit 618, both accessible to the processors 604 via the bus 640), comprising:
obtaining a geological sketch of a first region (Fig. 2; [0032]: receive a representative pair of images, i.e. geological sketch, includes a semantic map image of a geographical area and the corresponding satellite image of a same geographical area; [0033]: pairs of 512 pixels x 512 pixels map and satellite images of capital cities, i.e. geological sketch; [0034]: semantic maps of urban areas; Fig. 2; [0035]: the GAN model 204 receives a manipulated semantic map image, i.e. geological sketch, as input; an image is manipulated to replace a lake with a set of buildings;
PNG
media_image1.png
568
752
media_image1.png
Greyscale
; Fig. 3; [0038]: the processor accesses and receive map image data; [0039]: the processor accesses satellite image data associated with the map image data; [0043-0044]: the manipulated map image data is a geological sketch),
applying a first machine learning model to convert the geological sketch into a first virtual satellite image of the first region ([0034]: use manipulated building segmentation maps to generate deepfake satellite images, i.e. virtual satellite image; Fig. 2; [0035]: the GAN model 204 generates a GAN-generated satellite image 208, i.e. virtual satellite image; [0043]: the processor generates manipulated satellite image data based on the manipulated map image data using the trained machine learning framework; the output of the GAN model 204 is manipulated satellite image data; [0044]: the processor generates a blended satellite image, the blended satellite image generated based on a combination of the manipulated satellite image data and the accessed satellite image data); and
providing the first virtual satellite image for geological interpretation of the first region (Fig. 2; [0036]: the final blended image generated by the image generation system 118 is a deepfake satellite image 212; [0044]: the processor generates a blended satellite image; the blended satellite image is generated based on a combination of the manipulated satellite image data and the accessed satellite image data; [0045]: the processor stores the blended satellite image; Fig. 6; [0046]: the deepfake satellite image is blended with the pristine satellite image 408 to generate the blended satellite image 406;
PNG
media_image2.png
356
618
media_image2.png
Greyscale
; [0051]: the user output components 626 includes visual components, e.g., a display).
Gudavalli fails to explicitly disclose:
wherein the geological sketch comprises a color-labeled image of the first region, each color in the geological sketch represents a respective type of geological land cover in the first region;
In same field of endeavor, Abady teaches wherein the geological sketch comprises a color-labeled image of the first region, each color in the geological sketch represents a respective type of geological land cover in the first region (page 3; 2 Relevant Work: change the land-cover of a satellite image; page 4; Fig. 1; 3.3 Land Cover Datasets: create the land cover dataset by selecting images with different colors;
PNG
media_image3.png
280
144
media_image3.png
Greyscale
; Land cover images are labeled with different colors are illustrated in Fig. 1; page 5; Fig. 2; 3.5 World Dataset: images from various regions in the world contain are labeled different colors as illustrated in Fig. 2;
PNG
media_image4.png
198
624
media_image4.png
Greyscale
).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gudavalli to include wherein the geological sketch comprises a color-labeled image of the first region, each color in the geological sketch represents a respective type of geological land cover in the first region as taught by Abady. The motivation for doing so would have been to generate highly realistic, synthetic, and satellite images; to create the land cover dataset by selecting images with different colors as taught by Abady in Abstract, 3.3 Land Cover Datasets, and Conclusion.
Regarding to claim 2, Gudavalli in view of Abady discloses the computer-implemented method of claim 1, further comprising:
obtaining a first plurality of satellite images of one or more regions (Gudavalli; Fig. 2; [0032]: the image generation system 118 uses a training dataset 202 to train a GAN model 204; the training dataset 202 includes a set of semantic map image and satellite image pairs; a representative pair of images includes a semantic map image of a geographical area and the corresponding satellite image of a same geographical area; the corresponding satellite regions are highly variable depending on the geographical region in which the image pairs are taken; Fig. 2; [0033]: the training dataset includes pairs of 512 pixels x 512 pixels map and satellite images of capital cities; obtain multiple images for each city); and
generating the first machine learning model by using the first plurality of satellite images to train the first machine learning model (Gudavalli; Fig. 2; [0032]: the image generation system 118 uses a training dataset 202 to train a GAN model 204; [0041]: the processor trains a machine learning framework using a set of map and satellite image pairs).
Regarding to claim 4, Gudavalli in view of Abady discloses the computer-implemented method of claim 2, wherein generating the first machine learning model by using the first plurality of satellite images to train the first machine learning model (same as rejected in claim 2) comprises:
generating a plurality of satellite label images by labeling the first plurality of satellite images using colors respectively associated with geological land covers in the first plurality of satellite images (Abady; page 3; 2 Relevant Work: change the land-cover of a satellite image; page 4; Fig. 1; 3.3 Land Cover Datasets: create the land cover dataset by selecting images with different colors;
PNG
media_image3.png
280
144
media_image3.png
Greyscale
; Land cover images with different colors, i.e. labels, are illustrated in Fig. 1; page 5; Fig. 2; 3.5 World Dataset: images from various regions in the world contain different colors as illustrated in Fig. 2;
PNG
media_image4.png
198
624
media_image4.png
Greyscale
); and
generating the first machine learning model by using the first plurality of satellite images and the plurality of satellite label images to train the first machine learning model (Gudavalli; Fig. 2; [0032]: the image generation system 118 uses a training dataset 202 to train a GAN model 204; the training dataset 202 includes a set of semantic map image and satellite image pairs; a representative pair of images includes a semantic map image of a geographical area and the corresponding satellite image of a same geographical area).
Regarding to claim 5, Gudavalli in view of Abady discloses the computer-implemented method of claim 2, wherein generating the first machine learning model by using the first plurality of satellite images to train the first machine learning model (same as rejected in claim 2) comprises:
generating a second plurality of satellite images of the one or more regions based on the first plurality of satellite images (Abady; page 1; 1 Introduction: receive satellite images; page 4; 3.1 Alps Dataset: resize images into several 512 × 512 images; the image size is 512x512; page 4; 3.2 Scandinavian Dataset: the final dataset consists of 9000 paired images of size 512 × 512; page 4; Fig. 1; 3.3 Land Cover datasets:
PNG
media_image5.png
152
142
media_image5.png
Greyscale
; Fig. 1 illustrated square images; page 5; 3.5 World Dataset: image size equal to 10,980 × 10,980; extract nonoverlapping patches of size 512 × 512; page 12; Fig. 7; 4.4.1 Architecture: the original image size is 128 × 128); and
generating the first machine learning model by using the first plurality of satellite images and the second plurality of satellite images to train the first machine learning model (Gudavalli; Fig. 2; [0032]: the image generation system 118 uses a training dataset 202 to train a GAN model 204; [0035]: the GAN model 204 generates a GAN-generated satellite image 208; [0036]: the image generation system blends the GAN-generated satellite image 208 with a different satellite image from the training dataset 202; the blended satellite image can be added into the training dataset 202 to train GAN; [0040]: generate second images by removing, replacing, and inserting an object depicted in the accessed map image data; [0041]: the processor trains a machine learning framework using a set of map and satellite image pairs; [0043]: the processor generates manipulated satellite image data based on the manipulated map image data using the trained machine learning framework; [0044]: the blended satellite image generated based on a combination of the manipulated satellite image data and the accessed satellite image data).
Gudavalli in view of Abady discloses generating the first machine learning model by using the second plurality of satellite images to train the first machine learning model (Abady; page 6; 4.1 Season Transfer: a model is trained to transfer images in one direction only; page 7; 4.1.2 Training: 6000 images were used for training, 2000 for testing, and 1000 for validation; To create the season transferred images, the pix2pix model was, separately, trained on the Alps and Scandinavian datasets; page 8; 4.2.1. Architecture: training is obtained by finding a good trade-off between three partial losses.).
Same motivation of claim 1 is applied here.
Regarding to claim 6, Gudavalli in view of Abady discloses the computer-implemented method of claim 5, wherein generating the second plurality of satellite images of the one or more regions based on the first plurality of satellite images (same as rejected in claim 5) comprises:
generating the second plurality of satellite images of the one or more regions based on the plurality of square images (Gudavalli; Fig. 4; [0046]: the deepfake satellite image is blended with the pristine satellite image 408 to generate the blended satellite image 406;
PNG
media_image6.png
348
610
media_image6.png
Greyscale
).
Gudavalli in view of Abady further discloses:
resizing the first plurality of satellite images into a plurality of square images (Abady; page 1; 1 Introduction: receive satellite images; page 4; 3.1 Alps Dataset: resize images into several 512 × 512 images; the image size is 512x512; page 4; 3.2 Scandinavian Dataset: the final dataset consists of 9000 paired images of size 512 × 512; page 4; Fig. 1; 3.3 Land Cover datasets:
PNG
media_image5.png
152
142
media_image5.png
Greyscale
; Fig. 1 illustrated square images; page 5; 3.5 World Dataset: image size equal to 10,980 × 10,980; extract nonoverlapping patches of size 512 × 512; page 12; Fig. 7; 4.4.1 Architecture: the original image size is 128 × 128); and
generating the second plurality of satellite images of the one or more regions based on the plurality of square images (Abady; page 1; 1 Introduction: receive satellite images; page 4; 3.1 Alps Dataset: resize original images into several 512 × 512 images; the generated image size is 512x512 and a square image; page 4; 3.2 Scandinavian Dataset: the final dataset consists of 9000 paired images of size 512 × 512; page 4; Fig. 1; 3.3 Land Cover datasets:
PNG
media_image5.png
152
142
media_image5.png
Greyscale
; plurality of square images as illustrated in Fig. 1; page 5; 3.5 World Dataset: image size equal to 10,980 × 10,980; extract nonoverlapping patches of size 512 × 512; page 6; 4.1 Season Transfer: the cGAN takes one image as input and translates it into an image belonging to the target domain; page 12; Fig. 7; 4.4.1 Architecture: the original image size is 128 × 128; page 9; Fig. 4; 4.2.2. Training: the cycleGAN model was trained for 200 epochs with an input size of 512 × 512 × 4;
PNG
media_image7.png
160
112
media_image7.png
Greyscale
).
Regarding to claim 9, Gudavalli in view of Abady discloses the computer-implemented method of claim 1, wherein providing the first virtual satellite image for the geological interpretation of the first region (same as rejected in claim 1) comprises:
providing, through a graphical user interface (GUI), the first virtual satellite image for the geological interpretation of the first region (Gudavalli; Fig. 4; [0046]: the deepfake satellite image is blended with the pristine satellite image 408 to generate the blended satellite image 406;
PNG
media_image6.png
348
610
media_image6.png
Greyscale
; [0051]: the user output components 626 includes visual components, e.g., a display).
Regarding to claim 10, Gudavalli in view of Abady discloses the computer-implemented method of claim 1, wherein the first machine learning model is a conditional generative adversarial network (CGAN) (Abady; page 6; 4.1 Season Transfer: a conditional GAN (cGAN); the cGAN takes one image as input and translates it into an image belonging to the target domain).
Same motivation of claim 1 is applied here.
Regarding to claim 11, Gudavalli discloses a non-transitory computer-readable medium storing one or more instructions executable by a computer system to perform operations ([0032]: produce satellite images that are indistinguishable from the original satellite images; [0036]: the image blending system 210 blends the GAN-generated satellite image 208 with the original, pristine satellite image; Fig. 3; [0037]: a process 300 for generating a manipulated image based on a corresponding semantic segmentation image; [0044]: the processor generates a blended satellite image; Fig. 6; [0049]: the machine 600 includes processors 604, memory 606, and input/output I/O components 602; the processors; Fig. 6; [0050]: the memory 606 includes a main memory 614, a static memory 616, and a storage unit 618, both accessible to the processors 604 via the bus 640) comprising:
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 11.
Regarding to claim 12, Gudavalli in view of Abady discloses the non-transitory computer-readable medium of claim 11, wherein the operations further comprise:
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 12.
Regarding to claim 14, Gudavalli in view of Abady discloses the non-transitory computer-readable medium of claim 12,
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 14.
Regarding to claim 15, Gudavalli in view of Abady discloses the non-transitory computer-readable medium of claim 12,
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 15.
Regarding to claim 16, Gudavalli discloses a computer-implemented system (Fig. 1; [0022]: image analysis system; Fig. 1; [0023-0025]: a client application 104 and the server system 106) comprising:
one or more computers (Fig. 1; [0023-0025]: a client application 104 and the server system 106); and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, cause the computer-implemented system to perform one or more operations ([0032]: produce satellite images that are indistinguishable from the original satellite images; [0036]: the image blending system 210 blends the GAN-generated satellite image 208 with the original, pristine satellite image; Fig. 3; [0037]: a process 300 for generating a manipulated image based on a corresponding semantic segmentation image; [0044]: the processor generates a blended satellite image; Fig. 6; [0049]: the machine 600 includes processors 604, memory 606, and input/output I/O components 602; the processors; Fig. 6; [0050]: the memory 606 includes a main memory 614, a static memory 616, and a storage unit 618, both accessible to the processors 604 via the bus 640) comprising:
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 16.
Regarding to claim 17, Gudavalli in view of Abady discloses the computer-implemented system of claim 16, wherein the one or more operations further comprise:
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 17.
Regarding to claim 19, Gudavalli in view of Abady discloses the computer-implemented system of claim 17,
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 19.
Regarding to claim 20, Gudavalli in view of Abady discloses the computer-implemented system of claim 17,
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 20.
Claims 3, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Gudavalli (US 20230419655 A1) in view of Abady (NPL; 06-24-2026-IDS; Manipulation and generation of synthetic satellite images using deep learning models), and further in view of Bailey (US 20210165938 A1).
Regarding to claim 3, Gudavalli in view of Abady discloses the computer-implemented method of claim 2, wherein an environment of the first region is part of a plurality of environments of the one or more regions (Gudavalli; Fig. 2; [0032]: a representative pair of images includes a semantic map image of a geographical area and the corresponding satellite image of a same geographical area; the corresponding satellite regions are highly variable depending on the geographical region in which the image pairs are taken).
Gudavalli in view of Abady fails to explicitly disclose a depositional environment.
In same field of endeavor, Bailey teaches a depositional environment ([0106]: generate properties of the depositional environment such as sea level change rate and relative sea level; extract the depositional environment; [0115]: seismic based conditions are supplied based on attribute interpretation; the depositional environment includes continental shelf, or deep water; [0144]: receive 206 one or more training images including a distribution of patterns associated with a depositional environment).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gudavalli in view of Abady to include a depositional environment as taught by Bailey. The motivation for doing so would have been to improve the accuracy of predictions of the geology at new locations where wells have not yet been drilled; to generate properties of the depositional environment such as sea level change rate and relative sea level; to extract the depositional environment; to receive 206 one or more training images including a distribution of patterns associated with a depositional environment as taught by Bailey in paragraphs [0006], [0106] and [0144].
Regarding to claim 13, Gudavalli in view of Abady discloses the non-transitory computer-readable medium of claim 12,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 13.
Regarding to claim 18, Gudavalli in view of Abady discloses the computer-implemented system of claim 17,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 18.
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Gudavalli (US 20230419655 A1) in view of Abady (NPL; 06-24-2026-IDS; Manipulation and generation of synthetic satellite images using deep learning models), and further in view of Denli (US 20200183047 A1).
Regarding to claim 7, Gudavalli in view of Abady discloses the computer-implemented method of claim 1, further comprising:
obtaining an attribute map of the first region (Gudavalli: Fig. 2; [0032]: receive a representative pair of images, includes a semantic map image of a geographical area and the corresponding satellite image of a same geographical area; [0033]: pairs of 512 pixels x 512 pixels map and satellite images of capital cities; [0034]: semantic maps of urban areas; Fig. 2; [0035]: the GAN model 204 receives a manipulated semantic map image as input; an image is manipulated to replace a lake with a set of buildings;
PNG
media_image1.png
568
752
media_image1.png
Greyscale
; Fig. 3; [0038]: the processor accesses and receive map image data; [0039]: the processor accesses satellite image data associated with the map image data; [0043-0044]: the manipulated map image data is a geological sketch),
applying a second machine learning model to convert the attribute map into a second virtual satellite image of the first region (Gudavalli: [0034]: use manipulated building segmentation maps to generate deepfake satellite images; Fig. 2; [0035]: the GAN model 204 generates a GAN-generated satellite image 208, i.e. virtual satellite image; [0043]: the processor generates manipulated satellite image data based on the manipulated map image data using the trained machine learning framework; the output of the GAN model 204 is manipulated satellite image data; [0044]: the processor generates a blended satellite image, the blended satellite image generated based on a combination of the manipulated satellite image data and the accessed satellite image data); and
providing the second virtual satellite image for the geological interpretation of the first region (Gudavalli: Fig. 2; [0036]: the final blended image generated by the image generation system 118 is a deepfake satellite image 212; [0044]: the processor generates a blended satellite image, the blended satellite image is generated based on a combination of the manipulated satellite image data and the accessed satellite image data; [0045]: the processor stores the blended satellite image; Fig. 6; [0046]: the deepfake satellite image is blended with the pristine satellite image 408 to generate the blended satellite image 406;
PNG
media_image2.png
356
618
media_image2.png
Greyscale
).
Gudavalli in view of Abady fails to explicitly disclose:
seismic attribute map and wherein the seismic attribute map comprises a plurality of seismic attributes of the first region;
In same field of endeavor, Denli teaches:
seismic attribute map and wherein the seismic attribute map comprises a plurality of seismic attributes of the first region (Fig. 1; [0008]: construct subsurface images, such as seismic images 130, typically using the seismic reflection events and the inverted geophysical models to migrate the events from surface locations to their subsurface locations;
PNG
media_image8.png
412
360
media_image8.png
Greyscale
; [0076]: the applicable geological and geophysical data may comprise seismic data generated from exploration surveying and simulated seismic data generated by geological models of sites similar to the current site; Fig. 6A; [0081]: multiple seismic images; [0091]: synthetic seismic images are generated using seismic wave simulations; [0108]: data representations, e.g., seismic images, feature probability maps, feature objects, etc.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Gudavalli in view of Abady to include seismic attribute map and wherein the seismic attribute map comprises a plurality of seismic attributes of the first region as taught by Denli. The motivation for doing so would have been to construct subsurface images, such as seismic images 130, typically using the seismic reflection events and the inverted geophysical models to migrate the events from surface locations to their subsurface locations; to improve the understanding of the subsurface including acquiring additional data regarding the subsurface, which may be prohibitively expensive as taught by Denli in paragraphs [0008] and [0055].
Regarding to claim 8, Gudavalli in view of Abady and Denli discloses the computer-implemented method of claim 7, further comprising:
training the second machine learning model based on the third plurality of satellite images (Gudavalli; Fig. 2; [0032]: the image generation system 118 uses a training dataset 202 to train a GAN model 204; [0035]: the GAN model 204 generates a GAN-generated satellite image 208; [0036]: the image generation system blends the GAN-generated satellite image 208 with a different satellite image from the training dataset 202; the blended satellite image can be added into the training dataset 202 to train GAN; [0040]: generate second images by removing, replacing, and inserting an object depicted in the accessed map image data; [0041]: the processor trains a machine learning framework using a set of map and satellite image pairs; [0043]: the processor generates manipulated satellite image data based on the manipulated map image data using the trained machine learning framework; [0044]: the blended satellite image generated based on a combination of the manipulated satellite image data and the accessed satellite image data).
Gudavalli in view of Abady and Denli further discloses:
obtaining a third plurality of satellite images of one or more regions (Abady; page 1; 1 Introduction: receive satellite images; page 4; 3.1 Alps Dataset: resize images into several 512 × 512 images; page 4; Fig. 1; 3.3 Land Cover datasets:
PNG
media_image5.png
152
142
media_image5.png
Greyscale
; Fig. 1 illustrated square images); and
training the second machine learning model based on the third plurality of satellite images (Abady; page 6; 4.1 Season Transfer: a conditional GAN (cGAN); the cGAN takes one image as input and translates it into an image belonging to the target domain; a model is trained to transfer images in one direction only; page 7; 4.1.2 Training: 6000 images were used for training, 2000 for testing, and 1000 for validation; the pix2pix model was, separately, trained on the Alps and Scandinavian datasets.).
Same motivation of claim 1 is applied here.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights