Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/4/2026 has been entered.
Response to Amendment
The amendment filed on 6/4/2026 has been entered and made of record. Claims 1, 16 and 19 are amended. Claim 20 is cancelled. Claim 21 is new. Claims 1-19 and 21 are pending.
Response to Arguments
Applicant’s arguments with respect to the rejections of independent claim 1 have been fully considered but they are moot because the arguments do not apply to the references being used in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 12-13 and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2023/0304826 A1) in view of Shu et al. (US 2024/0233086 A1) and Wang et al. (Adaptive Patch Deformation for Textureless-Resilient Multi-View Stereo, 2023 CVPR), further in view of Li et al. (US 2022/0099855 A1).
As to Claim 1, Zhang teaches A computer-implemented method for landcover map generation, comprising:
receiving, by a communication network and from a digital camera situated at an aerial position above an environment, an input image comprising pixels and representing the environment; obtaining coordinate information of the environment corresponding to the pixels of the input image (Zhang discloses “In 405, acquiring road usage information specifying which parts of a geographical area have been used for driving a vehicle; In 406, acquiring a satellite image of the geographical area… In 407, forming a road usage image of the geographical area which has pixels, each pixel corresponding to a respective part of the geographical area, such each pixel has a pixel value indicating whether the part of the geographical area, to which the pixel corresponds…” in [0076-0078], see also communication interface 501 in [0087]);
determining, based on the coordinate information of the pixels of the input image, one or more coordinate measures for the input image, wherein the one or more coordinate measures provide a measure of spatial characteristics for the pixels of the input image (Zhang discloses “The GPS image 204 has the same number of pixels as its paired satellite image 202. The covered space for each pixel in GPS image is the same as the corresponding pixel in the satellite image” in [0055]; mean absolute pixel difference loss in [0067]; “each pixel corresponding to a respective part of the geographical area, such each pixel has a pixel value indicating whether the part of the geographical area, to which the pixel corresponds, is specified by the road usage information to have been used for driving a vehicle” in [0078]).
Zhang doesn’t explicitly teach input image patches. The combination of Shu further teaches following limitations:
generating, from the input image and based on the one or more coordinate measures, a plurality of input image patches, each input image patch from the plurality of input image patches representing a portion of pixels in the input image (Shu discloses “At operation 204, the image tile generation unit 220 subdivides the geospatial imagery into a grid of image tiles. Such a grid of image tiles may be a regular grid of equally sized cells of any appropriate dimensions… Such a grid may be established with reference to the geospatial information contained in metadata of the geospatial imagery (e.g., geospatial coordinates, image scale, resolution, etc.)” in [0030]; see also Fig 3);
applying, for the plurality of input image patches, one or more image processing techniques to adjust pixel values of a respective input image patch from the plurality of input image patches to generate a plurality of adjusted image patches (Zhang discloses “The satellite image 304 and the GPS image 303 are concatenated in a channel-wise fashion by image concatenation 305. Here, "channel-wise" means that a GPS image channel is added as a fourth channel to the RGB channels” in [0059]. Shu teaches input image tiles in Fig 3);
providing the plurality of adjusted image patches to a generative adversarial network, wherein the generative adversarial network is trained to generate a corresponding landcover map patch for each adjusted image patch in the plurality of adjusted image patches (Zhang discloses “The result of image concatenation 305 (i.e. the concatenated image data) is imported to a (trainable) U-Net 306” in [0059]; “The output of the generator is a map image 307 that contains RGB three colour channels” in [0060]; see also Fig 3 below.
PNG
media_image1.png
447
941
media_image1.png
Greyscale
Shu teaches input image tiles in Fig 3).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang with the teaching of Shu so as to subdivide a large area of geospatial imagery into smaller image tiles and process tasks on image tiles rather than the entire image for more manageable and better suited to parallel processing (Shu, [0028]).
Zhang and Shu don’t explicitly teach anchoring pixel. The combination of Wang further teaches following limitation:
wherein each input image patch comprises a respective anchoring pixel from the portion of the pixels (Wang, Fig 4 as shown below:
PNG
media_image2.png
664
707
media_image2.png
Greyscale
);
generating a landcover map from the plurality of landcover map patches, wherein generating the landcover map comprises assembling the plurality of landcover map patches according to the coordinate information of the environment by using the respective anchoring pixels from the plurality of input image patches; and providing the landcover map corresponding to the input image for output (Zhang discloses “According to various embodiments, in other words, map data is generated by jointly using a satellite image and road usage information (e.g. GPS traces of vehicles of an e-hailing service) by means of a generative adversarial network (GAN)” in [0080]; “It should be noted that more than one satellite images may be acquired which may be seen to form, together, a big satellite image (even if being stored in separate image parts)” in [0081]. Shu teaches merged vector map in Fig 10. Wang further discloses “For each unreliable pixel, we adaptively deform its corresponding patch to cover enough anchor pixels with high reliability (Sec. 3.3)” at p. 1623; see also Fig 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang and Shu with the teaching of Wang so that more anchor pixels can bring a better-fitting deformable patch and improve depth estimation accuracy (Wang, p. 1625).
Shu teaches land classification maps without detailed description on classification label. The combination of Li further teaches following limitations:
generating, by the generative adversarial network and using the plurality of adjusted image patches, a plurality of landcover map patches comprising classification labels indicative of locations in the environment likely to be a source of interference in seismic imaging of the environment based on visual features represented by the adjusted pixels in the plurality of adjusted image patches, the generative adversarial network comprises: a generator network trained to generate training landcover map patches from an input image patch and a discriminator network trained to classify training landcover map patches from the generator network, wherein a classification of a training landcover map patch comprises a label indicating that the training landcover map patch represents one or more landcover classes of a corresponding input image patch, the label in each classification indicating locations in the environment likely to be a source of interference in seismic imaging of an environment based on visual features represented by pixels in the corresponding input image patch (Zhang discloses “The GAN-based Image Translation 205 converts a pair of rendered GPS image 204 and satellite image 202 into a map image 206 using a GAN (or GAN-based model)” in [0056]; “The GAN 300 is a neural network which contains two parts (i.e. two sub-networks): a generator (network) 301 and a discriminator (network) 302” in [0058]; “The CNN 310 outputs a value between 0 and 1 where a larger value indicates that the map image of the input image pair is more likely to be a plausible transformation from the satellite image of the input image pair (i.e. is likely no fake)” in [0065]. Shu teaches input image tiles in Fig 3. Li further discloses “As an example, supervised training (e.g., supervised learning) can be based on output of a trained ML model trained using unsupervised training (e.g., unsupervised learning) and can be based on output of an interactive interpretation process that generates labels for seismic image data” in [0156]; “whereas, if some portion of the earth model is unacceptable, the method 900 can continue to the label block 930 for labeling, re-labeling, adjusting labels, etc., for one or more portions of the seismic data and/or the processed seismic data ( e.g., of the interpretation block 920)” in [0166]; “As an example, a training process may align tiles with windows or other markings as labeled by an interpreter. In such a maimer, there may be assured that a tile (e.g., for training purposes) includes a label” in [0172]; “As an example, an approach can include input labels (training data for supervised learning) to be pixel-wise categorical labels selectively marked on seismic inline and/ or crossline slices. The amount of labeled slices can be predetermined to be of an amount that is suitable for purposes of training” in [0175]; “As mentioned, a system can optionally utilize output from an unsupervised process to assist in labeling” in [0181]; “As an example, a trained CNN model can be applied to predict the pixel-wise categorical classes for remaining unlabeled portions of a seismic cube” in [0184]; “In the example of FIG. 11, the output of the processes 1120 and 1140, as appropriate, can be provided to a system that can perform neural network training per a process 1150 that can generate a trained neural network model 1160 that can be utilized for in a prediction process 1170 to generate output labels for an entire seismic cube 1180 (e.g., or one or more portions thereof that may have not been labeled in the process 1140)” in [0188]; “As mentioned, a workflow can include using a general ML model, generating labels, using the labels to generate a specific ML model and then using the specific ML model to generate output” in [0201].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu and Wang with the teaching of Li so as to perform neural network training to generate a trained neural network model that can be utilized for in a prediction process to generate output labels for an entire seismic cube (Li, [0188]).
As to Claim 2, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein providing the landcover map comprises transmitting the landcover map to a client device, for display of the landcover map on the client device (Zhang discloses “To achieve this, a map server selects map data from a geospatial database for a certain geographic area, generates multiple map image layers using the selected map data, and transmits them separately to the client device” in [0044], see also Shu’s Fig 1.)
As to Claim 3, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1. The combination of Sargent further teaches wherein providing the landcover map comprises transmitting the landcover map to a computing device configured to determine positions of seismic imaging equipment according to the landcover map (Zhang discloses “generating map data for a geographical region by acquiring road usage information specifying which parts of a geographical area have been used for driving a vehicle…” in [0003], see also Shu’s Fig 10. Zhang and Shu are silent on seismic equipment. Li further discloses “Where the position of an LWD module is mentioned, as an example, it may refer to a module at the position of the LWD module 454, the module 456, etc. An LWD module can include capabilities for measuring, processing… the LWD module 454 may include a seismic measuring device” in [0092], see also [0145].)
As to Claim 4, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein the landcover map comprises one or more landcover classification labels, each classification label from the one or more landcover classification labels corresponding to a landcover class of the environment (Shu discloses “Raster data can also be used to present semantic information extracted from imagery (i.e., raster labels) such as in land classification maps.” in [0001]; “a machine learning model may be trained to recognize the landcover features that tend to demarcate such boundaries” in [0018]; “Further, the geospatial image 620 depicts several landcover features 628 that may mark the boundaries of legal land parcels, including roads, fences, and tree lines… legal land parcels tend to be marked by such landcover features” in [0062]; see also Fig 10.)
As to Claim 5, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 4, wherein at least one classification label from the one or more landcover classification labels represent an obstructed portion of the environment, the obstructed portion of the environment being likely to cause an interference in seismic imaging of the environment (Shu discloses “Further, unlike physical landcover, such as roads, trees, and buildings, which may be directly depicted in geospatial imagery, legal land parcels have boundaries which are legal constructs and which are not depicted directly in geospatial imagery. Rather, the boundaries of legal land parcels tend to be loosely demarcated by a multitude of markers, if any, such as roads, fences, tree lines, or other features that may indirectly suggest the boundaries of legal land parcels.” in [0018]; “Such a machine learning model may be trained to recognize, across a broad range of contexts, the visual features that tend to demarcate legal land parcel boundaries, such as fences, roads, curbs, tree lines, and other features that tend to be visible in geospatial imagery, to produce artificial parcel data” in [0020]. Here, it is obvious that Shu’s classification is not limited to the cited landcover features and can include seismic imaging device.)
As to Claim 6, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein the input image is an orthomosaic image captured by an unmanned aerial vehicle and the coordinate information is obtained from the unmanned aerial vehicle (Shu discloses “An image capture device 110 may include any suitable sensor (e.g., camera) onboard an aircraft, satellite, drone, observation balloon, or other device capable of capturing imagery of an area of interest from an overhead point of view (i.e., geospatial imagery).” in [0021]; “The geospatial imagery may comprise, for example, a single aerial or satellite image, an orthophoto, or an orthomosaic generated from several images” in [0029]; “geospatial imagery containing geospatial coordinate information” in [0070].)
As to Claim 7, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein the input image is a satellite image captured by a satellite and the coordinate information is obtained from the satellite (Zhang discloses “acquiring a satellite image of the geographical area” in [0003]. Shu also discloses “The geospatial imagery may comprise, for example, a single aerial or satellite image, an orthophoto, or an orthomosaic generated from several images” in [0029]; “geospatial imagery containing geospatial coordinate information” in [0070].)
As to Claim 8, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein the coordinate measures comprise one or more of (i) a pixel size, (ii) a pixel coordinate, (iii) a center of a pixel coordinate, (iv) a pixel rotation about an axis of the input image, for one or more pixels of the input image (Zhang discloses number of pixel of the satellite or GPS image in [0055].)
As to Claim 9, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein generating the plurality of input image patches from the input image comprises:
removing the coordinate information from the input image; selecting, for each input image patch in the plurality of input image patches, a portion of pixels of the input image, wherein each input image patch in the plurality of input image patches share an identical resolution; and generating a second set of input image patches from the plurality of input image patches, wherein each input image patch in the second set of input image patches is a duplicate of a corresponding input image patch in the plurality of input image patches (Shu discloses “In such cases, the machine learning model may be prevented from generating artificial parcel data over such areas (e.g., by applying a mask to the geospatial imagery prior to processing by the machine learning model), or, alternatively, the machine learning model may be allowed to generate artificial parcel data over such areas, only to be removed and/or overridden when merged with the ground truth parcel data at a later stage” in [0071]; “The method 200 involves subdividing a large area of geospatial imagery into smaller image tiles” in [0028]; “For illustrative purposes, reference may be had to FIG. 3B which shows the grid of image tiles 304 with a parcel mask 310 overlaid which covers the areas where there is ground truth parcel data available. There remain several buildings 303 which are not covered by the ground truth parcel data” in [0038]. Here, each image tile has an identical resolution, and the remaining input image tiles refers to a duplicate of a corresponding input image tile.)
As to Claim 10, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 9, wherein the plurality of input image patches are non-overlapping (Shu, Fig 3.)
As to Claim 12, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein the discriminator network of the generative adversarial network comprises a plurality of convolutional neural network layers and a plurality of activation layers (Zhang discloses “According to one embodiment, the generative adversarial network comprises a discriminator comprising a convolutional network” in [0011]; activation layers in [0060].)
As to Claim 13, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein one or more channels of the input image comprises a frequency band of visible light (Zhang discloses “According to one embodiment, the input of the generator 301 is a pair of GPS image 303 and a satellite image 304 where the GPS image 303 contains a single colour channel and the satellite image contains RGB three colour channels.” in [0059].)
As to Claim 15, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1, wherein the generative adversarial network is a condition generative adversarial network, and comprising:
obtaining a training set of input landcover map patches corresponding to the environment captured by the input image (Zhang discloses “When training the GAN 300, the satellite image 304 and the GPS image 303 come from training data” in [0061]. Shu further discloses “The method 200 involves subdividing a large area of geospatial imagery into smaller image tiles and performing various data processing tasks” in [0028], see also Fig 3);
generating, by the generator network and using the plurality of input image patches, a plurality of output landcover map patches, each output landcover map patch from the plurality of output landcover map patches corresponding to an input image patch from the plurality of input image patches (Zhang discloses “a map image 307 generated by the generator 301 from the satellite image 304 and the GPS image 303 of the training data element” in [0064]; “It should be noted that more than one satellite images may be acquired which may be seen to form, together, a big satellite image” in [0081]. Shu, Fig 10);
providing the plurality of output landcover map patches to the discriminator network (Zhang, Fig 3);
determining, by the discriminator network and for each output landcover map patch from the plurality of output landcover map patches, a classification label indicating that the output landcover map patch correctly identifies one or more landcover classes of the corresponding input image patch (Zhang discloses “The CNN 310 outputs a value between 0 and 1 where a larger value indicates that the map image of the input image pair is more likely to be a plausible transformation from the satellite image of the input image pair (i.e. is likely no fake)” in [0065]. Li teaches classification label in [0184, 0188, 0201]);
determining, based on the classification label, an error between the plurality of output landcover map patches and the training set of input landcover map patches (Zhang discloses “The generator 301 may be trained using both an adversarial loss (punishing that the discriminator 302 recognizes an image generated by the generator 301 as fake) and an L1 or mean absolute pixel difference loss between an image generated from a satellite image of training data element and the map image included in the training data element (i.e. the expected target image or ground truth image). The adversarial loss is for example a binary cross-entropy loss (applied to the output, which is between 0 and 1, of the CNN 310)” in [0067], see also [0068]); and
in response to determining that the error exceeds a threshold value, updating one or more parameters of the generator network or the discriminator network (Zhang discloses “The discriminator 302 may for example be trained to minimize the negative log likelihood of identifying real and fake images. This may be done by updating its weights using backpropagation” in [0066], see also [0068]. Shu further discloses “It should also be noted here that in some examples, an output from the machine learning model that falls below a threshold confidence level may be excluded from the resulting artificially-generated parcel dataset.” in [0075].)
Claim 16 recites similar limitations as claim 1 but in a system form. Therefore, the same rationale used for claim 1 is applied.
Claim 17 is rejected based upon similar rationale as Claim 2.
Claim 18 is rejected based upon similar rationale as Claim 3.
Claim 19 recites similar limitations as claim 1 but in a computer readable media form. Therefore, the same rationale used for claim 1 is applied.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Shu and Wang, further in view of Li and Alwon (US 2019/0302290A1).
As to Claim 11, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1. The combination of Alwon further teaches wherein the generator network of the generative adversarial network comprises a plurality of convolutional neural network layers, a plurality of skip connections, and a plurality of activation layers (Zhang teaches a U-Net 306 in Fig 3. Alwon further discloses “As an example, a U-Net architecture can include such features, for example, with skip connections. Such skip connections can concatenate activations ( e.g., from layer i to layer n), which may alter the number of channels in a decoder. As to a discriminator architecture, after a last layer, a convolution may be applied to map to a 1-dimensional output” in [0102], see also Fig 7.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu, Wang and Li with the teaching of Alwon so as to explain a general GAN model with the corresponding structure.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Shu and Wang, further in view of Li and Sargent et al. (EP 3614308 A1).
As to Claim 14, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1. The combination of Sargent further teaches wherein one or more channels of the input image comprises a frequency band of infrared (Sargent discloses “Aerial photos of S1 and S2 were captured using Vexcel UltraCam Xp digital aerial cameras on 22/07/2012 and 20/04/2016, respectively. The images have four multispectral bands (Red, Green, Blue and Near Infrared) with a spatial resolution of 50 cm.” in [0060].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu, Wang and Li with the teaching of Sargent so as to capture input image with infrared light.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Shu and Wang, further in view of Li and Goldstein et al. (US 2024/0428568 A1).
As to Claim 21, Zhang in view of Shu, Wang and Li teaches The computer-implemented method of claim 1. The combination of Goldstein further teaches wherein the generative adversarial network comprises a plurality of discriminator networks, wherein each discriminator network from the plurality of discriminator networks is trained on a different set of features associated with different types of landcover classes (Goldstein discloses “In some embodiments, the use of multiple discriminators 604 may be supported, such as where different discriminators 604 analyze image data at different scales” in [0052], see also [0053].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu, Wang and Li with the teaching of Goldstein so as to use multiple discriminators to analyze image data at different scales (Goldstein, [0052]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIMING HE whose telephone number is (571)270-1221. The examiner can normally be reached on Monday-Friday, 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached on 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEIMING HE/
Primary Examiner, Art Unit 2611
Prosecution Insights