Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 2/18/26 has been entered and made of record. Claims 1, 8, 16 and 19 are amended. Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments with respect to claims 1, 16 and 19 have been fully considered but they are moot because the arguments do not apply to the references being used in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4-10, 12-13, 15-17 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2023/0304826 A1) in view of Shu et al. (US 2024/0233086 A1) and Zhang et al. (CN 116009092 A, hereinafter Zhang2).
As to Claim 1, Zhang teaches A computer-implemented method for landcover map generation, comprising:
receiving, by a communication network and from a digital camera situated at an aerial position above an environment, an input image comprising pixels and representing the environment; obtaining coordinate information of the environment corresponding to the pixels of the input image (Zhang discloses “In 405, acquiring road usage information specifying which parts of a geographical area have been used for driving a vehicle; In 406, acquiring a satellite image of the geographical area; In 407, forming a road usage image of the geographical area which has pixels, each pixel corresponding to a respective part of the geographical area, such each pixel has a pixel value indicating whether the part of the geographical area, to which the pixel corresponds…” in [0076-0078], see also communication interface 501 in [0087]);
determining, based on the coordinate information of the pixels of the input image, one or more coordinate measures for the input image (Zhang discloses “The GPS image 204 has the same number of pixels as its paired satellite image 202. The covered space for each pixel in GPS image is the same as the corresponding pixel in the satellite image” in [0055]; mean absolute pixel difference loss in [0067]).
Zhang doesn’t explicitly teach input image patches. The combination of Shu further teaches following limitations:
generating, from the input image and based on the one or more coordinate measures, a plurality of input image patches, each input image patch from the plurality of input image patches representing a portion of pixels in the input image (Shu discloses “At operation 204, the image tile generation unit 220 subdivides the geospatial imagery into a grid of image tiles. Such a grid of image tiles may be a regular grid of equally sized cells of any appropriate dimensions… Such a grid may be established with reference to the geospatial information contained in metadata of the geospatial imagery (e.g., geospatial coordinates, image scale, resolution, etc.)” in [0030]; see also Fig 3);
applying, for the plurality of input image patches, one or more image processing techniques to adjust pixel values of a respective input image patch from the plurality of input image patches to generate a plurality of adjusted image patches (Zhang discloses “The satellite image 304 and the GPS image 303 are concatenated in a channel-wise fashion by image concatenation 305. Here, "channel-wise" means that a GPS image channel is added as a fourth channel to the RGB channels” in [0059]. Shu teaches input image tiles in Fig 3);
providing the plurality of adjusted image patches to a generative adversarial network, wherein the generative adversarial network is trained to generate a corresponding landcover map patch for each adjusted image patch in the plurality of adjusted image patches (Zhang discloses “The result of image concatenation 305 (i.e. the concatenated image data) is imported to a (trainable) U-Net 306” in [0059]; “The output of the generator is a map image 307 that contains RGB three colour channels” in [0060]; see also Fig 3 below:
PNG
media_image1.png
447
941
media_image1.png
Greyscale
);
generating, by the generative adversarial network and using the plurality of adjusted image patches, a plurality of landcover map patches, wherein the generative adversarial network comprises a generator network trained to generate training landcover map patches from an input image patch and a discriminator network trained to classify training landcover map patches from the generator network, wherein a classification of a training landcover map patch comprises a label indicating that the training landcover map patch represents one or more landcover classes of a corresponding input image patch (Zhang discloses “The GAN-based Image Translation 205 converts a pair of rendered GPS image 204 and satellite image 202 into a map image 206 using a GAN (or GAN-based model)” in [0056]; “The GAN 300 is a neural network which contains two parts (i.e. two sub-networks): a generator (network) 301 and a discriminator (network) 302” in [0058]; “The CNN 310 outputs a value between 0 and 1 where a larger value indicates that the map image of the input image pair is more likely to be a plausible transformation from the satellite image of the input image pair (i.e. is likely no fake)” in [0065]. Shu teaches input image tiles in Fig 3.);
generating a landcover map from the plurality of landcover map patches, wherein generating the landcover map comprises assembling the plurality of landcover map patches according to the coordinate information of the environment; and providing the landcover map corresponding to the input image for output (Zhang discloses “According to various embodiments, in other words, map data is generated by jointly using a satellite image and road usage information ( e.g. GPS traces of vehicles of an e-hailing service) by means of a generative adversarial network (GAN)” in [0080]; “It should be noted that more than one satellite images may be acquired which may be seen to form, together, a big satellite image (even if being stored in separate image parts)” in [0081]. Shu teaches merged vector map in Fig 10.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang with the teaching of Shu so as to subdivide a large area of geospatial imagery into smaller image tiles and process tasks on image tiles rather than the entire image for more manageable and better suited to parallel processing (Shu, [0028]).
Zhang and Shu don’t explicitly teach classification labels indicative of locations in the environment likely to be a source of interference in seismic imaging of the environment, and the label in each classification indicating locations in the environment likely to be a source of interference in seismic imaging of an environment represented by the corresponding input image patch. Zhang2 further discloses “finishing the location identification of the remote source interference in the seismic data, for the amplitude and fidelity of the seismic data, only interfering and suppressing the seismic data identified by the location” at p. 5. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang and Shu with the teaching of Zhang2 so as to identify location of the source of interference in seismic data.
As to Claim 2, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein providing the landcover map comprises transmitting the landcover map to a client device, for display of the landcover map on the client device (Zhang discloses “To achieve this, a map server selects map data from a geospatial database for a certain geographic area, generates multiple map image layers using the selected map data, and transmits them separately to the client device” in [0044], see also Shu’s Fig 1.)
As to Claim 4, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein the landcover map comprises one or more landcover classification labels, each classification label from the one or more landcover classification labels corresponding to a landcover class of the environment (Shu discloses “Raster data can also be used to present semantic information extracted from imagery (i.e., raster labels) such as in land classification maps.” in [0001]; “a machine learning model may be trained to recognize the landcover features that tend to demarcate such boundaries” in [0018]; “Further, the geospatial image 620 depicts several landcover features 628 that may mark the boundaries of legal land parcels, including roads, fences, and tree lines… legal land parcels tend to be marked by such landcover features” in [0062]; see also Fig 10.)
As to Claim 5, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 4, wherein at least one classification label from the one or more landcover classification labels represent an obstructed portion of the environment, the obstructed portion of the environment being likely to cause an interference in seismic imaging of the environment (Shu discloses “Further, unlike physical landcover, such as roads, trees, and buildings, which may be directly depicted in geospatial imagery, legal land parcels have boundaries which are legal constructs and which are not depicted directly in geospatial imagery. Rather, the boundaries of legal land parcels tend to be loosely demarcated by a multitude of markers, if any, such as roads, fences, tree lines, or other features that may indirectly suggest the boundaries of legal land parcels.” in [0018]; “Such a machine learning model may be trained to recognize, across a broad range of contexts, the visual features that tend to demarcate legal land parcel boundaries, such as fences, roads, curbs, tree lines, and other features that tend to be visible in geospatial imagery, to produce artificial parcel data” in [0020]. Here, it is obvious that Shu’s classification is not limited to the cited landcover features and can include seismic imaging device.)
As to Claim 6, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein the input image is an orthomosaic image captured by an unmanned aerial vehicle and the coordinate information is obtained from the unmanned aerial vehicle (Shu discloses “An image capture device 110 may include any suitable sensor (e.g., camera) onboard an aircraft, satellite, drone, observation balloon, or other device capable of capturing imagery of an area of interest from an overhead point of view (i.e., geospatial imagery).” in [0021]; “The geospatial imagery may comprise, for example, a single aerial or satellite image, an orthophoto, or an orthomosaic generated from several images” in [0029]; “geospatial imagery containing geospatial coordinate information” in [0070].)
As to Claim 7, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein the input image is a satellite image captured by a satellite and the coordinate information is obtained from the satellite (Zhang discloses “acquiring a satellite image of the geographical area” in [0003]. Shu also discloses “The geospatial imagery may comprise, for example, a single aerial or satellite image, an orthophoto, or an orthomosaic generated from several images” in [0029]; “geospatial imagery containing geospatial coordinate information” in [0070].)
As to Claim 8, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein the coordinate measures comprise one or more of (i) a pixel size, (ii) a pixel coordinate, (iii) a center of a pixel coordinate, (iv) a pixel rotation about an axis of the input image, for one or more pixels of the input image (Zhang discloses number of pixel of the satellite or GPS image in [0055].)
As to Claim 9, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein generating the plurality of input image patches from the input image comprises:
removing the coordinate information from the input image; selecting, for each input image patch in the plurality of input image patches, a portion of pixels of the input image, wherein each input image patch in the plurality of input image patches share an identical resolution; and generating a second set of input image patches from the plurality of input image patches, wherein each input image patch in the second set of input image patches is a duplicate of a corresponding input image patch in the plurality of input image patches (Shu discloses “In such cases, the machine learning model may be prevented from generating artificial parcel data over such areas (e.g., by applying a mask to the geospatial imagery prior to processing by the machine learning model), or, alternatively, the machine learning model may be allowed to generate artificial parcel data over such areas, only to be removed and/or overridden when merged with the ground truth parcel data at a later stage” in [0071]; “The method 200 involves subdividing a large area of geospatial imagery into smaller image tiles” in [0028]; “For illustrative purposes, reference may be had to FIG. 3B which shows the grid of image tiles 304 with a parcel mask 310 overlaid which covers the areas where there is ground truth parcel data available. There remain several buildings 303 which are not covered by the ground truth parcel data” in [0038]. Here, each image tile has an identical resolution, and the remaining input image tiles refers to a duplicate of a corresponding input image tile.)
As to Claim 10, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 9, wherein the plurality of input image patches are non-overlapping (Shu, Fig 3.)
As to Claim 12, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein the discriminator network of the generative adversarial network comprises a plurality of convolutional neural network layers and a plurality of activation layers (Zhang discloses “According to one embodiment, the generative adversarial network comprises a discriminator comprising a convolutional network” in [0011]; activation layers in [0060].)
As to Claim 13, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein one or more channels of the input image comprises a frequency band of visible light (Zhang discloses “According to one embodiment, the input of the generator 301 is a pair of GPS image 303 and a satellite image 304 where the GPS image 303 contains a single colour channel and the satellite image contains RGB three colour channels.” in [0059].)
As to Claim 15, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1, wherein the generative adversarial network is a condition generative adversarial network, and comprising:
obtaining a training set of input landcover map patches corresponding to the environment captured by the input image (Zhang discloses “When training the GAN 300, the satellite image 304 and the GPS image 303 come from training data” in [0061]. Shu further discloses “The method 200 involves subdividing a large area of geospatial imagery into smaller image tiles and performing various data processing tasks” in [0028], see also Fig 3);
generating, by the generator network and using the plurality of input image patches, a plurality of output landcover map patches, each output landcover map patch from the plurality of output landcover map patches corresponding to an input image patch from the plurality of input image patches (Zhang discloses “a map image 307 generated by the generator 301 from the satellite image 304 and the GPS image 303 of the training data element” in [0064]; “It should be noted that more than one satellite images may be acquired which may be seen to form, together, a big satellite image” in [0081]. Shu, Fig 10);
providing the plurality of output landcover map patches to the discriminator network (Zhang, Fig 3);
determining, by the discriminator network and for each output landcover map patch from the plurality of output landcover map patches, a classification label indicating that the output landcover map patch correctly identifies one or more landcover classes of the corresponding input image patch (Zhang discloses “The CNN 310 outputs a value between 0 and 1 where a larger value indicates that the map image of the input image pair is more likely to be a plausible transformation from the satellite image of the input image pair (i.e. is likely no fake)” in [0065]);
determining, based on the classification label, an error between the plurality of output landcover map patches and the training set of input landcover map patches (Zhang discloses “The generator 301 may be trained using both an adversarial loss (punishing that the discriminator 302 recognizes an image generated by the generator 301 as fake) and an L1 or mean absolute pixel difference loss between an image generated from a satellite image of training data element and the map image included in the training data element (i.e. the expected target image or ground truth image). The adversarial loss is for example a binary cross-entropy loss (applied to the output, which is between 0 and 1, of the CNN 310)” in [0067], see also [0068]); and
in response to determining that the error exceeds a threshold value, updating one or more parameters of the generator network or the discriminator network (Zhang discloses “The discriminator 302 may for example be trained to minimize the negative log likelihood of identifying real and fake images. This may be done by updating its weights using backpropagation” in [0066], see also [0068]. Shu further discloses “It should also be noted here that in some examples, an output from the machine learning model that falls below a threshold confidence level may be excluded from the resulting artificially-generated parcel dataset.” in [0075].)
Claim 16 recites similar limitations as claim 1 but in a system form. Therefore, the same rationale used for claim 1 is applied.
Claim 17 is rejected based upon similar rationale as Claim 2.
Claim 19 recites similar limitations as claim 1 but in a computer readable media form. Therefore, the same rationale used for claim 1 is applied.
Claim 20 is rejected based upon similar rationale as Claim 2.
Claim 3, 14 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Shu and Zhang2, further in view of Sargent et al. (EP 3614308 A1).
As to Claim 3, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1. The combination of Sargent further teaches wherein providing the landcover map comprises transmitting the landcover map to a computing device configured to determine positions of seismic imaging equipment according to the landcover map (Zhang discloses “generating map data for a geographical region by acquiring road usage information specifying which parts of a geographical area have been used for driving a vehicle…” in [0003], see also Shu’s Fig 10. Zhang and Shu are silent on seismic equipment. Sargent further discloses “The disclosure relates to a method and system that uses a joint deep learning framework to determine land cover and land use classifications from remote sensed image data of the land” in [0001]; “An object-based CNN (OCNN) was proposed recently for the urban LU classification using remotely sensed imagery. The OCNN is trained as for the standard CNN model with labelled image patches, whereas the model prediction labels each segmented object derived from image segmentation… The OCNN was trained on the LU classes, in which the semantic information of LU was learnt through the deep network, while the boundaries of the objects were retained through the process of segmentation.” in [0043]; “With respect to the OCNN model structure and parameters, for each segmented object, the centre point of the object was taken as the centre of the input image patch, where a standard CNN was trained to classify the object into a specific LU category” in [0065]; “The method starts with pixel-based classification using MLP applied to the original image to obtain the pixel-level characteristics (LC). Then this information (LC conditional 45 probabilities) was fed into the LU classification using the CNN model as part of modelling the joint distributions between LC and LU, and to infer LU categories through patch-based contextual neighbourhoods” in [0085]. Here, both landcover and land use classifications are determined from the sensed image data of the land via machine learning model, which can include a seismic imaging equipment.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu and Zhang2 with the teaching of Sargent so as to joint landcover and land use classification in an automatic fashion, and increase the classification accuracy, model robustness and generalization capability (Sargent, [0058]).
As to Claim 14, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1. The combination of Sargent further teaches wherein one or more channels of the input image comprises a frequency band of infrared (Sargent discloses “Aerial photos of S1 and S2 were captured using Vexcel UltraCam Xp digital aerial cameras on 22/07/2012 and 20/04/2016, respectively. The images have four multispectral bands (Red, Green, Blue and Near Infrared) with a spatial resolution of 50 cm.” in [0060].)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu and Zhang2 with the teaching of Sargent so as to capture input image with infrared light.
Claim 18 is rejected based upon similar rationale as Claim 3.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Shu and Zhang2, further in view of Alwon (US 2019/0302290A1).
As to Claim 11, Zhang in view of Shu and Zhang2 teaches The computer-implemented method of claim 1. The combination of Alwon further teaches wherein the generator network of the generative adversarial network comprises a plurality of convolutional neural network layers, a plurality of skip connections, and a plurality of activation layers (Zhang teaches a U-Net 306 in Fig 3. Alwon further discloses “As an example, a U-Net architecture can include such features, for example, with skip connections. Such skip connections can concatenate activations ( e.g., from layer i to layer n), which may alter the number of channels in a decoder. As to a discriminator architecture, after a last layer, a convolution may be applied to map to a 1-dimensional output” in [0102], see also Fig 7.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the invention of Zhang, Shu and Zhang2 with the teaching of Alwon so as to explain a general GAN model with the corresponding structure.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEIMING HE whose telephone number is (571)270-1221. The examiner can normally be reached Monday-Friday, 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached on 571-272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Weiming He/
Primary Examiner, Art Unit 2611