DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/29/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Preliminary Amendment
Applicant submitted a preliminary amendment on 4/22/2025. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Claim Objections
Claim 2 is objected to because of the following informalities:
Claim 2 states “wherein the set of metadata associated with the image is absent metadata required to geolocalize the image, wherein outputting the geographic features file further comprises:”. The term absent metadata is unclear and needs to be further specified. Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6, 9-12, 14, 17-20, and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Flynn (AU-2013248183-B2) in view of Cheng (U.S. Patent Pub. No. 2012/0314935).
Regarding Claim 1, Flynn teaches a computer-implemented method for geolocalizing (¶2 Aspects of the invention relate generally to digital imagery. More particularly, aspects are directed to matching a received image with geolocation information against selected reference images:)
receiving an image file recording an image and a set of metadata associated with the image (¶5 The method comprises receiving an image request from a user device, the image request including an image of interest and location metadata for the image of interest;)
determining a search space using-one or more of at least a portion of the set of metadata and auxiliary data (¶5 analyzing the location metadata to select one or more cells to evaluate against the image of interest, each cell having one or more geolocated images and index data associated therewith;)
generating, (¶5 for each selected cell, comparing the image of interest against the index data of that cell; identifying any matches from the geolocated images of the selected cells based on the compared index data;)
identifying a simulated candidate image in the set of simulated candidate images as a best matching image relative to the image, the simulated candidate image being associated with a set of candidate metadata (¶5 for each selected cell, comparing the image of interest against the index data of that cell; identifying any matches from the geolocated images of the selected cells based on the compared index data; and providing the matches; ¶50 Each cell match server 504 matches the received image against its respective index data. One or more matching references (if any) are returned to the front end server 502. These results preferably include a match confidence indicator.)
providing a set of augmented metadata for the image generated from the set of metadata and the set of candidate metadata, the set of augmented metadata comprising at least a portion of the set of candidate metadata; and (¶23 In addition to being associated with geographic images such as street level image 100 may be associated with information indicating the orientation of the image; ¶42 Once the received image is matched to a known image, a more refined location can be associated with the received image. Or, alternatively, the location and orientation of the mobile user device can be corrected. This may be done by solving for the relative pose or relative location and orientation of the received image based on correspondences with image information from the database.)
outputting a geographic features file that is generated using the set of augmented metadata, the geographic features file comprising data representing one or more geographic features represented in the image file (¶44 The imagery of each image each cell has certain features. For instance, each image may be associated with the location information such as latitude/longitude, orientation and height. The image also includes image details. The images details may include corners, edges or lines, brightness changes, histograms or other image filtering outputs from known image processing techniques.)
Flynn does not explicitly disclose geolocalizing aerial images; generating, using a multi-dimensional model, a set of simulated candidate images, each simulated candidate image corresponding to the search space.
Cheng is in the same field of art of image analysis. Further, Cheng teaches geolocalizing aerial images; generating, using a multi-dimensional model, a set of simulated candidate images, each simulated candidate image corresponding to the search space (¶28 Embodiments of the present invention generally relate to determining the geographic location of a captured depiction whose location is unknown, using other geo-referenced depiction data captured from a different perspective. According to one embodiment, the captured depictions are narrow field of view (NFOV), ground plane, and/or street view (SV) images, and the method determines the geographic location of the scene depicted in a captured image by extracting a set of features from a database of reference depictions, which, according to some embodiments include satellite (SAT) imagery, three-dimensional (3D) model data and oblique bird's eye view (BEV) images, i.e., oblique aerial imagery, of an area of interest.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Flynn by geolocalizing aerial images and using a multidimensional model that is taught by Cheng; thus, one of ordinary skilled in the art would be motivated to combine the references to geolocate from a different perspective (Cheng ¶8).
Regarding Claim 2, Flynn in view of Cheng discloses the method of claim 1, wherein the set of metadata associated with the image is absent metadata required to geolocalize the image, wherein outputting the geographic features file further comprises (It is unclear what this limitation means. The method of Cheng teaches geolocalizing without metadata ¶6:)
determining bounding box data using the set of augmented data, wherein the one or more geographic features represented within the geographic features file are at least partially located within a bounding box defined by the bounding box data (using a bounding box to define features is well known in the art. Flynn teaches ¶44 The images details may include corners, edges or lines, brightness changes, histograms or other image filtering outputs from known image processing techniques; Cheng claims 9 and 10 teach outlining the detecting buildings.)
Regarding Claim 3, Flynn in view of Cheng discloses the method of claim 1, wherein determining a search space, generating a set of candidate images, identifying a candidate image in the set of candidate images as a best matching image, and providing a set of augmented metadata for the image are performed in response to determining that the set of augmented data is absent at least a portion of pose data (Flynn, ¶50 Each cell match server 504 matches the received image against its respective index data. One or more matching references (if any) are returned to the front end server 502. These results preferably include a match confidence indicator; This is optionally done without pose data ¶42)
Regarding Claim 4, Flynn in view of Cheng discloses the method of claim 1, wherein each candidate image in the set of candidate images is generated using a multi-dimensional model of Earth (Cheng, ¶28 the captured depictions are narrow field of view (NFOV), ground plane, and/or street view (SV) images, and the method determines the geographic location of the scene depicted in a captured image by extracting a set of features from a database of reference depictions, which, according to some embodiments include satellite (SAT) imagery, three-dimensional (3D) model data and oblique bird's eye view (BEV) images, i.e., oblique aerial imagery, of an area of interest. In an exemplary embodiment, feature extraction includes annotating those images with the objects that they are determined to contain such as trees, bushes, houses, and the like.)
The reasons for combining Flynn and Cheng are similar to that stated in the rejection of claim 1.
Regarding Claim 6, Flynn in view of Cheng discloses the method of claim 1, wherein the search space comprises sets of parameters and each candidate image in the set of candidate images is generated based on a respective set of parameters (Flynn, ¶5 for each selected cell, comparing the image of interest against the index data of that cell; identifying any matches from the geolocated images of the selected cells based on the compared index data; and providing the matches; ¶50 Each cell match server 504 matches the received image against its respective index data. One or more matching references (if any) are returned to the front end server 502. These results preferably include a match confidence indicator.)
Regarding claim 9, claim 9 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Flynn further teaching on: A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations (Claim 15: A non-transitory computer-readable medium storing software comprising instructions executable by one or more computing devices which, upon such execution, cause the one or more computing devices to perform operations)
Claim 10 recites limitations similar to claim 2 and is rejected under the same rationale and reasoning
Claim 11 recites limitations similar to claim 3 and is rejected under the same rationale and reasoning
Claim 12 recites limitations similar to claim 4 and is rejected under the same rationale and reasoning
Claim 14 recites limitations similar to claim 6 and is rejected under the same rationale and reasoning
Regarding claim 17, claim 17 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Flynn further teaching on:
A system, comprising: one or more processors; and a computer-readable storage device coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations (¶35 As shown in diagram 400 of FIG. 7, the devices contain a processor 402, memory/storage 404 and other components typically present in a computer. Memory 404 stores information accessible by processor 402, including instructions 406 that may be executed by the processor 402.)
Claim 18 recites limitations similar to claim 2 and is rejected under the same rationale and reasoning
Claim 19 recites limitations similar to claim 3 and is rejected under the same rationale and reasoning
Claim 20 recites limitations similar to claim 4 and is rejected under the same rationale and reasoning
Claim 22 recites limitations similar to claim 6 and is rejected under the same rationale and reasoning
Claims 5, 7, 8, 13, 15, 16, 21, 23, 24 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Flynn (AU-2013248183-B2) in view of Cheng (U.S. Patent Pub. No. 2012/0314935) in view of Rodrigues (U.S. Patent No. 12536680).
Regarding Claim 5, Flynn in view of Cheng teaches the method of claim 1.
Flynn in view of Cheng does not explicitly disclose wherein the search space is determined by processing the image through a search space machine learning (ML) model that outputs the search space.
Rodrigues is in the same field of art of image analysis. Further, Rodrigues teaches wherein the search space is determined by processing the image through a search space machine learning (ML) model that outputs the search space (Col 9 Lines 18-22: the discriminator 10 includes one or more machine learning-based models each of which includes trainable parameters, such as weights of neural networks. For example, the discriminator 10 has two feature extractors each of which is formed as a machine learning-based model; Col 7 Lines 28-30: Specifically, the discriminator 10 extracts a plurality of partial aerial regions 40 from the aerial-view image 30.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Flynn in view of Cheng by using a machine learning model for the processes that is taught by Rodrigues; thus, one of ordinary skilled in the art would be motivated to combine the references as it would be obvious to try, by one of ordinary skill in the art, a machine learning model for the processes of segmenting and comparing (Rodrigues).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 7, Flynn in view of Cheng in view of Rodrigues discloses the method of claim 1, wherein identifying a candidate image in the set of candidate images as a best matching image relative to the image comprises processing the image and each candidate image through an image similarity ML model that determines similarity scores, each similarity score representing a similarity between the image and a respective candidate image (Rodrigues, Col 7 Lines 34-40: the discriminator 10 determines whether the features of the partial aerial region 40 is substantially similar to the feature of the ground-view image 20. To do so, the discriminator 10 computes an index (hereinafter, similarity score) indicating a degree of similarity between the features of the ground-view image 20 and those of the partial aerial region 40.) (Flynn ¶50 teaches a cell match confidence indicator, but not using a ML model)
The reasons for combining Flynn, Cheng and Rodrigues are similar to that stated in the rejection of claim 5. In addition, this same reasoning is pertinent and applicable to the rejections of claims 8 and 25 below.
Regarding Claim 8, Flynn in view of Cheng in view of Rodrigues discloses the method of claim 7, wherein the candidate image is identified as the best matching image in response to the candidate image having a highest similarity score (Rodrigues Col 7 Lines 41-47: In the case where there is a partial aerial region 40 whose features are substantially similar to the features of the ground-view image 20 (e.g. the similarity score computed for the features of the ground-view image 20 and those of the partial aerial region 40 is equal to or larger than a predefined threshold), the discriminator 10 determines that the ground-view image 20 matches the aerial-view image 30.) (Flynn, ¶53 The front end server 502 is configured to collate results returned by the cell match servers 504. The front end server may threshold the match scores provided by the cell match servers. The result(s) with the highest correlation and/or confidence is (are) identified as a (possible) match.)
Claim 13 recites limitations similar to claim 5 and is rejected under the same rationale and reasoning.
Claim 15 recites limitations similar to claim 7 and is rejected under the same rationale and reasoning.
Claim 16 recites limitations similar to claim 8 and is rejected under the same rationale and reasoning.
Claim 21 recites limitations similar to claim 5 and is rejected under the same rationale and reasoning.
Claim 23 recites limitations similar to claim 7 and is rejected under the same rationale and reasoning.
Claim 24 recites limitations similar to claim 8 and is rejected under the same rationale and reasoning.
Regarding Claim 25, Flynn in view of Cheng in view of Rodrigues discloses the method of claim 1, wherein determining the search space comprises, providing the one or more of at least the portion of the set of metadata and auxiliary data to a trained machine-learning module and obtaining, from the trained machine-learning model, the search space including a set of parameters (Rodrigues, Col 8 Lines 20-31 The discriminator 10 may extracts the partial aerial regions 40 in such a manner that a part of a partial aerial region 40 overlaps a part of one or more other partial aerial regions 40. In this case, for instance, the discriminator 10 may use a sliding window method to define radial lines by which a partial aerial region 40 is extracted from the aerial-view image 30. Hereinafter, two radial lines by which a partial aerial region 40 is extracted from the aerial-view image 30 are described as being “a first radial line” and “a second radial line” respectively. Note that the angle between the first radial line and the x axis is smaller than the angle between the second radial line and the x axis; Col 9 Lines 18-22: the discriminator 10 includes one or more machine learning-based models each of which includes trainable parameters, such as weights of neural networks. For example, the discriminator 10 has two feature extractors each of which is formed as a machine learning-based model; Col 7 Lines 28-30: Specifically, the discriminator 10 extracts a plurality of partial aerial regions 40 from the aerial-view image 30.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
Prosecution Insights