DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 3 and 11 are objected to because of the following informalities:
In Claim 3, “training the second machine learning model based on the one or more one or more other annotations” should read as “training the second machine learning model based on the one or more other annotations.”
In Claim 11, “train the second machine learning model based on the one or more one or more other annotations” should read as “train the second machine learning model based on the one or more other annotations.”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The method of claim 1 is directed to a process, which is one of the statutory categories of invention, and passes Step 1: Statutory Category- MPEP § 2106.03. However, the following steps of Claim 1 recite limitations that constitute mental processes because they describe acts of observation, evaluation, and judgement that can practically be performed in the human mind, or by a human using pen and paper as a physical aid, therefore failing Step 2A Prong One. These acts are mental processes because a human can mark or label things in a geospatial image (“annotate”), create a collection of labeled examples (“training dataset’), and then learn how to identify features in the images.
providing a user interface for annotating one or more geospatial images;
receiving at least one annotation for the one or more geospatial images from a user device;
generating a training dataset based on the at least one annotation;
and training a first machine learning model for features identification in geospatial images via the training dataset.
Claim 1 fails Step 2A Prong Two because the additional elements beyond the judicial exception, including a user interface and machine learning model, do not integrate the judicial exception into a practical application. There are no improvements to the functioning of a computer or any other technology or technical field (MPEP § 2106.05(a)) as the user interface and machine learning model merely apply the abstract idea on a computer (MPEP § 2106.05(f)). Furthermore, the claim does not impose meaningful limits on the computer components such that they are tied to a particular machine; the additional elements are described at a high level of generality and can be implemented on any generic computing system (MPEP § 2106.05(b)). Claim 1 also fails Step 2B, as these additional elements are well-understood, routine, and conventional (WURC), adding nothing significantly more than the abstract idea itself (MPEP § 2106.07(a)((III)). A user interface is WURC (see MPEP § 2106.05(d)), as well as a machine learning model (see Abstract and Introduction section of Boutayeb et. al, “When Machine Learning Meets Geospatial Data: A Comprehensive GeoAI Review”).
Claims 9 and 17 contain this identical ineligible subject matter, with the only additional elements beyond the judicial exception being a non-transitory computer-readable medium, processor, user interface subsystem, data processing component, and training component, which also do not integrate the judicial exception into a practical application (see claim 1 analysis above) and are WURC (see MPEP § 2106.05(d)). Therefore, they are rejected.
Claims 2, 3, 4, 6, and 7 recite limitations that constitute mental processes because they describe acts of observation, evaluation, and judgement that can practically be performed in the human mind, or by a human using pen and paper as a physical aid, therefore failing Step 2A Prong One. These acts are mental processes because a human can suggest labels for the image (“recommend annotation”), provide feedback or confirmation (“receiving an acceptance of the annotation recommendation”), mark or label things in a geospatial image (“annotate”), circle a region of interest (“selecting areas of interest”), identify features in the images, and check a folder and search a catalog for an image (“determining whether images are available in a database”).
These claims, alongside claims 5 and 8, also fail Step 2A Prong Two and Step 2B because the additional elements beyond the judicial exception do not integrate the judicial exception into a practical application and are WURC (see claim 1 analysis above); therefore, they are rejected. As claims 10-16 contain this identical ineligible subject matter, with the only additional elements beyond the judicial exception being a non-transitory computer-readable medium and processor, they are also rejected (see claim 9 analysis above).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-17 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Dubroy et. al (US2021303849A1).
Regarding Claim 1, Dubroy teaches a method for geospatial image processing, comprising:
Paragraph [0007]: “In one aspect, the disclosed technology may take the form of a first method that involves (i) obtaining a first layer of map data associated with sensor data capturing a geographical area, the first layer of map data comprising an aggregated overhead-view image of the geographical area...”
providing a user interface for annotating one or more geospatial images (Fig. 7 (shown below));
PNG
media_image1.png
541
1141
media_image1.png
Greyscale
Explanation: FIG. 7 illustrates an overview of the user interface tool used and the distribution of tasks to manual curators. Thus, the user platform 710 and associated user interface provide tools for annotating geospatial images.
receiving at least one annotation for the one or more geospatial images from a user device (Fig. 5 (shown below) and Fig. 7 (shown above));
PNG
media_image2.png
727
942
media_image2.png
Greyscale
Explanation: The reference explicitly discloses receiving annotations (user input/adjustments) from a user device (user platform 710).
generating a training dataset based on the at least one annotation;
Paragraph [0040]: “Additionally, once images have been verified and submitted following verification, the images can be stored for further distribution for quality assurance or be used as training data for an automated data-curation system.”
Paragraph [0061]: “Additionally, once the semantic data for the map or areas/sections thereof have been verified and submitted as complete following verification, the images and semantic data can be stored for further distribution for quality assurance or be used as training data for an automated data curation system.”
and training a first machine learning model for features identification in geospatial images via the training dataset.
Paragraph [0065]: “In this respect, the function of creating the map may involve automatically extracting the label data (e.g., semantic map data) from the aggregated overhead-view image using one or more of: machine learning models; classifiers; or Generative Adversarial Networks.”
Paragraph [0084]: “One can use a combination of heuristics, computer vision, and point classification algorithms to generate hypotheses for these semantic objects and their metadata… feedback from the human curation and quality assurance steps can be used to keep these up to date.”
Regarding Claim 2, Dubroy teaches the method of claim 1, further comprising:
generating, via a second machine learning model, an annotation recommendation for the one or more geospatial images (Paragraph [0065] (shown above in claim 1));
Paragraph [0069]: “In some embodiments, the automated processing function can further suggest one or more corrections for each of the highlighted high-confidence defects (for example a new location of a semantic label can be recommended and the difference between the label in the semantic layer and the recommended new location for the semantic label can be displayed to the data curators).”
Explanation: The “suggest one or more corrections” constitutes an annotation recommendation generated by automated ML-based processing.
and providing the annotation recommendation to the user device via the user interface, wherein receiving the at least one annotation includes receiving an acceptance of the annotation recommendation.
Paragraph [0068]: “At block 610, the map may be refined based on user input received via the user platform, where such user input may reflect reasoned or quality-based judgements, annotations, and/or visual manipulations of the section or area of the map.”
Paragraph [0069]: “The difference between the label in the semantic layer and the recommended new location for the semantic label can be displayed to the data curators.”
Paragraph [0074]: “In example embodiments, the user platform 710 can provide a map layer presentation which can display to the curator a centered view of the map, a semantic layer and/or geometric layer of the map based on the overhead-view image generated of the ground map or drivable surface.”
Paragraph [0079]: “Existing semantic labels that are left without amendment may be assumed by the user platform 710 to be considered correct by the curator.”
Explanation: The recommendations are presented via the user platform 710, and user validation/refinement constitutes acceptance of recommendations.
Regarding Claim 3, Dubroy teaches the method of claim 2, further comprising:
receiving, via the user interface, one or more other annotations for the one or more geospatial images;
Paragraph [0068]: “In this respect, the user input may comprise a requested adjustment to label data included in the second layer of the map, and the function of refining the map may involve (i) refining the map data based on adjusted label data that we created locally by the user platform or (ii) adjusting label data at the mapping system based on user input received from the user platform.”
Paragraph [0078]: “In example embodiments, the curators can make edits or adjustments and/or manually label data by any one or any combination of: visual manipulation; determining abnormalities; determining alignments/misalignments; inputting one or more annotations…”
and training the second machine learning model based on the one or more one or more other annotations (Paragraphs [0061] and [0040] (shown above in claim 1)).
Explanation: The additional annotations are used as training data for ML systems.
Regarding Claim 4, Dubroy teaches the method of claim 1, further comprising:
selecting an area of interest for identifying a geospatial feature based on a first geospatial image;
Paragraph [0058]: “In example embodiments, therefore, a pre-processed or pre-generated overhead-view map or overhead-view image is generated or received, and portions of the map or image are extracted and determined whether validation is required.”
Paragraph [0071]: “In example embodiments, the one or more units 708 can include more than one section or area of the map and in some embodiments can be allocated in accordance with the time it takes to verify the map data or based on the contextual analysis of each section or area.”
Explanation: Area extraction = selecting area of interest
receiving a second geospatial image of the area of interest (Fig. 4 (shown below));
PNG
media_image3.png
731
951
media_image3.png
Greyscale
Paragraph [0062]: “In some cases, the human user may be provided with a corresponding limited field-of-view image(s) for each section of the geographical area to use for further verification.”
Explanation: Fig. 4 shows trajectory lines and associated field-of-view images.
and identifying the geospatial feature via the second geospatial image.
Paragraph [0084]: “Semantic objects can include various traffic, two- and three-dimensional objects such as lane boundaries, intersections, crosswalks, parking spots, stop signs, traffic lights, etc. that are used for driving safely… For example, to identify traffic lights, one can first run a traffic light detector on the camera images.
Explanation: Features are identified using another image.
Regarding Claim 5, Dubroy teaches the method of claim 4, wherein a resolution associated with the second geospatial image is greater than a resolution associated with the first geospatial image.
Paragraph [0043]: “Particularly, example embodiments seek to generate content rich aggregated overhead-view images of geographical areas on top of ground map data, the ground map data providing a representation of the surface topology of a geographical area, using limited field-of-view images captured from a substantially ground level perspective. This results in higher-resolution overhead-view images being generated without, or with substantially fewer, occlusions compared to using satellite or other aerially-captured images.”
Explanation: Aggregated overhead images provide higher resolution than limited field-of-view images.
Regarding Claim 6, Dubroy teaches the method of claim 4, wherein the area of interest is selected via the first machine learning model.
Paragraph [0065]: “In this respect, the function of creating the map may involve automatically extracting the label data (e.g., semantic map data) from the aggregated overhead-view image using one or more of: machine learning models; classifiers; or Generative Adversarial Networks.”
Paragraph [0069]: “For instance, in some embodiments, a checklist of predetermined errors can be used by an automated processing function to determine if there are any errors in any portion of a semantic layer of the map data and these errors can be used to highlight high-confidence defects to curators.”
Explanation: Automated processing identifies areas needing review which corresponds to ML-based selection of area.
Regarding Claim 7, Dubroy teaches the method of claim 1, further comprising:
receiving, from the user device and via the user interface, a request for geospatial imagery of an area of interest;
Paragraph [0063]: “In some embodiments, once a task is verified, complete, and submitted, the results can be saved or stored, and the same task can be accessed with a given uniform resource locator (URL) for example.”
Paragraph [0073]: “alternatively the curator may be capable of selecting tasks to be performed.”
determining whether the one or more geospatial images are available in a local database of geospatial imagery in response to the request;
Paragraph [0090]: “In particular embodiments, storage 806 includes mass storage for data or instructions.”
Paragraph [0096]: “In embodiments, raw and/or processed image data may be stored within a cloud storage which may be accessed through a web service application programming interface (API) or by applications that utilize the API, such as a cloud desktop storage, a cloud storage gateway, or web-based content management systems. Typically, data may be stored locally or remotely in order to efficiently access data.”
Explanation: Storage systems determine availability and provide images.
and obtaining the one or more geospatial images based on the determination (paragraph [0090] and [0096] (shown above)).
Regarding Claim 8, Dubroy teaches the method of claim 7, wherein the one or more geospatial images are obtained from a third-party geospatial imagery source based on the one or more geospatial images being unavailable in the local database (paragraph [0096] (shown above in claim 7)).
Paragraph [0092]: “In particular embodiments, communication interface 810 includes hardware or software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 800 and one or more other computer systems (or other network devices) via one or more networks.”
Explanation: Cloud/API -based retrieval supports third-party sourcing when local is unavailable.
Regarding Claim 9, Dubroy teaches all the limitations as in the consideration of claim 1 above. Dubroy further teaches the non-transitory computer-readable medium and one or more processors that perform the same steps as claim 1.
Paragraph [0022]: “In yet another aspect, the disclosed technology may take the form of a computer system comprising at least one processor, a non-transitory computer-readable medium, and program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computer system is configured to carry out the functions of the aforementioned first and/or second method.”
Regarding Claim 10, Dubroy teaches the non-transitory computer-readable medium of claim 9, and additional limitations are met as in the consideration of claim 2 above.
Regarding Claim 11, Dubroy teaches the non-transitory computer-readable medium of claim 10, and additional limitations are met as in the consideration of claim 3 above.
Regarding Claim 12, Dubroy teaches the non-transitory computer-readable medium of claim 9, and additional limitations are met as in the consideration of claim 4 above.
Regarding Claim 13, Dubroy teaches the non-transitory computer-readable medium of claim 12, and additional limitations are met as in the consideration of claim 5 above.
Regarding Claim 14, Dubroy teaches the non-transitory computer-readable medium of claim 12, and additional limitations are met as in the consideration of claim 6 above.
Regarding Claim 15, Dubroy teaches the non-transitory computer-readable medium of claim 9, and additional limitations are met as in the consideration of claim 7 above.
Regarding Claim 16, Dubroy teaches the non-transitory computer-readable medium of claim 15, and additional limitations are met as in the consideration of claim 8 above.
Regarding Claim 17, Dubroy teaches all of the limitations of claim 1 above because claim 17 recites a system that performs substantially the same steps as claim 1.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Eno (US 20200104646 A1) teaches systems and methods for training and validating a computer vision model for geospatial imagery. In certain examples, a geospatial image processing system provides a user interface that includes user interface content displayed in a graphical user interface view and one or more user interface tools configured to facilitate user tagging of geospatial imagery and/or user validation of computer vision model detections of objects of interest in geospatial imagery.
Estrada (US 10733759 B2) teaches a system for broad area geospatial object detection using auto-generated deep learning models, comprising a deep learning model training software module and an image analysis software module.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM ADU-JAMFI whose telephone number is (571)272-9298. The examiner can normally be reached M-T 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM ADU-JAMFI/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677