DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims s 1, 3–9, 11–14, and 16–20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a non-statutory judicial exception (an abstract idea) without reciting an inventive concept sufficient to transform the exception into a patent-eligible application.
Examination Framework (Alice/Mayo Two-Step Test)
Step 1: Statutory Category Determination
Independent claim 1 sets forth a method. Independent claim 9 sets forth a non-transitory computer readable storage medium (an article of manufacture). Independent claim 14 sets forth a system (a machine).Because the claims fall within statutory categories of 35 U.S.C. § 101, the examination proceeds to Step 2A.
Step 2A (Prong 1): Is the Claim Directed to a Judicial Exception?
A claim is disqualified under Step 2A (Prong 1) if its structural limitations, viewed as a whole, are "directed to" a recognized judicial exception: Mathematical Concepts, Mental Processes, or Certain Methods of Organizing Human Activity.
Independent claims 1, 9, and 14 are directed to a Mathematical Concept and a Mental Process for the reasons set forth below.
1. Recitation of Mathematical Concepts
The claims explicitly recite a mathematical formula and a series of discrete mathematical calculations designed to compute statistical variance. Specifically, the claims direct a processor to calculate a model's variance. The individual steps elements required to solve this formula comprise: "multiplying the square of a performance metric by a probability value summing the product of those performance metrics and probabilities subtracting the square of the aggregated sum from the aggregated sum of squares" Mathematical formulas, mathematical algorithms, and statistical calculations of variance represent quintessential "Mathematical Concepts." Under established federal precedent, mathematical equations do not become patent-eligible simply because they are restricted to a particular technological context, such as measuring machine learning prediction accuracy (see Parker v. Flook, 437 U.S. 584 (2014); Gottschalk v. Benson, 409 U.S. 63 (1972)).
2. Recitation of Mental Processes
The steps of "determining which of the plurality of features are strongly correlated with performance of the model" and "creating a plurality of parameterized sub-models that, in aggregate, approximate the input space" recite steps that can be performed entirely in the human mind or via pencil-and-paper data manipulation. Analyzing data correlations, grouping data into parameterized subsets (tessellations), and choosing representative examples ("prototype exemplars") are cognitive acts of data manipulation and classification. Because the claims fail to recite any physical, physical-world transformations or dedicated physical limits to achieve these data groupings, they describe a conceptual "Mental Process" (see Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350 (Fed. Cir. 2016)).
Step 2A (Prong 2): Is the Abstract Idea Integrated into a Practical Application?
A claim directing attention to an abstract idea may still be patent-eligible if the claim elements integrate the judicial exception into a "practical application" that improves a technological field or fixes a technical problem with a specific technical solution. The claims do not integrate the abstract mathematical idea into a practical application for the following reasons:
General Processing Framework and Failure to Improve Underlying Technology.
The specification describes that the technical problem to be solved relates to measuring the expected performance of an AI model on unseen images. However, the solution proposed by the independent claims is simply to execute a math formula. The claims do not improve the actual physical operation of a computer hardware circuit, nor do they detail an altered, more efficient software architecture that changes how a system processes memory. Rather, the claims present generic algorithmic data manipulation where the final output is a new number (the quantified variance/accuracy metric). Merely producing or calculating a piece of statistical data, even highly useful diagnostic data about an AI model, is an administrative data exercise rather than a technical improvement to physical or virtual computer operations.
The Context of "Identifying Objects in Images" is Field-of-Use Limitation
The recitation that the prediction model "has been trained to identify objects in images" is merely a field-of-use limitation. Limiting the calculation of a mathematical formula or the execution of a mental concept to a specific environment (machine learning image analysis) does not integrate an abstract idea into a practical application (see Bilski v. Kappos, 561 U.S. 593 (2010)).
Step 2B: Does the Claim Recite an "Inventive Concept" (Search for Something More)?
Because the independent claims are directed to an abstract idea without integration into a practical application, they must be checked for an "inventive concept" under Step 2B. This step determines whether the claim features, considered individually or as an ordered combination, add "something more" than what is conventional, routine, and generic in the relevant industry.
Evaluation of Independent Claims 1, 9, and 14
The physical elements of the claims consist entirely of generic, off-the-shelf computer architecture components. Claim 9 requires a standard "non-transitory computer readable storage medium" and a general "hardware processor." Claim 14 recites a generic "electronic interface," "at least one processor," and a standard "memory." The steps performed by these processing assets are stated in high-level functional prose: determining features, creating sub-models, generating prototype exemplars, and quantifying accuracy. The claims contain no specialized programming logic or physical hardware adaptations. When the abstract mathematical logic is stripped away, nothing remains except instructions to take standard processing machinery and use it to execute routine functional mathematical calculations. Instructing an ordinary computer to perform math or automate a mental process does not supply an inventive concept (see Alice, 573 U.S. at 218).
Evaluation of Dependent Claims
The dependent claims add further parameters, metadata constraints, or functional data processing methods, but they fail to elevate the overall combination to a patent-eligible inventive concept: Claims 3, 4, 11, and 16 specify that the input data types are "satellite images" and that the parameters include "spatial extent, NIIRS, off-nadir angle, SNR, and/or cloud coverage." These limitations simply specify the data parameters being fed into the abstract calculation. Restricting the abstract idea to specific types of data feeds or data contexts does not form an inventive step.Claims 5 and 12 introduce a secondary abstract mathematical tool: "defining the set of one or more performance metrics... as a regression." This replaces one statistical calculation with a different mathematical approach (regression mapping), which is itself a mathematical exception. Adding one abstract concept on top of another abstract concept does not yield an inventive concept. Claims 6, 7, 8, and 13 expand on the evaluation scenarios (e.g., changing the input space to all valid datasets or testing the model across different application domains or geospatial/geotemporal image types). These represent conceptual analytical testing methods that do not change the core generic computer system logic. Claim 17 details that the features are determined by receiving a "user-specified list" (a basic manual human entry step) or by "running a residual network feature extractor." Utilizing a routine machine learning structure like a residual network (ResNet) to extract data points for a formula is an entry-level, conventional application of AI tools. Claim 18 states that the prototype exemplars are created "using synthetics." Generating synthetic simulation data is a well-known, conventional technique within artificial intelligence testing frameworks and lacks an inventive step. Claims 19 and 20 specify that the objects are image collections parameterized on the features or involve a "non-linear mapping" such as Kernel PCA. Applying conventional mathematical data-transformation techniques (non-linear mapping) to standard image attributes amounts to routine, well-known data engineering practices.
Neither the elements of the independent claims nor the added restrictions of the dependent claims, whether viewed individually or in an ordered combination, contribute an inventive concept. The claims simply automate a series of mathematical and analytical calculations using conventional hardware tools. Accordingly, Claims 1, 3–9, 11–14, and 16–20 are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4-6, 9, 11, 12, 14, 16, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pai et al. (US 20200234158 A1; hereinafter “PAI”) in view of Paglieroni et al. (US 20160260222 A1; hereinafter “PAGLIERONI“).
Regarding claim 1, PAI teaches a method of quantifying accuracy of a prediction model that has been trained to identify objects in images on a data set parameterized by a plurality of features, the model operating over an input space in connection with the features (PAI source describes analyzing machine-learning models (e.g., neural networks) that provide data-driven predictions. Specifically, the source discusses applications such as self-driving vehicles "that use machine-learning to identify objects (e.g., other vehicles or pedestrians)". The system maps the features of these data points to a multi-dimensional "feature space" (i.e., an input space). Furthermore, the source evaluates the model's accuracy and behavior using calculations like mean squared error, para. 0025-35,83-88), the method comprising:
determining which of the plurality of features are strongly correlated with performance of the model (PAI determines the impact and sensitivity of the features within the machine-learning model. The system calculates local directions of sensitivity to act as a gradient, which allows it to "rank order directions (e.g., features of the prototypes) by sensitivity" and determine how features impact outputs, para. 0021-25,91-95);
based on the features determined to be strongly correlated with performance of the model, creating a plurality of parameterized sub-models that, in aggregate, approximate the input space (PAI partitions the input feature space into "epsilon balls" (regions) or "neighborhoods". The system assesses regions within the feature space where data points behave similarly and are covered by prototypes, effectively treating these local regions as sub-approximations of the overall input space, para. 0048,73-78);
generating prototype exemplars for each of the created sub-models, the prototype exemplars for each created sub-model being objects to which the model can be applied to result in a match with the respective sub-model (PAI identifies "prototypes" (representative exemplars) from the dataset that represent other data points for which the machine-learning model behaves similarly. The system selects prototypes that cover the data points falling within a specific epsilon ball size, ensuring that the prototypes match the targeted outputs and features of their local space, para. 0073-78); and
quantifying the accuracy of the model using the generated prototype exemplars (PAI calculates statistical metrics like the mean squared error (MSE) of the predictions across the prototypes to determine how well the model fits the data. Specifically, it states that "models for which the prototypes fit well with a low mean squared error have a high interpretability", para. 0083).
PAI is silent to teaching that wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are representative of the input space, approximating a variance of the model on a new data set as:(a) a sum of a set of one or more performance metrics for the model on each of the prototype exemplars squared multiplied by a probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) a square of the sum of the set of the one or more performance metrics for the model on each of the prototype exemplars, multiplied by the probability of the respective prototype exemplar matching its respective sub-model.
In the same field of endeavor, PAGLIERONI teaches a method wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are representative of the input space, approximating a variance of the model on a new data set as:(a) a sum of a set of one or more performance metrics for the model on each of the prototype exemplars squared multiplied by a probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) a square of the sum of the set of the one or more performance metrics for the model on each of the prototype exemplars, multiplied by the probability of the respective prototype exemplar matching its respective sub-model (PAGLIERONI teaches the mathematical identity for calculating standard statistical variance on the exemplars within its classes effectively acts as a probability weight resulting in (sum of squares × prob) - (square of sum × prob), the source uses this formula to compute within-class variance for the training set to find an optimal threshold boundary, para. 0047-55).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI with the teaching of PAGLIERONI in order to enable object recognition under various environment (PAGLIERONI, para. 0003).
Regarding claim 4, the combination of PAI and PAGLIERONI teaches the method of claim 1 claim 2, wherein the quantified accuracy reflects the variance expected performance of the model identifying an object of a given type from new and/or unseen images (PAI states that machine-learning models can "approximate unknown functions" and "generate data-driven predictions or decisions from the known input data". In the context of images, it explicitly describes models that "use machine-learning to identify objects (e.g., other vehicles or pedestrians)". By analyzing how the model behaves on test points relative to prototype examples to "estimate a model prediction" and determine how it reacts to "noise, or richness of the model," the quantified accuracy inherently reflects the model's expected performance on new or unseen data, para. 0025,65-68).
Regarding claim 5, the combination of PAI and PAGLIERONI teaches the method of claim 1, wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are completely parameterized by the features:
defining the set of one or more performance metrics for the model as a regression on the model for the prototype exemplars; and (i) approximating the variance of the model on the new data set as a sum of a regression on each prototype exemplar multiplied by the probability of the respective prototype exemplar matching its respective sub-model; and/or (ii) approximating the variance of the model on the new data set as (a) the sum of the regression on each prototype exemplar squared multiplied by the probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) the square of the sum of the regression on each prototype exemplar multiplied by the probability of the respective prototype exemplar matching its respective sub-model (PAGLIERONI teaches the mathematical identity for calculating standard statistical variance on the exemplars within its classes effectively acts as a probability weight resulting in (sum of squares × prob) - (square of sum × prob), the source uses this formula to compute within-class variance for the training set to find an optimal threshold boundary, para. 0047-55).
Regarding claim 6, the combination of PAI and PAGLIERONI teaches the method of claim 1, wherein the input space represents all valid data sets to which the model can be applied (PAI teaches the dataset to a "feature space" (i.e., an input space) where it "maps each feature vector for all of the data points to a corresponding point in the three-dimensional feature space". Since this multi-dimensional space reflects all the features of the data points, it effectively operates as a space representing all valid data points (and therefore valid data sets) to which the particular machine-learning model can be applied, para. 0025,43).
Regarding claim 9, PAI teaches a non-transitory computer readable storage medium tangibly storing instructions that (PAI, fig. 7,8, para. 0105), when executed by at least one hardware processor of a computing system, quantify accuracy of a prediction model that has been trained to identify objects in images on a data set parameterized by a plurality of features and that operates over an input space in connection with the features, by performing functionality comprising:
determining which of the plurality of features are strongly correlated with performance of the model (PAI determines the impact and sensitivity of the features within the machine-learning model. The system calculates local directions of sensitivity to act as a gradient, which allows it to "rank order directions (e.g., features of the prototypes) by sensitivity" and determine how features impact outputs, para. 0021-25,91-95);
based on the features determined to be strongly correlated with performance of the model, creating a plurality of parameterized sub-models that, in aggregate, approximate the input space (PAI partitions the input feature space into "epsilon balls" (regions) or "neighborhoods". The system assesses regions within the feature space where data points behave similarly and are covered by prototypes, effectively treating these local regions as sub-approximations of the overall input space, para. 0048,73-78);
generating prototype exemplars for each of the created sub-models, the prototype exemplars for each created sub-model being objects to which the model can be applied to result in a match with the respective sub-model (PAI identifies "prototypes" (representative exemplars) from the dataset that represent other data points for which the machine-learning model behaves similarly. The system selects prototypes that cover the data points falling within a specific epsilon ball size, ensuring that the prototypes match the targeted outputs and features of their local space, para. 0073-78); and
quantifying the accuracy of the model using the generated prototype exemplars (PAI calculates statistical metrics like the mean squared error (MSE) of the predictions across the prototypes to determine how well the model fits the data. Specifically, it states that "models for which the prototypes fit well with a low mean squared error have a high interpretability", para. 0083).
PAI is silent to teaching that wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are representative of the input space, approximating a variance of the model on a new data set as:(a) a sum of a set of one or more performance metrics for the model on each of the prototype exemplars squared multiplied by a probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) a square of the sum of the set of the one or more performance metrics for the model on each of the prototype exemplars, multiplied by the probability of the respective prototype exemplar matching its respective sub-model.
In the same field of endeavor, PAGLIERONI teaches a device wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are representative of the input space, approximating a variance of the model on a new data set as:(a) a sum of a set of one or more performance metrics for the model on each of the prototype exemplars squared multiplied by a probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) a square of the sum of the set of the one or more performance metrics for the model on each of the prototype exemplars, multiplied by the probability of the respective prototype exemplar matching its respective sub-model (PAGLIERONI teaches the mathematical identity for calculating standard statistical variance on the exemplars within its classes effectively acts as a probability weight resulting in (sum of squares × prob) - (square of sum × prob), the source uses this formula to compute within-class variance for the training set to find an optimal threshold boundary, para. 0047-55).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI with the teaching of PAGLIERONI in order to enable object recognition under various environment (PAGLIERONI, para. 0003).
Regarding claims 11 and 12, the dependent claim is interpreted and rejected for the same reasons as set forth above in claims 4 and 5, respectively.
Regarding claim 14, PAI teaches a system for quantifying accuracy of a prediction model that has been trained to identify objects in images on a data set parameterized by a plurality of features, the model operating over an input space in connection with the features, the system comprising:
an electronic interface over which the model is received (PAI, fig. 8); and
processing resources including at least one processor and a memory coupled thereto (PAI, fig. 8), the processing resources being configured to at least:
determine which of the plurality of features are strongly correlated with performance of the model (PAI determines the impact and sensitivity of the features within the machine-learning model. The system calculates local directions of sensitivity to act as a gradient, which allows it to "rank order directions (e.g., features of the prototypes) by sensitivity" and determine how features impact outputs, para. 0021-25,91-95);
based on the features determined to be strongly correlated with performance of the model, create a plurality of parameterized sub-models that, in aggregate, approximate the input space (PAI partitions the input feature space into "epsilon balls" (regions) or "neighborhoods". The system assesses regions within the feature space where data points behave similarly and are covered by prototypes, effectively treating these local regions as sub-approximations of the overall input space, para. 0048,73-78);
generate prototype exemplars for each of the created sub-models, the prototype exemplars for each created sub-model being objects to which the model can be applied to result in a match with the respective sub-model (PAI identifies "prototypes" (representative exemplars) from the dataset that represent other data points for which the machine-learning model behaves similarly. The system selects prototypes that cover the data points falling within a specific epsilon ball size, ensuring that the prototypes match the targeted outputs and features of their local space, para. 0073-78); and
quantify the accuracy of the model using the generated prototype exemplars (PAI calculates statistical metrics like the mean squared error (MSE) of the predictions across the prototypes to determine how well the model fits the data. Specifically, it states that "models for which the prototypes fit well with a low mean squared error have a high interpretability", para. 0083).
PAI is silent to teaching that wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are representative of the input space, approximating a variance of the model on a new data set as:(a) a sum of a set of one or more performance metrics for the model on each of the prototype exemplars squared multiplied by a probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) a square of the sum of the set of the one or more performance metrics for the model on each of the prototype exemplars, multiplied by the probability of the respective prototype exemplar matching its respective sub-model.
In the same field of endeavor, PAGLIERONI teaches a device wherein the quantifying of the accuracy of the model further comprises, provided that the prototype exemplars are representative of the input space, approximating a variance of the model on a new data set as:(a) a sum of a set of one or more performance metrics for the model on each of the prototype exemplars squared multiplied by a probability of the respective prototype exemplar matching its respective sub-model, subtracting (b) a square of the sum of the set of the one or more performance metrics for the model on each of the prototype exemplars, multiplied by the probability of the respective prototype exemplar matching its respective sub-model (PAGLIERONI teaches the mathematical identity for calculating standard statistical variance on the exemplars within its classes effectively acts as a probability weight resulting in (sum of squares × prob) - (square of sum × prob), the source uses this formula to compute within-class variance for the training set to find an optimal threshold boundary, para. 0047-55).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI with the teaching of PAGLIERONI in order to enable object recognition under various environment (PAGLIERONI, para. 0003).
Regarding claim 18, the combination of PAI and PAGLIERONI teaches the system of claim 14, wherein the prototype exemplars are generated using synthetics (PAI, fig. 7, 708, para. 0110-113).
Regarding claim 20, the combination of PAI and PAGLIERONI teaches the system of claim 14, wherein at least one of the features determined to be strongly correlated with part of the model includes a non-linear mapping based on a feature from the data set on which the prediction model is trained (PAI, fig. 7, 710, para. 0114-117).
Regarding claims 16 and 19, the dependent claim is interpreted and rejected for the same reasons as set forth above in claims 4 and 6, respectively.
Claim(s) 3, 7, 8, 13 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over PAI and PAGLIERONI as applied to claims 1, 9 and 14 above, and further in view of Marchisio et al. (US 20150371115 A1; hereinafter “MARCHISIO”).
Regarding claim 3, the combination of PAI and PAGLIERONI teaches the method of claim 1.
The combination of PAI and PAGLIERONI is silent to teaching that wherein the images are satellite images, and wherein the features include spatial extent, National Imagery Interpretability Rating Scale (NIIRS), off-nadir angle, signal-to-noise ratio (SNR), and/or cloud coverage values.
In the same field of endeavor, MARCHISIO teaches a method wherein the images are satellite images, and wherein the features include spatial extent, National Imagery Interpretability Rating Scale (NIIRS), off-nadir angle, signal-to-noise ratio (SNR), and/or cloud coverage values (MARCHISIO teaches the use of "VHR remotely-sensed multispectral Earth imagery" and image data collected by a "satellite imaging system 415". Features include off-nadir angle: MARCHISIO extracts features using metadata that includes "satellite parameters (e.g., off nadir satellite angles, satellite attitudes, solar elevation angles, etc.)" Features include spatial extent: MARCHISIO teaches region-level features related to morphology, including "shape properties (such as, area, perimeter, compactness, elongation, eccentricity, etc.), spatial relationships (such as, arrangement, distance, etc.)". (para. 0050-70) Features include cloud coverage values: MARCHISIO teaches generating training data related to weather-based occlusions, noting this is "particularly useful in the context of VHR remotely-sensed multispectral imagery where cloud cover or the like is present", para. 0145).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI and PAGLIERONI with the teaching of MARCHISIO in order to keep pace with the advance in image acquisition and facilitate new image technologies (MARCHISIO, para. 0005).
Regarding claim 7, the combination of PAI and PAGLIERONI teaches the method of claim 1.
The combination of PAI and PAGLIERONI is silent to teaching that wherein the data set on which the prediction model is trained is for a first application, the accuracy of the model is quantified for a data set for a second application, and the first and second applications are different from one another.
In the same field of endeavor, MARCHISIO teaches a method wherein the data set on which the prediction model is trained is for a first application, the accuracy of the model is quantified for a data set for a second application, and the first and second applications are different from one another (MARCHISIO teaches that "classification models 462 may have limited portability or accuracy when classifying images from different geographies and/or images that were acquired at different times (e.g., certain periods of the year corresponding to seasons or the like)". Because the system quantifies the accuracy/portability of a model trained on a first dataset (e.g., a specific geography or season) when applied to a second distinct dataset (e.g., a different geography or season), it teaches training on a first application and quantifying accuracy for a different, second application, para. 0142).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI and PAGLIERONI with the teaching of MARCHISIO in order to keep pace with the advance in image acquisition and facilitate new image technologies (MARCHISIO, para. 0005).
Regarding claim 8, the combination of PAI and PAGLIERONI teaches the method of claim 1.
The combination of PAI and PAGLIERONI is silent to teaching that wherein the data set on which the prediction model is trained is for a first geospatial and/or geotemporal image type, the accuracy of the model is quantified for a data set for a second geospatial and/or geotemporal image type, and the first and second geospatial and/or geotemporal image types are different from one another.
In the same field of endeavor, MARCHISIO teaches a method wherein the data set on which the prediction model is trained is for a first geospatial and/or geotemporal image type, the accuracy of the model is quantified for a data set for a second geospatial and/or geotemporal image type, and the first and second geospatial and/or geotemporal image types are different from one another (MARCHISIO teaches that models trained on a first geospatial or geotemporal dataset suffer accuracy drops when applied to a second, different geospatial or geotemporal dataset. Specifically, it notes the limited accuracy "when classifying images from different geographies [first and second geospatial image types] and/or images that were acquired at different times (e.g., certain periods of the year corresponding to seasons or the like) [first and second geotemporal image types]". To address this, the system generates targeted classification models based on geographic regions and temporal periods (seasons), inherently requiring the system to quantify accuracy across these different, distinctly defined geospatial and geotemporal boundaries, para. 0142).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI and PAGLIERONI with the teaching of MARCHISIO in order to keep pace with the advance in image acquisition and facilitate new image technologies (MARCHISIO, para. 0005).
Regarding claim 13, the dependent claim is interpreted and rejected for the same reasons as set forth above in claim 8.
Regarding claim 17, the combination of PAI and PAGLIERONI teaches the system of claim 14.
The combination of PAI and PAGLIERONI is silent to teaching that wherein the processing resources are further configured to at least determine which features are strongly correlated with performance of the model by receiving a user-specified list of one or more features and/or by running a residual network feature extractor.
In the same field of endeavor, MARCHISIO teaches a system wherein the processing resources are further configured to at least determine which features are strongly correlated with performance of the model by receiving a user-specified list of one or more features and/or by running a residual network feature extractor, and discloses that "a user developing a model may be capable of selecting one or more of the feature extraction modules 450 for execution during development of a classification model 462". Furthermore, it teaches that the user's "selection of a limited set of the feature extraction modules 450" prevents the generation of "unused and/or unhelpful feature data layers". Allowing a user to select specific feature extraction modules to omit unhelpful data layers and tailor the model's execution broadly equates to determining features correlated with the model's performance via a user-specified list, para. 0148-155).
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teaching of PAI and PAGLIERONI with the teaching of MARCHISIO in order to keep pace with the advance in image acquisition and facilitate new image technologies (MARCHISIO, para. 0005).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kaplow (20150186800) and Tesauro (20080154817) teach machine learning systems.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEN WU HUANG whose telephone number is (571)272-7852. The examiner can normally be reached Mon-Fri 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wesley Kim can be reached at (571) 272-7867. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEN W HUANG/Primary Examiner, Art Unit 2648
Prosecution Insights