Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The following is in response to the amendment filed on February 3, 2026.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 9 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “recently” in claim 9 is a relative term which renders the claim indefinite. The term “recently” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Examiner notes that the Specification never strictly defines the metes and bounds of “recently”, and only gives one example, which does not amount to a closed definition (Spec [0063]: “a recently stored feature, e.g., a feature stored in machine learning feature store 204 by user 220 in the past week”).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
Claims 1-10 are directed to a method. Therefore, each of the claims is directed to one of the four statutory categories of patent eligible subject matter.
Step 2A Prong 1:
Claim 1 recites:
“using[, by the device,] the first input to identify a feature among features [stored in a machine learning feature store]”; identifying is a mental process
“wherein the identifying is based on a probability of use of the feature [by the machine learning model] being higher than other probabilities of use of other features [stored in the machine learning feature store] so as to be a more relevant [machine learning] feature for the [machine learning] model under development by the user”; identifying based on a probability is a mental process
“determining[, by the device] a similarity between the first user profile and the second user profile”; determining a similarity is a mental process
“determining[, by the device], the probability of use of the feature based on the similarity between the first user profile and the second user profile and the feature selections identifying features selected by the second user;” determining a probability based on similarity is a mental process
“sorting[,by the device,] candidate features including the feature based on respective probabilities of use determined for the candidate features;” sorting features is a mental process
“responsive to the sorting, selecting the feature for the recommending based on the probability of use, the selecting to increase the probability of use for features selected for use and to reduce the probability of use for features not selected for use” selecting a feature is a mental process.
Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“receiving … a first input, from a user seeking to discover and select relevant machine learning feature for a machine learning model under development by the user, wherein the first input comprises data descriptive of the machine learning model under development by the user, the machine learning model being associated with a machine learning model domain, wherein the first input includes information indicative of the machine learning model domain”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g)
“by a device comprising a processor”; “by the device”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f)
“storing, by the device, a first user profile associated with the user, the first user profile comprising a user identifier and the first input” this amounts to insignificant extra solution activity, mere data storage, as per MPEP 2106.05(g)
“storing, by the device, a second user profile associated with a second user, the second user profile associated with feature selections identifying features selected by the second user;” this amounts to insignificant extra solution activity, mere data storage, as per MPEP 2106.05(g)
“stored in a machine learning feature store”; “for a machine learning model under development by the user”; “by the machine learning model”; “wherein the features stored in the machine learning feature store are associated with multiple different machine learning model domains, wherein each respective feature of the features is stored along with metadata indicating one or more associated machine learning domains”; these limitations amount to merely indicating a field of use or technological environment in which to apply a judicial exception; in other words, these merely describe a technological environment in which to practice the abstract idea of identifying a feature based on probability of the use of the feature
“recommending, by the device to the user, the feature for inclusion in a group of features used by the machine learning model under development by the user”; this amounts to insignificant extra solution activity, mere data outputting, as per MPEP 2106.05(g)
Step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“receiving … a first input, from a user seeking to discover and select relevant machine learning feature for a machine learning model under development by the user, wherein the first input comprises data descriptive of the machine learning model under development by the user, the machine learning model being associated with a machine learning model domain, wherein the first input includes information indicative of the machine learning model domain”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g); furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”)
“by a device comprising a processor”; “by the device”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f)
“storing, by the device, a first user profile associated with the user, the first user profile comprising a user identifier and the first input” this amounts to insignificant extra solution activity, mere data storage, as per MPEP 2106.05(g)
“storing, by the device, a second user profile associated with a second user, the second user profile associated with feature selections identifying features selected by the second user;” this amounts to insignificant extra solution activity, mere data storage, as per MPEP 2106.05(g)
“stored in a machine learning feature store”; “for a machine learning model under development by the user”; “by the machine learning model”; “wherein the features stored in the machine learning feature store are associated with multiple different machine learning model domains, wherein each respective feature of the features is stored along with metadata indicating one or more associated machine learning domains”; these limitations amount to merely indicating a field of use or technological environment in which to apply a judicial exception; in other words, these merely describe a technological environment in which to practice the abstract idea of identifying a feature based on probability of the use of the feature
“recommending, by the device to the user, the feature for inclusion in a group of features used by the machine learning model under development by the user”; this amounts to insignificant extra solution activity, mere data outputting, as per MPEP 2106.05(g); furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”)
Dependent Claims
Claims 2-10 are also rejected under 35 USC 101 for the following reasons:
Claim 2 recites:
“wherein using the first input to identify the feature comprises using the first input to determine the probability of use of the feature”; determining probability is a mental process
Claim 3 recites:
“wherein using the first input to identify the feature comprises using the first input to identify multiple features, comprising the feature, among the features stored in the machine learning feature store, wherein multiple probabilities of use, comprising the probability of use, of the multiple features by the machine learning model are higher than the other probabilities of use of the other features stored in the machine learning feature store”; identifying features with higher probabilities is a mental process
“wherein the recommending comprises recommending the multiple features for inclusion in the group of features used by the machine learning model”; this amounts to insignificant extra solution activity, mere data outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
“receiving, by the device, feature selections from among the multiple features”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
“storing, by the device, the feature selections for subsequent probability of use determinations associated with the multiple features”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“iv. Storing and retrieving information in memory”) under Step 2B
Claim 4 recites:
“receiving, by the device, feature importance information indicating an importance of the feature determined by the machine learning model”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
“storing, by the device, the feature importance information for subsequent probability of use determinations associated with the feature”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“iv. Storing and retrieving information in memory”) under Step 2B
Claim 5 recites:
“receiving, by the device, a second input, wherein the second input comprises a feature search input”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
“searching[, by the device,] the features stored in the machine learning feature store to identify search results, the search results comprising result features associated with the feature search input”; searching is a mental process
“sorting[, by the device,] the search results based on respective probabilities of use of the search results by the machine learning model”; sorting based on probabilities is a mental process
“by the device”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f) under Steps 2A Prong 2 and 2B
Claim 6 recites:
“storing, by the device, the second input for subsequent probability of use determinations”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“iv. Storing and retrieving information in memory”) under Step 2B
Claim 7 recites:
“wherein the probability of use of the feature is a first probability of use of a first feature, wherein the other probabilities of the other features are first other probabilities of first other features”; calculating a probability is a mental process
“storing, by the device, a user profile comprising a user identifier and the first input”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“iv. Storing and retrieving information in memory”) under Step 2B
“using[, by the device,] the user profile to identify a second feature among the features stored in the machine learning feature store”; identifying is a mental process
“by the device”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f) under Steps 2A Prong 2 and 2B
“wherein the second feature is identified based on a second probability of use of the second feature in connection with the user profile being higher than second other probabilities of use of second other features stored in the machine learning feature store”; identifying based on probability is a mental process
“recommending, by the device, the second feature in connection with the user profile”; this amounts to insignificant extra solution activity, mere data outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
Claim 8 recites:
“wherein the user profile is a first user profile”; this amounts to insignificant extra solution activity, mere data gathering and outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“iv. Storing and retrieving information in memory”) under Step 2B
“determining[, by the device,] the second probability of use of the second feature at least in part by evaluating a similarity of the first user profile and a second user profile, wherein the second feature is associated with the second user profile”; determining probability based on similarity is a mental process
“by the device”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f) under Steps 2A Prong 2 and 2B
Claim 9 recites:
“recommending, by the device, a recently stored feature in connection with the user profile”; this amounts to insignificant extra solution activity, mere data outputting, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
Claim 10 recites:
“wherein the data descriptive of the machine learning model associated with the machine learning model domain comprises at least one of: a first indication of the machine learning model domain; a second indication of whether the group of features used by the machine learning model comprises real-time features available in real-time; a third indication of a target associated with the machine learning model; and a fourth indication of which features are of interest in connection with the machine learning model”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
Claims 16-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1:
Claims 16-20 are directed to a non-transitory machine-readable medium. Therefore, each of the claims is directed to one of the four statutory categories of patent eligible subject matter.
Step 2A Prong 1:
Claim 16 recites:
“determining a probability of selection and use of a feature in the machine learning model under development by the user based on the user profile data” determining a probability is a mental process.
“using the data descriptive of the machine learning model the feature importance information, and the probability of selection and use, identifying recommended features among features [stored in a machine learning feature store, forming candidate features, wherein the recommended features are represented in the machine learning model domain, wherein the recommended features are selected so as to be a more relevant machine learning feature for the machine learning model under development]; identifying is a mental process
“filtering the candidate features based on metadata indicating one or more associated machine learning model domains sored for respective features and based on the information identifying the machine learning model domains” filtering features is a mental process
“selecting a candidate feature from the candidate features based on the filtering, forming a recommended feature; and” selecting a feature is a mental process
“recommending the recommended feature for inclusion in a group of features to be used by the machine learning model, the recommending to increase the probability of use for features selected for use and to reduce the probability of use for features not selected for use” recommending the feature is a mental process
Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements are as follows:
“a non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f)
“receiving feature importance information determined by machine learning models, wherein the feature importance information comprises feature importance information associated with multiple features, and wherein the multiple features are from multiple machine learning model domains”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g)
“receiving, from a user seeking to discover and select relevant machine learning features for a machine learning model under development by the user, data descriptive of the machine learning model associated with a machine learning model domain of the multiple machine learning model domains, wherein the receiving the data comprises receiving information identifying the machine learning model domain”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g)
“storing user profile data associated with a user identity, the user profile data comprising at least search history data associated with a user identity” this amounts to insignificant extra solution activity, mere data storage, as per MPEP 2106.05(g)
“stored in a machine learning feature store, wherein the recommended feature is represented in the machine learning model domain, wherein the recommended feature is selected so as to be a more relevant machine learning feature for the machine learning model under development”; these limitations amount to merely indicating a field of use or technological environment in which to apply a judicial exception; in other words, these merely describe a technological environment in which to practice the abstract idea of identifying a feature based on probability of the use of the feature
Step 2B:
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements are as follows:
“a non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations”; these limitations amount to nothing more than an instruction to apply the abstract idea using a generic computer as per MPEP 2106.05(f)
“receiving feature importance information determined by machine learning models, wherein the feature importance information comprises feature importance information associated with multiple features, and wherein the multiple features are from multiple machine learning model domains”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g); furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”)
“storing user profile data associated with a user identity, the user profile data comprising at least search history data associated with a user identity” this amounts to insignificant extra solution activity, mere data storage, as per MPEP 2106.05(g)
“receiving, from a user seeking to discover and select relevant machine learning features for a machine learning model under development by the user, data descriptive of the machine learning model associated with a machine learning model domain of the multiple machine learning model domains, wherein the receiving the data comprises receiving information identifying the machine learning model domain”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g); furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”)
“stored in a machine learning feature store, wherein the recommended feature is represented in the machine learning model domain, wherein the recommended feature is selected so as to be a more relevant machine learning feature for the machine learning model under development”; these limitations amount to merely indicating a field of use or technological environment in which to apply a judicial exception; in other words, these merely describe a technological environment in which to practice the abstract idea of identifying a feature based on probability of the use of the feature
over a network”)
Dependent Claims
Claims 17-20 are also rejected under 35 USC 101 for the following reasons:
Claim 17 recites:
“wherein using the data and the feature importance information to identify the recommended feature comprises determining a probability of use of the recommended feature in the machine learning model based on the feature importance information”; determining probability is a mental process
Claim 18 recites:
“wherein determining the probability of use of the recommended feature in the machine learning model is further based on user profile data associated with the data descriptive of the machine learning model.”; determining probability is a mental process
Claim 19 recites:
“wherein the user profile data is associated with a user identity, and wherein the user profile data comprises search history data associated with the user identity.”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
Claim 20 recites:
“wherein the data descriptive of the machine learning model comprises at least one of: a first indication of the machine learning model domain; a second indication of whether a group of features used by the machine learning model comprises real-time features that consume real-time data; a third indication of a target associated with the machine learning model; and a fourth indication of ones of the features that are of interest for inclusion in the machine learning model”; this amounts to insignificant extra solution activity, mere data gathering, as per MPEP 2106.05(g) under Step 2A Prong 2; furthermore, this amounts to well-understood, routine, and conventional activity as per MPEP 2106.05(d) (“i. Receiving or transmitting data over a network”) under Step 2B
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6 and 10-15 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2020/0210881 A1; hereinafter “Wu”) in view of Misra et al. (US 2018/0357511 A1; hereinafter “Misra”) and “Build a Recommendation Engine With Collaborative Filtering”, hereinafter “Ajitsaria”.
As per Claim 1, Wu teaches a method, comprising:
receiving, by a device comprising a processor (Wu [0007]: “a processor configured to receive data of an asset included in a target domain and information about an evaluation attribute associated with the asset in the target domain.”)
a first input, from a user seeking to discover and select relevant machine learning feature for a machine learning model under development by the user (Wu [0038]: “The system may auto-recommend features or a feature set based on input data received from the user. The system herein can be used to streamline the process from raw data to predictive model.”)
wherein the first input comprises data descriptive of the machine learning model under development by the user (Wu [0005]: “Accordingly, the example embodiments greatly expand the amount of features available to users (e.g., data scientists, machine learning engineers, etc.) thereby enabling the creation of potentially more accurate, efficient, and robust machine learning models.” Wu [0049]: “In this example, the host platform 330 receives input data from the target domain which in this example is the wind turbine domain. The input data may include raw data as well as metrics to be used to evaluate a machine learning model designed for predicting information from the raw data.”)
the machine learning model being associated with a machine learning model domain, wherein the first input includes information indicative of the machine learning model domain (Wu [0049]: “In this example, the host platform 330 receives input data from the target domain which in this example is the wind turbine domain.”)
using, by the device, the first input to identify a feature among features stored in a machine learning feature store (Wu [0054]: “In 430, the method may include determining predictive features in the received data based on previously used predictive features stored in the cross-domain feature store which is associated with machine learning models in a different domain and the evaluation attributes”)
wherein the features stored in the machine learning feature store are associated with multiple different machine learning model domains, wherein each respective feature of the features is stored along with metadata indicating one or more associated machine learning domains (Wu [0054]: “In 430, the method may include determining predictive features in the received data based on previously used predictive features stored in the cross-domain feature store which is associated with machine learning models in a different domain and the evaluation attributes.” Wu [0027]: “As described in various examples herein, data may include a raw collection of related values of an asset or a process/operation including the asset, for example, in the form of a stream (in motion) or in a data storage system (at rest). Individual data values may include descriptive metadata as to a source of the data and an order in which the data was received, but may not be explicitly correlated.”)
wherein the identifying is based on a [probability of] use of the feature by the machine learning model being higher than [other probabilities of] use of other features stored in the machine learning feature store so as to be a more relevant machine learning feature for the machine learning model under development by the user (Wu [0054]: “In 430, the method may include determining predictive features in the received data based on previously used predictive features stored in the cross-domain feature store which is associated with machine learning models in a different domain and the evaluation attributes.” Wu [0038]: “Furthermore, the system can continue to monitor performance of a model over time to thereby ensure that most optimal features are still relevant or to suggest new features that may be more relevant.”
Examiner notes that here, Wu discloses identifying more relevant features, but does not explicitly disclose probability.)
recommending, by the device to the user, the feature for inclusion in a group of features used by the machine learning model under development by the user (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.”)
However, Wu does not teach
Storing, by the device, a first user profile associated with the user, the first user profile comprising a user identifier and the first input;
Storing, by the device, a second user profile associated with a second user, the second user profile associated with feature selections identifying features selected by the second user;
wherein the identifying is based on a probability of use of the feature by the machine learning model being higher than other probabilities of use of other features stored in the machine learning feature store so as to be a more relevant machine learning feature for the machine learning model under development by the user;
wherein the recommending comprises:
determining, by the device, a similarity between the first user profile and the second user profile;
determining, by the device, the probability of use of the feature based on the similarity between the first user profile and the second user profile and the feature selections identifying features selected by the second user;
sorting, by the device, candidate features including the feature based on respective probabilities of use determined for the candidate features; and
responsive to the sorting, selecting the feature for the recommending based on the probability of use, the selecting to increase the probability of use for features selected for use and to reduce the probability of use for features not selected for use.
Misra teaches wherein the identifying is based on a probability of use of the feature by the machine learning model being higher than other probabilities of use of other features stored in the machine learning feature store so as to be a more relevant machine learning feature for the machine learning model under development by the user (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance.)
Misra is analogous art because it is in the field of endeavor of assisted machine learning development (Misra [0011]: “a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use”). It would have been obvious before the effective filing date of the claimed invention to combine the feature engineering with relevant feature identification of Wu with the likelihood-based relevance score of Misra. One of ordinary skill in the art would have been motivated to do so in order to assist users and save time on developing models (Misra [0011]: “However, manually selecting the machine learning technique, features, and relevance scores … may require that the developer possess specialized knowledge regarding a field associated with the corpus of digital objects to be analyzed … Moreover, the developer may be required to have specialized knowledge of machine learning techniques and features. Additionally, training a model for use in an analytics application may often involve significant iterations and testing to identify a well-trained model. Furthermore, custom selection of machine learning techniques, features, and relevance scores may be time-consuming, error prone, and resource intensive.”)
The combination of Wu and Misra does not appear to explicitly disclose:
Storing, by the device, a first user profile associated with the user, the first user profile comprising a user identifier and the first input;
Storing, by the device, a second user profile associated with a second user, the second user profile associated with feature selections identifying features selected by the second user;
wherein the identifying is based on a probability of use of the feature by the machine learning model being higher than other probabilities of use of other features stored in the machine learning feature store so as to be a more relevant machine learning feature for the machine learning model under development by the user;
wherein the recommending comprises:
determining, by the device, a similarity between the first user profile and the second user profile;
determining, by the device, the probability of use of the feature based on the similarity between the first user profile and the second user profile and the feature selections identifying features selected by the second user;
sorting, by the device, candidate features including the feature based on respective probabilities of use determined for the candidate features; and
responsive to the sorting, selecting the feature for the recommending based on the probability of use, the selecting to increase the probability of use for features selected for use and to reduce the probability of use for features not selected for use.
Ajitsaria teaches:
Storing, by the device, a first user profile associated with the user, the first user profile comprising a user identifier and the first input; (Section “The dataset”, teaches a profile for each user that identifies the user and an input (viewing, etc.)
Storing, by the device, a second user profile associated with a second user, the second user profile associated with feature selections identifying features selected by the second user; (Section “The dataset”, teaches a profile of users that have reacted (selecting) to items)
wherein the recommending comprises:
determining, by the device, a similarity between the first user profile and the second user profile; (Section “Steps Involved in Collaborative Filtering”, discloses determining similarities between user profiles)
determining, by the device, the probability of use of the feature based on the similarity between the first user profile and the second user profile and the feature selections identifying features selected by the second user; (Section “User-Based vs Item-Based Collaborative Filtering”, discloses determining and item ranking based on the similarity of the user profiles)
sorting, by the device, candidate features including the feature based on respective probabilities of use determined for the candidate features; and responsive to the sorting, selecting the feature for the recommending based on the probability of use, the selecting to increase the probability of use for features selected for use and to reduce the probability of use for features not selected for use. (Section “User-Based vs Item-Based Collaborative Filtering”, discloses sorting an item based on the user profiles and the item that has the highest ranking would be the one recommended based on probability of like (use))
It would have been obvious before the effective filing date of the claimed invention to combine the feature engineering with relevant feature identification of Wu with the likelihood-based relevance score of Misra and the collaborative filtering of AJitsaria. One of ordinary skill in the art would have been motivated to do so in order to allow for accurately recommending features for use based on user profiles. (Section “User-Based vs Item Based Collaborative Filtering”)
As per Claim 2, the combination of Wu, Misra and Ajitsaria teaches the method of claim 1. Misra further teaches wherein using the first input to identify the feature comprises using the first input to determine the probability of use of the feature (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Misra with Wu for at least the reasons recited in the rejection to Claim 1.
As per Claim 3, the combination of Wu, Misra and Ajitsaria teaches the method of claim 1. Wu further teaches wherein using the first input to identify the feature comprises using the first input to identify multiple features, comprising the feature, among the features stored in the machine learning feature store (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.”)
wherein the recommending comprises recommending the multiple features for inclusion in the group of features used by the machine learning model, and the method further comprising: receiving, by the device, feature selections from among the multiple features; (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.” Examiner notes that here “recommended … to be used by the developer” suggests that the developer may select the recommended features, and thus the system would receive such selections.)
and storing, by the device, the feature selections for subsequent [probability] of use determinations associated with the multiple features (Wu [0021]: “When new predictive features are determined for an asset, the newly determined features may be propagated or otherwise fed to pre-existing modules for continuously increasing the cross-domain feature store implemented therein with the new features.”)
However, Wu does not teach wherein multiple probabilities of use, comprising the probability of use, of the multiple features by the machine learning model are higher than the other probabilities of use of the other features stored in the machine learning feature store
Misra teaches wherein multiple probabilities of use, comprising the probability of use, of the multiple features by the machine learning model are higher than the other probabilities of use of the other features stored in the machine learning feature store (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Misra with Wu for at least the reasons recited in the rejection to Claim 1.
As per Claim 4, the combination of Wu, Misra and Ajitsaria teaches the method of claim 1. Wu further teaches further comprising: receiving, by the device, feature importance information indicating an importance of the feature determined by the machine learning model (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.” Examiner notes that here “recommended … to be used by the developer” suggests that the developer may select the recommended features, and thus the system would receive such selections.)
and storing, by the device, the feature importance information for subsequent probability of use determinations associated with the feature. (Wu [0021]: “When new predictive features are determined for an asset, the newly determined features may be propagated or otherwise fed to pre-existing modules for continuously increasing the cross-domain feature store implemented therein with the new features.”)
As per Claim 5, the combination of Wu, Misra and Ajitsaria teaches the method of claim 1. Wu further teaches further comprising: receiving, by the device, a second input, wherein the second input comprises a feature search input (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.” Wu [0021]: “When new predictive features are determined for an asset, the newly determined features may be propagated or otherwise fed to pre-existing modules for continuously increasing the cross-domain feature store implemented therein with the new features.” Here, Examiner notes that Wu discloses an ongoing process, where feature search inputs are used to keep growing the feature store, and thus teaches a second input that is a feature search input.)
searching, by the device, the features stored in the machine learning feature store to identify search results, the search results comprising result features associated with the feature search input (Wu [0054]: “In 430, the method may include determining predictive features in the received data based on previously used predictive features stored in the cross-domain feature store.”)
sorting, by the device, the search results [based on respective probabilities of use] of the search results by the machine learning model (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.” Here, Examiner notes that the “search results”, during the process of identifying relevant features, must be sorted to some extent in order to determine which ones go into the final list that are auto-recommended to be used by the developer.)
However, Wu does not teach sorting, by the device, the search results based on respective probabilities of use of the search results by the machine learning model
Misra teaches sorting, by the device, the search results based on respective probabilities of use of the search results by the machine learning model (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance. Furthermore, Examiner notes that the list recited by Misra above is sorted (0.8, 0.6, 0.4)).
As per Claim 6, the combination of Wu, Misra and Ajitsaria teaches the method of claim 5. Wu further teaches further comprising storing, by the device, the second input for subsequent probability of use determinations. (Wu [0021]: “When new predictive features are determined for an asset, the newly determined features may be propagated or otherwise fed to pre-existing modules for continuously increasing the cross-domain feature store implemented therein with the new features.”)
As per Claim 10, the combination of Wu, Misra and Ajitsaria teaches the method of claim 1. Wu further teaches wherein the data descriptive of the machine learning model associated with the machine learning model domain comprises at least one of: a first indication of the machine learning model domain; a second indication of whether the group of features used by the machine learning model comprises real-time features available in real-time; a third indication of a target associated with the machine learning model; and a fourth indication of which features are of interest in connection with the machine learning model. (Wu [0049] teaches, at the very least, a first indication of the machine learning model domain, and a third indication of a target: “In this example, the host platform 330 receives input data from the target domain which in this example is the wind turbine domain. The input data may include raw data as well as metrics to be used to evaluate a machine learning model designed for predicting information from the raw data.”)
As per Claim 11, Wu teaches a device, comprising: a processor; and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising: (Wu [0007]: “a processor configured to receive data of an asset included in a target domain and information about an evaluation attribute associated with the asset in the target domain.” Wu [0057]: “The storage device 540 is not limited to any particular storage device and may include any known memory device such as RAM, ROM, hard disk, and the like.”)
receiving, from a user seeking to discover and select relevant machine learning features for a machine learning model under development by the user, a feature search input, wherein the feature search input is associated with a user profile (Wu [0038]: “The system may auto-recommend features or a feature set based on input data received from the user. The system herein can be used to streamline the process from raw data to predictive model.” Wu [0049]: “In this example, the host platform 330 receives input data from the target domain which in this example is the wind turbine domain. The input data may include raw data as well as metrics to be used to evaluate a machine learning model designed for predicting information from the raw data.” Examiner notes that the broadest reasonable interpretation of a “user profile” includes a user’s preferences, such as the ones expressed by the inputs of “raw data as well as metrics”).
based on the feature search input, searching a machine learning feature store in order to identify search results, the machine learning feature store including recommendable machine learning features that may be relevant to a machine learning model of the user, the search results comprising features associated with the feature search input (Wu [0054]: “In 430, the method may include determining predictive features in the received data based on previously used predictive features stored in the cross-domain feature store which is associated with machine learning models in a different domain and the evaluation attributes”)
based on data associated with the user profile, determining respective [probabilities] of use of the search results, wherein each respective probability of use corresponds to a [probability] that the user will select and retrieve a recommended feature for use in the machine learning model of the user from among the search results so as to be a more relevant machine learning feature for the machine learning model under development by the user (Wu [0054]: “In 430, the method may include determining predictive features in the received data based on previously used predictive features stored in the cross-domain feature store which is associated with machine learning models in a different domain and the evaluation attributes.” Wu [0038]: “Furthermore, the system can continue to monitor performance of a model over time to thereby ensure that most optimal features are still relevant or to suggest new features that may be more relevant.”
Examiner notes that here, Wu discloses identifying more relevant features, but does not explicitly disclose probability.)
sorting the search results based on the respective [probabilities of] use of the search results (Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.” Here, Examiner notes that the “search results”, during the process of identifying relevant features, must be sorted to some extent in order to determine which ones go into the final list that are auto-recommended to be used by the developer.)
However, Wu does not teach based on data associated with the user profile, determining respective probabilities of use of the search results, wherein each respective probability of use corresponds to a probability that the user will select and retrieve a recommended feature for use in the machine learning model of the user from among the search results so as to be a more relevant machine learning feature for the machine learning model under development by the user; sorting the search results based on the respective probabilities of use of the search results receiving feature importance information associated with features used by a second user;
Storing the feature importance information in association with a second user profile for the second user;
Responsive to the receiving the feature importance information, determining the respective probabilities of user based on the feature importance information;
Responsive to the sorting of search results, presenting, via a search interface, the search results ordered based on the respective probabilities; and
presenting, via the search interface, additional recommendations comprising other features selected by users having interests similar to the feature search input, wherein a user selection increases the probability of use for features not selected for use.
Misra teaches based on data associated with the user profile, determining respective probabilities of use of the search results, wherein each respective probability of use corresponds to a probability that the user will select and retrieve a recommended feature for use in the machine learning model of the user from among the search results so as to be a more relevant machine learning feature for the machine learning model under development by the user; sorting the search results based on the respective probabilities of use of the search results (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance. Furthermore, Examiner notes that the list recited by Misra above is sorted (0.8, 0.6, 0.4)).
Misra is analogous art because it is in the field of endeavor of assisted machine learning development (Misra [0011]: “a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use”). It would have been obvious before the effective filing date of the claimed invention to combine the feature engineering with relevant feature identification of Wu with the likelihood-based relevance score of Misra. One of ordinary skill in the art would have been motivated to do so in order to assist users and save time on developing models (Misra [0011]: “However, manually selecting the machine learning technique, features, and relevance scores … may require that the developer possess specialized knowledge regarding a field associated with the corpus of digital objects to be analyzed … Moreover, the developer may be required to have specialized knowledge of machine learning techniques and features. Additionally, training a model for use in an analytics application may often involve significant iterations and testing to identify a well-trained model. Furthermore, custom selection of machine learning techniques, features, and relevance scores may be time-consuming, error prone, and resource intensive.”) The combination of Wu and Misra does not appear to explicitly disclose: Receiving feature importance information associated with features used by a second user;
Storing the feature importance information in association with a second user profile for the second user;
Responsive to the receiving the feature importance information, determining the respective probabilities of user based on the feature importance information;
Responsive to the sorting of search results, presenting, via a search interface, the search results ordered based on the respective probabilities; and
presenting, via the search interface, additional recommendations comprising other features selected by users having interests similar to the feature search input, wherein a user selection increases the probability of use for features not selected for use.
Ajitsaria teaches: Receiving feature importance information associated with features used by a second user; Storing the feature importance information in association with a second user profile for the second user; (Section “The Dataset”, discloses each item having a ranking (importance information) form a second user)
Responsive to the receiving the feature importance information, determining the respective probabilities of user based on the feature importance information; (Section “User-Based vs Item-Based Collaborative Filtering”, discloses item based rating based on another user profile)
Responsive to the sorting of search results, presenting, via a search interface, the search results ordered based on the respective probabilities; and presenting, via the search interface, additional recommendations comprising other features selected by users having interests similar to the feature search input, wherein a user selection increases the probability of use for features not selected for use. (Sections “Steps in Collaborative Filtering” and “When Can Collaborative Filtering be Used?”, discloses sorting and presenting items based on probabilities when the probability is based on similarities between users, this would increase the probability of like (use) of the items for the recommendations)
It would have been obvious before the effective filing date of the claimed invention to combine the feature engineering with relevant feature identification of Wu with the likelihood-based relevance score of Misra and the collaborative filtering of AJitsaria. One of ordinary skill in the art would have been motivated to do so in order to allow for accurately recommending features for use based on user profiles. (Section “User-Based vs Item Based Collaborative Filtering”)
As per Claim 12, the combination of Wu, Misra and Ajitsaria teaches the device of claim 11. Wu further teaches wherein the data associated with the user profile comprises model data descriptive of a machine learning model associated with a machine learning model domain. (Wu [0049]: “In this example, the host platform 330 receives input data from the target domain which in this example is the wind turbine domain. The input data may include raw data as well as metrics to be used to evaluate a machine learning model designed for predicting information from the raw data.”)
As per Claim 13, the combination of Wu and Misra teaches the device of claim 11. Wu further teaches wherein the user profile is a first user profile, wherein the data associated with the user profile is first data, and wherein determining the respective probabilities of use of the search results is further based on second data associated with a second user profile. (Wu [0048]: “FIG. 3B illustrates a process 300B of the system described herein determining a feature for a first domain (wind turbine) based on a previously used features in a second domain (aviation).” Examiner reiterates the explanation from Claim 11, that the broadest reasonable interpretation of a user profile is merely the user’s preferences as expressed by the user’s input, and if the recommended features are based on previously used features, then these were based on previous users’ models and metrics, and therefore second data associated with a second user profile.)
As per Claim 14, the combination of Wu, Misra and Ajitsaria teaches the device of claim 13. Wu further teaches wherein the second data comprises feature selections associated with the second user profile. (Wu [0048]: “FIG. 3B illustrates a process 300B of the system described herein determining a feature for a first domain (wind turbine) based on a previously used features in a second domain (aviation).” Examiner reiterates the explanation from Claim 11, that the broadest reasonable interpretation of a user profile is merely the user’s preferences as expressed by the user’s input, and if the recommended features are based on previously used features, then these were based on previous users’ models and metrics, and therefore second data associated with a second user profile. Also, Examiner notes that the “previously used” features were thus feature selections associated with the second user profile, which were previous user preferences.)
As per Claim 15, the combination of Wu and Misra teaches the device of claim 13. Wu further teaches wherein the second data associated with the second user profile comprises feature importance information determined by a machine learning model associated with the second user profile. (Wu [0048]: “FIG. 3B illustrates a process 300B of the system described herein determining a feature for a first domain (wind turbine) based on a previously used features in a second domain (aviation).” Examiner reiterates the explanation from Claim 11, that the broadest reasonable interpretation of a user profile is merely the user’s preferences as expressed by the user’s input, and if the recommended features are based on previously used features, then these were based on previous users’ models and metrics, and therefore second data associated with a second user profile. Also, Examiner notes that the “previously used” features were thus feature selections associated with the second user profile, which were previous user preferences, and amount to feature importance information since they were important enough to previous be used.)
Claims 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Wu and Misra and Ajitsaria, further in view of Ashlock et al. (US 2021/0350276 A1; hereinafter “Ashlock”)
As per Claim 7, the combination of Wu, Misra and Ajitsaria teaches the method of claim 1. Wu teaches recommending, by the device, the second feature in connection with the user profile ((Wu [0051]: “Although not shown in the example of FIG. 3A or 3B, the output of the system may be a list of auto-recommended features to be used by a developer based on the input data received via the user interface.”)
However, Wu does not teach wherein the probability of use of the feature is a first probability of use of a first feature, wherein the other probabilities of the other features are first other probabilities of first other features; further comprising: storing, by the device, a user profile comprising a user identifier and the first input; using, by the device, the user profile to identify a second feature among the features stored in the machine learning feature store, wherein the second feature is identified based on a second probability of use of the second feature in connection with the user profile being higher than second other probabilities of use of second other features stored in the machine learning feature store; and
Misra teaches wherein the probability of use of the feature is a first probability of use of a first feature, wherein the other probabilities of the other features are first other probabilities of first other features (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance.)
using, by the device, the user profile to identify a second feature among the features stored in the machine learning feature store, wherein the second feature is identified based on a second probability of use of the second feature in connection with the user profile being higher than second other probabilities of use of second other features stored in the machine learning feature store (Misra [0011]: “Often, a developer may be assigned to determine, for an analytics application (e.g., an application designed to provide information identifying meaningful patterns in data), which machine learning technique to use (e.g., a natural language processing technique, a computer vision technique, and/or the like), which features to use (e.g., a feature being a measurable property of a digital object, such as a numeric property, a string-based property, a graph-based property, and/or the like), and how much of a relevance score (e.g., weight) to assign each feature.” Misra [0030]: “By way of example, the measure of similarity between the first and second analytics applications may be a 0.75 on a 0 to 1 scale. A relevance score, for the first analytics application, may be determined for a machine learning technique used by the second analytics application as a function of the measure of similarity. For example, given the 0.75 measure of similarity, the analytics recommendation platform may determine that the machine learning technique used by the second analytics application has a 0.75 relevance score which, in this situation, may indicate a measure of likelihood that the machine learning technique is relevant to the first analytics application. The analytics recommendation platform may determine relevance scores for features in a similar manner. For example, the second analytics application may be associated with three different features, a relevance score of 0.8 for a first feature, a relevance score of 0.6 for a second feature, and a relevance score of 0.4 for a third feature. By combining (e.g., multiplying) the relevance scores associated with the features by the measure of similarity, the analytics recommendation platform may determine relevance scores associated with the three different features, for the first analytics application. For example, the analytics recommendation platform may determine that the first feature has a 0.6 relevance score (e.g., 0.8*0.75=0.6), the second feature has a 0.45 relevance score (e.g., 0.6*0.75=0.45), and the third feature has a 0.3 relevance score (e.g., 0.4*0.75).” Here, Misra discloses using probability (“likelihood”) as a measure of relevance.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Misra with Wu and Ajitsaria for at least the reasons recited in the rejection to Claim 1.
However, the combination does not teach further comprising: storing, by the device, a user profile comprising a user identifier and the first input
Ashlock teaches further comprising: storing, by the device, a user profile comprising a user identifier and the first input (Ashlock [0033]: “Stored data in the entity profile database 205 includes entity characteristic information and activity history. As referred to herein, “characteristic information” represents characteristics of an entity that is not directly related to the online document system 140 and “activity history” represents activities performed by the entity involving the online document system 140.”)
Ashlock is analogous art because it is in the field of endeavor of feature recommendation and machine learning (Ashlock, Abstract: “An online document system provides a recommendation for one or more features within the online document system to an entity. The online document system accesses a set of feature training data to train a machine learning model.”) It would have been obvious before the effective filing date of the claimed invention to combine the feature recommendation of Wu and Misra with the storage of user profiles of Ashlock. One of ordinary skill in the art would have been motivated to do so in order to improve feature recommendation accuracy (Ashlock [0005]: “The online document system may obtain information on how entities use features of the online document system to improve feature management. For example, a feature recommendation engine of the online document system may train a machine learning model with the feature use information to improve the accuracy of feature recommendations made by the online document system.”)
As per Claim 8, the combination of Wu, Misra, Ajitsaria and Ashlock teaches the method of claim 7. Ashlock further teaches wherein the user profile is a first user profile, and further comprising determining, by the device, the second probability of use of the second feature at least in part by evaluating a similarity of the first user profile and a second user profile, wherein the second feature is associated with the second user profile. (Ashlock [0037]: “In some embodiments, the featurization module 210 applies weights to training sets or feature vectors input to a machine learning model. For example, the featurization module 210 may determine degrees of similarities between entities and apply weights based on the determined degrees, weighting a more similar entity greater than a less similar entity.” Ashlock [0041]: “Additionally, or alternatively, the feature recommendation engine 220 may determine similarities based on comparisons of activity history.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ashlock with Wu, Misra and Ajitsaria for at least the reasons recited in the rejection to Claim 7.
As per Claim 9, the combination of Wu, Misra, Ajitsaria and Ashlock teaches the method of claim 7. Ashlock further teaches further comprising recommending, by the device, a recently stored feature in connection with the user profile. (Ashlock [0037]: “For example, the featurization module 210 encodes data that has been collected within the past year to improve the likelihood that outputs from the machine learning models reflect recent entity profiles and activity history.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Ashlock with Wu, Misra and Ajitsaria for at least the reasons recited in the rejection to Claim 7.
Response to Arguments
Claim Rejections - 35 USC § 101
With respect to claims 1 and 16:
Applicant argues “The amended claim clearly recites how an AI system itself may be technically improved. Claim 1 as amended recites “the selecting to increase the probability of use for features selected for use and to reduce the probability of use for features not selected for use”” Examiner respectfully disagrees that the amendment overcomes the 101 rejection. The amendment do show the improvement stated in the specification, however the improvement is recited as part of a limitation that has been identified as an abstract idea. Per MPEP 2106.05(a)(II): However, it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology and that the improvement needs to be provided by the one or additional elements.
Claim Rejections - 35 USC § 103
Applicant’s arguments have been considered and are moot in view of the new ground(s) of rejection.
Allowable Subject Matter
Claims 16-20 are allowed under prior art. The recitation of “filtering the candidate features based on metadata indicating one or more associated machine learning model domains stored for respective features and based on the information identifying the machine learning model domains; selecting a candidate feature from the candidate features based on the filtering, forming a recommended feature; and recommending the recommended feature for inclusion in a group of features to be used by the machine learning model, the recommending to increase the probability of use for features selected for use and to reduce the probability of use and to reduce the probability of use for features not selected for use.” has been found to be allowable under prior art.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIELA D REYES whose telephone number is (571)270-1006. The examiner can normally be reached Monday-Friday, 7:30 am -5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Wiley can be reached at (571) 272-4150. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights