Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Application
The following is a Final Office Action. In response to Examiner's communication on 12/29/2025, Applicant on 03/17/2026, amended Claims 1, 11-14. Claims 1-20 are now pending in this application and have been rejected below.
Response to Amendment
Applicant’s amendments are insufficient to overcome the 35 USC 112(b) rejections set forth in the previous action.
Applicants’ amendments are insufficient to overcome the 35 USC 101 rejections set forth in the previous action.
Applicants’ amendments render moot the 35 USC 103 rejections set forth in the previous action in view of new and updated grounds for rejection necessitated by Applicants’ amendments. Therefore, these rejections are withdrawn in view of the new grounds for rejection necessitated by Applicants’ amendments, as set forth below.
Response to Arguments – 35 USC § 112(b)
Applicant's arguments with respect to the 35 USC 112(b) rejections have been fully considered but they are not persuasive.
Applicant argues that the “.clip()” function would be apparent to one of ordinary skill in the art as a Python function. Examiner respectfully disagrees.
As .clip is a function in a plurality of programming languages, and specific libraries pertaining to mathematics and data science, the specific interpretation of .clip() as corresponding to the function in Python would not be apparent to one of ordinary skill in the art. This would also require the specification to support the usage of the NumPy package, as .clip() is not standard to Python but rather an implementation supported by an additional library. Further, as the mechanics of the implementation of .clip() can differ from one library and/or language to another, further clarification is required.
The rejections under 35 USC 112(b) have therefore been maintained below.
Response to Arguments – 35 USC § 101
Applicant's arguments with respect to the 35 USC 101 rejections have been fully considered but they are not persuasive.
Applicant firstly argues that limitations do not recite abstract ideas that could practically be performed by a human in the mind. Examiner respectfully disagrees. Firstly, the aggregation of different scores are a mathematical operation; analyzing the heart of Applicant’s claims with regard to taking different scores and combining them, it is clear that abstract ideas are present that are mentally performable by a human with pen and paper. Further, creating a score for a lead manages sales and marketing activities and the human behavior of marketers/advertisers and consumers, and further represents a Certain Method of Organizing Human Activity.
Examining the additional elements with respect to a practical application, if the basis for integration is improvement to technology, it cannot be said that what is claimed is a specific means of improving the training of models or deployment of artificial models; the mechanics of doing so are somewhat underspecified as “training” using "xGBoost and balanced random forest techniques". These techniques are well known in the art and only serve to generally link the idea of aggregating scores to using machine learning models. The distinction between what applicant has put forth in subject matter eligible cases and here is the specificity of the improvement to particular technology. In this context, what is claimed is not a novel method of training a machine learning model as Applicant asserts, but rather an application of the abstract idea of score aggregation with generic computing components.
The MPEP makes clear that "[m]ere automation of manual processes” is not an improvement in computer technology. See MPEP 2106.05(a). As in the claims at issue in Electric Power Group, the present claims are not focused on a specific improvement in computers or any other technology, but instead on certain independently abstract ideas that simply invokes computers as tools to implement the abstract idea. Electric Power Group, LLC v. Alstom S.A., et al., No. 2015-1778, slip op. at 8 (Fed. Cir. Aug. 1, 2016); MPEP 2106.05(a).
The rejections have therefore been updated to address the amendments and maintained below.
Response to Arguments – 35 USC § 103
Applicant' s arguments with respect to the rejections under 35 USC 103 have been considered but are moot in light of new grounds of rejections necessitated by applicant’s amendments.
Applicant’s arguments seem to be directed to the environment in which the teachings operate as opposed to the actual features of the inventions. The environment, ie sparse or dense data, does not preclude using the teachings of Chiao to teach the limitations of the claims, as the Broadest Reasonable Interpretation of the claims is not limited to be within the confines of such an environment. If Applicant’s claimed language were specified to capture this difference in data availability, that would be a relevant consideration to obviousness, but the notion that pooling is intended to address sparse data challenges is currently missing from the explicit language of the claims.
Regarding the employment of two or more machine learning models, see [0107] of Chiao, "In an example embodiment, lead prioritization can be determined based on results from multiple modeling methods, such as the modeling methods described above. The system and method of an example embodiment can generate a plurality of scores for each lead using a plurality of different ensemble machine learning techniques or processing models. Then, the system and method of an example embodiment can generate scores using a linear parametric machine learning technique or other type of modeling technique or processing model. The notion of using a plurality of different ensemble machine learning techniques inherently addresses using two or more.
Further arguments are rendered moot by the new grounds of rejection necessitated by Applicant’s amendments. Examiner respectfully notes them as outlined below.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 10, 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 10, 20 recite the limitation "s=s.clip(0,1)" in Line 4 of Claim 10 and Line 3 of Claim 20. The specification does not offer any definition of this “clip” function beyond its recitation in Equation 7 on Page 10 of Applicant’s specification, nor does it implicitly follow from the context of the Claims. For purpose of examination, this “clip” function has been understood to correspond to the Python“clip” function that limits a value to be within a prescribed range by changing its value to the lower bound if it is below that lower bound, changing its value to the upper bound if it exceeds that upper bound, and leaving its value unchanged if it is already within that range. However, as this would not necessarily be apparent to one of ordinary skill in the art, appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
101 Analysis – Step 1
The claims are directed to an apparatus and method. Therefore, the claims are directed to at least one of the four statutory categories.
101 Analysis – Step 2A
Regarding Prong 1 of the Step 2A analysis in the MPEP, the claims are to be analyzed to determine whether they recite subject matter that is directed to a judicial expectation, namely a law of nature, a natural phenomenon, or one of the follow groups of abstract ideas: a) mathematical concepts, b) certain methods of organizing human activity, and/or c) mental processes.
Independent Claim 1 includes limitations that recite an abstract idea and will henceforth be used as a representative claim for the 101 rejection until otherwise noted. Claim 1 recites:
A lead pooling and ranking system, comprising: at least one processor; and memory comprising a set of instructions, wherein the set of instructions is configured to cause the at least one processor to execute implementing a pooling technique enhancing and optimizing lead scoring machine learning techniques for a set of leads; generating a lead score for each configured rule; combining two or more machine learning (ML) scores from two one or more trained ML models and one or more rules based scores to create a unitary score for each corresponding lead in the set of leads, wherein the ML models are trained using xgBoost and balanced random forest techniques and generating a rank and rating for each lead in the set of leads.
The examiner submits that the foregoing bolded limitation(s) constitute an abstract idea because under its broadest reasonable interpretation, the claim recites mental processes that could be
performed by a human with a pen and paper, per the MPEP, merely
adapting them into the context of a technological environment with computing parts does not preclude them from being abstract. Further, the aggregation of certain scores amounts to mathematical relationships. The claim further recites Certain Methods of Organizing Human Activity, namely Commercial or Legal Interactions, as the scope of the claim pertains to organizing information to enhance the performance of sales.
Accordingly, the claim recites at least one abstract idea.
Claim 11 recites at least one abstract idea by virtue of reciting substantially similar limitations.
Claims 5-6, 8-10, 15-16, 18-20 further recite Mathematical Concepts, namely a Mathematical Calculation. The relationships recited in said claims reflect a mathematical formula that amounts to computing a weighted sum as well as the mechanics for doing so.
101 Analysis – Step 2A, Prong II
Regarding Prong II of the Step 2A analysis in the MPEP, the claims are to be analyzed to determine whether the claim, as a whole, integrates the abstract into practical application. As noted in the MPEP, it must be determined whether any additional elements in the claim beyond the judicial exception integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements, such as merely using a computer to implement an abstract idea, adding insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”):
A lead pooling and ranking system, comprising: at least one processor; and memory comprising a set of instructions, wherein the set of instructions is configured to cause the at least one processor to execute implementing a pooling technique enhancing and optimizing lead scoring machine learning techniques for a set of leads; generating a lead score for each configured rule; combining two or more machine learning (ML) scores from two one or more trained ML models and one or more rules based scores to create a unitary score for each corresponding lead in the set of leads, wherein the ML models are trained using xgBoost and balanced random forest techniques and generating a rank and rating for each lead in the set of leads.
For the following reason(s), the examiner submits that the above identified additional limitations do not integrate the above-noted abstract idea into a practical application.
As it pertains to Claim 1, the additional elements in the claims include “A lead pooling and ranking system, comprising: at least one processor; and memory comprising a set of instructions, wherein the set of instructions is configured to cause the at least one processor to execute”, “machine learning”, “two or more machine learning (ML) scores”, and “wherein the ML models are trained using xgBoost and balanced random forest techniques”.
When considered in view of the claim as a whole, the additional elements do not integrate the abstract idea into a practical application because the additional elements are generic computing components that are merely used as a tool to perform the recited abstract idea and/or do no more than generally link the use of the recited abstract idea to a particular technological environment or field of use under Step 2A Prong Two.
Thus, taken alone, the additional elements do not integrate the abstract idea into a practical application. Further, looking at the additional limitation(s) as an ordered combination or as a whole, the limitation(s) add nothing that is not already present when looking at the elements taken individually. For instance, there is no indication that the additional elements, when considered as a whole, reflect an improvement in the functioning of a computer or an improvement to another technology or technical field, apply or use the above-noted judicial exception to effect a particular treatment or prophylaxis for a disease or medical condition, implement/use the above-noted judicial exception with a particular machine or manufacture that is integral to the claim, effect a transformation or reduction of a particular article to a different state or thing, or apply or use the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is not more than a drafting effort designed to monopolize the exception (MPEP § 2106.05).
Accordingly, the additional limitation(s) does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing an abstract idea.
Claim 11 does not serve to integrate recited abstract ideas into a practical application by analogous reasoning.
Claims 3, 13 recite “a single artificial intelligence (AI) score”, “a fit model, an engagement model, and a semantic model”.
Claims 9, 19 recite “static, interest, semantic models”.
These limitations do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing an abstract idea.
Claims 2, 4-8, 10, 12, 14-18, 20 do not recite additional limitations beyond those found in claims from which they depend and therefore do not integrate the recited abstract ideas into a practical application.
101 Analysis – Step 2B
Regarding Step 2B of the MPEP, representative independent claim 1 does not include additional elements (considered both individually and as an ordered combination) that are sufficient to amount to significantly more than the judicial exception for the same reasons to those discussed above with respect to determining that the claim does not integrate the abstract idea into a practical application. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements amount to generic computing components that are merely used
as a tool to perform the recited abstract idea and/or do no more than
generally link the use of the recited abstract idea to a particular
technological environment or field of use. Further, looking at the additional
elements as an ordered combination adds nothing that is not already
present when considering the additional elements individually.
Claim 11 does not serve to integrate recited abstract ideas into a practical application or amount to significantly more by analogous reasoning.
Claims 3, 13 recite “a single artificial intelligence (AI) score”, “a fit model, an engagement model, and a semantic model”.
Claims 9, 19 recite “static, interest, semantic models”.
These limitations do not integrate the abstract idea into a practical application or amount to significantly more because they do not impose any meaningful limits on practicing an abstract idea.
Claims 2, 4-8, 10, 12, 14-18, 20 do not recite additional limitations beyond those found in claims from which they depend and therefore do not integrate the recited abstract ideas into a practical application or amount to significantly more.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 8-13, 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chiao(US 20160071118 A1) in view of Brannon(US 20220108222 A1) in further view of Robinson(US 20200311601 A1) in further view of Drapeau(US 20230026121 A1).
Claims 1, 11
As to Claim 1,
Chiao teaches:
A lead … and ranking system, comprising: at least one processor; and memory comprising a set of instructions, wherein the set of instructions is configured to cause the at least one processor to execute implementing a … technique enhancing and optimizing lead scoring machine learning techniques for a set of leads; combining two or more machine learning (ML) scores from two or more trained ML models ... to create a unitary score for each corresponding lead in the set of leads; and generating a rank and rating for each lead in the set of leads;
Hardware components are described in [0125-0126]. In [0107], "In an example embodiment, lead prioritization can be determined based on results from multiple modeling methods, such as the modeling methods described above. The system and method of an example embodiment can generate a plurality of scores for each lead using a plurality of different ensemble machine learning techniques or processing models. Then, the system and method of an example embodiment can generate scores using a linear parametric machine learning technique or other type of modeling technique or processing model. The system and method of the example embodiment can evaluate the results from each of the plurality of models and stack rank the list of leads based on a set of criteria (e.g., heuristics), including: 1) a sorting of the list from highest to lowest score based on the stronger of the plurality of ensemble machine learning methods; 2) a sorting of the list of leads from highest to lowest score based on the linear parametric machine learning model or other type of modeling technique; 3) an assignment of a composite score for each lead from 0-100 based on its percentile rank within the sorted list; and 4) a re-evaluation of the composite score for each lead relative to the scores for each lead from the plurality of individual modeling methods. If the composite score for each lead is at least as strong as the strongest individual model score, the system and method can use the composite score as the final score for the lead. The details of an example embodiment are described below". Notably, we consider step 2), involving a linear parametric machine learning model of model outputs, to encompass a linear regression that would amount to a linear transformation. Our rank and rating is the position of the lead in the ranked list, and its percentile score respectively.
Chiao does not expressly disclose the remaining limitations.
However, Brannon teaches:
pooling; implementing a pooling technique enhancing and optimizing ... machine learning techniques;
In [0059], it is outlined that our techniques for dataset curation pertain to “training, testing, and/or validating machine-learning models”. The mechanics of doing so are outlined in [0061], "Additionally or alternatively, a computing system that modifies the data set (e.g., the bias evaluation computing system 200 and/or the third-party computing system 235) could address the prejudice bias by adding data instances to the data set for a sub-category that is underrepresented in the data set. For instance, the computing system may be provided with a pool of data instances to pull from to add instances to the data set. For example, the entity that has uploaded the data set to the bias evaluation system 200 for analysis may also provide a pool of data instances along with the data set. According to some aspects, the bias evaluation system 200 may also process the data instances in the pool to identify applicable sub-categories for the different bias categories so that they may be used to supplement the data set".
Chiao discloses a system for combining different modelling methods to predict sales lead conversion. Brannon discloses a system meant to augment machine learning implementations by detecting areas of insufficient data and fetching analogous references. Each reference discloses means of optimizing machine learning model methods. Extending the data pooling as recorded in Brannon to the system of Chiao is applicable as they are from the shared field of endeavor, namely implementing machine learning architectures.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the data pooling as taught in Brannon and apply that to the system as taught in Chiao. Motivation to do so comes from the fact that the claim is plainly directed to the predictable result of combining known items in the prior art, with the expected benefit that adopting said methods would enable users to proactively identify and mitigate disproportionate datasets.
Chiao combined with Brannon does not expressly disclose the remaining limitations.
However, Robinson teaches:
generating a lead score for each configured rule; combining … and one or more rules based scores
We understand a lead score to be a probability here, with classification being a binary indicating conversion or not. In [0079], "In some embodiments, the satisfaction of a predictive rule in relation to a prediction input may indicate association of the prediction input with the prediction categories indicates by the prediction rule in accordance with the predictive weights associated with the prediction rule. The rules engine 115 may aggregate the predictive weights of the prediction rules satisfied by a prediction input to determine rule-based classification scores for the prediction input. The rules engine 115 may further determine the rule-based classification based on the rule-based classification scores". In [0085], "Returning to FIG. 11, at step/operation 1102, the rules engine 115 aggregates per-rule prediction scores for each prediction category to generate per-category prediction scores".
Chiao combined with Brannon discloses a system for applying a plurality of statistical modelling methods to predict sales lead conversion. Robinson discloses a system meant to leverage rules based methods in machine learning implementations. Each reference discloses means for improve machine learning methods by leveraging other approaches. Extending the rules methodology as recorded in Robinson to the system of Chiao combined with Brannon, is applicable as they are both directed to the shared problem of enriching machine learning methods by combining models with other approaches.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the rules based methodology and semantic machine learning as taught in Robinson and apply that to the system as taught in Chiao combined with Brannon. Motivation to do so comes from the fact that the claim is plainly directed to the predictable result of combining known items in the prior art, with the expected benefit that adopting said rules based methodology would enable users to both have a more visible insight into the prediction process, in [0066], "With respect to those use cases, the user can benefit from a more interpretable output that allows insights into the prediction process", and incorporate human expertise, in [0068], "Because some users, such as SMEs, are generally well-trained and understand the prediction domain intimately, such users are able to specify logical rules that can have high degrees of accuracy and can proper account for post-training discoveries and changes in data. Thus, even after training and deployment of a prediction system, new prediction rules can be manually created and modified by end users to generalize classification capabilities without the need to incur costs associated with re-training the machine learning model".
Chiao combined with Brannon and Robinson does not expressly disclose the remaining limitations.
However, Drapeau teaches:
wherein the ML models are trained using xgBoost and balanced random forest techniques;
In [0055], “In embodiment, ensemble methods may also be provided in embodiments, such as combining ML analysis performed by ML engines 214A and 214B. In statistics and machine learning, ensemble methods can use multiple ML model analysis techniques to obtain better predictive performance than could be obtained from any of the constituent learning algorithms. In an example, two models are generated and combined to create one prediction, with one model using the XGBoost technique and one model using a deep neural network (DNN) technique. However, other and additional models may be used including, but not limited to, regular random forest, one balanced random forest, support vector machine, etc.”
generate per-category prediction scores".
Chiao combined with Brannon and Robinson discloses a system for applying a plurality of statistical modelling methods to predict sales lead conversion. Drapeau discloses a system meant to leverage a plurality of models to assess fraud risk. Each reference discloses means for improving machine learning methods by leveraging other approaches. Extending the model training methodology as recorded in Drapeau to the system of Chiao combined with Brannon and Robinson, is applicable as they are both directed to the shared problem of enriching machine learning methods by combining models with other approaches.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the plurality of model training methods as taught in Drapeau and apply that to the system as taught in Chiao combined with Brannon and Robinson. Motivation to do so comes from the fact that the claim is plainly directed to the predictable result of combining known items in the prior art, with the expected benefit that adopting said training techniques being to improve predicting modeling when using machine learning for classification and analysis.
Claim 11 is rejected as disclosing substantially similar limitations as Claim 1.
Claims 2, 12
As to Claim 2, Chiao combined with Brannon, Robinson and Drapeau teaches all the limitations of Claim 1 as discussed above.
Chiao combined with Brannon does not expressly disclose the remaining limitations.
However, Robinson teaches:
The system of claim 1, wherein the set of instructions is further configured to cause the at least one processor to execute assigning a category to an entity for which a lead scoring feature is to be enabled, wherein the category comprises an indecisive tag, a blurry tag, and a limpid tag; and determining if data pooling is required for an account based on the assigned category.
We understand the following labels to analogize to the claims: underrepresentation to indecisive, overrepresented to limpid, appropriate representation, or the lack of a characterization here, to blurry in [0070], "The model analysis module 220 then analyzes the sub-categories found in the bias categories identified for the results to determine whether any of the bias categories are significantly impacting the output found in the results in Operation 425. For example, similar to analyzing a data set, a determination may be made as to whether a proportion of the results (e.g., number of result instances) that represents a sub-category for any particular category satisfies a threshold amount or percentage of the part of the results (e.g., total number of result instances) that represents the particular category. Therefore, similar to the data set analysis module 210, the model analysis module 220 may analyze the sub-categories found in the different bias categories identified for the results to determine whether any particular sub-category found in a bias category is underrepresented or overrepresented in the results. Again, the model analysis module 220 may analyze each of the sub-categories of the different bias categories by comparing the proportion of the results (e.g., the number of result instances) associated with the sub-category to one or more thresholds to determine whether the proportion of the results associated with the sub-category satisfies either of the thresholds". The underrepresented category is used to determine the need for data pooling with similar instances, in [0061], “Additionally or alternatively, a computing system that modifies the data set (e.g., the bias evaluation computing system 200 and/or the third-party computing system 235) could address the prejudice bias by adding data instances to the data set for a sub-category that is underrepresented in the data set. For instance, the computing system may be provided with a pool of data instances to pull from to add instances to the data set. For example, the entity that has uploaded the data set to the bias evaluation system 200 for analysis may also provide a pool of data instances along with the data set. According to some aspects, the bias evaluation system 200 may also process the data instances in the pool to identify applicable sub-categories for the different bias categories so that they may be used to supplement the data set”.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the rules based methodology and semantic machine learning of Robinson and apply that to the system of Chiao combined with Brannon. Motivation to do so comes from the same rationale as outlined above with respect to Claim 1.
Claim 12 is rejected as disclosing substantially similar limitations as Claim 2.
Claims 3, 13
As to Claim 3, Chiao combined with Brannon, Robinson and Drapeau teaches all the limitations of Claim 1 as discussed above.
Chiao teaches:
The system of claim 1, wherein the set of instructions is further configured to cause the at least one processor to execute executing a ML based approach by combining a plurality of models and providing a single artificial intelligence (AI) score for each lead in the set of leads, wherein the plurality of models comprises a fit model, an engagement model
As outlined above, [0107] teaches the ensemble machine learning method. We consider the different machine learning models to be attuned to different features of the model. Regarding engagement and fit, in [0061], “As with conventional lead scoring, the type of features present are of broadly two kinds static (or fit) features and behavioral (or activity) features. The static features are demographical information about either the individual contact or the company for which the individual works. Examples would be information about customer location, number of employees, the contact's job title, industry type, number of open job postings for different departments, and about the technologies used by the customer, and represent the “fit” of the individual and the product. Behavioral features represent actions taken by an individual. For example, the number of times a lead has visited a product website, or whether the lead has filled out a particular form. All of the behavioral features are represented as counts, while the majority of the static features are binary or categorical variables”.
Chiao combined with Brannon does not expressly disclose the remaining limitations.
However, Robinson teaches:
and a semantic model
Pertaining to parsing natural language inputs in [0091], “The process 600 of FIG. 6 begins when the machine learning engine 117 extracts a set of machine learning features 602 from the prediction input 501. For example, if the prediction input 501 is a text input, the set of machine learning features may be a vector of one or more words of the text input. As another example, if the prediction input 501 is a text input, the set of machine learning features may be a vector that indicates frequency of occurrence of one or more index phrases in the text input…The machine learning engine 117 then provides the set of machine learning features 602 as an input to various machine learning nodes of an input layer 603 of the artificial neural network of the machine learning engine 117. For example, if the set of machine learning features 602 include n machine learning features, the machine learning engine 117 may provide each feature to one of n machine learning nodes of the input layer 603.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the rules based methodology and semantic machine learning of Robinson and apply that to the system of Chiao combined with Brannon. Motivation to do so comes from the same rationale as outlined above with respect to Claim 1.
Claim 13 is rejected as disclosing substantially similar limitations as Claim 3.
Claim 8, 18
As to Claim 8, Chiao combined with Brannon, Robinson and Drapeau teaches all the limitations of Claim 1 as discussed above.
Chiao combined with Brannon does not expressly disclose the remaining limitations.
However, Robinson teaches:
The system of claim 1, wherein the set of instructions is further configured to cause the at least one processor to execute matching a plurality of rules with a predefined attribute and calculating a matched score as defined in sr=(totalmatchedpositiveweight-totalmatchednegativeweight).
Regarding the aggregation of applicable rules in [0085], Returning to FIG. 11, at step/operation 1102, the rules engine 115 aggregates per-rule prediction scores for each prediction category to generate per-category prediction scores. For example, the rules engine 115 may aggregate (e.g., sum) all generated per-rule prediction scores for a first prediction category to generate a per-category prediction score for the first category; aggregate all generated per-rule prediction scores for a second prediction category to generate a per-category prediction score for the second category, and so on. As another example, if the prediction rules repository 900 of FIG. 9 is applied to a prediction input 501 and only prediction rules 901-903 are satisfied by the prediction input, the rules engine 115 may generate the following per-category prediction scores: for prediction category PGN, 100+15=115; for prediction category CONS, 15; and for prediction category DS, −100". The plurality of rules are implicitly matched based on their applicability to a given attribute that they pertain to; rules that aren't matched to that attributed are left unsatisfied, and contribute neither positive or negative weight and satisfy the claimed equation. These values are arrived at by aid of defined weights, in [0081], “In some embodiments, a user may define a prediction rule 502, e.g., by supplying at least one component of the prediction rule 502, such as the condition for the prediction rule 502 and/or one or more predictive weight values for the prediction rule 502”.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the rules based methodology and semantic machine learning of Robinson and apply that to the system of Chiao combined with Brannon. Motivation to do so comes from the same rationale as outlined above with respect to Claim 1.
Claim 18 is rejected as disclosing substantially similar limitations as Claim 8.
Claims 9, 19
As to Claim 9, Chiao combined with Brannon, Robinson and Drapeau teaches all the limitations of Claim 8 as discussed above.
Chiao teaches:
The system of claim 8, wherein the set of instructions is further configured to cause the at least one processor to execute generating the ML based score using a plurality of models as defined by sML=(w1*static_model_score)+(w2*interest_model_score)+(w3*semantic_model_score)where , w1, w2, and w3 are predetermined weights
In [0107], "In an example embodiment, lead prioritization can be determined based on results from multiple modeling methods, such as the modeling methods described above. The system and method of an example embodiment can generate a plurality of scores for each lead using a plurality of different ensemble machine learning techniques or processing models. Then, the system and method of an example embodiment can generate scores using a linear parametric machine learning technique or other type of modeling technique or processing model. The system and method of the example embodiment can evaluate the results from each of the plurality of models and stack rank the list of leads based on a set of criteria (e.g., heuristics), including: 1) a sorting of the list from highest to lowest score based on the stronger of the plurality of ensemble machine learning methods; 2) a sorting of the list of leads from highest to lowest score based on the linear parametric machine learning model or other type of modeling technique; 3) an assignment of a composite score for each lead from 0-100 based on its percentile rank within the sorted list; and 4) a re-evaluation of the composite score for each lead relative to the scores for each lead from the plurality of individual modeling methods. If the composite score for each lead is at least as strong as the strongest individual model score, the system and method can use the composite score as the final score for the lead. The details of an example embodiment are described below". As would be well known to one of ordinary skill in the art, one such linear parametric machine learning technique would be a linear combination of values, in this case model outputs, with different parameters. Said parameters of the linear combination would correspond to predetermined values that would be used by the trained linear parametric model.
and represent the contributions from static, interest,
As outlined above, [0107] teaches the ensemble machine learning method. We consider the different machine learning models to be attuned to different features of the model. Regarding engagement and fit, in [0061], “As with conventional lead scoring, the type of features present are of broadly two kinds static (or fit) features and behavioral (or activity) features. The static features are demographical information about either the individual contact or the company for which the individual works. Examples would be information about customer location, number of employees, the contact's job title, industry type, number of open job postings for different departments, and about the technologies used by the customer, and represent the “fit” of the individual and the product. Behavioral features represent actions taken by an individual. For example, the number of times a lead has visited a product website, or whether the lead has filled out a particular form. All of the behavioral features are represented as counts, while the majority of the static features are binary or categorical variables”.
Chiao combined with Brannon does not expressly disclose the remaining limitations.
However, Robinson teaches:
and semantic models, respectively.
As above, we consider the different machine learning models to be attuned to different features of the model. Pertaining to parsing natural language inputs in [0091], “The process 600 of FIG. 6 begins when the machine learning engine 117 extracts a set of machine learning features 602 from the prediction input 501. For example, if the prediction input 501 is a text input, the set of machine learning features may be a vector of one or more words of the text input. As another example, if the prediction input 501 is a text input, the set of machine learning features may be a vector that indicates frequency of occurrence of one or more index phrases in the text input…The machine learning engine 117 then provides the set of machine learning features 602 as an input to various machine learning nodes of an input layer 603 of the artificial neural network of the machine learning engine 117. For example, if the set of machine learning features 602 include n machine learning features, the machine learning engine 117 may provide each feature to one of n machine learning nodes of the input layer 603.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the rules based methodology and semantic machine learning of Robinson and apply that to the system of Chiao combined with Brannon. Motivation to do so comes from the same rationale as outlined above with respect to Claim 1.
Claim 19 is rejected as disclosing substantially similar limitations as Claim 9.
Claims 10, 20
As to Claim 10, Chiao combined with Brannon, Robinson and Drapeau teaches all the limitations of Claim 9 as discussed above.
Chiao teaches:
The system of claim 9, wherein the set of instructions is further configured to cause the at least one processor to execute calculating the unitary score as defined by s=sR+sML, s=s.clip(0,1).
With reference to our linear combination interpretation of the linear parametric model in [0112], "2) Next, use the probability scores output by the LR model (or other modeling techniques) to perform a secondary sort, which has the effect of breaking ties resulting from the primary sort". In line with our 112b) interpretation above, clip is understood to act as a bound between 0 and 1. Given that our final score is intended to represent a probability, this bound would be implicitly present. Further, what is described is a linear combination with no further weighting being applied to the plurality of machine learning models beyond w_1, w_2, and w_3 of Claim 9, and a weight of 1 being applied to our rules model.
Claim 20 is rejected as disclosing substantially similar limitations as Claim 10.
Claims 4, 14 are rejected under 35 U.S.C. 103 as being unpatentable over Chiao(US 20160071118 A1) in view of Brannon(US 20220108222 A1) in further view of Robinson(US 20200311601 A1) in further view of Drapeau(US 20230026121 A1) in further view of Field-Darragh(US 20160042315 A1).
Claims 4, 14
As to Claim 4, Chiao combined with Brannon, Robinson and Drapeau teaches all the limitations of Claim 1 as discussed above.
Chiao combined with Brannon does not expressly disclose the remaining limitations.
However, Field-Darragh teaches:
The system of claim 1, wherein the set of instructions is further configured to cause the at least one processor to execute executing a rules based approach by using an algorithm to automatically compute and assign different weights to each lead.
In [0204], “As noted, in one embodiment a value termed a “fulfillment confidence score” may be used as part of a process of one or more of (a) determining whether to use a particular item to fulfill an order, (b) determining the optimal sequence in which a set of items should be picked, or (c) determining the fulfillment options to provide to a customer. The score may be expressed as a number or set of numbers, or along with other parameters. The score may be determined based on a weighted sum of multiple terms, an average of multiple values, a sum of multiple terms, or by any other suitable method or process. The weights and/or terms may be selected based on a customer profile (thereby emphasizing those criteria of most importance to a particular customer), on an available inventory (thereby adjusting a fulfillment decision based on inventory levels or changes in inventory levels), delivery schedules (thereby adjusting a fulfillment decision based on availability of delivery or time of year), or any other suitable factor or factors”. In [0261], “Additionally, as described herein, the various values may be assigned relative weights. Each of the weights, either singly or in combination, may be dynamically adjustable, manually adjustable, heuristically determined, determined by application of a suitable rule base, predetermined, or the like”. We consider the dynamic adjustment according to some set of heuristics to encompass an algorithm.
Chiao combined with Brannon and Robinson discloses a system for applying a plurality of statistical modelling methods to predict sales lead conversion. Field-Darragh discloses a system meant to personalize services to customers with the goal of marketing prediction. Each reference discloses means for leveraging data for the purpose of marketing and sales. Extending the tailored rule development as recorded in Field-Darragh to the system of Chiao combined with Brannon and Robinson, is applicable as they are both directed to the shared field of endeavor of data analysis for marketing.
It would have been obvious to one having ordinary skill in the art at the effective filling date of the invention to apply the tailored rule development as taught in Field-Darragh and apply that to the system as taught in Chiao combined with Brannon and Robinson. Motivation to do so comes from the fact that the claim is plainly directed to the predictable result of combining known items in the prior art, with the expected benefit that doing so would refine the output of the rules based methodology by avoiding overgeneralizing from a concrete rules schema.
Claim 14 is rejected as disclosing substantially similar limitations as Claim 4.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THEODORE L XIE whose telephone number is (571)272-7102. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached at 571-272-6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THEODORE XIE/Examiner, Art Unit 3623
/CHARLES GUILIANO/Primary Examiner, Art Unit 3623