DETAILED ACTION
This action is responsive to the claims filed on 12/29/2025. Claims 1-20 are pending for examination.
Response to Remarks
Applicant’s arguments have been fully considered but are not persuasive. Although claims 1 and 9 have been amended to recite “running an application comprising a user interface with a control that, when selected by a user, perform steps,” the claims still recite the same abstract idea previously identified, namely, obtaining candidate-related data, evaluating confidence and probability values, comparing those values to thresholds, selecting candidates based on those evaluations, and outputting a list of candidates. These limitations are still directed to observations, evaluations, judgments, and comparisons, which fall within the category of mental processes. The added user-interface/control language merely states that the abstract process is carried out using generic computer components and does not amount to an improvement in the functioning of a computer or any other technology. Applicant’s argument that the claims improve prediction accuracy is also not persuasive because any such improvement is directed to the abstract process of selecting or ranking people based on data, rather than to a specific improvement in computer technology. The claims do not recite any particular technological mechanism that improves model training, data storage, networking, processor operation, or user-interface technology, but instead recite results at a high level of generality. Accordingly, the amendments do not materially change the eligibility analysis, and the rejection under 35 U.S.C. 101 is maintained.
Regarding applicants arguments that the enrichment data may not provide sufficient information to ascertain the identity of all possible candidates (Remarks, page 10), the examiner respectfully disagrees. Applicant’s argument is not persuasive. Applicant argues that claim 1 may involve a population such as a city or company, and that the enrichment data may therefore be insufficient to identify all possible candidates. However, such a scenario is not positively recited in claim 1. Claim 1 broadly recites obtaining a list of possible candidates and enrichment data, determining an identity confidence based at least on the enrichment data, applying a threshold condition, and then performing further predictive processing. The applied references teach that sequence. Garg teaches obtaining candidate-related data from multiple sources and generating enriched talent profiles including profile data, supplemental candidate data, related-entity data, similarity data, and derived insights. See, e.g., Garg, Fig. 3 and col. 3, line 26. Orun teaches determining confidence in identity-related data such as name, email address, phone number, and address, and also teaches excluding low-confidence data using a threshold. See, e.g., Orun, “the database system applies an output threshold of 0.4 for individual data elements” and outputs combined data except for data “that has a confidence score that is less than the output threshold.” Schwarm teaches using a trained predictive model to calculate probability scores and output prioritized lists based on score thresholds. See, e.g., Schwarm paragraphs 92, 128. Accordingly, Applicant’s discussion of a possible embodiment or alleged efficiency gain does not persuasively distinguish the claimed invention from the applied combination.
Applicant argues that Garg’s disclosed “pair” is a job-candidate/job-position pair rather than a possible-contact/possible-candidate pair (Remarks, pages 10-11). The examiner respectfully disagrees. The rejection is based on the combined teachings of the references, not Garg alone. Schwarm teaches an outreach and response-prediction context in which a user may search scored prospect companies or employees, and the system calculates response-related probability scores and outputs prioritized prospect lists. See, e.g., Schwarm paragraph 128 (“calculate a probability score for company employees and/or employee positions likely to respond to a message”) and paragraph 92 (outputting a prioritized target list). Garg teaches candidate identification and enrichment. See, e.g., Garg, Fig. 3 and claim 17. When considered together, the references teach or at least suggest the claimed interaction between a possible contact and a possible candidate. Accordingly, Applicant’s argument is not persuasive because it focuses on Garg in isolation rather than on the combination relied upon.
Applicant argues that Garg’s “job requirements 120” do not disclose an identity of a possible contact because they do not refer to a person, and further argues that Caraviello does not cure this deficiency (Remarks, pages 11-12), the examiner respectfully disagrees. The rejection of claim 14 is based on the combined teachings of Garg, Schwarm, and Caraviello, not on “job requirements 120” by itself. As discussed above, Schwarm teaches a communication and response-prediction framework involving identified prospects and predicted responses to communications. See, e.g., Schwarm paragraph 128. Applicant’s argument therefore does not address the combination actually applied by the previous office action. Further, the statement that Caraviello does not cure the deficiency is conclusory because it does not explain why the references, when considered together, fail to teach or suggest the claimed training-data and contact-related limitations. Accordingly, the rejection of claim 14, and the claims depending therefrom, is maintained.
Applicant’s arguments with respect to claims 1 and 9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Statutory Categories
Claims 1-8 are directed to a system.
Claims 9-13 are directed to a method.
Claim 14-20 is directed to a method.
Independent Claims – Claim 1
Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes. Independent claim 1 recites limitations that are abstract ideas in the form of mental processes:
Claim 1 recites:
for each possible candidate on the list of possible candidates, determining a confidence regarding an identity of the possible candidate based at least on the enrichment data, wherein the identity of the possible candidate comprises one or more of a name, an email address, a contact number, or an address, (a mental process of evaluation of values which can reasonably be performed in one’s mind or with aid of pen and paper)
when the confidence regarding the identity of the possible candidate satisfies a threshold condition, determining . . . , a probability that the possible candidate will take an action of interest, and when the probability meets a threshold probability, (a mental process of evaluation which can reasonably be performed in one’s mind or with aid of pen and paper)
then adding the possible candidate to a list of candidates; (a mental process of evaluation which can reasonably be performed in one’s mind or with aid of pen and paper)
This claim further recites the following additional elements for the purposes of Step 2A Prong Two analysis:
A computing system, comprising: a logic subsystem; and a storage subsystem comprising instructions executable by the logic subsystem to (a recitation of using components stated at a high level of generality and with little context as to how they are implemented to perform the function is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
run an application comprising a user interface with a control that, when selected by a user, perform steps of: (a recitation of using a user interface stated at a high level of generality to perform the function is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
obtaining a list of possible candidates and enrichment data for each possible candidate on the list of possible candidates; (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
by inputting information regarding the identity of the possible candidate and the enrichment data for the possible candidate into a trained machine learning model, (a recitation of using a trained machine learning model stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
and outputting the list of candidates. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
The additional limitations fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice.
This claim recites the following additional elements for the purposes of Step 2B analysis:
A computing system, comprising: a logic subsystem; and a storage subsystem comprising instructions executable by the logic subsystem to (a recitation of using components stated at a high level of generality and with little context as to how they are implemented to perform the function is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
run an application comprising a user interface with a control that, when selected by a user, perform steps of: (a recitation of using a user interface stated at a high level of generality to perform the function is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
obtaining a list of possible candidates and enrichment data for each possible candidate on the list of possible candidates; (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.)
by inputting information regarding the identity of the possible candidate and the enrichment data for the possible candidate into a trained machine learning model, (a recitation of using a trained machine learning model stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
then add the possible candidate to a list of candidates; and output the list of candidates. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity.)
The claim also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claim is unpatentable.
Claims 9 and 18 have limitations substantially similar to claim 1, therefore a similar analysis applies.
Dependents of Claim 1
The remaining dependent claims corresponding to independent claims 1 do not recite additional elements, whether considered individually or in combination, that are sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. The analysis of which is shown below:
The claims below recite additional limitations which fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice.
The claims also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claims are unpatentable.
Claim 2 recites the further limitation of:
The computing system of claim 1, wherein the enrichment data comprises one or more of demographic information, financial information, or lifestyle preferences information. (this is merely additional information for an aforementioned judicial exception which further describes the enrichment data in terms of the type of data and is considered insignificant extra-solution activity because it does not integrate the exception in Step 2A Prong 2. For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
Claim 3 recites the further limitation of:
The computing system of claim 1, wherein the machine learning model comprises a gradient boosting classifier comprising an ensemble of decision trees. (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 4 recites the further limitation of:
The computing system of claim 1, wherein the machine learning model comprises a plurality of parameters having values determined based at least on an area under a receiver operating characteristic curve metric. (recites mathematical concepts that comprise mathematical algorithms, formulas, or calculations, paragraph [0068] of the specification contains the related mathematical disclosure)
Claim 5 recites the further limitation of:
The computing system of claim 1, wherein the machine learning model is trained based at least on a data set regarding a plurality of previous possible candidates, the data set including, for each previous possible candidate of the plurality of previous possible candidates, enrichment data regarding the previous possible candidate, and a label comprising an indication of whether the previous possible candidate took the action of interest. (a recitation of training by only stating data used, stated at a high level of generality, is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 6 recites the further limitation of:
The computing system of claim 5, wherein the plurality of previous possible candidates are members of a first organization, and wherein the possible candidate is a member of a second organization different from the first organization. (this is merely additional information that directs the aforementioned judicial exception into a field of use, see MPEP 2106.05(h).)
Claim 7 recites the further limitation of:
The computing system of claim 1, further comprising instructions executable to determine the threshold probability based at least on one or both of a precision or a recall of the machine learning model. (a determination by using data of a machine learning model stated at a high level of generality, is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 8 recites the further limitation of:
The computing system of claim 1, wherein the possible candidate comprises one of an individual or an organization. (this is merely additional information that directs the aforementioned judicial exception into a field of use, see MPEP 2106.05(h).)
Independent Claims – Claim 9
Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes. Independent claim 9 recites limitations that are abstract ideas in the form of mental processes:
Claim 1 recites:
determining a confidence regarding an identity of the possible candidate based at least on the enrichment data, wherein the identity of the possible candidate comprises one or more of a name, an email address, a contact number, or an address, (a mental process of evaluation which can reasonably be performed in one’s mind or with aid of pen and paper)
and when the probability meets a threshold probability, then adding the possible candidate to a list of candidates for the possible contact for whom the probability is maximized (a mental process of evaluation which can reasonably be performed in one’s mind or with aid of pen and paper)
This claim further recites the following additional elements for the purposes of Step 2A Prong Two analysis:
A method, comprising running an application comprising a user interface with a control that, when selected by a user, perform steps of: (a recitation of using a user interface stated at a high level of generality to perform the function is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
obtaining a list of possible candidates; for each possible candidate on the list of possible candidates, obtaining enrichment data for the possible candidate, (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
when the confidence regarding the identity of the possible candidate satisfies a threshold condition, for a plurality of possible contact/possible candidate pairs, inputting information regarding the identity of the possible candidate, the enrichment data for the possible candidate, an identity of the possible contact, and enrichment data for the possible contact, (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
into a trained machine learning model to obtain a probability that the possible candidate will take an action of interest when contacted by the possible contact, (a recitation of using a trained model stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
and outputting the list of candidates for the possible contact. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
The additional limitations fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice.
This claim recites the following additional elements for the purposes of Step 2B analysis:
A method, comprising running an application comprising a user interface with a control that, when selected by a user, perform steps of: (a recitation of using a user interface stated at a high level of generality to perform the function is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
obtaining a list of possible candidates; for each possible candidate on the list of possible candidates, obtaining enrichment data for the possible candidate, (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
when the confidence regarding the identity of the possible candidate satisfies a threshold condition, for a plurality of possible contact/possible candidate pairs, inputting information regarding the identity of the possible candidate, the enrichment data for the possible candidate, an identity of the possible contact, and enrichment data for the possible contact, (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
into a trained machine learning model to obtain a probability that the possible candidate will take an action of interest when contacted by the possible contact, (a recitation of using a trained model stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
and outputting the list of candidates for the possible contact. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
The claim also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claim is unpatentable.
Claims 9 and 18 have limitations substantially similar to claim 1, therefore a similar analysis applies.
Dependents of Claim 9
The remaining dependent claims corresponding to independent claims 9 do not recite additional elements, whether considered individually or in combination, that are sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. The analysis of which is shown below:
The claims below recite additional limitations which fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice.
The claims also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claims are unpatentable.
Claim 10 recites the further limitation of:
The method of claim 9, wherein the machine learning model comprises a gradient boosting classifier comprising an ensemble of decision trees. (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 11 recites the further limitation of:
The method of claim 9, wherein the machine learning model comprises a plurality of parameters having values determined based at least on an area under a receiver operating characteristic curve metric. (recites mathematical concepts that comprise mathematical algorithms, formulas, or calculations, paragraph [0068] of the specification provides the related mathematical disclosure.)
Claim 12 recites the further limitation of:
The method of claim 9, further comprising outputting the list of candidates to a device associated with the possible contact and not to a device associated with another possible contact in the list of possible contacts. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
Claim 13 recites the further limitation of:
The method of claim 9, wherein the probability is obtained based further on information regarding an event occurring between a last contact to contact the possible candidate and the possible candidate. (a mental process of evaluation which can reasonably be performed in one’s mind or with aid of pen and paper)
Independent Claims – Claim 14
Step 2A Prong 1: Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes. Independent claim 14 recites limitations that are abstract ideas in the form of mental processes:
Claim 14 recites:
tuning one or more parameters of the machine learning model based at least on a portion of the data set and an evaluation metric; (a mental process of evaluation which can reasonably be performed in one’s mind or with aid of pen and paper)
This claim further recites the following additional elements for the purposes of Step 2A Prong Two analysis:
A method, comprising obtaining a training data set regarding a plurality of previous possible candidates and a plurality of previous contacts for the previous possible candidates, (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
the data set including, for each previous possible candidate of the plurality of previous possible candidates, enrichment data regarding the previous possible candidate, enrichment data regarding a previous contact that contacted the previous possible candidate, and a label comprising a binary indication of whether the previous possible candidate eventually took an action of interest when contacted by the previous contact; (this is merely additional information for an aforementioned judicial exception which further describes the enrichment data in terms of the type of data and is considered insignificant extra-solution activity because it does not integrate the exception in Step 2A Prong 2.)
training a machine learning model comprising a classifier based at least on a portion of the data set to train the machine learning model to (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
output a prediction, for an input possible candidate and input contact, regarding whether the input possible candidate eventually will take the action of interest; (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g).)
and outputting the machine learning model. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g))
The additional limitations fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice.
This claim recites the following additional elements for the purposes of Step 2B analysis:
A method, comprising obtaining a training data set regarding a plurality of previous possible candidates and a plurality of previous contacts for the previous possible candidates, (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
the data set including, for each previous possible candidate of the plurality of previous possible candidates, enrichment data regarding the previous possible candidate, enrichment data regarding a previous contact that contacted the previous possible candidate, and a label comprising a binary indication of whether the previous possible candidate eventually took an action of interest when contacted by the previous contact; (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
training a machine learning model comprising a classifier based at least on a portion of the data set to train the machine learning model to (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
output a prediction, for an input possible candidate and input contact, regarding whether the input possible candidate eventually will take the action of interest; (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
and outputting the machine learning model. (limitation is merely inputting/outputting information and is considered insignificant extra-solution activity, see MPEP 2106.05(g). For the purposes of step 2B, it should be noted that the courts have recognized receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information) as well-understood, routine, and conventional activity)
The claim also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claim is unpatentable.
Claims 9 and 18 have limitations substantially similar to claim 1, therefore a similar analysis applies.
Dependents of Claim 14
The remaining dependent claims corresponding to independent claims 14 do not recite additional elements, whether considered individually or in combination, that are sufficient to integrate the judicial exception into a practical application or amount to significantly more than the judicial exception. The analysis of which is shown below:
The claims below recite additional limitations which fail step 2A Prong 2 of the 101 analysis because they do not transform the claim into a practical application. These limitations are too abstract or lack technical improvement that would make the concept practically useful. Without clear utility or integration into a specific field, the claim does not relate to any particular application. It does not meet the requirements of Step 2A Prong 2, as it fails to make the concept meaningfully applicable in practice.
The claims also fails Step 2B of the analysis because the additional limitations do not amount to significantly more than the abstract idea itself. The additional limitations do not enhance the claim in a way that would move it beyond its abstract ideas as they minimally elaborate on the core concept without adding any inventive or technical substance. The claims are unpatentable.
Claim 15 recites the further limitation of:
The method of claim 14, wherein training the machine learning model comprises using gradient boosting. (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 16 recites the further limitation of:
The method of claim 14, wherein each previous possible candidate of the plurality of previous possible candidates belongs to one of two imbalanced classes, (this is merely additional information that directs the aforementioned judicial exception into a field of use, see MPEP 2106.05(h).)
further comprising balancing the imbalanced classes. (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 17 recites the further limitation of:
The method of claim 14, wherein tuning the one or more parameters comprises applying k-fold cross validation to at least the portion of the data set. (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim 18 recites the further limitation of:
The method of claim 14, wherein the evaluation metric comprises an area under a receiver operating characteristic curve metric. (recites mathematical concepts that comprise mathematical algorithms, formulas, or calculations, paragraph [0068] of the specification contains the related mathematical disclosure)
Claim 19 recites the further limitation of:
The method of claim 18, wherein the input contact is a member of an organization, (this is merely additional information that directs the aforementioned judicial exception into a field of use, see MPEP 2106.05(h).)
further comprising collecting a replacement data set regarding a plurality of previous possible candidates that are members of the organization, and replacing at least a portion of the data set with the replacement data. (a mental process of evaluation which can reasonably be performed.)
Claim 20 recites the further limitation of:
The method of claim 14, wherein training the machine learning model comprises using a logistic loss function. (a recitation of using a known classifier stated at a high level of generality is being considered merely as instructions to apply an exception under MPEP 2106.05(f))
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 3, 5, 6, 8, 9, 10, 12, 13 are rejected under 35 U.S.C. 103 as being unpatentable by Garg et al. (US 10803421 B1; patented Oct. 13, 2020), hereinafter referred to as Garg, in view of Orun et al. (US 20190114354 A1), hereafter referred to as Orun, and in further view of Schwarm et al. (US 20180101771 A1; published Apr. 12, 2018), hereafter referred to as Schwarm.
PNG
media_image1.png
689
638
media_image1.png
Greyscale
Figure 3 of Garg
Claim 1: Garg teaches the following limitations:
A computing system, comprising: a logic subsystem; and a storage subsystem comprising instructions executable by the logic subsystem (Garg, col. 18, line 17, “A non - transitory computer - readable medium comprising machine - executable code that , when executed by a computing system comprising one or more processors , enables the computing system to perform operations that automatically predict a match between a candidate and a job position for an enterprise”, describes a “computing system” with memory (storage subsystem) and processors running stored instructions to perform each step (logic subsystem).)
to run an application comprising a user interface with a control that, when selected by a user, perform steps of: (Garg, col. 3, line 33, “providing a first user interface via which a user can provide job requirements for an open job position, as well as one or more preferred or required personality and/or talent traits, wherein the personality and/or talent traits selectable in the user interface correspond to one or more of the personality and/or talent insights derived by the system in creating the enriched talent profiles… displaying the top n candidates for the job position based on the match score in a second user interface , where n is a positive integer”, Garg further teaches that “If a user selects a particular skill for a job requirement, the system displays other similar skills in the user interface for consideration to add to the job requirement. ” (Garg, col. 9, line 25) Garg’s disclosure further teaches an application having a user interface with user-selectable interface options. In particular, Garg’s disclosure that traits are “selectable in the user interface” and that “if a user selects a particular skill” the system responds by performing further processing teaches the claimed user interface with a control that, when selected by a user, perform[s] steps. Garg also teaches that, through that UI-driven workflow, the system identifies candidates and displays resulting candidates in a user interface. )
obtaining a list of possible candidates (Garg, claim 17, “identify a plurality of candidates for the job position , each of the plurality of candidates being associated with a corresponding enriched talent profile”, a plurality of possible candidates are obtained/identified.)
and enrichment data for each possible candidate on the list of possible candidates; (Garg, col. 3, line 26, “creating an enriched talent profile for the candidate includes the profile data , the supplemental candidate data , the related - entity data , the similarity data , the derived personality and talent insights for the candidate , and the predicted next role”, Garg’s chart in Figure 3 (shown above claim 1) further shows retrieving, for each candidate, multiple enrichment-data types.)
Orun, in the same field of likelihood estimation teaches the following limitation which the above prior art fails to teach:
for each possible candidate on the list of possible candidates, determining a confidence regarding an identity of the possible candidate based at least on the enrichment data, wherein the identity of the possible candidate comprises one or more of a name, an email address, a contact number, or an address, (Orun, paragraph 20, “The customer resolution engine can cleanse, normalize, and enrich entity data as needed… For this example, the customer resolution engine captures and leverages data validation and enrichment attributes as part of the identifier attributes to determine that “11 Main St” is not a valid street address for “San Francisco, Calif. 94105,” infer a data entry error in the street number, and then identify the nearest string or geo-proximity match as a reliable candidate”
Orun, paragraph 22, “The database system generates context scores for contact elements. For example, the customer resolution engine generates a moderate confidence score of 0.6 for the name “Samantha Smith” because 60% of Acme's community webpage users enter their names correctly, a high confidence score of 1.0 for the Twitter handle “StylishSam” because of its use to authenticate Smith, and a moderate confidence score of 0.5 for the email address “sam@mystyle.com” because this unverified email address was entered by an authenticated user. Although this example describes a context score that ranges from a minimum of 0.0 to a maximum of 1.0, any range and any type of context score may be used.”, Orun’s disclosure teaches determining a confidence regarding a person’s identity, because Orun expressly scores identifying/contact data elements used to resolve whether data belongs to the same person. Orun further teaches that the confidence is based at least on enrichment data, because the engine “cleanses, normalizes, and enriches entity data,” and “leverages data validation and enrichment attributes” when determining a reliable match. Orun also expressly teaches the recited identity fields: name, email address, and contact number/phone number. And Orun’s address example—“John Smith/1 Main St …” versus “John Smith/11 Main St …” with use of validation/enrichment attributes to identify the reliable candidate—teaches the recited address. Accordingly, Orun teaches determining a confidence regarding an identity of the possible candidate, based at least on enrichment data, where the identity comprises one or more of a name, an email address, a contact number, or an address.)
when the confidence regarding the identity of the possible candidate satisfies a threshold condition, (Orun, paragraph 15, “The database system outputs Smith's combined data to the representative, with the exception of the phone number “+00 800-7253-3333” that has a confidence score that is less than the output threshold, even though Smith had provided the names “Samantha,” “Samanta,” and “Sam” that do not match.”
Orun, paragraph 48, “In this example, the output data did not include the phone number “+00 800-7253-3333 because the customer resolution engine previously generated a confidence score of 0.3 for this phone number and the database system applies an output threshold of 0.4 for individual data elements.”, Orun’s disclosure teaches the claimed threshold condition because it expressly compares the identity-related confidence score of a data element against a threshold and conditionally uses or excludes that element based on whether the threshold is met. In other words, Orun teaches that downstream use of identity-related information is permitted only when the confidence satisfies a threshold condition, and low-confidence identity/contact data is excluded when it does not satisfy the threshold.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Garg’s candidate-evaluation system with Orun’s identity-confidence techniques because Garg expressly obtains and combines candidate-related data from multiple sources into enriched talent profiles, including profile data, supplemental data, related-entity data, and similarity data. Where multiple-source candidate data is combined, a person of ordinary skill would have recognized the desirability of first determining whether particular name, email, phone, and address information actually corresponds to the same underlying candidate before relying on that information in downstream scoring. Orun expressly teaches resolving identity using such data elements, generating confidence scores for those elements, leveraging validation and enrichment attributes, and excluding low-confidence identity/contact data under a threshold. Applying Orun’s identity-confidence determination to Garg’s multi-source candidate enrichment would therefore have been a predictable use of known techniques to improve data quality, reduce false associations, and ensure that enrichment data is associated with the correct candidate before further prediction is performed.
Schwarm, in the same field of machine learning candidate selection, teaches the following limitations which the above fails to teach:
determine, by inputting information regarding the identity of the possible candidate and the enrichment data for the possible candidate into a trained machine learning model, a probability that the possible candidate will take an action of interest, and (Schwarm, paragraph 128, “Once the new customer database is enriched with firmographic data as well as the employee data , at operation 511 the prediction engine is configured to employ the trained classifier to calculate a probability score for company employees and / or employee positions (e.g., management code , title ) likely to respond to a message and by what channels”, the identity (employee data) and the enrichment data (firmographic data) are inputted into a trained machine learning model to compute a probability that the candidate will take an action of interest (in this case a response to a message).)
when the probability meets a threshold probability, then add the possible candidate to a list of candidates and output the list of candidates. (Schwarm, paragraph 92, “At operation 606 the system then outputs the prioritized target list to the client user . to an output configured to output a list of classified , prioritized prospects . For example , the client user can receive an ordered set of 100 prospects based on the scores calculated .”, prioritized targets (list of candidates) are selected based on the probability scores calculated. It is interpreted that a certain level of score will classify prioritized targets, forming as the ‘threshold probability’ for selecting possible candidates for the list of candidates. The final list is then outputted to the users/clients.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg and Orun by incorporating teachings of Schwarm and include methods of calculating probability scores for prioritized candidate selection. A motivation for the combination would have to be the use of quantifying the prospects likelihood of response to dynamically rank and select the most promising targets. (Schwarm, paragraph 119, “Once the new customer database is enriched with firmographic data , at operation 406 the prediction engine is configured to employ the trained classifier to calculate a probability score for companies from the database of company names 405 likely to purchase a given product or a given service .”)
Claim 2: Garg, Orun, and Schwarm teaches limitations of claim 1. Garg further teaches:
The computing system of claim 1, wherein the enrichment data comprises one or more of demographic information, financial information, or lifestyle preferences information. (Garg, col. 6, line 23, “In certain embodiments , profile data also includes the candidate's hobbies and interests.”, it is interpreted that the enrichment data of Garg comprises at least one of a candidates hobbies and interests (lifestyle preferences).)
Claim 3: Garg, Orun, and Schwarm teaches limitations of claim 1. Schwarm further teaches:
The computing system of claim 1, wherein the machine learning model comprises a gradient boosting classifier comprising an ensemble of decision trees. (Schwarm, paragraph 7, “In at least one of the various embodiments , the classifier engine can comprise a machine learning classifier model builder selected from the group of : a decision tree , a random forest modeler , a cluster modeler , a K Means Cluster modeler , a neural net , a gradient boosted trees machine modeler , and a support vector machine ( SVM ) .”, it is interpreted that the enrichment data of Garg comprises at least one of a candidates hobbies and interests (lifestyle preferences).)
Claim 5: Garg, Orun, and Schwarm teaches limitations of claim 1. Schwarm further teaches:
The computing system of claim 1, wherein the machine learning model is trained based at least on a data set regarding a plurality of previous possible candidates, the data set including, for each previous possible candidate of the plurality of previous possible candidates, enrichment data regarding the previous possible candidate, and a label comprising an indication of whether the previous possible candidate took the action of interest. (Schwarm, paragraph 101, “In at least one embodiment , at operation 305 the linked company win / loss database 301 data 310 , 312 , 314 and company entity data 316 , 318 from company entity database 304 are compiled into a training database 306 . In an embodiment , the database includes , for each company , one or more win / loss values 310 , firmographic and score data 316 , and company data 318 .”, it is interpreted that the data of Schwarm comprises at least one of a candidates enrichment data, such as firmographic or company entity data, and a win/loss label indicating if the candidate took the action of interest or not (win or loss).)
Claim 6: Garg, Orun, and Schwarm teaches limitations of claim 1. Schwarm further teaches:
The computing system of claim 5, wherein the plurality of previous possible candidates are members of a first organization, and wherein the possible candidate is a member of a second organization different from the first organization. (Schwarm, paragraph 109, “The classifier thus identifies and can generate list of companies firmographically similar to those sold or marketed to the past and grouped by profile characteristics trained on the client user ' s own data . Such lists can be generated in seconds using the trained classifier and can include from thousands to tens of thousands of companies as well as associated people and their contact information , per profile , all being Al qualified prospects .”, the model identifies targets not from just a single company/organization.)
Claim 8: Garg, Orun, and Schwarm teaches limitations of claim 1. Schwarm further teaches:
The computing system of claim 1, wherein the possible candidate comprises one of an individual or an organization. (Garg, col. 1, line 16 “This invention relates generally to machine learning in human resource applications, and more specifically to using machine learning and enriched candidate and job profiles to predict the candidates most likely to be hired and successful in a job.”, the candidates in Garg are individuals.)
Claim 9: Garg teaches the following limitations:
A method, comprising running an application comprising a user interface with a control that, when selected by a user, perform steps of: (Garg, col. 3, line 33, “providing a first user interface via which a user can provide job requirements for an open job position, as well as one or more preferred or required personality and/or talent traits, wherein the personality and/or talent traits selectable in the user interface correspond to one or more of the personality and/or talent insights derived by the system in creating the enriched talent profiles… displaying the top n candidates for the job position based on the match score in a second user interface , where n is a positive integer”, Garg further teaches that “If a user selects a particular skill for a job requirement, the system displays other similar skills in the user interface for consideration to add to the job requirement. ” (Garg, col. 9, line 25) Garg’s disclosure further teaches an application having a user interface with user-selectable interface options. In particular, Garg’s disclosure that traits are “selectable in the user interface” and that “if a user selects a particular skill” the system responds by performing further processing teaches the claimed user interface with a control that, when selected by a user, perform[s] steps. Garg also teaches that, through that UI-driven workflow, the system identifies candidates and displays resulting candidates in a user interface. )
obtaining a list of possible candidates; (Garg, claim 17, “identify a plurality of candidates for the job position , each of the plurality of candidates being associated with a corresponding enriched talent profile”, a plurality of possible candidates are obtained/identified.)
for each possible candidate on the list of possible candidates, obtaining enrichment data for the possible candidate, (Garg, col. 3, line 26, “creating an enriched talent profile for the candidate includes the profile data , the supplemental candidate data , the related - entity data , the similarity data , the derived personality and talent insights for the candidate , and the predicted next role”, Garg’s chart in Figure 3 (shown above claim 1) further shows retrieving, for each candidate, multiple enrichment-data types.)
Orun, in the same field of likelihood estimation teaches the following limitation which the above prior art fails to teach:
determining a confidence regarding an identity of the possible candidate based at least on the enrichment data, wherein the identity of the possible candidate comprises one or more of a name, an email address, a contact number, or an address, (Orun, paragraph 20, “The customer resolution engine can cleanse, normalize, and enrich entity data as needed… For this example, the customer resolution engine captures and leverages data validation and enrichment attributes as part of the identifier attributes to determine that “11 Main St” is not a valid street address for “San Francisco, Calif. 94105,” infer a data entry error in the street number, and then identify the nearest string or geo-proximity match as a reliable candidate”
Orun, paragraph 22, “The database system generates context scores for contact elements. For example, the customer resolution engine generates a moderate confidence score of 0.6 for the name “Samantha Smith” because 60% of Acme's community webpage users enter their names correctly, a high confidence score of 1.0 for the Twitter handle “StylishSam” because of its use to authenticate Smith, and a moderate confidence score of 0.5 for the email address “sam@mystyle.com” because this unverified email address was entered by an authenticated user. Although this example describes a context score that ranges from a minimum of 0.0 to a maximum of 1.0, any range and any type of context score may be used.”, Orun’s disclosure teaches determining a confidence regarding a person’s identity, because Orun expressly scores identifying/contact data elements used to resolve whether data belongs to the same person. Orun further teaches that the confidence is based at least on enrichment data, because the engine “cleanses, normalizes, and enriches entity data,” and “leverages data validation and enrichment attributes” when determining a reliable match. Orun also expressly teaches the recited identity fields: name, email address, and contact number/phone number. And Orun’s address example—“John Smith/1 Main St …” versus “John Smith/11 Main St …” with use of validation/enrichment attributes to identify the reliable candidate—teaches the recited address. Accordingly, Orun teaches determining a confidence regarding an identity of the possible candidate, based at least on enrichment data, where the identity comprises one or more of a name, an email address, a contact number, or an address.)
when the confidence regarding the identity of the possible candidate satisfies a threshold condition, (Orun, paragraph 15, “The database system outputs Smith's combined data to the representative, with the exception of the phone number “+00 800-7253-3333” that has a confidence score that is less than the output threshold, even though Smith had provided the names “Samantha,” “Samanta,” and “Sam” that do not match.”
Orun, paragraph 48, “In this example, the output data did not include the phone number “+00 800-7253-3333 because the customer resolution engine previously generated a confidence score of 0.3 for this phone number and the database system applies an output threshold of 0.4 for individual data elements.”, Orun’s disclosure teaches the claimed threshold condition because it expressly compares the identity-related confidence score of a data element against a threshold and conditionally uses or excludes that element based on whether the threshold is met. In other words, Orun teaches that downstream use of identity-related information is permitted only when the confidence satisfies a threshold condition, and low-confidence identity/contact data is excluded when it does not satisfy the threshold.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Garg’s candidate-evaluation system with Orun’s identity-confidence techniques because Garg expressly obtains and combines candidate-related data from multiple sources into enriched talent profiles, including profile data, supplemental data, related-entity data, and similarity data. Where multiple-source candidate data is combined, a person of ordinary skill would have recognized the desirability of first determining whether particular name, email, phone, and address information actually corresponds to the same underlying candidate before relying on that information in downstream scoring. Orun expressly teaches resolving identity using such data elements, generating confidence scores for those elements, leveraging validation and enrichment attributes, and excluding low-confidence identity/contact data under a threshold. Applying Orun’s identity-confidence determination to Garg’s multi-source candidate enrichment would therefore have been a predictable use of known techniques to improve data quality, reduce false associations, and ensure that enrichment data is associated with the correct candidate before further prediction is performed.
Garg further teaches
for a plurality of possible contact/possible candidate pairs, (Garg, col. 12, line 5, “For each job candidate pair , data from the calibrated job profile and the candidates enriched talent profile are inputted into the DNN 750”)
inputting information regarding the identity of the possible candidate, the enrichment data for the possible candidate, (Garg, col. 9, line 7, “2.7 Enriched Talent Profile: The system creates an enriched talent profile for the candidate that includes the profile data ( 335 ) ; the similarity data 355 ; the supplemental candidate data ( 340 ) ; the related entity data ( 345 ) , the system - derived insights ( 350 ).”, identity of the possible candidate (profile data) and the enrichments data of the possible candidate (supplemental data, similarity data, etc) are encapsulated into a enriched talent profile and used as input as shown further below.)
an identity of the possible contact, and enrichment data for the possible contact, (Garg, col. 5, line 36, “(Creating a Calibrated Job Profile ): The system obtains a job description and job requirements ( 120 ) for an open job position … The system creates a calibrated job profile with the job description and requirements , the ideal candidates and their corresponding experiences , skills , and traits , and the required / preferred personality and talent traits ( 270 ) .”, the identity of the possible contact is encapsulated in the “Calibrated Job Profile” created for each job. Identity of the possible contact (job requirements 120) shows identity information of the contact as well as enrichment data in the form of other encapsulated fields like job description, ideal candidates, etc.)
into a trained machine learning model to obtain a probability that the possible candidate will take an action of interest when contacted by the possible contact (Garg, col. 12, line 8, “The DNN 750 outputs one or more hiring – related predictions ( 980 ) for each pair , resulting in a match score for each of the inputted candidates at Organization X. Examples of hiring - related predictions are the probability of the candidate being hired , and the probability of the candidate being successful in a job after a period of time ( e.g. , 1 year ). ”, the candidate identity and enrichment data (enriched talent profiles) and contact identity and enrichment data (job profiles) are input into a machine learning (DNN) to compute a probability that a candidate will take an action of interest (in this case the probability of hiring or retention).)
Schwarm, in the same field of machine learning candidate selection, teaches the following limitations which the above fails to teach:
and when the probability meets a threshold probability, then adding the possible candidate to a list of candidates for the possible contact for whom the probability is maximized and outputting the list of candidates for the possible contact. (Schwarm, paragraph 92, “At operation 606 the system then outputs the prioritized target list to the client user . to an output configured to output a list of classified , prioritized prospects . For example , the client user can receive an ordered set of 100 prospects based on the scores calculated .”, prioritized targets (list of candidates) are selected based on the probability scores calculated. It is interpreted that a certain level of score will classify prioritized targets, forming as the ‘threshold probability’ for selecting possible candidates for the list of candidates. The final list is then outputted to the users/clients.
Paragraph 94, “The client user can then select targets companies and / or employees identified with the predictive score as the highest likelihood to be in “ win ” classes .”, the selection of target companies or employees is explicitly done by maximizing the highest likelihood (when probability is maximized))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg and Orun by incorporating teachings of Schwarm and include methods of calculating probability scores for prioritized candidate selection. A motivation for the combination would have to be the use of quantifying the prospects likelihood of response to dynamically rank and select the most promising targets. (Schwarm, paragraph 119, “Once the new customer database is enriched with firmographic data , at operation 406 the prediction engine is configured to employ the trained classifier to calculate a probability score for companies from the database of company names 405 likely to purchase a given product or a given service .”)
Claim 10 is substantially similar to claim 3, as such a similar analysis applies.
Claim 12: Garg, Orun, and Schwarm teaches limitations of claim 9. Garg further teaches:
The method of claim 9, further comprising outputting the list of candidates to a device associated with the possible contact and not to a device associated with another possible contact in the list of possible contacts. (Garg, col. 9, line 17, “The system initiates the process of creating a calibrated job profile in response to receiving a job description and job requirements for an open job position ( step 510 ) . This data is entered by a user , such as an HR manager , via a user interface provided by the system.”, a user such as a hiring manager uses a user interface to input job related information for creating job profiles.
col. 14, line 10, “presenting, in a first user interface, the candidate in a ranked list including a plurality of candidates at a ranking position determined according to the calculated match score value.”, Garg specifically outputs the candidate list to a first device (user interface) rather than to another device.)
Claim 13: Garg, Orun, and Schwarm teaches limitations of claim 9. Schwarm further teaches:
The method of claim 9, wherein the probability is obtained based further on information regarding an event occurring between a last contact to contact the possible candidate and the possible candidate. (Schwarm, paragraph 122, “In an embodiment , the list of won company engagements 510W comprises a marketing response . In an embodiment , the won company engagement can include for each engagement a time value , for example , in an embodiment , when the company and / or company employee was sent a marketing message and when a response was received. The user can also provide list of lost company engagements 510W . In an embodiment the list can comprise for each lost company engagement 510W , if available , a when lost value , and a why lost value . For example , the user can provide , if available , when the marketing message was sent and , if available , why there was no response ( e . g . message channel no longer available ) .” , Schwarm specifically tracks events that are time stamped based on communication/contact received/given. This comprises the list of won/loss company engagements used as enrichment data for the trained classifier to calculate a probability.)
Claims 4, 11, are rejected under 35 U.S.C. 103 as being unpatentable by Garg in view of Orun, Schwarm, and further in view of Caraviello et al. (US 10102476 B2), hereafter referred to as Caravielo.
Claim 4: Garg, Orun, and Schwarm teaches limitations of claim 1. Caraviello, in the same field of machine learning classifier implementation further teaches:
The computing system of claim 1, wherein the machine learning model comprises a plurality of parameters having values determined based at least on an area under a receiver operating characteristic curve metric. (Caraviello, paragraph 45, “In an embodiment, cross-validation is used to compare algorithms and sets of parameter values. In an embodiment, receiver operating characteristic (ROC) curves are used to compare algorithms and sets of parameter values.”, Caraviello employs ROC analysis to select among different hyperparameter configurations.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg, Orun, and Schwarm by incorporating teachings of Caraviello and include methods of calculating machine learning parameters based on the ROC (receiver operating characteristics) curve. A motivation for the combination would have to create a system suited for large datasets, allowing for optimal parameter configuration for a certain dataset. (Caraviello, paragraph 76, “Moreover, data mining using the approaches described herein provides a flexible, scalable framework for modeling with datasets that include features based on molecular genetic markers. This framework is flexible because it includes tests (i.e. cross-validation and ROC curves) to determine which algorithm and specific parameter settings should be used for the analysis of a data set. This framework is scalable because it is suitable for very large datasets.”)
Claim 11 is substantially similar to claim 4, as such a similar analysis applies.
Claims 14, 15, 17, 18 are rejected under 35 U.S.C. 103 as being unpatentable by Garg in view of Schwarm and in further view of Caraviello.
Claim 14: Garg teaches the following limitations:
A method, comprising obtaining a training data set regarding a plurality of previous possible candidates and a plurality of previous contacts for the previous possible candidates, (Garg, col. 5, line 36, “(Creating a Calibrated Job Profile ): The system obtains a job description and job requirements ( 120 ) for an open job position. Examples of ideal candidates are people who are currently or previously in the position , as well as previous successful applicants .”, the identity of the possible contact is encapsulated in the “Calibrated Job Profile” created for each job. Identity of the possible contact (job requirements 120) shows identity information of the contact as well as enrichment data in the form of other encapsulated fields like job description, ideal candidates, etc. This is previous information since each job profile is created first before being used for training for identifying ideal candidates.
claim 1, “receiving , by a computing system comprising one or more processors , a neural network module trained by adjusting at least one parameter associated with the neural network module based on a training enriched talent profile , a training calibrated job profile , and a training match score between the training enriched talent profile and the training calibrated job profile ;”)
the data set including, for each previous possible candidate of the plurality of previous possible candidates, enrichment data regarding the previous possible candidate, enrichment data regarding a previous contact that contacted the previous possible candidate, (Garg, col. 9, line 7, “2.7 Enriched Talent Profile: The system creates an enriched talent profile for the candidate that includes the profile data ( 335 ) ; the similarity data 355 ; the supplemental candidate data ( 340 ) ; the related entity data ( 345 ) , the system - derived insights ( 350 ).”, identity of the possible previous candidate (profile data) and the enrichments data of the possible candidate (supplemental data, similarity data, etc) are encapsulated into a enriched talent profile and used as input as shown further below.
Garg, col. 5, line 36, “(Creating a Calibrated Job Profile ): The system obtains a job description and job requirements ( 120 ) for an open job position. Examples of ideal candidates are people who are currently or previously in the position , as well as previous successful applicants … The system creates a calibrated job profile with the job description and requirements , the ideal candidates and their corresponding experiences , skills , and traits , and the required / preferred personality and talent traits ( 270 ) .”, the identity of the possible contact is encapsulated in the “Calibrated Job Profile” created for each job. Identity of the possible contact (job requirements 120) shows identity information of the contact as well as enrichment data in the form of other encapsulated fields like job description, ideal candidates, etc.)
Schwarm, in the same field of machine learning candidate selection, teaches the following limitations which the above fails to teach:
and a label comprising a binary indication of whether the previous possible candidate eventually took an action of interest when contacted by the previous contact; (Schwarm, paragraph 80, “the system accepts as input a company win/loss database 301 including a list of company engagements, and, for each engagement, a company name value, and a company address value into a classifier modeler configured train and output at least one classifier. The list can include further identification data for the company, and for example a company address value, a website, a phone number, an email address, and so on. In at least one of the various embodiments, the company win/loss database 301 can also comprise employee identifiers for employees and agents of the company, for example names, job titles, contact information (e.g., email, mobile identification, social network profile). The database 301 comprises , for each company , one or more success metrics for each company engagement , for example , wins / losses or response / no response . As described herein , positive event observations ( e . g . customers , sales , marketing responses ) for companies are described as “ win ” ( W ) engagements and negative event observations ( e . g . lost sale , no response ) as “ loss ” ( L ) engagements”,
Schwarm, paragraph 122, “the won company engagement can include for each engagement a time value, for example, in an embodiment, when the company and/or company employee was sent a marketing message and when a response was received.”,
Scwarm, paragraph 125, “including training data comprising a company list of responders 510W and non-responders 510L”, Schwarm teaches a binary label for each prior engagement, namely response win (W) versus no response (L), and further teaches that the label corresponds to whether the company and/or company employee responded after being sent a marketing message. Thus, Schwarm teaches a label comprising a binary indication of whether the previous possible candidate eventually took an action of interest when contacted by the previous contact.)
training a machine learning model comprising a classifier based at least on a portion of the data set to train the machine learning model to output a prediction, for an input possible candidate and input contact, regarding whether the input possible candidate eventually will take the action of interest; (Schwarm, paragraph 86, “For another example , in at least one embodiment , the classification engine trains a predictive classifier on the training database 306 , 403 , 503 using a decision tree classifier , for example random forest classifier . The classifier model builder then outputs the trained company predictive profile classifier to a prediction engine . The classifier modeler then outputs the predictive classifier trained on client “ win ” and “ losses ” to run on and prioritize prospect company lists”,
Schwarm, paragraph 119, “the prediction engine is configured to employ the trained classifier to calculate a probability score for company employees and/or employee positions … likely to respond to a message and by what channels”;
Schwarm, paragraph 80, “the training database 306 includes, for each company and/or company employee, one or more win/loss values … and company data.”, Schwarm teaches training a classifier on prior response/non-response data together with associated company/employee data, and then using the trained classifier to output a prediction for a given company employee as to whether that employee is likely to respond to a message. Thus, Schwarm teaches outputting a prediction, for an input possible candidate and input contact, regarding whether the input possible candidate eventually will take the action of interest.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg by incorporating teachings of Schwarm and include methods of calculating probability scores for prioritized candidate selection. A motivation for the combination would have to be the use of quantifying the prospects likelihood of response to dynamically rank and select the most promising targets. (Schwarm, paragraph 119, “Once the new customer database is enriched with firmographic data , at operation 406 the prediction engine is configured to employ the trained classifier to calculate a probability score for companies from the database of company names 405 likely to purchase a given product or a given service .”)
Caraviello, in the same field of machine learning classifier implementation further teaches:
tuning one or more parameters of the machine learning model based at least on a portion of the data set and an evaluation metric; (Caraviello, paragraph 140, “An aspect of the modeling methods disclosed herein is that because a single algorithm may not always be the best option for modeling every data set, the framework presented herein uses cross-validation techniques, ROC curves and precision and recall to choose the best algorithm for each data set from various options within the field of machine learning. In an embodiment, several algorithms and parameter settings may be compared using cross-validation, ROC curves and precision and recall, during model development.”, the evaluation metric here is the ROC curve constructed by plotting the true-positive rate versus false-positive rate across thresholds. This teaches splitting off part of the data for testing (cross-validation), generating ROC curves for each candidate parameter configuration, and then selecting the configuration whose ROC performance (Area-Under-Curve) is highest as the tuned parameter set.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg and Schwarm by incorporating teachings of Caraviello and include methods of calculating machine learning parameters based on the ROC (receiver operating characteristics) curve. A motivation for the combination would have to create a system suited for large datasets, allowing for optimal parameter configuration for a certain dataset. (Caraviello, paragraph 76, “Moreover, data mining using the approaches described herein provides a flexible, scalable framework for modeling with datasets that include features based on molecular genetic markers. This framework is flexible because it includes tests (i.e. cross-validation and ROC curves) to determine which algorithm and specific parameter settings should be used for the analysis of a data set. This framework is scalable because it is suitable for very large datasets.”)
Schwarm further teaches:
and outputting the machine learning model. (Schwarm, paragraph 86, “the classifier model builder then outputs the trained company predictive profile classifier to a prediction engine”; Schwarm, paragraph 119, “the classifier engine outputs the trained classifier with learned responder characteristics to a prediction engine.” Schwarm expressly identifies the claimed output as the trained classifier, i.e., the trained machine learning model, being output to the prediction engine after training.)
Claim 15 is substantially similar to claim 3, as such a similar analysis applies.
Claim 17: Garg, Schwarm and Caraviello teaches limitations of claim 14. Caraviello further teaches:
The method of claim 14, wherein tuning the one or more parameters comprises applying k-fold cross validation to at least the portion of the data set. (Caraviello, paragraph 253, “In an embodiment of cross-validation, a k-fold cross-validation method is an improvement over the holdout method. The data set is divided into k subsets, and the holdout method is repeated k times. The average error across the k trials is then computed. Each record is part of the testing set once, and is part of the training set k-1 times. This method is less sensitive to the way in which the data set is divided, but the computational cost is k times greater than with the holdout method.”, Caraviello employs k-fold cross validation to at least a portion of the data set.)
The rationale to combine Garg with Caraviello is similar to that applied in claim 14 above.
Claim 18: Garg, Schwarm and Caraviello teaches limitations of claim 14. Caraviello further teaches:
The method of claim 14, wherein the evaluation metric comprises an area under a receiver operating characteristic curve metric. (Caraviello, paragraph 140, “An aspect of the modeling methods disclosed herein is that because a single algorithm may not always be the best option for modeling every data set, the framework presented herein uses cross-validation techniques, ROC curves and precision and recall to choose the best algorithm for each data set from various options within the field of machine learning. In an embodiment, several algorithms and parameter settings may be compared using cross-validation, ROC curves and precision and recall, during model development.”, the evaluation metric is explicitly an ROC curve in Caraviello.)
The rationale to combine Garg with Caraviello is similar to that applied in claim 14 above.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable by Garg in view of Orun, Schwarm, and further in view of Borisyuk et al. (US20170236073A1), hereafter referred to as Borisyuk.
Claim 7: Garg, Orun, and Schwarm teaches limitations of claim 1. Borisyuk, in the same field of candidate selection via machine learning, further teaches:
The computing system of claim 1, further comprising instructions executable to determine the threshold probability based at least on one or both of a precision or a recall of the machine learning model. (Borisyuk, paragraph 70, “In operation 350, the tuning component 250 generates a tuned behavioral index model from the behavioral index model. The tuned behavioral index model may be generated by tuning a threshold of the behavioral index model using a precision recall curve. The precision recall curve can be constructed by measuring precision and recall values. The precision recall curve and the values for precision and recall may depend on different values of a threshold parameter of the process used to generate the behavioral index model by the model component 240. The tuning component 250 may use one or more inputs for threshold selection and generation of the tuned behavioral index model. In some instances, the one or more inputs comprise a specified recall value and a training data set.”,)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg, Orun, and Schwarm by incorporating the teachings of Borisyuk and include methods of threshold tuning using precision-recall curves. The motivation for this combination is to enable the system to automatically optimize its selection threshold for better performance by leveraging well-established machine learning evaluation metrics (precision and recall). (Borisyuk, paragraph 70, “The tuned behavioral index model may be generated by tuning a threshold of the behavioral index model using a precision recall curve. The precision recall curve can be constructed by measuring precision and recall values. The precision recall curve and the values for precision and recall may depend on different values of a threshold parameter of the process used to generate the behavioral index model…”)
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable by Garg in view of Schwarm and Caraviello and further in view of Chawla et al. (Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.), hereafter referred to as Chawla.
Claim 16: Garg, Schwarm, and Caraviello teaches limitations of claim 14. Chawla, in the same field of machine learning class balancing, further teaches:
The method of claim 14, wherein each previous possible candidate of the plurality of previous possible candidates belongs to one of two imbalanced classes, further comprising balancing the imbalanced classes. (Chawla, page 331, section 4.3, “The majority class is under-sampled by randomly removing samples from the majority class population until the minority class becomes some specified percentage of the majority class. This forces the learner to experience varying degrees of under-sampling and at higher degrees of under-sampling the minority class has a larger presence in the training set. In describing our experiments, our terminology will be such that if we under-sample the majority class at 200%, it would mean that the modified dataset will contain twice as many elements from the minority class as from the majority class; that is, if the minority class had 50 samples and the majority class had 200 samples and we under-sample majority at 200%, the majority class would end up having 25 samples. By applying a combination of under-sampling and over-sampling, the initial bias of the learner towards the negative (majority)class is reversed in the favor of the positive (minority)class.”, this explicitly teaches balancing two imbalanced classes by reducing the majority-class size to a target ratio and over-sampling the minority class via SMOTE.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg, Schwarm , and Caraviello by incorporating the teachings of Chawla, to apply state-of-the-art class balancing techniques such as over-sampling the minority class and under-sampling the majority class. The motivation for this combination is to correct class imbalance and improve classifier sensitivity and accuracy in candidate selection, as demonstrated by Chawla. (Chawla, page 331, section 4.3, “The majority class is under-sampled by randomly removing samples from the majority class population until the minority class becomes some specified percentage of the majority class… By applying a combination of under-sampling and over-sampling, the initial bias of the learner towards the negative (majority) class is reversed in the favor of the positive (minority) class.”, Integrating Chawla’s SMOTE and class balancing methods into Schwarm’s approach would address real-world data imbalance, improving both model fairness and predictive performance.)
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable by Garg in view of Schwarm and Caraviello and further in view of Wu et al., (Wu, Y., Dobriban, E., & Davidson, S. (2020, November). Deltagrad: Rapid retraining of machine learning models. In International Conference on Machine Learning (pp. 10355-10366). PMLR.), hereafter referred to as Wu.
Claim 19: Garg, Schwarm, and Caraviello teaches limitations of claim 18. Schwarm further teaches:
The method of claim 18, wherein the input contact is a member of an organization, (Schwarm, paragraph 75, “In at least one of the various embodiments the business entity information data base 304 can include one or more databases linked to business entity information , for example an employee data base 37 including employee names and title or management code , a contact database 38 of contacts for employees of companies ( e . g . email , mobile device IDs , phone ) ,”)
Wu, in the same field of machine learning data updating, further teaches:
further comprising collecting a replacement data set regarding a plurality of previous possible candidates that are members of the organization, and replacing at least a portion of the data set with the replacement data. (Wu, page 3, section 2.1, “After training on the full dataset, the training samples with indices R = {i1, i2, . . . , ir} are removed, where r < n. Our goal is to efficiently update the model parameter to the minimizer of the new empirical loss. Our algorithm also applies when r new datapoints are added.”, While Wu describes removing or adding arbitrary datapoints to update a machine learning model, Schwarm’s system is directed to candidate selection for organizations—where the “previous possible candidates” and their data belong to the client organization’s records. Combining Wu’s DeltaGrad approach with Schwarm’s system would mean periodically collecting an updated or “replacement” dataset of candidate outcomes (for purposes of updating candidate enrichment data), then retraining or updating the model on this revised set. Thus, when Schwarm’s system substitutes older candidate data with a fresh set of previous candidates from the organization, Wu’s method provides an efficient way to replace at least a portion of the original data with this new “replacement” set—thereby satisfying Claim 19’s step of “collecting a replacement data set regarding a plurality of previous possible candidates that are members of the organization, and replacing at least a portion of the data set with the replacement data.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg, Schwarm, and Caraviello by incorporating the teachings of Wu, enabling efficient retraining of models on updated or replacement candidate data. The motivation for this combination is to support scalable, up-to-date machine learning candidate selection systems by allowing for the addition or removal of candidate data points and rapid retraining, as taught by Wu. (Wu, abstract, “Machine learning models are not static and may need to be retrained on slightly changed datasets, for instance, with the addition or deletion of a set of datapoints… We propose the DeltaGrad algorithm for rapid retraining of machine learning models based on information cached during the training phase.”)
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable by Garg and Schwarm in view of Caraviello and further in view of Borisyuk.
Claim 20: Garg, Schwarm, and Caraviello teaches limitations of claim 14. Borisyuk, in the same field of candidate selection via machine learning, further teaches:
The method of claim 14, wherein training the machine learning model comprises using a logistic loss function. (Borisyuk, paragraph 69, “the model component 240 may learn the one or more weights using the logistic regression classifier. In such instances, the model component 240 may use a logistic regression classifier, which may be represented in whole or in part by Equation 3 below.”, training to learn weights involve using a logistic regression classifier.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Garg, Schwarm, and Caraviello by incorporating the teachings of Borisyuk, using logistic regression and associated loss functions to train candidate selection models. The motivation for this combination is to leverage established, interpretable machine learning techniques for candidate scoring and ranking, as described by Borisyuk (Borisyuk, paragraph 69, “The model component 240 may learn the one or more weights using the logistic regression classifier. In such instances, the model component 240 may use a logistic regression classifier, which may be represented in whole or in part by Equation 3 below.”, Combining this with Schwarm would enhance model transparency and reliability in candidate selection scenarios.)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200065857 A1
US 20210097493 A1
US 10540607 B1
US 20140066044 A1
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HYUNGJUN B YI whose telephone number is (703)756-4799. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.B.Y./Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights