DETAILED ACTION
This action is in response to the application filed 30 November 2023.
Claims 1–20 are pending. Claims 1, 9, and 15 are independent.
Claims 1–20 are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after 16 March 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections—35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1, 2, 5–7, and 9–12 are rejected under 35 U.S.C. § 103 as being unpatentable over Ranjan et al. (US 2024/0020717 A1) [hereinafter Ranjan] in view of Wu et al. (US 2020/0151746 A1) [hereinafter Wu].
Regarding independent claim 1, Ranjan teaches [a] method comprising: applying a propensity model to a subject vector to generate a propensity value estimating a probability that a subject will perform an action, wherein the subject vector comprises a data structure having a plurality of features storing information regarding the subject; One or more machine learning models that output a churn score indicating the likelihood of a churn event (Ranjan, ¶ 36). The churn event may be an action, e.g., canceling a membership (Ranjan, ¶ 36). applying, after applying the propensity model to the subject vector, a Shapley additive explanation tool to the propensity model to generate a subset of the plurality of features that contributed to the propensity value more than a remaining set of the plurality of features; A SHAP [Shapley additive explanation] model may be applied to the trained and validated machine learning model (Ranjan, ¶ 40). The SHAP model generates explainability data, including Shapley values that characterize a contribution of each feature of the features of the machine learning model (Ranjan, ¶¶ 43, 79, 80). selecting an actionable feature from the subset of the plurality of features, wherein the actionable feature comprises a feature in the subset that an entity is able to influence; For each feature and associated explainability data, a determination is made whether to implement an operation to modify the feature in order to adjust the churn score (Ranjan, ¶ 44). […] presenting the actionable feature and the output. An analyst receives a graphical representation of the output, including membership users having a particular churn score, the features and feature values, the explainability data, etc. (Ranjan, ¶¶ 49–52).
Ranjan teaches generating explainability data for a machine learning model that predicts user behavior, but does not expressly teach a correlation model as claimed. However, Wu teaches: applying a correlation model to labels for training data and the actionable feature to generate an output that describes a reason why the subject performs the action; and A propensity model is used to generate a user–reason code matrix that provides a contribution value for each reason code indicating the impact of each variable/reason code on a particular outcome, e.g., churn (Wu, ¶ 69). The reason code is a label for a variable/feature (Wu, ¶¶ 17, 56). A topic model [correlation model] is applied to a user–reason code matrix to determine clusters of users and associated latent characteristics, with contribution levels for each reason code for each topic (Wu, ¶¶ 70–71). The topic matrix defines segments, e.g., groups of customers, having similar reasons for a particular outcome [action], e.g., customers with similar reasons to churn (Wu, ¶ 85). The topic matrix can be translated into descriptive terms and displayed as a table visualization (Wu, ¶¶ 86, 91, FIG. 7D).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Ranjan with those of Wu. One would have been motivated to do so in order to improve the analysis, e.g., by providing better information regarding the reasons behind the outcomes (Wu, ¶ 2).
Regarding dependent claim 2, the rejection of claim 1 is incorporated and Ranjan/Wu further teaches: further comprising: displaying, on a display device, the propensity value, the actionable feature, and the output. The output data is transmitted to a mobile computing device of an analyst, which generates a graphical representation (Ranjan, ¶ 49). The membership computing device also includes a display (Ranjan, ¶ 53). A presentation component presents data indications to a user, e.g., on a display device (Wu, ¶ 94).
Regarding dependent claim 5, the rejection of claim 1 is incorporated and Ranjan/Wu further teaches: further comprising: generating a target plot from the training data corresponding to the actionable feature; and A mobile computing device of an analyst generates a graphical representation of the distribution of features of, e.g., a group of users having a certain churn score range (Ranjan, ¶ 51). displaying the target plot on a display device. The graphical representation is displayed on the mobile computing device (Ranjan, ¶ 51).
Regarding dependent claim 6, the rejection of claim 5 is incorporated and Ranjan/Wu further teaches: wherein applying the correlation model comprises: applying the correlation model to the labels for training data and the actionable feature to correlate a highest average propensity value for the action by a plurality of users to a common attribute among the plurality of users; and The topic model output may be used to sort reason codes by contribution level, with users sorted into segments based on sharing similar reasons; customers can be assigned to the topic having the highest attribution value (Wu, ¶ 85). returning the common attribute as the output describing the reason why the subject performs the action. The topic matrix are translated into descriptive terms to convey why the customer was assigned to the topic (Wu, ¶ 86).
Regarding dependent claim 7, the rejection of claim 1 is incorporated and Ranjan/Wu further teaches: further comprising: generating a target plot prior to applying the correlation model, wherein generating the target plot comprises: A mobile computing device of an analyst generates a graphical representation of the distribution of features of, e.g., a group of users having a certain churn score range (Ranjan, ¶ 51). receiving feature data from a plurality of subjects, wherein the feature data corresponds to the actionable feature, and A plurality of churn scores are generated, each score for a user (Ranjan, ¶ 36). distributing the feature data into a plurality of bins, wherein each of the plurality of bins represents a range of feature values for the actionable feature for the plurality of subjects, and wherein each of the plurality of bins is associated with a value representing an average propensity value for the action for the plurality of subjects; and Users are placed into bins/cohorts based on their churn score, wherein each bin/cohort corresponds to a range of churn score (Ranjan, ¶ 41). displaying the target plot on a display device. The graphical representation is generated and displayed by the mobile computing device (Ranjan, ¶ 49).
Regarding independent claim 9, Ranjan teaches [a] system comprising: a processor; A membership computing device having one or more processors (Ranjan, ¶¶ 53–54). a data repository in communication with the processor and storing: A data repository in communication with the membership computing device (Ranjan, ¶¶ 20–22). a subject vector, comprising a data structure having a plurality of features storing information regarding a subject, The data repository stores data regarding users, including user transaction data, membership data, purchase interval data, etc. (Ranjan, ¶¶ 64–65, FIG. 3). a propensity value estimating a probability that the subject will perform an action, The data repository stores output data, which includes churn scores [propensity values], which indicate the likelihood of an occurrence of a churn event [action], such as a user canceling their membership (Ranjan, ¶¶ 72–73). a subset of the plurality of features that contributed to the propensity value more than a remaining set of the plurality of features, The data repository stores explainability data, including subsets of values that are associated with features of a trained machine learning model [that outputs the churn scores], including values that indicate the magnitude of contribution a feature had to the model output (Ranjan, ¶ 77). an actionable feature comprising a feature in the subset that an entity is able to influence, For each feature and associated explainability data, a determination is made whether to implement an operation to modify the feature in order to adjust the churn score (Ranjan, ¶ 44). labels for training data, and The model may be a supervised model [i.e., trained using labeled data] (Ranjan, ¶ 103). an output that describes a reason why the subject performs the action; a propensity model which, when applied by the processor to the subject vector, generates the propensity value; The data repository stores a machine learning database, which includes the machine learning models [that output the churn scores], and machine learning data (Ranjan, ¶ 72). a Shapley additive explanation tool which, when applied by the processor to the propensity model after generating the propensity value, generates the subset of the plurality of features; and The data repository stores a machine learning database, which includes SHAP model data of a SHAP model [Shapley additive explanation tool] (Ranjan, ¶ 76). […]
Ranjan teaches generating explainability data for a machine learning model that predicts user behavior, but does not expressly teach a correlation model as claimed. However, Wu teaches: a correlation model which, when applied by the processor to the labels for training data and the actionable feature, generates the output. A propensity model is used to generate a user–reason code matrix that provides a contribution value for each reason code indicating the impact of each variable/reason code on a particular outcome, e.g., churn (Wu, ¶ 69). The reason code is a label for a variable/feature (Wu, ¶¶ 17, 56). A topic model [correlation model] is applied to a user–reason code matrix to determine clusters of users and associated latent characteristics, with contribution levels for each reason code for each topic (Wu, ¶¶ 70–71). The topic matrix defines segments, e.g., groups of customers, having similar reasons for a particular outcome [action], e.g., customers with similar reasons to churn (Wu, ¶ 85). The topic matrix can be translated into descriptive terms and displayed as a table visualization (Wu, ¶¶ 86, 91, FIG. 7D).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Ranjan with those of Wu. One would have been motivated to do so in order to improve the analysis, e.g., by providing better information regarding the reasons behind the outcomes (Wu, ¶ 2).
Regarding dependent claim 10, the rejection of claim 9 is incorporated and Ranjan/Wu further teaches: further comprising: a display device for displaying the actionable feature and the output. The output data is transmitted to a mobile computing device of an analyst, which generates a graphical representation (Ranjan, ¶ 49). The membership computing device also includes a display (Ranjan, ¶ 53).
Regarding dependent claim 11, the rejection of claim 9 is incorporated and Ranjan/Wu further teaches: further comprising: a server controller which, when applied by the processor to the actionable feature and the output, generates a graphical user interface that displays the actionable feature, the output, and a suggested action. The membership computing device can display a user interface for interaction (Ranjan, ¶ 59).
Regarding dependent claim 12, the rejection of claim 9 is incorporated and Ranjan/Wu further teaches: wherein the subject comprises a subscriber to a software program, and wherein the action comprises the subject canceling a subscription to the software program. The churn scores, churn events, etc. may relate to membership programs, such as membership to an e-commerce platform [software] (Ranjan, ¶¶ 3, 4). The membership may be, e.g., annual or monthly [subscription] (Ranjan, ¶ 24).
Claims 3, 4, and 13 are rejected under 35 U.S.C. § 103 as being unpatentable over Ranjan et al. (US 2024/0020717 A1) [hereinafter Ranjan] in view of Wu et al. (US 2020/0151746 A1) [hereinafter Wu], further in view of Sobolev et al. (US 2023/0412475 A1) [hereinafter Sobolev].
Regarding dependent claim 3, the rejection of claim 1 is incorporated. Ranjan/Wu teaches selecting actionable features, but does not expressly teach using a library of actionable features. However, Sobolev teaches: wherein selecting the actionable feature comprises: comparing the subset of the plurality of features to a library of actionable features, and Tickets are clustered based on, e.g., features (Sobolev, ¶ 29). selecting, from among the subset, the actionable feature corresponding to an entry in the library. A corrective action may be chosen for a ticket based on a mapping from the ticket cluster to a database [library] of actions (Sobolev, ¶ 34).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Ranjan/Wu with those of Sobolev. One would have been motivated to do so in order to increase the likelihood of choosing an action that would result in the customer continuing to use the product (i.e., reduce churn) (Sobolev, ¶ 17).
Regarding dependent claim 4, the rejection of claim 3 is incorporated and Ranjan/Wu/Sobolev further teaches: further comprising: building the library of actionable features from user input. Successful corrective actions are input by users and stored in the database of actions (Sobolev, ¶¶ 24–25).
Regarding dependent claim 13, the rejection of claim 9 is incorporated. Ranjan/Wu teaches selecting actionable features, but does not expressly teach using a library of actionable features. However, Sobolev teaches: wherein the data repository further stores a library of actionable features, and wherein the system further comprises: Successful corrective actions are input by users and stored in the database of actions (Sobolev, ¶¶ 24–25). a server controller which, when applied by the processor to the library and the actionable feature, selects, from among the subset, the actionable feature corresponding to an entry in the library. Tickets are clustered based on, e.g., features (Sobolev, ¶ 29). A corrective action may be chosen for a ticket based on a mapping from the ticket cluster to a database [library] of actions (Sobolev, ¶ 34).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Ranjan/Wu with those of Sobolev. One would have been motivated to do so in order to increase the likelihood of choosing an action that would result in the customer continuing to use the product (i.e., reduce churn) (Sobolev, ¶ 17).
Claims 8, 14, 15, and 18–20 are rejected under 35 U.S.C. § 103 as being unpatentable over Ranjan et al. (US 2024/0020717 A1) [hereinafter Ranjan] in view of Wu et al. (US 2020/0151746 A1) [hereinafter Wu], further in view of Horesh et al. (US 2022/0138592 A1) [hereinafter Horesh].
Regarding dependent claim 8, the rejection of claim 1 is incorporated. Ranjan/Wu teaches training machine learning models, but does not expressly teach explicit feedback. However, Horesh teaches: further comprising: displaying, on a display device, a graphical user interface comprising the propensity value, the actionable feature, and the output; displaying, on the graphical user interface, a widget requesting user input whether the display was helpful; A graphical user interface displays insights generated by a machine learning model; the GUI includes a feedback input prompt [widget] asking the user whether the insight was helpful (Horesh, ¶ 120, FIG. 4). receiving a user response from activation of the widget; and The user selection is taken as feedback (Horesh, ¶ 121). retraining the correlation model based on the user response. The feedback is used as input to the machine learning model, such that the ML model learns what the user deems relevant [is retrained] (Horesh, ¶ 121).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Ranjan/Wu with those of Horesh. One would have been motivated to do so in order to make the analysis more useful to the user by providing them data that they feel is more relevant (Horesh, ¶¶ 120–121).
Regarding dependent claim 14, the rejection of claim 9 is incorporated. Ranjan/Wu teaches training machine learning models, but does not expressly teach explicit feedback. However, Horesh teaches: wherein the data repository further stores a user response, and wherein the system further comprises: A graphical user interface displays insights generated by a machine learning model; the GUI includes a feedback input prompt [widget] asking the user whether the insight was helpful (Horesh, ¶ 120, FIG. 4). The user selection is taken as feedback (Horesh, ¶ 121). a training controller which, when applied by the processor to the user response and the correlation model, retrains the correlation model based on the user response. The feedback is used as input to the machine learning model, such that the ML model learns what the user deems relevant [is retrained] (Horesh, ¶ 121).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Ranjan/Wu with those of Horesh. One would have been motivated to do so in order to make the analysis more useful to the user by providing them data that they feel is more relevant (Horesh, ¶¶ 120–121).
Regarding independent claim 15, this claim recites limitations similar to those of claims 1, 2, and 8, and therefore is rejected for the same reasons.
Regarding dependent claim 18, this claim recites limitations similar to those of claim 5, and therefore is rejected for the same reasons.
Regarding dependent claim 19, this claim recites limitations similar to those of claim 6, and therefore is rejected for the same reasons.
Regarding dependent claim 20, this claim recites limitations similar to those of claim 7, and therefore is rejected for the same reasons.
Claims 16 and 17 are rejected under 35 U.S.C. § 103 as being unpatentable over Ranjan et al. (US 2024/0020717 A1) [hereinafter Ranjan] in view of Wu et al. (US 2020/0151746 A1) [hereinafter Wu], further in view of Horesh et al. (US 2022/0138592 A1) [hereinafter Horesh] and Sobolev et al. (US 2023/0412475 A1) [hereinafter Sobolev].
Regarding dependent claim 16, this claim recites limitations similar to those of claim 3, and therefore is rejected for the same reasons.
Regarding dependent claim 17, this claim recites limitations similar to those of claim 4, and therefore is rejected for the same reasons.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tyler Schallhorn whose telephone number is 571-270-3178. The examiner can normally be reached Monday through Friday, 8:30 a.m. to 6 p.m. (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached at 571-272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (in the USA or Canada) or 571-272-1000.
/Tyler Schallhorn/Examiner, Art Unit 2144
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
Prosecution Insights