DETAILED ACTION
Status of the Claims
The following is a non-final Office Action in response to claims filed 24 April 2024.
Claims 1-20 are pending.
Claims 1-20 have been examined.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-5, 8-14, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupta (US PG Pub. 20190102820) further in view of Noh et al. (US PG Pub. 2021/0319375).
As per claims 1, 9, and 17, Gupta discloses a method comprising: at a computer system comprising a processor and a computer-readable medium; a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to perform actions comprising; and a computing system, comprising one or more processors; and a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by the one or more processors, cause the one or more processors to perform actions comprising (machine readable medium, software, memory, processor, system, Gupta ¶43; network, ¶45):
accessing user data describing characteristics of a plurality of users of an online system (For example, in some embodiments, a churn prediction engine 112 on recommendation server 108 can receive as input the activity indicators from application 104 and other applications during trial periods for the applications. Information received from an application 104 can be stored as statistics 118. For example, statistics 118 may include statistics derived from the actions described above such as counts of various indicators of a user's interaction with an application interface, counts of application functions performed, application results statistics, etc., Gupta ¶21; see also activity indicators, ¶17-¶18);
applying a first machine-learning model on the user data to identify a churn score of each of the plurality of users, the churn score of each user indicating a probability that a corresponding user will discontinue their use of the online system within a time period (For example, statistics 118 may include statistics derived from the actions described above such as counts of various indicators of a user's interaction with an application interface, counts of application functions performed, application results statistics, etc. Additionally, churn prediction engine 112 can receive external information 128. External information 128 can comprise date, time, day of week, weekend info, etc. External information 128 can also include demographic information about users (age, sex, race, income etc), pay dates, weather information etc. All of the above-described indicators—local and global, immediate and historic, user specific and general population, can be analyzed by a machine learning component of churn prediction engine 112 to determine a score that indicates a likelihood that the user will churn within some time period (e.g. in the next day, week or month). Churn prediction engine 112 can use the input data to update one or more machine learning models 120. In some embodiments, there is a machine learning model 120 per application being measured. Churn prediction engine 112 can compare a particular user's activities during a trial period of an application 104 with information from a model 120 for the application 104 in order to determine a probability regarding whether or not the particular user will go on to subscribe or purchase the application, Gupta ¶21);
applying a second machine-learning model on user data describing characteristics of the user, the churn score of the user, and the error signal from the user to select one or more corrective actions from a set of corrective actions that are applicable to users, wherein the second machine-learning model is trained to (machine learning algorithm to learn desirable actions, Gupta ¶26):
access the set of corrective actions that are applicable to users, wherein an actual impact of each corrective action in the set of corrective actions on reducing a churn score of the user is uncertain when the corrective action were to be applied to the user (the server recommendation engine can select a best known action at the time of the selection. The best known action can be based on a score associated with the actions in the library of actions. Selection of actions after an initial time period can be based on the best known action at the time. At block 204, the action selected in block 202 is manifested in the client, thereby changing the application environment for the user. This can take the form of a popup or a change in the color, text or screen of the application. At block 206, user engagement is measured after the computing device environment is updated. In some embodiments, various types of user engagement can be measured and a score associated with an action can be updated based on the type of user engagement. For example, if the recommended action results in more user interaction with the application, then the score associated with the action can be adjusted upwards. If the recommend action results in less interaction with the application, the score can be adjusted downwards. If the recommended action results in a paid conversion (i.e., purchase or subscription of the application), the score can be adjusted upwards. If the recommended action results in churn (i.e., the user uninstalls or does not purchase/subscribe to the application) the score can be adjusted downwards. The following table illustrates example score adjustments in particular embodiments, Gupta ¶26-¶28);
generate an error correction score for each corrective action in the set of corrective actions based on the user data describing the characteristics of the user, the churn score of the user, and the error signal from the user, wherein the error correction score indicates a reduction of churn score of the user after applying the corrective action to the user (the server recommendation engine can select a best known action at the time of the selection. The best known action can be based on a score associated with the actions in the library of actions. Selection of actions after an initial time period can be based on the best known action at the time. At block 204, the action selected in block 202 is manifested in the client, thereby changing the application environment for the user. This can take the form of a popup or a change in the color, text or screen of the application. At block 206, user engagement is measured after the computing device environment is updated. In some embodiments, various types of user engagement can be measured and a score associated with an action can be updated based on the type of user engagement. For example, if the recommended action results in more user interaction with the application, then the score associated with the action can be adjusted upwards. If the recommend action results in less interaction with the application, the score can be adjusted downwards. If the recommended action results in a paid conversion (i.e., purchase or subscription of the application), the score can be adjusted upwards. If the recommended action results in churn (i.e., the user uninstalls or does not purchase/subscribe to the application) the score can be adjusted downwards. The following table illustrates example score adjustments in particular embodiments, Gupta ¶26-¶28) (Examiner notes the score associated with an action as the error correction score);
select a corrective action from the set of corrective actions based on the generated error correction scores (best known action, Gupta ¶26);
applying the selected one or more corrective actions to the user (best known action, Gupta ¶26);
collecting updated user data describing user engagement or disengagement with the online system following the applying the selected one or more corrective actions (learn desirable actions, Gupta ¶26; updating scores, ¶28-¶29 and Table 1); and
retraining the first machine-learning model or the second machine-learning model based on the updated user data (learn desirable actions, Gupta ¶26; updating scores, ¶28-¶29 and Table 1).
Gupta does not expressly disclose receiving an error signal from a client device of a user among the plurality of users, the error signal indicating a system error that occurred at the online system; responsive to receiving the error signal.
However, Noh teaches receiving an error signal from a client device of a user among the plurality of users, the error signal indicating a system error that occurred at the online system; responsive to receiving the error signal (error counts, customer calls, Noh ¶25).
Both the Noh and Gupta references are analogous in that both are directed towards/concerned with churn prediction and remedial actions. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use Noh’s ability to account for users encounter errors or issues in Gupta’s churn prevention system to improve the system and method with reasonable expectation that this would result in a system that is able to predict and prevent churn.
The motivation being that while network operators typically have access to a great amount of historical data, it is not always clear what data to use, or how to interpret the data to predict future behavior of other customers. Additionally, while certain empirical data may be available for a churner, the disconnecting customers do not always provide an indication of their reason(s) for leaving. Thus, there may be a great deal of data available, and yet not a specific reason as to why a customer became dissatisfied (Noh ¶5).
As per claims 2, 10, and 18, Gupta and Noh disclose as shown above. Gupta further discloses wherein the method further comprises sending the selected one or more corrective actions to a client device of the user, causing the client device of the user to display the selected one or more corrective actions (provide action to user, Gupta ¶22; tips, insights, ¶23).
As per claims 3, 11, and 19, Gupta and Noh disclose as shown above. Noh further teaches the first model comprises a logistic regression model or an extreme gradient boosting (XGB) model (logistic regression, Noh ¶31).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use Noh’s ability to utilize logistic regression in Gupta’s churn prevention system to improve the system and method with reasonable expectation that this would result in a system that is able to utilize a specific type of machine learning algorithm.
The motivation being that while network operators typically have access to a great amount of historical data, it is not always clear what data to use, or how to interpret the data to predict future behavior of other customers. Additionally, while certain empirical data may be available for a churner, the disconnecting customers do not always provide an indication of their reason(s) for leaving. Thus, there may be a great deal of data available, and yet not a specific reason as to why a customer became dissatisfied (Noh ¶5).
As per claims 4, 12, and 20, Gupta and Noh disclose as shown above. Noh further teaches wherein the method further comprises: identifying whether the churn score of the user is greater than a threshold, and responsive to identifying that the churn score is greater than the threshold, applying the second machine-learning model on user data describing characteristics of the user and the churn score of the user (threshold, churner, non-churner, Noh ¶63).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use Noh’s ability to use thresholds in Gupta’s churn prevention system to improve the system and method with reasonable expectation that this would result in a system that is able to predict and prevent churn.
The motivation being that while network operators typically have access to a great amount of historical data, it is not always clear what data to use, or how to interpret the data to predict future behavior of other customers. Additionally, while certain empirical data may be available for a churner, the disconnecting customers do not always provide an indication of their reason(s) for leaving. Thus, there may be a great deal of data available, and yet not a specific reason as to why a customer became dissatisfied (Noh ¶5).
As per claims 5 and 13, Gupta and Noh disclose as shown above. Gupta further discloses wherein the set of corrective actions comprise coupons available at current time (date and time, over a time period, Gupta ¶21).
As per claims 8 and 16, Gupta and Noh disclose as shown above. Gupta further discloses wherein the second machine-learning model is trained to identify one or more corrective actions, how many times each of the one or more corrective actions is to be applied, and an order in which the one or more corrective actions are to be applied (the server recommendation engine can select a best known action at the time of the selection. The best known action can be based on a score associated with the actions in the library of actions. Selection of actions after an initial time period can be based on the best known action at the time. At block 204, the action selected in block 202 is manifested in the client, thereby changing the application environment for the user. This can take the form of a popup or a change in the color, text or screen of the application. At block 206, user engagement is measured after the computing device environment is updated. In some embodiments, various types of user engagement can be measured and a score associated with an action can be updated based on the type of user engagement. For example, if the recommended action results in more user interaction with the application, then the score associated with the action can be adjusted upwards. If the recommend action results in less interaction with the application, the score can be adjusted downwards. If the recommended action results in a paid conversion (i.e., purchase or subscription of the application), the score can be adjusted upwards. If the recommended action results in churn (i.e., the user uninstalls or does not purchase/subscribe to the application) the score can be adjusted downwards. The following table illustrates example score adjustments in particular embodiments, Gupta ¶26-¶28).
Claim(s) 6-7 and 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupta (US PG Pub. 20190102820) and Noh et al. (US PG Pub. 2021/0319375) further in view of Gomes et al. (US PG Pub. 2024/0029088).
As per claims 6 and 14, Gupta and Noh disclose as shown above. The combination of Gupta and Noh do not expressly disclose wherein the second machine-learning model is a reinforcement learning model.
However, Gomes teaches wherein the second machine-learning model is a reinforcement learning model (reinforcement model for selecting retention actions for customers, Gomes ¶7).
The Gomes, Noh, and Gupta references are analogous in that both are directed towards/concerned with churn prediction and remedial actions. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use Gomes’ ability to utilize different types of models such as reinforcement learning in Noh’s and Gupta’s churn prevention system to improve the system and method with reasonable expectation that this would result in a system that is able to predict and prevent churn.
The motivation being that while network operators typically have access to a great amount of historical data, it is not always clear what data to use, or how to interpret the data to predict future behavior of other customers. Additionally, while certain empirical data may be available for a churner, the disconnecting customers do not always provide an indication of their reason(s) for leaving. Thus, there may be a great deal of data available, and yet not a specific reason as to why a customer became dissatisfied (Noh ¶5).
As per claims 7 and 15, Gupta and Noh disclose as shown above. The combination of Gupta and Noh do not expressly disclose wherein the second machine-learning model is a multi-armed bandit contextual learning model, and wherein the multi-armed bandit contextual learning model is trained to predict a likelihood of user interaction with the online system in response to taking a corrective action.
However, Gomes teaches wherein the second machine-learning model is a multi-armed bandit contextual learning model, and wherein the multi-armed bandit contextual learning model is trained to predict a likelihood of user interaction with the online system in response to taking a corrective action (Some embodiments also explore and learn the best actions to be taken in order to prevent the predicted churns (referred to as customer retention). Some embodiments model the problem as a Multi-Armed Bandit (MAB) problem and treat it using reinforcement learning techniques, Gomes ¶31).
The Gomes, Noh, and Gupta references are analogous in that both are directed towards/concerned with churn prediction and remedial actions. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use Gomes’ ability to utilize different types of models such as multi-armed bandit contextual learning model in Noh’s and Gupta’s churn prevention system to improve the system and method with reasonable expectation that this would result in a system that is able to predict and prevent churn.
The motivation being that while network operators typically have access to a great amount of historical data, it is not always clear what data to use, or how to interpret the data to predict future behavior of other customers. Additionally, while certain empirical data may be available for a churner, the disconnecting customers do not always provide an indication of their reason(s) for leaving. Thus, there may be a great deal of data available, and yet not a specific reason as to why a customer became dissatisfied (Noh ¶5).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure (additional art can be located on the PTO-892):
Orr et al. (US PG Pub. 2022/0138780) Churn prediction with machine learning.
Zhu et al. (US PG Pub. 2023/0206155) Data driven customer churn analytics.
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to ANDREW B WHITAKER whose telephone number is (571)270-7563. The examiner can normally be reached on M-F, 8am-5pm, EST.
If attempts to reach the examiner by telephone are unsuccessful, the Examiner’s supervisor, Lynda Jasmin can be reached on (571) 272-6782. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-
automated- interview-request-air-form
/ANDREW B WHITAKER/Primary Examiner, Art Unit 3629
Prosecution Insights