Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
DETAILED ACTION
The following FINAL Office Action is in response to communication filed on 4/8/2026.
Priority
The Examiner has noted the Applicant claiming Priority from Provisional Application 63/594,688 filed 10/31/2023.
Status of Claims
Claims 1-7, 9-21 are currently pending.
Claims 1-7, 9 are currently amended.
Claim 8 is cancelled.
Claims 10-21 are new.
Claims 1-7, 9-21 are currently under examination and have been rejected as follows.
IDS
The information disclosure statement filed on 4/25/2023 complies with the provisions of 37 CFR 1.97, 1.98 and MPEP § 609 and is considered by the Examiner.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Response to Amendment
The previously pending statutory double patenting rejection is withdrawn in view of the amendments.
The previously pending rejections under 35 USC 112 are withdrawn in view of the amendments.
The previously pending rejections under 35 USC 101 will be maintained. The 101 rejection is updated in view of the amendments.
New grounds for rejection under 35 USC 103 are applied as necessitated by the amendments.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Response to Arguments
Regarding Applicant’s remarks pertaining to 35 USC 101:
Step 2A Prong 1:
Applicant argues starting on page 4 of remarks 4/8/2026:
“Applicant respectfully disagrees as the relevant recitations do not, per se, recite a mathematical concept, methods of organizing human activity, or a mental process….
“The claim in Example 39 relates to the collection of data, the creation of two training data sets, and an iterative training process with two training processes using the two training data sets….
“In identical fashion to the above example, amended claim 1 does not recite, per se, any mathematical concept (i.e., mathematical relationships, formulas, or calculations), method of organizing human activity, or mental process….
“Applying the revised standards to amended claim 1, Applicant respectfully submits that the claim does not recite, per se, an abstract idea and is, therefore, patent eligible under Prong One of Step 2A.”
Examiner respectfully disagrees. Though similarities are apparent with regard to training a neural network, Example 39 asserts eligibility as no abstract idea as a whole being claimed. As a whole, the present application claims recite, describe, or set forth analyzing customer experience, satisfaction, value, and churn data to assign scores to customers (see Applicant specification ¶ [0007]), thereby prescribing targeted customer engagement based on individual and group characteristics of the customers, falling within mitigating risk as it pertains to fundamental economic principles; marketing or sales activities or behaviors and business relations as they pertain to commercial or legal interactions; and managing relationships or interactions between people. Further, the claims as a whole recite, describe, or set forth mathematical relationships and calculations, including penalty function minimization, computing metrics, normalizing metrics, computing weighted sums, computing experience scores, maximizing correlation, and computing regression values. Accordingly, the claims recite an abstract idea.
Step 2A Prong 2:
Applicant argues starting on page 7 of remarks 4/8/2026:
“Applicant respectfully submits that these additional elements, individually and in ordered combination, integrate any alleged judicial exception into a practical application under Step 2A Prong Two. The claims recite a specific, technical process that constitutes an improvement to existing computer technology for computing and calibrating client experience scores. The improvement is rooted in the particular manner in which a neural network is trained, with specific backpropagation steps recited in the claims, and the trained neural network is then deployed within a penalty function to optimize weighting factors that calibrate the client experience score computation to actual client behavior….
“Applicant further submits that the claimed invention reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field…. The specific combination of neural network training, multi-metric computation from heterogeneous data sources, logistic regression, and penalty function optimization using the trained neural network constitutes a technological improvement in the manner in which client experience scores are computed and calibrated - not merely an abstract business concept applied to a generic computer.”
Examiner respectfully disagrees. The independent claims as amended introduce the additional computer based elements “neural network”, “weighting factor database”, “score database”, “churn database”, and “user interface”. Along with the original additional computer-based elements “semantic knowledge database”, “computer”, “processors”, “memory”, “internal and external data sources”, various “input channels”, and “network connection”, the functions of these additional elements include examples such as analyzing data, connecting the computer to data sources, computing metrics with weighted sums, computing scores, normalizing metrics, computing regression values, minimizing penalty functions, adjusting weight coefficients, prescribing client interactions based on scores, and updating weighting factors. The additional elements are recited at a high level of generality (i.e. as a generic computer performing functions of analyzing data; calculating metrics, scores, and regression values, optimizing functions, etc.) such that they amount to no more than mere instructions to apply the exception using generic computer components. Though Applicant asserts the combination of neural network training, multi-metric computation from heterogeneous data sources, logistic regression, and penalty function optimization using the trained neural network constitutes a technological improvement, the explicit details of the technological improvement itself and descriptive benefits of the alleged improvement remain unclear.
Applicant argues on page 9 of remarks 4/8/2026:
“Similarly [to Enfish v. Microsoft], the claims presented herewith are directed to a specific improvement in the way a computer system trains and deploys a neural network within an optimization pipeline to calibrate multi-metric client experience scores against actual behavioral data….
“Here, the amended claims similarly [to McRo v. Bandai Namco] recite a combined order of specific technical steps, training a neural network with specific backpropagation methodology….”
Examiner respectfully disagrees. Enfish achieved eligibility by demonstrating a specific asserted improvement in computer capability, the self-referential table facilitating faster data searching, more effective data storage, and automated database configuration. McRo achieved eligibility in its decision by demonstrating improvement in character animation technology more accurately automating facial expressions to match speech, as well as avoiding an abstract idea entirely. Similar to above, pertaining to the present invention, the explicit details of the technological improvement itself and descriptive benefits of the alleged improvement remain unclear.
Step 2B:
Applicant argues on page 11 of remarks 4/8/2026:
“These are not generic recitations of well-understood, routine, or conventional activities. While neural networks and logistic regression are known techniques individually, the specific ordered combination claimed, specifically: training a neural network with the recited backpropagation steps, and then integrating the trained neural network into a penalty function that operates on logistic regression outputs and client churn data to optimize the weighting factors used in a multi-metric client experience score calculation, is a specific and unconventional arrangement of known components that provides the specific improvement discussed above….
“As stated at paragraph [0179], the disclosed techniques "provide capabilities for client insight which were not previously available to businesses." The integration of a trained neural network into the penalty function for weighting factor optimization, where the neural network was specifically trained via supervised learning with the recited backpropagation methodology, represents a specific technological contribution that goes
well beyond routine or conventional computer implementation.”
Examiner respectfully disagrees. Training a neural network with backpropagation methodology and logistic regression, incorporating penalty functions and client churn data, etc. are conventional technological methods demonstrated in the references Hall, Matam, Scholz and Gupta (see 103 rejection section for details), as well as elsewhere in the field of modeling consumer sentiment and behavior. Applicant specification discusses several conventional types and applications of neural networks used in the field at ¶ [0070]-[0078]. The specific arrangement of known additional elements in the present invention is asserted by Applicant to provide significantly more than the judicial exception; however, as above, the explicit details of the technological improvement itself and descriptive benefits of the alleged improvement remain unclear.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Regarding Applicant’s remarks pertaining to 35 USC 103:
Applicant argues on page 13 of remarks 4/8/2026:
“Applicant respectfully submits that none of the cited references (Gupta, Scholz, or Matam) individually or in combination, teach or suggest the specific combination of limitations recited in the amended independent claims.”
Examiner respectfully finds the argument moot on new grounds. Examiner provides additional support from original references Gupta, Matam, and particularly Sholtz; and further points to new reference Hall et al. US 20230376981 A1, hereinafter Hall, which in combination or modification teach independent claims 1, 3 as amended and new independent claim 21. See 103 rejection section below for citations and additional details.
Applicant argues on page 17 of remarks 4/8/2026:
“In addition, several of the dependent claims recite further limitations that are not taught or suggested by the cited references.”
Examiner respectfully disagrees. Examiner provides additional support from primary reference Gupta which teaches new claims 10, 13-18; additional support from Sholtz which teaches new claims 11-12; and additional support from Hall which teaches claim 19. Examiner points to new reference Beaufays et al. US 20230177382 A1, hereinafter Beaufays, which teaches claim 20. See 103 rejection section below for citations and additional details.
Accordingly, new grounds for rejection under 35 USC 103 are applied as necessitated by the amendments.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7, 9-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1-2 are directed to a system or machine which is a statutory category.
Claims 3-7, 9-20 are directed to a method or process which is a statutory category.
Claim 21 is directed to a non-transitory computer-readable medium or article of manufacture which is a statutory category.
Step 2A Prong One: The claims recite, describe, or set forth a judicial exception of an abstract idea (see MPEP 2106.04(a)). Specifically, the claims recite, describe or set forth mitigating risk, marketing or sales activities or behaviors, business relations, and managing relationships or interactions between people, including: “computing a first metric… where the plurality of first data sources is client feedback data… and the first metric is computed as a weighted sum of client sentiments… derived from each of a call center… an online chat… client complaints… voice of the customer…”; “computing a second metric… where the plurality of second data sources is client experience key performance indicators (KPIs)… and the second metric is computed as a weighted sum of the client experience KPIs”; “computing a third metric… where the plurality of third data sources is client value key performance indicators… and the third metric is computed as a weighted sum of the client value KPIs”; “computing the client experience score”; “prescribing targeted interactions with clients based on the client experience score”; “periodically updating the weighting factors using an optimization process which maximizes a correlation between the client experience score for each of the clients and a behavioral parameter of a group of the clients, where the behavioral parameter comprises client churn data”; and “updating, based on the client experience score being recomputed, a display of the client experience score”. Analyzing customer experience, satisfaction, value, and churn data to assign scores to customers, thereby prescribing targeted customer engagement based on individual and group characteristics of the customers falls within mitigating risk as it pertains to fundamental economic principles; marketing or sales activities or behaviors and business relations as they pertain to commercial or legal interactions; and managing relationships or interactions between people; each under the larger abstract grouping of Certain Methods of Organizing Human Activity (MPEP 2106.04(a)(2) II).
Furthermore, the claims recite, describe, or set forth mathematical relationships and calculations, including: “minimize a total value of a penalty function”; “the first metric is computed as a weighted sum”; “the second metric is computed as a weighted sum”; “the third metric is computed as a weighted sum”; “normalizing the first metric, the second metric, and the third metric to a common range of values”; “computing the client experience score using a calculation including the first, second and third metrics and first, second and third weighting factors, each of the metrics being multiplied by its corresponding weighting factor in the calculation”; “using an optimization process which maximizes a correlation”; “computing a regression value using a logistic regression equation and minimizing the penalty function, the penalty function penalizing, using the trained neural network, differences between the regression value and the behavioral parameter”; and “wherein, after updating the first, second and third weighting factors, … recomputes the client experience score of all of the clients using the updated weighting factors”. Computing metrics as products of a set of factors such as KPIs and weighted sums, normalizing metrics, maximizing correlation, computing regression values, and minimizing penalty functions fall withing mathematical relationships and calculations under the larger abstract grouping of Mathematical Concepts (MPEP 2106.04(a)(2) I).1 Accordingly, the claims recite an abstract idea.
Step 2A Prong Two: Independent claims 1, 3, 21 recite the following additional elements: “semantic knowledge database”, “computer”, “processors”, “memory”, “internal and external data sources”, various “input channels”, “network connection”, “neural network”, “weighting factor database”, “score database”, “churn database”, and “user interface”. The functions of these additional elements include examples such as analyzing data, connecting the computer to data sources, computing metrics with weighted sums, computing scores, normalizing metrics, computing regression values, minimizing penalty functions, adjusting weight coefficients, prescribing client interactions based on scores, and updating weighting factors. The additional elements are recited at a high level of generality (i.e. as a generic computer performing functions of analyzing data; calculating metrics, scores, and regression values, optimizing functions, etc.) such that they amount to no more than mere instructions to apply the exception using generic computer components. Therefore, these functions can be viewed as not meaningfully different than a business method or mathematical algorithm being applied on a general-purpose computer as tested per MPEP 2106.05(f)(2)(i). The claims are directed to an abstract idea and the judicial exception does not integrate the abstract idea into a practical application.
Step 2B: According to MPEP 2106.05(f)(1), considering whether the claim recites only the idea of a solution or outcome i.e., the claims fail to recite the technological details of how the actual technological solution to the actual technological problem is accomplished. The recitation of claim limitations that attempt to cover an entrepreneurial and thus abstract solution to an entrepreneurial problem with no technological details on how the technological result is accomplished and no description of the mechanism for accomplishing the result do not provide significantly more than the judicial exception.
The additional element “neural network” and “supervised learning” language merely requires execution of an algorithm that can be performed by a generic computer component and provides no detail regarding the operation of that algorithm. As such, the claim requirement amounts to mere instructions to implement the abstract idea on a computer, and, therefore, is not sufficient to make the claim patent eligible. See Alice, 573 U.S. at 226 (determining that the claim limitations “data processing system,” “communications controller,” and “data storage unit” were generic computer components that amounted to mere instructions to implement the abstract idea on a computer); October 2019 Guidance Update at 11–12 (recitation of generic computer limitations for implementing the abstract idea “would not be sufficient to demonstrate integration of a judicial exception into a practical application”). Such a generic recitation of “machine learning model” alone is insufficient to show a practical application of the recited abstract idea.
Dependent claims 2, 4-7, 9-20 do not appear to provide any further additional computer-based elements, let alone for such additional computer-based elements to integrate the abstract idea into practical application (Step 2A Prong Two) or providing significantly more (Step 2B).
Further, dependent claims 2, 4-7, 9-20 merely incorporate the additional elements recited in claims 1, 3 along with further narrowing of the abstract idea of claims 1, 3 and their execution of the abstract idea. Specifically, the dependent claims narrow the “semantic knowledge database”, “computer”, “processors”, “memory”, “internal and external data sources”, various “input channels”, “network connection”, “neural network”, “weighting factor database”, “score database”, “churn database”, and “user interface” to capabilities such as computing, summing, deriving, including, receiving, normalizing, maximizing, minimizing, and identifying various forms of data such as clients, feedback, sentiments, online chat, complaints, voices, KPIs, factors, ranges, correlations, parameters, scores, values etc. which, when evaluated per MPEP 2106.05(f)(2) represent mere invocation of computers to perform existing processes. Therefore, the additional elements recited in the claimed invention individually and in combination fail to integrate a judicial exception into a practical application (Step 2A prong two) and for the same reasons they also fail to provide significantly more (Step 2B). Thus, claims 1-7, 9-21 are reasoned to be patent ineligible.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
REJECTIONS BASED ON PRIOR ART
Examiner Note: Some rejections will contain bracketed comments preceded by an “EN” that will denote an examiner note. This will be placed to further explain a rejection.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-8 are rejected under 35 U.S.C. 103 as being unpatentable over:
Gupta et al. US 20220253777 A1, hereinafter Gupta in view of
Scholz US 20170257285 A1, hereinafter Scholz,
Matam et al. US 20180268318 A1, hereinafter Matam, and
Hall et al. US 20230376981 A1, hereinafter Hall. As per,
Regarding claim 1: Gupta teaches:
A system for computing a client experience score from structured and unstructured datatypes of a semantic knowledge database (Gupta ¶ [0024]: Embodiments disclosed herein relate to systems, devices, and methods for generating and dynamically updating an experience score for a client, where the experience score operates as a quantitative indicator describing a relationship between the client and an entity. Mid-¶ [0051]: the system is configured to recognize one or more predefined keywords that are known to be associated with particular sentiment of the client [EN: semantic knowledge] (i.e. where certain keywords indicate or contribute to a positive or negative sentiment score). ¶ [0052]: The data 300 can also be in the form of structured data 310 or unstructured data 315. ¶ [0107]: Act 1010 includes using natural language processing (NLP) to provide structure to the unstructured sentiment data. As a consequence, a second set of structured sentiment data (e.g., the set of structured sentiment data 415A [EN: semantic knowledge] from FIG. 4) is acquired), said system comprising:
a computer with one or more processors and memory, where the computer is configured to analyze data received from internal and external data sources via multiple input channels (Gupta ¶ [0046]: In some cases, the sources can be linked to an entity's platform [EN: internal data sources], such as a website that offers clients the opportunities to leave feedback. In other scenarios, the sources are independent relative to the entity [EN: external data sources], such as the case where the source is in the form of a social media platform or some other independent entity. In such scenarios, the interactions engine 140 can be configured to crawl 150 a public network (e.g., the Internet) to identify other sources (e.g., indirect sources or third-party sources) where a user may have expressed his/her viewpoints about a particular entity. For instance, FIG. 1 shows the interactions engine 140 crawling a network 155 to identify a source 160 that is entirely independent of the entity 130 and that can be a third-party source. From this source 160, the interactions engine 140 can acquire additional sentiment data. Accordingly, in this sense, the disclosed interactions engine 140 and ML engine 140A can be configured to perform big data mining 165 and analysis); and
a network connection operatively connecting the external data sources to the computer (Gupta mid-¶ [0046]: For instance, FIG. 1 shows the interactions engine 140 crawling a network 155 to identify a source 160 that is entirely independent [EN: external] of the entity 130 and that can be a third party source),
where the computer is configured to perform steps including:
[..]
the training including:
[..]; and
for each client in a client base:
computing a first metric from a plurality of first data sources (See Gupta Fig. 5 showing multiple metrics, or scores, generated from various data sources) where
the plurality of first data sources is client feedback data received from the external data sources via the multiple input channels (Gupta ¶ [0064]: FIG. 5 shows a set of input 500, which is representative of the input 410 from FIG. 4. The input 500 can include data from any number of sources and can include expressions made by a client), and
the first metric is computed as a weighted [..] client sentiments each derived from a different one of the input channels (Gupta ¶ [0064]: Returning to FIG. 4, the process flow 400 includes the ML engine 405 generating and applying weight(s) 435 to each respective input type [EN: input channel]. That is, the ML engine generates the weighting factors…. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A. FIG. 5 provides a useful illustration), and where
the client sentiments include a sentiment derived from each of a call center input channel, an online chat input channel, a client complaints input channel, a voice of the customer input channel, and two different mobile device application store input channels (Gupta mid-¶ [0064]: As some examples, the input 500 [EN: input channel] can be acquired from messages 505 [EN: mobile device application], a webchat 510, a survey 515 [EN: complaints], reviews 520 [EN: complaints], social media 525 [EN: mobile device application], texts 530 [EN: mobile device application], voicemail 535 [EN: call center, customer voice], email 540 [EN: mobile device application]. ¶ [0047]: In some scenarios, such as where a client has enabled microphone access associated with the entity or a separate application that is linked to the entity, the system is able to gather feedback data, through automatic speech [EN: voice] recognition, by identifying relevant speech signals recorded by a client's microphone (i.e. if the client is speaking about his or her interaction with the entity));
computing a second metric from a plurality of second data sources (See Gupta Fig. 5 showing multiple metrics, or scores, generated from various data sources), where
the plurality of second data sources is client experience key performance indicators (KPIs) received from the internal data sources (Gupta ¶ [0046]: ¶ [0046]: In some cases, the sources can be linked to an entity's platform [EN: internal data sources], such as a website that offers clients the opportunities to leave feedback [EN: experience KPI]. Mid-¶ [0049]: For instance, one type of interaction can be a user's response to a survey [EN: experience KPI], and the [EN: internal data] "source" would be the survey… Another type of interaction can be a user providing feedback in a website, and the [EN: internal data] source would be the website), and
the second metric is computed as [..] weighted [..] client experience KPIs (Gupta ¶ [0064]: Returning to FIG. 4, the process flow 400 includes the ML engine 405 generating and applying weight(s) 435 to each respective input type. That is, the ML engine generates the weighting factors…. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A. FIG. 5 provides a useful illustration);
computing a third metric from a plurality of third data sources (See Gupta Fig. 5 showing multiple metrics, or scores, generated from various data sources), where
[..];
normalizing the first metric, the second metric, and the third metric to a common range of values (See Gupta Fig. 5: normalized scores, or metrics, from various data sources);
computing the client experience score using a calculation including the first, second and third metrics and first, second and third weighting factors, retrieved from a weighting factor database (Gupta ¶ [0079]: Returning to FIG. 4, the ML engine 405 aggregates the normalized and weighted scores to generate an aggregate score 440, which is then provided as output 445 in the form of an experience score. Claim 1: …one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to:… after normalizing the initial set of scoring data, apply the weighting factors to the initial set of scoring data to generate a set of weighted scores),
each of the metrics being multiplied by its corresponding weighting factor in the calculation (See Gupta equation below ¶ [0079] multiplying corresponding metrics and weights. ¶ [0080]: In the above algorithm, M is the number of inputs for this particular customer, wi is the importance weight for the ith input, t; is the history weight for the ith input based on the time of feedback or interaction, and si is the normalized score [EN: metric] from that input. FIG. 5 shows an aggregated score 570, which is representative of the aggregate score 440);
storing the client experience score and historical values of the client experience score in a score database (Gupta ¶ [0052]: The data 300 can also be in the form of structured data 310 or unstructured data 315. As used herein, structured data 310 refers to data that is stored in a predefined format while unstructured data 315 can be a conglomeration of varied data types that are stored together in their native formats. An example of structured data 310 would be a specific 1-5 star rating for a product. An example of unstructured data 315 would be a user's typewritten comments on the quality of a product. In some instances, the system is configured to convert unstructured data into structured data (e.g., a pre-defined summary template that is populated with the relevant information extracted from the unstructured data));
prescribing targeted interactions with respective clients based on the client experience score attributed to an individual client (Gupta ¶ [0115]: Act 1035 then includes using the experience score to modify a subsequent interaction the client has with the entity. For instance, modify experience 175 from FIG. 1 and modify experience 700 from FIG. 7 are representative of example options for modifying the user's subsequent interactions with the entity…. Another way to dynamically influence interactions is by initiating [EN: prescribing] a new interaction, such as by prompting a client to refer another client. ¶ [0087]: FIG. 7 shows various examples of how the disclosed embodiments can modify a user's experience with an entity, as represented by modify experience 700. In one scenario, various campaigns 705 can be triggered. A campaign can include targeted promotions or advertisements that are directed to a client); and
periodically updating the first, second and third weighting factors [..] and where updating the first, second and third weighting factors includes computing a regression value using a logistic regression equation and [optimizing] the [..] function, the [..] function [..] using the trained neural network, differences between the regression value and the behavioral parameter (Gupta ¶ [0048]: As used herein, any type of ML engine, algorithm, model, or neural network may be used to perform the disclosed operations. As used herein, reference to "machine learning" or to a ML model or to a "neural network" may Include… logistic regression model(s). ¶ [0081]: In some embodiments, the ML engine 405 can also perform a regression analysis 450. The regression analysis 450 generally refers to a technique for identifying trends in data, such as possible dependencies between variables. As will be discussed in more detail shortly, the ML engine 405 can perform the regression analysis 450 in an effort to identify which specific interaction types or which specific events contributed most heavily to a user's particular experience score. ¶ [0066]: Optionally, the ML engine can continuously or periodically update the weighting factors over time based on newly learned data, such as newly acquired sentiment data. The weights can also be set manually or perhaps refined. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A);
[..]
wherein, after updating the first, second and third weighting factors, the system recomputes the client experience score for all of the clients using the updated weighting factors (Gupta ¶ [0116]: Beneficially, an interactions engine can be configured to at least periodically monitor for new sentiment data. As a further benefit, the client's experience score can be updated based on the new sentiment data that is acquired); and
updating, based on the client experience score being recomputed, a display of the client experience score on a user interface (Gupta ¶ [0096]: In the scenario shown in FIG. 8, the client 805 has provided sentiment data 810. The disclosed embodiments are able to acquire this sentiment data 810 and use it to generate and/or update a client's score 815. That score 815 is then displayed at a location proximate to the client's name in the user interface 800).
Although Gupta teaches calculating multiple weighted client metrics from internal data sources and multiple channels of external data sources, Gupta does not specifically teach identifying weighting factors that minimize a penalty function, an internal data source comprising client value KPIs used to calculate a third metric and summing the metrics, maximizing correlation between client experience scores and client behavior, determining error in output of a neural network, incorporating client churn data in optimization, determining output error in a neural network and backpropagating the error until minimization, or writing updated weighting factors to a database and calibrating accurate metrics for client experience.
However, Scholz in analogous art of modeling consumer sentiment and behavior teaches or suggests:
training a neural network using labeled datasets to identify values of a plurality of weighting factors that [optimize] a total value of a [..] function (Scholz [0051]: The data analytics processes performed in step 502 may include, in various embodiments, machine learning activities, classification activities using large volumes of raw data, a Bayesian Belief Network (BBN) engine [EN: used in supervised learning]…. In some cases a neural network may be generated and trained, using input data (e.g., service-specific data metrics) and output data (e.g., customer-initiated actions) during the training [EN: optimization] process. Such processes may involve pattern recognition and adaptive machine learning, to form correlations between the various inputs and outputs in order to train the neural network to predict customer actions and/or to perform other predictive analyses based on future data metrics. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics),
the training including:
comparing generated output produced by the neural network in response to training data with a desired output (Scholz [0051]: The data analytics processes performed in step 502 may include, in various embodiments, machine learning activities, classification activities using large volumes of raw data, a Bayesian Belief Network (BBN) engine [EN: used in supervised learning]…. In some cases a neural network may be generated and trained, using input data (e.g., service-specific data metrics) and output data (e.g., customer-initiated actions) during the training [EN: optimization] process. Such processes may involve pattern recognition and adaptive machine learning, to form correlations between the various inputs and outputs in order to train the neural network to predict customer actions and/or to perform other predictive analyses based on future data metrics. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics);
[..];
[..] the plurality of third data sources is client value key performance indicators received from the internal data sources (Scholz mid-¶ [0053]: Other types of compound service metrics may be measured unambiguously based on data available to the service provider, such as the value or profitability of a particular customer to the service provider, the amount of physical/network resources of the service provider used a particular customer, etc.), and
the third metric is computed as [..] weighted [..] client value KPIs (Scholz end-¶ [0053]: For instance, in each of these examples of compound service metrics, a set of service-specific data metrics may be identified, combined/grouped and/or weighted to serve as the compound service metric definition from which values may be calculated for individual customers or groups of customers);
[..] using an optimization process which maximizes a correlation between the client experience score for each of the clients and a behavioral parameter of a group of the clients, where the behavioral parameter comprises client churn data stored in a churn database, [..] (Scholz end-¶ [0022]: The data store 115, in various embodiments, also may include support for data mining models such as churn prediction…. ¶ [0048]: In step 502, the compound service metric generator 110 may use analytics processes to identify correlations between the customer action [EN: behavior] data received in step 501, and one or more corresponding data metrics [EN: client experience score]. For example, the generator 110 may perform various computational analyses on the service-specific data metrics received from various services 120 in step 301, and the subsequent customer action data received in step 501, in order to identify particular data metrics that are positively or negatively correlated with particular customer actions. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics);
[first/second/third metric is computed as a weighted] sum [of client [sentiments/KPIs] (Scholz mid-¶ [0060]: The generator 110 may sum the weighted data metrics 401-403 to calculate a customer sentiment signaling data metric 406 for the voice service 120a.The generator 110 may sum the weighted data metrics 401-403 to calculate a customer sentiment signaling data metric 406 for the voice service 120a. Using similar processes… the generator 110 may sum the weighted data metrics 404 and 405 to calculate a customer sentiment media quality data metric 407 for the voice service 120a. The generator 110 then may… sum these category data metrics 406 and 407….).
Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta’s techniques for generating and dynamically updating client experience scores to have included Scholz’s teachings around an internal data source comprising client value KPIs used to calculate a third metric, maximizing correlation between client experience scores and client behavior, the weighting calculations for the metrics involving a sum, and incorporating client churn data in optimization. The benefit of these additional features would have provided additional insight into the relationship between customer experience metrics and customer behavior (Scholz abstract). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Furthermore, Matam in analogous art of modeling consumer sentiment and behavior teaches or suggests:
[the optimization function being a] penalty minimizing function (Matam ¶ [0081]: In the AIC and BIC, the 2log (L) term rewards the latent space model 422 for the goodness of the fit on the training non-conversational data 218… each of the AIC and BIC also includes penalty terms (e.g., the 2q term for the AIC and the q log(n) term for the BIC). Each penalty term is a function whose value increases if the number of estimated parameters in the latent space model 422 increases. The penalty term penalizes over-fitting in the model. In this manner, the 2log(L) term and the penalty term in the AIC and the BIC create a trade-off between fitting the model to the selected training data and reducing [EN: minimizing] the complexity of the model).
Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz’s techniques for generating and dynamically updating client experience scores to have included Matam’s teachings around producing a regression value in a normalized range and the optimization function being a penalty function. The benefit of these additional features would have improved accuracy of predicting consumer intentions, dispositions, and behavior (Matam ¶ [0004]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz and Matam (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz and Matam above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Further still, Hall in analogous art of modeling consumer sentiment and behavior teaches or suggests:
determining an associated error amount for the generated output (Hall mid-¶ [0066]: At step 218, the process 200 includes determining if the current iteration model 119 meets one or more predetermined performance thresholds based on the training output of step 215 (e.g., one or more generated test predictions). The model service 107 can compute one or more performance metrics (e.g., accuracy, error, deviation, precision,
etc.) by comparing the training output of step 215 to the known outcomes of the corresponding training dataset); and
communicating the associated error amount back through the neural network as an error signal, where weight coefficients assigned in a hidden layer of the neural network are adjusted based on the error signal until the associated error amount is less than a predetermined acceptable level (Hall mid-¶ [0040]: Neural networks can include, but are not limited to, uni- or multilayer perceptron, convolutional neural networks, recurrent neural networks… auto-encoders… back-propagations, stochastic gradient descents…. Mid-¶ [0041]: Non-limiting examples of properties 121 include coefficients or weights… stochastic gradient descent… choice of cost or loss function, number of hidden layers in a neural network…. The properties 121 can include thresholds for evaluating model performance, such as… error thresholds);
[..]
writing updated weighting factors to the weighting factor database, whereby the computation of client experience scores using the first, second and third metrics is calibrated to be an accurate indicator of actual client experience (Hall mid-¶ [0006]: In various embodiments, the disclosed prediction system generates an aggregated psychographic database with selectable filters of consumer personas, interests, and lifestyles. ¶ [0053]: The model service 107 can evaluate model performance by a) executing the model 119 on training data to generate experimental output, and b) determining model performance metrics by comparing the experimental output to known outcomes associated with the training data. The model service 107 can modify the model towards improving model accuracy until an optimal model 119 is generated (e.g., the optimal model 119 meeting a predetermined accuracy and/or other performance threshold)).
Hall, Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz / Matam’s techniques for generating and dynamically updating client experience scores to have included Hall’s teachings around determining output error in a neural network and backpropagating the error until minimization, writing updated weighting factors to a database, and calibrating accurate metrics for client experience. The benefit of these additional features would have enhanced prediction, analysis, and comprehension of many factors affecting consumer satisfaction (Hall ¶ [0003]-[0006]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz, Matam, and Hall (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz, Matam, and Hall above, the to-be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Regarding claim 2: Gupta / Scholz / Matam / Hall teaches all the limitations of claim 1 above.
Although Gupta teaches applying a logistic regression function to the client experience scores to produce a regression value, Gupta does not specifically teach producing a regression value in a range of zero to one; optimizing correlation between the regression value of the experience scores and the client behavior, and adjusting the weighting factors to optimize the function for the clients in the group; nor the optimization function being a penalty minimization function.
However, Hall in analogous art of modeling consumer sentiment and behavior teaches or suggests:
wherein the optimization process further includes applying a gradient descent iterative computation to identify values of the plurality of weighting factors which [optimize] the total value of the [..] function for the group of clients (Hall mid-¶ [0040]: Non-limiting examples of properties 121 include coefficients or weights of linear and logistic regression models, weights and biases of neural network-type models… learning rate (e.g. gradient descent)… choice of optimization algorithm or other boosting technique (e.g., gradient descent, gradient boosting, stochastic gradient descent, Adam optimizer, etc.)).
Hall, Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz / Matam’s techniques for generating and dynamically updating client experience scores to have included Hall’s teachings around determining output error in a neural network and backpropagating the error until minimization. The benefit of these additional features would have enhanced prediction, analysis, and comprehension of many factors affecting consumer satisfaction (Hall ¶ [0003]-[0006]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz, Matam, and Hall (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz, Matam, and Hall above, the to-be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Furthermore, Matam in analogous art of modeling consumer sentiment and behavior teaches or suggests:
[the optimization function being a] penalty minimizing function (Matam ¶ [0081]: In the AIC and BIC, the 2log (L) term rewards the latent space model 422 for the goodness of the fit on the training non-conversational data 218… each of the AIC and BIC also includes penalty terms (e.g., the 2q term for the AIC and the q log(n) term for the BIC). Each penalty term is a function whose value increases if the number of estimated parameters in the latent space model 422 increases. The penalty term penalizes over-fitting in the model. In this manner, the 2log(L) term and the penalty term in the AIC and the BIC create a trade-off between fitting the model to the selected training data and reducing [EN: minimizing] the complexity of the model).
Rationales for modifying/combining Gupta / Scholz / Matam / Hall are above in claim 1 and reincorporated.
Regarding claim 3: Gupta teaches:
A computer-implemented method for computing a client experience score from structured and unstructured datatypes of a semantic knowledge database (Gupta ¶ [0024]: Embodiments disclosed herein relate to systems, devices, and methods for generating and dynamically updating an experience score for a client, where the experience score operates as a quantitative indicator describing a relationship between the client and an entity. Mid-¶ [0051]: the system is configured to recognize one or more predefined keywords that are known to be associated with particular sentiment of the client [EN: semantic knowledge] (i.e. where certain keywords indicate or contribute to a positive or negative sentiment score). ¶ [0052]: The data 300 can also be in the form of structured data 310 or unstructured data 315. ¶ [0107]: Act 1010 includes using natural language processing (NLP) to provide structure to the unstructured sentiment data. As a consequence, a second set of structured sentiment data (e.g., the set of structured sentiment data 415A [EN: semantic knowledge] from FIG. 4) is acquired),
said method comprising:
[..]
and
for each client in a client base:
computing a first metric from a plurality of first data sources, a second metric from a plurality of second data sources and a third metric from a plurality of third data sources (See Gupta Fig. 5 showing multiple metrics, or scores, generated from various data sources);
normalizing the first metric, the second metric, and the third metric to a common range of values (See Gupta Fig. 5: normalized scores, or metrics, from various data sources);
computing the client experience score using a calculation including the first, second and third metrics, and first, second and third weighting factors retrieved from a weighting factor database (Gupta ¶ [0079]: Returning to FIG. 4, the ML engine 405 aggregates the normalized and weighted scores to generate an aggregate score 440, which is then provided as output 445 in the form of an experience score. Claim 1: …one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to:… after normalizing the initial set of scoring data, apply the weighting factors to the initial set of scoring data to generate a set of weighted scores),
each of the metrics being multiplied by its corresponding weighting factor in the calculation (See Gupta equation below ¶ [0079] multiplying corresponding metrics and weights. ¶ [0080]: In the above algorithm, M is the number of inputs for this particular customer, wi is the importance weight for the ith input, t; is the history weight for the ith input based on the time of feedback or interaction, and si is the normalized score [EN: metric] from that input. FIG. 5 shows an aggregated score 570, which is representative of the aggregate score 440);
[..]; and
prescribing targeted interactions with individual clients based on the client experience score (Gupta ¶ [0115]: Act 1035 then includes using the experience score to modify a subsequent interaction the client has with the entity. For instance, modify experience 175 from FIG. 1 and modify experience 700 from FIG. 7 are representative of example options for modifying the user's subsequent interactions with the entity…. Another way to dynamically influence interactions is by initiating [EN: prescribing] a new interaction, such as by prompting a client to refer another client. ¶ [0087]: FIG. 7 shows various examples of how the disclosed embodiments can modify a user's experience with an entity, as represented by modify experience 700. In one scenario, various campaigns 705 can be triggered. A campaign can include targeted promotions or advertisements that are directed to a client);
periodically updating the first, second and third weighting factors [..], and where updating the weighting factors includes computing a regression value using a logistic regression equation and minimizing the penalty function, the penalty function penalizing, using the trained neural network, differences between the regression value and the behavioral parameter (Gupta ¶ [0048]: As used herein, any type of ML engine, algorithm, model, or neural network may be used to perform the disclosed operations. As used herein, reference to "machine learning" or to a ML model or to a "neural network" may Include… logistic regression model(s). ¶ [0081]: In some embodiments, the ML engine 405 can also perform a regression analysis 450. The regression analysis 450 generally refers to a technique for identifying trends in data, such as possible dependencies between variables. As will be discussed in more detail shortly, the ML engine 405 can perform the regression analysis 450 in an effort to identify which specific interaction types or which specific events contributed most heavily to a user's particular experience score. ¶ [0066]: Optionally, the ML engine can continuously or periodically update the weighting factors over time based on newly learned data, such as newly acquired sentiment data. The weights can also be set manually or perhaps refined. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A);
[..]
recomputing, after updating the first, second and third weighting factors, the client experience score for all of the clients using the updated weighting factors (Gupta ¶ [0116]: Beneficially, an interactions engine can be configured to at least periodically monitor for new sentiment data. As a further benefit, the client's experience score can be updated based on the new sentiment data that is acquired); and
updating, based on the client experience score being recomputed, a display of the client experience score on a user interface (Gupta ¶ [0096]: In the scenario shown in FIG. 8, the client 805 has provided sentiment data 810. The disclosed embodiments are able to acquire this sentiment data 810 and use it to generate and/or update a client's score 815. That score 815 is then displayed at a location proximate to the client's name in the user interface 800).
Although Gupta teaches calculating multiple weighted client metrics from internal data sources and multiple channels of external data sources, Gupta does not specifically teach identifying weighting factors that minimize a penalty function, an internal data source comprising client value KPIs used to calculate a third metric and summing the metrics, maximizing correlation between client experience scores and client behavior, determining error in output of a neural network, incorporating client churn data in optimization, determining output error in a neural network and backpropagating the error until minimization, or writing updated weighting factors to a database and calibrating accurate metrics for client experience.
However, Scholz in analogous art of modeling consumer sentiment and behavior teaches or suggests:
training a neural network using labeled datasets to identify values of a plurality of weighting factors that [optimize] a total value of a [..] function (Scholz [0051]: The data analytics processes performed in step 502 may include, in various embodiments, machine learning activities, classification activities using large volumes of raw data, a Bayesian Belief Network (BBN) engine [EN: used in supervised learning]…. In some cases a neural network may be generated and trained, using input data (e.g., service-specific data metrics) and output data (e.g., customer-initiated actions) during the training [EN: optimization] process. Such processes may involve pattern recognition and adaptive machine learning, to form correlations between the various inputs and outputs in order to train the neural network to predict customer actions and/or to perform other predictive analyses based on future data metrics. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics),
the training including:
comparing generated output produced by the neural network in response to training data with a desired output (Scholz [0051]: The data analytics processes performed in step 502 may include, in various embodiments, machine learning activities, classification activities using large volumes of raw data, a Bayesian Belief Network (BBN) engine [EN: used in supervised learning]…. In some cases a neural network may be generated and trained, using input data (e.g., service-specific data metrics) and output data (e.g., customer-initiated actions) during the training [EN: optimization] process. Such processes may involve pattern recognition and adaptive machine learning, to form correlations between the various inputs and outputs in order to train the neural network to predict customer actions and/or to perform other predictive analyses based on future data metrics. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics);
[..]
[..] using an optimization process which maximizes a correlation between the client experience score for each of the clients and a behavioral parameter of a group of the clients, where the behavioral parameter comprises client chum data stored in a chum database, [..] (Scholz end-¶ [0022]: The data store 115, in various embodiments, also may include support for data mining models such as churn prediction…. ¶ [0048]: In step 502, the compound service metric generator 110 may use analytics processes to identify correlations between the customer action [EN: behavior] data received in step 501, and one or more corresponding data metrics [EN: client experience score]. For example, the generator 110 may perform various computational analyses on the service-specific data metrics received from various services 120 in step 301, and the subsequent customer action data received in step 501, in order to identify particular data metrics that are positively or negatively correlated with particular customer actions. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics);
storing the client experience score and historical values of the client experience score in a score database (Scholz mid-¶ [0018]: The received data metrics may be stored in a compound service metric data store 115 which may operate independently or may be implemented within / integrated into the compound service metric generator 110. ¶ [0055]: In step 504, the compound service metric generator 110 may store and deploy the updated compound service data metric(s) defined in step 503).
writing updated weighting factors to the weighting factor database (Scholz mid-¶ [0018]: The received data metrics may be stored in a compound service metric data store 115 which may operate independently or may be implemented within / integrated into the compound service metric generator 110. ¶ [0055]: In step 504, the compound service metric generator 110 may store and deploy the updated compound service data metric(s) defined in step 503).
Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta’s techniques for generating and dynamically updating client experience scores to have included Scholz’s teachings around optimization by training a neural network to identify weighting factors and storing the client experience score in a score database. The benefit of these additional features would have provided additional insight into the relationship between customer experience metrics and customer behavior (Scholz abstract). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Furthermore, Matam in analogous art of modeling consumer sentiment and behavior teaches or suggests:
[the optimization function being a] penalty minimizing function (Matam ¶ [0081]: In the AIC and BIC, the 2log (L) term rewards the latent space model 422 for the goodness of the fit on the training non-conversational data 218… each of the AIC and BIC also includes penalty terms (e.g., the 2q term for the AIC and the q log(n) term for the BIC). Each penalty term is a function whose value increases if the number of estimated parameters in the latent space model 422 increases. The penalty term penalizes over-fitting in the model. In this manner, the 2log(L) term and the penalty term in the AIC and the BIC create a trade-off between fitting the model to the selected training data and reducing [EN: minimizing] the complexity of the model).
Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz’s techniques for generating and dynamically updating client experience scores to have included Matam’s teachings around producing a regression value in a normalized range and the optimization function being a penalty function. The benefit of these additional features would have improved accuracy of predicting consumer intentions, dispositions, and behavior (Matam ¶ [0004]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz and Matam (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz and Matam above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Further still, Hall in analogous art of modeling consumer sentiment and behavior teaches or suggests:
determining an associated error amount for the generated output (Hall mid-¶ [0066]: At step 218, the process 200 includes determining if the current iteration model 119 meets one or more predetermined performance thresholds based on the training output of step 215 (e.g., one or more generated test predictions). The model service 107 can compute one or more performance metrics (e.g., accuracy, error, deviation, precision,
etc.) by comparing the training output of step 215 to the known outcomes of the corresponding training dataset); and
communicating the associated error amount back through the neural network as an error signal, where weight coefficients assigned in a hidden layer of the neural network are adjusted based on the error signal until the associated error amount is less than a predetermined acceptable level (Hall mid-¶ [0040]: Neural networks can include, but are not limited to, uni- or multilayer perceptron, convolutional neural networks, recurrent neural networks… auto-encoders… back-propagations, stochastic gradient descents…. Mid-¶ [0041]: Non-limiting examples of properties 121 include coefficients or weights… stochastic gradient descent… choice of cost or loss function, number of hidden layers in a neural network…. The properties 121 can include thresholds for evaluating model performance, such as… error thresholds).
Hall, Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz / Matam’s techniques for generating and dynamically updating client experience scores to have included Hall’s teachings around determining output error in a neural network and backpropagating the error until minimization, writing updated weighting factors to a database, and calibrating accurate metrics for client experience. The benefit of these additional features would have enhanced prediction, analysis, and comprehension of many factors affecting consumer satisfaction (Hall ¶ [0003]-[0006]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz, Matam, and Hall (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz, Matam, and Hall above, the to-be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Regarding claim 4: Gupta / Scholz / Matam / Hall teaches all the limitations of claim 3 above.
Gupta further teaches:
wherein the method further includes receiving client feedback data from external data sources via multiple input channels (Gupta ¶ [0046]: … In other scenarios, the sources are independent relative to the entity [EN: external data sources], such as the case where the source is in the form of a social media platform or some other independent entity. In such scenarios, the interactions engine 140 can be configured to crawl 150 a public network (e.g., the Internet) to identify other sources (e.g., indirect sources or third-party sources) where a user may have expressed his/her viewpoints about a particular entity. For instance, FIG. 1 shows the interactions engine 140 crawling a network 155 to identify a source 160 that is entirely independent of the entity 130 and that can be a third-party source. From this source 160, the interactions engine 140 can acquire additional sentiment data), and
wherein the plurality of first data sources is client feedback data received from the external data sources via the multiple input channels (Gupta ¶ [0064]: FIG. 5 shows a set of input 500, which is representative of the input 410 from FIG. 4. The input 500 can include data from any number of sources and can include expressions made by a client), and
the first metric is computed as a weighted sum of client sentiments each derived from a different one of the input channels (Gupta ¶ [0064]: Returning to FIG. 4, the process flow 400 includes the ML engine 405 generating and applying weight(s) 435 to each respective input type [EN: input channel]. That is, the ML engine generates the weighting factors…. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A. FIG. 5 provides a useful illustration), and where
the client sentiments include a sentiment derived from each of a call center input channel, an online chat input channel, a client complaints input channel, a voice of the customer input channel, and two different mobile device application store input channels (Gupta mid-¶ [0064]: As some examples, the input 500 [EN: input channel] can be acquired from messages 505 [EN: mobile device application], a webchat 510, a survey 515 [EN: complaints], reviews 520 [EN: complaints], social media 525 [EN: mobile device application], texts 530 [EN: mobile device application], voicemail 535 [EN: call center, customer voice], email 540 [EN: mobile device application]. ¶ [0047]: In some scenarios, such as where a client has enabled microphone access associated with the entity or a separate application that is linked to the entity, the system is able to gather feedback data, through automatic speech [EN: voice] recognition, by identifying relevant speech signals recorded by a client's microphone (i.e. if the client is speaking about his or her interaction with the entity)).
Regarding claim 5: Gupta / Scholz / Matam / Hall teaches all the limitations of claim 3 above.
Gupta further teaches:
wherein the plurality of second data sources is client experience key performance indicators (KPIs) received from internal data sources (Gupta ¶ [0046]: ¶ [0046]: In some cases, the sources can be linked to an entity's platform [EN: internal data sources], such as a website that offers clients the opportunities to leave feedback [EN: experience KPI]. Mid-¶ [0049]: For instance, one type of interaction can be a user's response to a survey [EN: experience KPI], and the [EN: internal data] "source" would be the survey… Another type of interaction can be a user providing feedback in a website, and the [EN: internal data] source would be the website), and
the second metric is computed as a weighted sum of the client experience KPIs (Gupta ¶ [0064]: Returning to FIG. 4, the process flow 400 includes the ML engine 405 generating and applying weight(s) 435 to each respective input type. That is, the ML engine generates the weighting factors…. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A. FIG. 5 provides a useful illustration).
Regarding claim 6: Gupta / Scholz / Matam / Hall teaches all the limitations of claim 3 above.
Although Gupta teaches calculating multiple weighted client metrics from internal data sources and multiple channels of external data sources, Gupta does not specifically teach an internal data source comprising client value KPIs used to calculate a third metric.
However, Scholz in analogous art of modeling consumer sentiment and behavior teaches or suggests:
wherein the plurality of third data sources is client value key performance indicators (KPIs) received from internal data sources (Scholz mid-¶ [0053]: Other types of compound service metrics may be measured unambiguously based on data available to the service provider, such as the value or profitability of a particular customer to the service provider, the amount of physical/network resources of the service provider used a particular customer, etc.), and
the third metric is computed as a weighted sum of the client value KPIs (Scholz end-¶ [0053]: For instance, in each of these examples of compound service metrics, a set of service-specific data metrics may be identified, combined/grouped and/or weighted to serve as the compound service metric definition from which values may be calculated for individual customers or groups of customers).
Rationales to have modified / combined Gupta / Scholz are in claim 1 above and reincorporated.
Regarding claim 7: Gupta / Scholz / Matam / Hall teaches all the limitations of claim 3 above.
Gupta further teaches:
wherein the normalizing of the first, second and third metrics includes scaling each of the metrics to a common range of values before computing the client experience score (Gupta ¶ [0063]: After providing structure to any unstructured data, the ML engine 405 then normalizes the data, all of which should now have the same or matching structure (e.g., perhaps the structure is a numerical indicator), as represented by normalized score(s) 430. As one example, all scores can optionally be normalized to fall within a range between 0 and 10. The process of normalizing the data results in all of the data having the same scale).
Regarding claim 9: Gupta / Scholz / Matam / Hall teaches all the limitations of claim 7 above.
Although Gupta teaches applying a logistic regression function to the client experience scores to produce a regression value, Gupta does not specifically teach producing a regression value in a range of zero to one; optimizing correlation between the regression value of the experience scores and the client behavior, and adjusting the weighting factors to optimize the function for the clients in the group; nor the optimization function being a penalty minimization function.
However, Hall in analogous art of modeling consumer sentiment and behavior teaches or suggests:
wherein the optimization process includes applying a gradient descent iterative computation to identify values of the weighting factors which [optimize] the total value of the [..] function for the group of the clients (Hall mid-¶ [0040]: Non-limiting examples of properties 121 include coefficients or weights of linear and logistic regression models, weights and biases of neural network-type models… learning rate (e.g. gradient descent)… choice of optimization algorithm or other boosting technique (e.g., gradient descent, gradient boosting, stochastic gradient descent, Adam optimizer, etc.)).
Hall, Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz / Matam’s techniques for generating and dynamically updating client experience scores to have included Hall’s teachings around determining output error in a neural network and backpropagating the error until minimization. The benefit of these additional features would have enhanced prediction, analysis, and comprehension of many factors affecting consumer satisfaction (Hall ¶ [0003]-[0006]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz, Matam, and Hall (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz, Matam, and Hall above, the to-be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Furthermore, Matam in analogous art of modeling consumer sentiment and behavior teaches or suggests:
[the optimization function being a] penalty minimizing function (Matam ¶ [0068]: In the example depicted in FIG. 4, a trained regression model (or other predictive model) is generated for predicting the probability [EN: regression value in range of zero to one] of an end-user's behavior (e.g., a consumer reaction to a sales call). ¶ [0081]: In the AIC and BIC, the 2log (L) term rewards the latent space model 422 for the goodness of the fit on the training non-conversational data 218… each of the AIC and BIC also includes penalty terms (e.g., the 2q term for the AIC and the q log(n) term for the BIC). Each penalty term is a function whose value increases if the number of estimated parameters in the latent space model 422 increases. The penalty term penalizes over-fitting in the model. In this manner, the 2log(L) term and the penalty term in the AIC and the BIC create a trade-off between fitting the model to the selected training data and reducing [EN: minimizing] the complexity of the model).
Rationales to have modified / combined Gupta / Scholz / Matam / Hall are above in claim 3 and reincorporated.
Regarding claim 10: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
wherein the regression value computed using a logistic regression equation is used as the client experience score (Gupta ¶ [0048]: As used herein, any type of ML engine, algorithm, model, or neural network may be used to perform the disclosed operations. As used herein, reference to "machine learning" or to a ML model or to a "neural network" may include any type of machine learning algorithm or device… linear regression model( s) or logistic regression model(s)…).
Regarding claim 11: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Although Gupta teaches consumer model behavior parameters, Gupta does not specifically teach the implementation of a churn rate.
However, Scholz in analogous art of modeling consumer sentiment and behavior teaches or suggests:
The method according to Claim 3, wherein the behavioral parameter comprises a client churn rate defined per month for a group of the clients (Scholz mid-¶ [0017]: Compound service performance metrics may be used in different contexts to perform further analytics by service providers and/or third-party systems, in ad-hoc and automated processes, relating to customer churn predictions, targeted offers, campaign analysis (e.g., pre- and post-campaign), loyalty management, brand promotion, and customer cost/revenue analysis, etc. End ¶ [0022]: The data store 115, in various embodiments, also may include support for data mining models such as churn prediction, segmentation based on services 120 and/or customers, etc. End-¶ [0049]: The analytics performed in step 502 also may correlate data metrics relating to various point-of-contact systems 120e-j with different customer actions… that may affect individual communication services l20a-d (e.g., lower/higher service usage, lower/higher customer churn for a service 120)).
Examiner notes that the information of different data rates (monthly) in the portfolio in claim 11 is construed as nonfunctional descriptive material and is given no patentable weight because these items have no functional relationship with a computer in the claim. This is the same as the example in MPEP 2111.05 of a table containing batting averages, where the information only has significance to a user reading the information.
Rationales to modify / combine Gupta / Scholze are above in claim 3 and reincorporated.
Regarding claim 12: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Although Gupta teaches generating and dynamically updating experience scores for clients, Gupta does not specifically teach grouping clients based on regions of business.
However, Scholz in analogous art of modeling consumer sentiment and behavior teaches or suggests:
wherein the group of the clients is defined based on regions of the business (Scholz end-¶ [0044]: For example, service-agnostic customer sentiment metrics, customer value metrics, and customer resource usage metrics may be calculated for groups of customers within the same geographic region).
Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta’s techniques for generating and dynamically updating client experience scores to have included Scholz’s teachings around grouping clients based on regions of business. The benefit of these additional features would have provided additional insight into the relationship between customer experience metrics and customer behavior (Scholz abstract). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Regarding claim 13: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
wherein determining whether the weighting factors need to be updated is based on an amount of time elapsed since the weighting factors were last re-computed (Gupta ¶ [0073]: Here [equation above], alpha is the decay constant. The inputs can be sorted based on their timestamps or creation times, so the most recently received time-based weight factor is t0, the second is t1, etc. The time-based weight factor ti thus penalizes each input's score by its position over time).
Regarding claim 14: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
wherein determining whether the weighting factors need to be updated is based on availability of a new source of client feedback data (Gupta mid-¶ [0066]: Optionally, the ML engine can continuously or periodically update the weighting factors over time based on newly learned data, such as newly acquired sentiment data).
Regarding claim 15: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
further comprising recomputing the client experience score for an individual client responsive to a recorded client event (Gupta mid-mid-¶ [0081]: As will be discussed in more detail shortly, the ML engine 405 can perform the regression analysis 450 in an effort to identify which specific interaction types or which specific events contributed most heavily to a user's particular experience score).
Regarding claim 16: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
wherein the client experience score is scaled so that a maximum value is 100 (Gupta mid-¶ [0063]: As one example, all scores can optionally be normalized to fall within a range between 0 and 10. The process of normalizing the data results in all of the data having the same scale).
Examiner notes that the information of different data in the scale in claim 16 is construed as nonfunctional descriptive material and is given no patentable weight because these items have no functional relationship with a computer in the claim. This is the same as the example in MPEP 2111.05 of a table containing batting averages, where the information only has significance to a user reading the information.
Regarding claim 17: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
The method according to Claim 4, wherein determining the first metric further includes deriving the client sentiment for a feedback source that is anonymized by using an aggregate sentiment value for that feedback source (Gupta ¶ [0053]: Returning to FIG. 1, the interactions engine 140 is able to acquire both structured data and unstructured data from any number of sources, where the acquired data can be used to describe or infer a client's relationship with an entity and where the acquired data is acquired from different types of interactions the client had relative to the entity. The ML engine 140A is able to analyze, process, and synthesize the acquired data in any number of ways in order to generate an aggregated score 170, which is also referred to herein as an "experience score." FIG. 4 provides some additional clarification regarding some of the processes that can be performed to generate the aggregated score 170.).
Regarding claim 18: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Gupta further teaches:
The method according to Claim 4, wherein determining the first metric further includes factoring a number of client feedback records for each feedback source into the calculation of the client sentiment for that feedback source (See Gupta Fig. 11 step 1115: For each of the [EN: numerous] different types of interactions the user had relative to the online commerce entity, causing the ml engine to generate a corresponding weighting factor).
Regarding claim 19: Gupta / Scholz / Matam / Hall teach all the limitations of claim 5 above.
Although Gupta teaches calculating client experience scores from quantitative indicators, Gupta does not specifically teach the indicators including a client effort score, a net promoter score, a client satisfaction score, and a behavioral score derived from both the client's digital and in-person interactions.
However, Hall in analogous art of modeling consumer sentiment and behavior teaches or suggests:
wherein the client experience key performance indicators include a client effort score, a net promoter score, a client satisfaction score, and a behavioral score derived from both the client's digital and in-person interactions (Hall mid-¶ [0033]: The product data 113 can include econometric indicators 116, including, but not limited to, price point(s) of a product, product cost, product distribution levels, product volume (e.g., a desired volume, breakeven volume, minimum volume, etc.), product channel and/or location (e.g., virtual and physical sale locations)…. ¶ [0035]: Product data 113 and historical data 115 can include values for various macroeconomic indicators and search trends (e.g., values being sampled on a weekly, monthly, daily, or any suitable basis). The macroeconomic indicators and search trend values can be stored in association with additional product data 113 or historical data 115, such as data points associated with a time period, channel, or location corresponding to the data value. The intake service 103 can expand the amount of data gathered around each product attribute 114 (e.g., or other element of product data 113 or historical data 115) by capturing additional information, such as search data around the product attribute).
Examiner notes that the information of different data (types of scores) in the portfolio in claim 10 is construed as nonfunctional descriptive material and is given no patentable weight because these items have no functional relationship with a computer in the claim. This is the same as the example in MPEP 2111.05 of a table containing batting averages, where the information only has significance to a user reading the information.
Hall, Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz / Matam’s techniques for generating and dynamically updating client experience scores to have included Hall’s teachings around determining output error in a neural network and backpropagating the error until minimization, writing updated weighting factors to a database, and calibrating accurate metrics for client experience. The benefit of these additional features would have enhanced prediction, analysis, and comprehension of many factors affecting consumer satisfaction (Hall ¶ [0003]-[0006]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz, Matam, and Hall (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz, Matam, and Hall above, the to-be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Regarding claim 21:
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor (Gupta ¶ 0131]), cause the processor to:
[..]
the training including:
[..]; and
for each client in a client base:
compute a first metric from a plurality of first data sources, a second metric from a plurality of second data sources and a third metric from a plurality of third data sources (See Gupta Fig. 5 showing multiple metrics, or scores, generated from various data sources);
normalize the first metric, the second metric, and the third metric to a common range of values (See Gupta Fig. 5: normalized scores, or metrics, from various data sources);
compute the client experience score using a calculation including the first, second and third metrics, and first, second and third weighting factors retrieved from a weighting
factor database (Gupta ¶ [0079]: Returning to FIG. 4, the ML engine 405 aggregates the normalized and weighted scores to generate an aggregate score 440, which is then provided as output 445 in the form of an experience score. Claim 1: …one or more computer-readable hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to:… after normalizing the initial set of scoring data, apply the weighting factors to the initial set of scoring data to generate a set of weighted scores),
each of the metrics being multiplied by its corresponding weighting factor in the calculation (See Gupta equation below ¶ [0079] multiplying corresponding metrics and weights. ¶ [0080]: In the above algorithm, M is the number of inputs for this particular customer, wi is the importance weight for the ith input, t; is the history weight for the ith input based on the time of feedback or interaction, and si is the normalized score [EN: metric] from that input. FIG. 5 shows an aggregated score 570, which is representative of the aggregate score 440);
store the client experience score and historical values of the client experience score in a score database (Gupta ¶ [0052]: The data 300 can also be in the form of structured data 310 or unstructured data 315. As used herein, structured data 310 refers to data that is stored in a predefined format while unstructured data 315 can be a conglomeration of varied data types that are stored together in their native formats. An example of structured data 310 would be a specific 1-5 star rating for a product. An example of unstructured data 315 would be a user's typewritten comments on the quality of a product. In some instances, the system is configured to convert unstructured data into structured data (e.g., a pre-defined summary template that is populated with the relevant information extracted from the unstructured data));
prescribe targeted interactions with individual clients based on the client experience score (Gupta ¶ [0115]: Act 1035 then includes using the experience score to modify a subsequent interaction the client has with the entity. For instance, modify experience 175 from FIG. 1 and modify experience 700 from FIG. 7 are representative of example options for modifying the user's subsequent interactions with the entity…. Another way to dynamically influence interactions is by initiating [EN: prescribing] a new interaction, such as by prompting a client to refer another client. ¶ [0087]: FIG. 7 shows various examples of how the disclosed embodiments can modify a user's experience with an entity, as represented by modify experience 700. In one scenario, various campaigns 705 can be triggered. A campaign can include targeted promotions or advertisements that are directed to a client);
periodically update the first, second and third weighting factors [..] and where updating the weighting factors includes computing a regression value using a logistic regression equation and minimizing the penalty function, the penalty function penalizing, using the trained neural network, differences between the regression value and the behavioral parameter (Gupta ¶ [0048]: As used herein, any type of ML engine, algorithm, model, or neural network may be used to perform the disclosed operations. As used herein, reference to "machine learning" or to a ML model or to a "neural network" may Include… logistic regression model(s). ¶ [0081]: In some embodiments, the ML engine 405 can also perform a regression analysis 450. The regression analysis 450 generally refers to a technique for identifying trends in data, such as possible dependencies between variables. As will be discussed in more detail shortly, the ML engine 405 can perform the regression analysis 450 in an effort to identify which specific interaction types or which specific events contributed most heavily to a user's particular experience score. ¶ [0066]: Optionally, the ML engine can continuously or periodically update the weighting factors over time based on newly learned data, such as newly acquired sentiment data. The weights can also be set manually or perhaps refined. As a result of applying the weights, the ML engine 405 has generated a set of weighted scores 435A);
[..]
recompute, after updating the first, second and third weighting factors, the client experience score for all of the clients using the updated weighting factors (Gupta ¶ [0116]: Beneficially, an interactions engine can be configured to at least periodically monitor for new sentiment data. As a further benefit, the client's experience score can be updated based on the new sentiment data that is acquired); and
update, based on the client experience score being recomputed, a display of the client experience score on a user interface (Gupta ¶ [0096]: In the scenario shown in FIG. 8, the client 805 has provided sentiment data 810. The disclosed embodiments are able to acquire this sentiment data 810 and use it to generate and/or update a client's score 815. That score 815 is then displayed at a location proximate to the client's name in the user interface 800).
Although Gupta teaches calculating multiple weighted client metrics from internal data sources and multiple channels of external data sources, Gupta does not specifically teach identifying weighting factors that minimize a penalty function, an internal data source comprising client value KPIs used to calculate a third metric and summing the metrics, maximizing correlation between client experience scores and client behavior, determining error in output of a neural network, incorporating client churn data in optimization, determining output error in a neural network and backpropagating the error until minimization, or writing updated weighting factors to a database and calibrating accurate metrics for client experience.
However, Scholz in analogous art of modeling consumer sentiment and behavior teaches or suggests:
train a neural network using labeled datasets to identify values of a plurality of weighting factors that [optimize] a total value of a [..] function (Scholz [0051]: The data analytics processes performed in step 502 may include, in various embodiments, machine learning activities, classification activities using large volumes of raw data, a Bayesian Belief Network (BBN) engine [EN: used in supervised learning]…. In some cases a neural network may be generated and trained, using input data (e.g., service-specific data metrics) and output data (e.g., customer-initiated actions) during the training [EN: optimization] process. Such processes may involve pattern recognition and adaptive machine learning, to form correlations between the various inputs and outputs in order to train the neural network to predict customer actions and/or to perform other predictive analyses based on future data metrics. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics),
the training including:
comparing generated output produced by the neural network in response to training data with a desired output (Scholz [0051]: The data analytics processes performed in step 502 may include, in various embodiments, machine learning activities, classification activities using large volumes of raw data, a Bayesian Belief Network (BBN) engine [EN: used in supervised learning]…. In some cases a neural network may be generated and trained, using input data (e.g., service-specific data metrics) and output data (e.g., customer-initiated actions) during the training [EN: optimization] process. Such processes may involve pattern recognition and adaptive machine learning, to form correlations between the various inputs and outputs in order to train the neural network to predict customer actions and/or to perform other predictive analyses based on future data metrics. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics);
[..]
[..] using an optimization process which maximizes a correlation between the client experience score for each of the clients and a behavioral parameter of a group of the clients, where the behavioral parameter comprises client chum data stored in a chum database, [..] (Scholz end-¶ [0022]: The data store 115, in various embodiments, also may include support for data mining models such as churn prediction…. ¶ [0048]: In step 502, the compound service metric generator 110 may use analytics processes to identify correlations between the customer action [EN: behavior] data received in step 501, and one or more corresponding data metrics [EN: client experience score]. For example, the generator 110 may perform various computational analyses on the service-specific data metrics received from various services 120 in step 301, and the subsequent customer action data received in step 501, in order to identify particular data metrics that are positively or negatively correlated with particular customer actions. Mid-¶ [0052]: Updating compound service metric definitions in step 503 may include…updating the assigned weights for data metrics and for different groupings/intermediate layers of data metrics. Mid-¶ [0054]: In such cases, the analytics performed in step 502 and the updated data metrics, relationships, weights, etc., determined in step 503 may be used to tune and/or optimize the previously defined compound service metrics);
[..]
write updated weighting factors to the weighting factor database (Scholz mid-¶ [0018]: The received data metrics may be stored in a compound service metric data store 115 which may operate independently or may be implemented within / integrated into the compound service metric generator 110. ¶ [0055]: In step 504, the compound service metric generator 110 may store and deploy the updated compound service data metric(s) defined in step 503).
Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta’s techniques for generating and dynamically updating client experience scores to have included Scholz’s teachings around optimization by training a neural network to identify weighting factors and storing the client experience score in a score database. The benefit of these additional features would have provided additional insight into the relationship between customer experience metrics and customer behavior (Scholz abstract). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Furthermore, Matam in analogous art of modeling consumer sentiment and behavior teaches or suggests:
[the optimization function being a] penalty minimizing function (Matam ¶ [0081]: In the AIC and BIC, the 2log (L) term rewards the latent space model 422 for the goodness of the fit on the training non-conversational data 218… each of the AIC and BIC also includes penalty terms (e.g., the 2q term for the AIC and the q log(n) term for the BIC). Each penalty term is a function whose value increases if the number of estimated parameters in the latent space model 422 increases. The penalty term penalizes over-fitting in the model. In this manner, the 2log(L) term and the penalty term in the AIC and the BIC create a trade-off between fitting the model to the selected training data and reducing [EN: minimizing] the complexity of the model).
Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz’s techniques for generating and dynamically updating client experience scores to have included Matam’s teachings around producing a regression value in a normalized range and the optimization function being a penalty function. The benefit of these additional features would have improved accuracy of predicting consumer intentions, dispositions, and behavior (Matam ¶ [0004]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz and Matam (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of modeling consumer sentiment and behavior. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz and Matam above, the to- be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
Further still, Hall in analogous art of modeling consumer sentiment and behavior teaches or suggests:
determining an associated error amount for the generated output (Hall mid-¶ [0066]: At step 218, the process 200 includes determining if the current iteration model 119 meets one or more predetermined performance thresholds based on the training output of step 215 (e.g., one or more generated test predictions). The model service 107 can compute one or more performance metrics (e.g., accuracy, error, deviation, precision,
etc.) by comparing the training output of step 215 to the known outcomes of the corresponding training dataset); and
communicating the associated error amount back through the neural network as an error signal, where weight coefficients assigned in a hidden layer of the neural network are adjusted based on the error signal until the associated error amount is less than a predetermined acceptable level (Hall mid-¶ [0040]: Neural networks can include, but are not limited to, uni- or multilayer perceptron, convolutional neural networks, recurrent neural networks… auto-encoders… back-propagations, stochastic gradient descents…. Mid-¶ [0041]: Non-limiting examples of properties 121 include coefficients or weights… stochastic gradient descent… choice of cost or loss function, number of hidden layers in a neural network…. The properties 121 can include thresholds for evaluating model performance, such as… error thresholds).
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over:
Gupta / Scholz / Matam / Hall as applied above, in further view of
Beaufays et al. US 20230177382 A1, hereinafter Beaufays. As per,
Regarding claim 20: Gupta / Scholz / Matam / Hall teach all the limitations of claim 3 above.
Although Gupta teaches periodically updating weighting factors for a machine learning algorithm, Gupta does not specifically teach correlating the updating of the weighting factors with the update of the model itself.
However, Beaufays in analogous art to the instant application of machine learning and natural language understanding models teaches or suggests:
wherein determining whether the weighting factors need to be updated includes evaluating availability of an upcoming system maintenance window in which an updated version of the machine learning algorithm may be placed into operation (Beaufays ¶ [0038]: In other versions of these additional or alternative implementations, the ML update 106 can include the one or more updated first on-device weights for the one or more first updated on-device ML layers and an indication of one or more global weights a global ML model that should be replaced in the remote memory of the remote system 160 with the one or more updated first on-device weights. Continuing with the above example, the ML update 106 may include an indication that one or more first global weights of the one or more first global ML layers of the corresponding counterpart global ML model should be replaced with the one or more updated first on-device weights for the one or more first updated on-device ML layers. Similarly, the additional ML update 107 may include an indication that one or more second global weights of the one or more second global ML layers of the corresponding counterpart global ML model should be replaced with one or more updated second on-device weights for the one or more second updated on-device ML layers).
Beaufays is found as analogous art to the instant application of machine learning and natural language understanding models and Hall, Matam, Scholz and Gupta are found as analogous art of modeling consumer sentiment and behavior. It would have been obvious to one skilled in the art, before the effective filing date of the invention, to have modified Gupta / Scholz / Matam / Hall’s techniques for generating and dynamically updating client experience scores to have included Beaufays’s teachings around correlating the updating of the weighting factors with the updating of the machine learning model. The benefit of these additional features would have improved the efficiency of federated learning. (Beaufays ¶ [0002]). The predictability of such modifications and/or variations, would have been corroborated by the broad level of skill of one of ordinary skills in the art as articulated by Gupta in view of Scholz, Matam, Hall, and Beaufays (see MPEP 2143 G).
Further, the claimed invention could have also been viewed as a mere combination of old elements in a similar field of machine learning and natural language understanding models. In such combination each element would have merely performed the same function as it did separately. Thus, one of ordinary skill in the art would have recognized that, given existing technical ability to combine the elements, as evidenced by Gupta in view of Scholz, Matam, Hall, and Beaufays above, the to-be combined elements would have fit together like pieces of a puzzle in a logical, complementary, technologically feasible and/or economically desirable manner. Thus, it would have been reasoned that the results of the combination would have been predictable (see MPEP 2143 A).
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The following art is made of record and considered pertinent to Applicant’s disclosure:
Purohit; Nitin et al. US 20210201335 A1, Customer experience rating system and method.
Null; Bradley William et al. US 20230281678 A1, Impact-based strength and weakness determination.
Griebeler; Jeffrey et al. US 20150051957 A1, Measuring customer experience value.
SEKAR; ARCHANA et al. US 20210201359 A1, Systems and methods relating to automation for personalizing the customer experience.
Bjelajac; Cris et al. US 20160034918 A1, System and method for managing customer feedback.
BHATNAGAR; Saumya et al. US 20230351421 A1, Customer-intelligence predictive model.
Perepa; Vijay Rao et al. US 20150206157 A1, Methods and systems for estimating customer experience.
Kumar; Shailesh et al. US 20070094067 A1, Method and apparatus for recommendation engine using pair-wise co-occurrence consistency.
Cattaneo; Laura et al. US 20230206160 A1, Systems and methods for rapport determination.
Briancon; Alain Charles et al. US 20200356900 A1, Predictive, machine-learning, locale-aware computer models suitable for location- and trajectory-aware training sets.
Johnsen; Mads et al. US 20190043063 A1, Model-based assessment and improvement of relationships.
Muthusamy; Ganesh et al. US 11853938 B1, System and method of reinforced machine-learning retail allocation.
Clodore; Glen et al. WO 2021173611 A1, Intent analysis for call center response generation.
Fu, Teng, et al. "Modeling Customer Experience in a Contact Center through Process Log Mining." ACM Transactions on Intelligent Systems and Technology (TIST) 12.4 (2021): 1-21.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REED M. BOND whose telephone number is (571) 270-0585. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patricia Munson can be reached at (571) 270-5396. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REED M. BOND/Examiner, Art Unit 3624 May 19, 2026
/HAMZEH OBAID/Primary Examiner, Art Unit 3624 May 20, 2026
1 MPEP 2106.04(a): “examiners should identify at least one abstract idea grouping, but preferably identify all groupings to the extent possible”.
Prosecution Insights