DETAILED ACTION
This action is in response to claims filed 20 October 2023 for application 18491473 filed 20 October 2023. Currently claims 1-20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-6, 9-16, 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jungmeisteris et al. (US 20220398635 A1) in view of Acharya et al. (US 11393454 B1).
Regarding claims 1, 11 and 19, Jungmeisteris discloses:
A system to control performance of digital components executed on one or more client devices, comprising:
a computing system comprising one or more processors, coupled with memory, to:
extract, from a plurality of electronic evaluation surveys executed via a plurality of client devices over a network, a plurality of strings indicative of performance of one or more digital components configured to render via the plurality of client devices (“In step 510, the pretrained ML model is applied to the input text as to the text of the training set, and vector representation(s) of the text input is generated. The autoencoder 240 is applied in step 510 to generate a series of vector representations from and representing textual sentences or phrases. In some embodiments, a corpus of text is stored as individual string inputs in memory 210, for example as survey data 233” [0070], “Customer support tickets may in some embodiments be created dynamically at the start of any communication between a user and customer support system 210, regardless of who and how that interaction was initiated (e.g., the presentation of a survey or chatbot to the user), or the initiation of a survey instance. Ticket data 234 may also include one or more of severity data, product ID data, feature or defect ID data (when associated with a software product), ticket topic data, and/or customer support history data organized by user. Severity data may be obtained or inferred from the content of the user's input itself (the structured or unstructured text of the survey response). Product data may also be so obtained, or may be dynamically inferred, e.g., from the particular user activity of the user. For instance, where the user has visited multiple account management and/or payment screens, the problem may be inferred to be one broadly related to payment, or even more narrowly related to, e.g., adding a new payment account. This category of “product” may in one example be dynamically inferred from this content of the user's textual customer service inputs, through a natural language processing (NLP) analysis that may include semantic classification (e.g., topic classification) based on feature extraction from freeform text. In some embodiments, a feature (possibly with a smaller scope) may be identifiable from the application of the machine learning algorithm.” [0050], character string in freeform text [0054]);
construct, for input into a first model comprising a transformer neural network, a prompt data structure formulated based on: i) the plurality of strings, ii) an indication of the one or more digital components (“In step 510, the pretrained ML model is applied to the input text as to the text of the training set, and vector representation(s) of the text input is generated. The autoencoder 240 is applied in step 510 to generate a series of vector representations from and representing textual sentences or phrases. In some embodiments, a corpus of text is stored as individual string inputs in memory 210, for example as survey data 233, or in a separate memory or hard disk. … The specific method of generating the vector from the textual input may be performed through any of a variety of known methods. In one exemplary embodiment, Google's BERT (Bidirectional Encoder Representations from Transformers) model or a model based on BERT, such as XLM-RoBERTa, may be used as an applied natural language processing (NLP) model. However, in other embodiments, any other appropriate pre-trained model (e.g., Generative Pretrained Transformer (GPT), Text-to-Text Transfer Transformer (T5), ULMFiT, etc.) may be used.” [0070]), iii) an instruction to identify an aspect of the plurality of strings (“For each of these topics, one or more machine learning models may be applied to detect patterns in the input freeform text that suggest relevance. For example, for “payment”, the models might detect patterns such as currency symbols, numbers, related words such as credit/debit, expensive/cheap, refund, bank, worth, price, account, and/or combinations of words in particular relevant order and/or structure.” [0072]),
generate, via input of the prompt data structure into the first model, a first output comprising a plurality of aspects and a plurality of terms extracted from the plurality of strings that are associated with the plurality of aspects generated by the first model (“Steps 512 and 514 are directed to the application of machine learning models to classify topics and analyze sentiment respectively, based on the text input by the user in step 508. In step 512, a topic classification is performed to determine the semantic meaning of the input text. In one embodiment, the user may, in a plain text sentence, phrase, or passage, reference a concept or description connecting the input to a particular scenario, circumstance, product, problem type, or the like. Rather than a Boolean search or filtered search (e.g., where the user selects keywords, categories, or characteristics), a machine learning analysis is performed on the free entry of text in the user's natural language. This input may be used by sentiment analysis 124 to obtain from a database one or more responsive text results that are contextually related to the content of the freeform input, without having to identically or explicitly match word-for-word content in such stored data.” [0071]);
convert the plurality of terms of the first output into embedding vectors (“In some embodiments, this may involve feature extraction from freeform text e.g., to generate vectors for words or sentences (step 512). Topic classifiers are defined in advance, and stored in memory 210 as thematic response data 235. As examples, some predefined topics may include: “user account”, “payment”, “booking”, “cancellation”, “confirmation”, and so on. The specific topics can be generally understood to be specific to the purpose and use of the website or application. In some embodiments, the predefined topics may correspond to products, ticket topics or identifiers, or other delimiters created by a backend customer support system.” [0072]);
create, via input of the embedding vectors into a second model trained with machine learning to cluster, a plurality of clusters for the first output of the first model (“The models would then label the corresponding text in the input string appropriately. In some embodiments, exemplary algorithms are NLP topic classifications models such as Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA), and/or other text classification algorithms such as Naïve Bayes, Support Vector Machines (SVM).” [0072], “As an output of step 514, one or more topic classifiers may be assigned to the user input string. The topic analysis may be applied at one or more of a sentence level (i.e., for each individual input by the user), at the field level (i.e., for the whole of a single query or response entered in an input field by the user in a back-and-forth conversation via a chatbot or messaging app, or in another written survey response). In some embodiments, the topic analysis may additionally be applied at the session level, where the topics defined by the user's text input are refined, altered, rewritten, or otherwise revisited in view of the holistic whole of the inputs by the user during the session.” [0073], Fig 5 steps 510, 512, 514);
determine a metric indicative of performance of the one or more digital components based on a count of the plurality of strings associated with a cluster of the plurality of clusters (“The calculation of such a sentiment score is performed in step 516. Each time the user submits an additional character string, a real-time sentiment analysis may be performed thereon and a sentiment score may be calculated to reflect the user's current satisfaction or happiness. Where more than one text input has been submitted by the user (that is, second and subsequent inputs), step 516 may also involve the calculation of a delta sentiment (Δsentiment) or change in sentiment after the submission of each input. This delta sentiment value reflects a change in sentiment between the previous user input and the most recent user input. Where the value is positive (or above a certain threshold), the user sentiment is considered to improve. Where the value is negative (or below a certain threshold), the user sentiment is considered to have degraded (i.e., the user's experience is worsening and they are being increasingly unhappy). Where the value is zero (in within a predetermined range), the user sentiment is considered to have remained stable. The delta sentiment value reflects a trend in sentiment over the course of the interaction. In some embodiments, the delta sentiment score is overwritten when each subsequent input is evaluated, and in other embodiments, a series of delta sentiment scores is retained and stored in memory 210, so that the user's changing satisfaction can be evaluated over a longer period of time.” [0078]); and
execute an action based at least in part on the metric to control delivery of the one or more digital components (Fig 5 “Use feature-level metrics to compare against defined guardrails and make a feature shipment determination).
Jungmeisteris does not explicitly disclose: and iv) a constraint on a size of output by the first model.
Acharya teaches: iv) a constraint on a size of output by the first model (“The response of the model (e.g., the output text data) is determined based on how the model is trained to respond to certain input text data. Possible variations in responses include but are not limited to the number of words of output in each turn, word selection for each position of output, sentence type (e.g., statement or question), or other such variations; the content of the output may include greeting the user, confirming receipt of information, prompting the user for further information, or other such content.” C11L54-63).
Jungmeisteris and Acharya are in the same field of endeavor of machine learning models and are analogous. Jungmeisteris discloses a transformer chatbot-based system for sentiment analysis of digital products. Acharya discloses a dialog neural network system with constraints on output size. It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the prompt structure of Jungmeisteris with the constraint on prompt output as taught by Acharya to yield predictable results of a desired response characteristic.
Regarding claims 2, 12 and 20, Jungmeisteris discloses:
The system of claim 1, comprising:
the computing system to construct the prompt data structure with a second instruction to identify one or more sentiments associated with the plurality of strings (“In an exemplary embodiment, sentiment analysis from text is performed by one or more supervised or unsupervised algorithms. In an exemplary embodiment, a transformer-based deep learning technique is used for sentiment analysis and other large scale NLP processing tasks.” [0077]).
Regarding claims 3 and 13, Jungmeisteris discloses: The system of claim 1, comprising:
the computing system to construct the prompt data structure with a second instruction to identify a sentiment associated with each of the plurality of strings, wherein the sentiment indicatives one of positive, neutral, or negative (“In some embodiments, this score may then be compared to a range of possible scores to classify the sentiment as positive, negative, or neutral. In other embodiments, other classification schemes may be used, e.g., highly/slightly positive/negative. In an exemplary embodiment, the sentiment score is a value from 0-1, with 0 being the most negative, and 1 being the most positive.” [0077]).
Regarding claims 4 and 14, Jungmeisteris does not explicitly disclose, however, Acharya teaches: The system of claim 1, comprising:
the computing system to construct the prompt data structure with the constraint on the size of the first output comprising a number of words (“The response of the model (e.g., the output text data) is determined based on how the model is trained to respond to certain input text data. Possible variations in responses include but are not limited to the number of words of output in each turn, word selection for each position of output, sentence type (e.g., statement or question), or other such variations; the content of the output may include greeting the user, confirming receipt of information, prompting the user for further information, or other such content.” C11L54-63).
Regarding claims 5 and 15, Jungmeisteris discloses: The system of claim 1, comprising:
the computing system to identify, via the first model, a name for a first cluster of the plurality of clusters created by the second model (“Topic classifiers are defined in advance, and stored in memory 210 as thematic response data 235. As examples, some predefined topics may include: “user account”, “payment”, “booking”, “cancellation”, “confirmation”, and so on. The specific topics can be generally understood to be specific to the purpose and use of the website or application. In some embodiments, the predefined topics may correspond to products, ticket topics or identifiers, or other delimiters created by a backend customer support system.” [0072]).
Regarding claims 6 and 16, Jungmeisteris discloses: The system of claim 1, comprising the computing system to:
construct a second prompt structure comprising: i) instructions to generate a plurality of topics for the plurality of clusters created by the second model, and ii) instructions to generate a summary based on the plurality of topics and the plurality of terms extracted from the plurality of strings; and
generate, via input of the second prompt structure into the first model, a second output comprising the plurality of topics and the summary (“In contrast, the systems and methods described herein provide a single platform that manages and administers surveys through different support sources and channels to obtain customer feedback at various levels of granularity. Further still, customer workflows can be inferred and additional signals, such as customer sentiment, can be collected directly from and during a support interaction and/or other text-based survey data. These signals and workflows can be analyzed in real-time using NLP analysis, and tailored questions and responses can be provided to the user in real-time in a manner and language that is accessible to them, without delay and without changing medium. Further still, the systems and methods described herein allow for the ability to expose, aggregate and analyze the collected data in the analytical tools and processes used by business, operations, community support ambassadors and product teams” [0093], Fig 5 steps 540, 542, 544, 546, 548).
Regarding claims 9 and 18, Jungmeisteris discloses: The system of claim 1, comprising:
the computing system to reduce a frequency of delivery of the one or more digital components based on the metric less than or equal to a performance threshold (“Step 548 involves the generation of a feature-level report on the support success metrics, for example as a document or user sentiment metadata for export. Such reporting would be most usual and valuable in a testing environment where certain features (or products) have been rolled out to all or a subset of customers. These feature-level metrics can be applied, in step 550, in a determination of the impact of such features to overall customer satisfaction (and therefore customer support success). Based on this evaluation, it can be determined whether the features fall within certain customer satisfaction guardrails. That is, if customer satisfaction at the feature-level is not sufficiently high to surpass the guardrails, an automated recognition may be triggered that the feature should not be launched, shipped, or deployed to customers in its current form, and such shipment may be automatically delayed or withheld. Accordingly, customer sentiment may be tied to software development and release cycles in an automated manner.” [0091] note: a decision to deploy or not bases on the guardrails is interpreted as a change in frequency).
Regarding claim 10, Jungmeisteris discloses: The system of claim 1, comprising:
the computing system to increase a frequency of delivery of the one or more digital components based on the metric greater than or equal to a performance threshold (“Step 548 involves the generation of a feature-level report on the support success metrics, for example as a document or user sentiment metadata for export. Such reporting would be most usual and valuable in a testing environment where certain features (or products) have been rolled out to all or a subset of customers. These feature-level metrics can be applied, in step 550, in a determination of the impact of such features to overall customer satisfaction (and therefore customer support success). Based on this evaluation, it can be determined whether the features fall within certain customer satisfaction guardrails. That is, if customer satisfaction at the feature-level is not sufficiently high to surpass the guardrails, an automated recognition may be triggered that the feature should not be launched, shipped, or deployed to customers in its current form, and such shipment may be automatically delayed or withheld. Accordingly, customer sentiment may be tied to software development and release cycles in an automated manner.” [0091] note: a decision to deploy or not bases on the guardrails is interpreted as a change in frequency).
Claim(s) 7, 8, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jungmeisteris in view of Acharya and further in view of Parnaby et al. (US 20130018957 A1).
Regarding claims 7 and 17, Jungmeisteris does not explicitly disclose, however, Parnaby teaches: The system of claim 1, comprising the computing system to:
generate a bar chart based on the count of the plurality of strings associated with the cluster of the plurality of clusters (Fig 32); and
provide the bar chart for display via a graphical user interface (“an area 226 where the basic sentiment analysis of what the Jabfab.TM. community is saying about the topic can be presented, including, but not limited to, line or bar graphs of the sentiments collected over time and by category, pie charts of same, a global map of all the topic locations or users locations when jabbed or fabbed;” [0067]).
Jungmeisteris, Acharya and Parnaby are in the same field of endeavor of language analysis and are analogous. Jungmeisteris discloses a transformer chatbot-based system for sentiment analysis of digital products. Acharya discloses a dialog neural network system with constraints on output size. Parnaby teaches a system that uses graphs to display sentiment information. It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the sentiment output of Jungmeisteris and Acharya with the sentiment counts and bar charts as taught by Parnaby to yield predictable results of an easily understandable interface for users.
Regarding claim 8, Jungmeisteris does not explicitly disclose, however, Parnaby teaches: The system of claim 1, comprising the computing system to:
provide, for display via a graphical user interface, a bar chart that indicates the count of the plurality of strings associated with the cluster of the plurality of clusters (“an area 226 where the basic sentiment analysis of what the Jabfab.TM. community is saying about the topic can be presented, including, but not limited to, line or bar graphs of the sentiments collected over time and by category, pie charts of same, a global map of all the topic locations or users locations when jabbed or fabbed;” [0067]); and
overlay, on a bar of the bar chart, the metric for a corresponding string of the plurality of strings associated with the bar (“an area 226 where the basic sentiment analysis of what the Jabfab.TM. community is saying about the topic can be presented, including, but not limited to, line or bar graphs of the sentiments collected over time and by category, pie charts of same, a global map of all the topic locations or users locations when jabbed or fabbed;” [0067]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC NILSSON whose telephone number is (571)272-5246. The examiner can normally be reached M-F: 7-3.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571)-272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERIC NILSSON/ Primary Examiner, Art Unit 2151
Prosecution Insights