DETAILED ACTION
This final Office action is responsive to Applicant’s amendment filed July 6, 2026. Claims 1, 8, and 17 have been amended. Claims 2, 4-5, 9, 13, and 20-24 are canceled. Claims 1, 3, 6-8, 10-12, and 14-19 are presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed July 6, 2026 have been fully considered but they are not persuasive.
Preliminarily, it is noted that the previously-pending rejections under 35 U.S.C. § 112(a) are withdrawn in response to Applicant’s claim amendments.
Regarding the art rejections, Applicant’s arguments center about the assertion that the teachings of ensemble learning in the references are not specifically used in conjunction with the specific claim operations. For example, Applicant states, “The parallelism cited in the Office Action relates to classifier training, a feature extractor, and a classifier module operating in an item-attribute verification system for image/listing data. That disclosure does not teach or suggest the claimed text-intent architecture, the claimed extraction of actions, entities, and associated connections from social-media text, or the claimed feedback that uses a weighted balance between score-logic outputs and intent-identification outputs.” (Page 9 of Applicant’s response) The Examiner maintains that the claims perform various operations related to data aggregation, data extraction, intent identification, classification, and scoring, and then incorporates the known concept of ensemble learning to facilitate these operations. The Examiner further points out that the benefits of ensemble learning (e.g., performing machine learning processes in parallel and weighting the output of each machine learning process) are relevant to any related processes. Jezewski largely addresses the details of the analysis performed throughout the claims and Biessmann, Huang, and Givental collectively provide more explanation of the benefits of processing information in parallel, including in an ensemble learning environment. Also, Huang provides additional motivation to combine the teachings of the various references. For example, as explained in the rejection, while Biessmann does not explicitly link the classification and extraction to an environment in which an audience sentiment is evaluated, Huang sheds some additional light on the benefits of performing operations related to extraction (like sentiment analysis) simultaneously with classification in the area of user sentiment analysis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6-8, 10-12, and 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over Jezewski (US 2019/0251626) in view of Biessmann et al. (US 9,892,133) in view of Huang et al. (US 2014/0250032) in view of Givental et al. (US 2021/0281592).
[Claim 1] Jezewski discloses a system implemented as a text analysis application executed by a hardware controller to analyze text data and social media posts to determine an audience interest level associated with business target features (¶¶ 11-12, 120), the system comprising:
a data aggregation logic executed by the text analysis application to collect text data associated with at least one of the business target feature through a hardware network (¶¶ 15-20 – Indications of user sentiment toward companies may be gathered; ¶ 12 – “As shown in FIG. 1A, example implementation 100 may include a client device, comment sources, and a sentiment analysis platform. Assume that a user utilizes the client device to access one or more applications provided by the comment sources. In some implementations, the comment sources may include sources that provide social media applications, blog applications, chat room applications, message board applications, ratings system applications, and/or the like. As further shown in FIG. 1A, the user may utilize the client device to provide complaint information, about an entity (e.g., company A), to the one or more applications provided by the comment sources. For example, the complaint information may include a complaint indicating that the user is switching from company A to company B, a complaint indicating a negative statement about company A, a complaint indicating invalidation of a product or service of company A, and/or the like.”; ¶ 120 – “Some implementations described herein may provide a sentiment analysis platform that utilizes artificial intelligence to make a prediction about an entity based on user sentiment and transaction history. For example, the sentiment analysis platform may consider sentiments of users who have opinions, complaints, and predictions about the entity, and transactions conducted by users with the entities in order to predict a future stock price of the entity. The sentiment analysis platform may receive the opinions, the complaints, and the predictions of the users, about the entity, from social media sources, and may receive transaction information associated with the users and the entity from financial institutions. The sentiment analysis platform may determine correlations between the transaction information and the opinions, the complaints, and the predictions, in order to apply weights to the opinions, the complaints, and the predictions. The sentiment analysis platform may generate a prediction about the future stock price of the entity based on the opinions, the complaints, the predictions, the transaction information, and the correlations between the transaction information and the opinions, the complaints, and the predictions.” Comments made on/via social media and/or in association with a social media source are examples of social media posts.);
an intent identification logic executed by the text analysis application (¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶ 120 – AI sentiment analysis platform.), the intent identification logic including:
an information extraction logic configured to extract information from the collected text data including metadata, actions, and entities with associated connections using tools that identify a role or a set of features for each word (¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”), and
wherein the intent identification logic is configured to identify intent actions based on the extracted information that includes related entities by aggregating a general idea or action toward an object (¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶ 120 – AI sentiment analysis platform.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”);
a classification logic configured to assign at least one label to portions of the collected text data, the classification logic including a trained statistical model stored in non-transitory memory (¶¶ 44-48; ¶¶ 59-60 – “[0059] As shown in FIG. 1F, and by reference number 155, the sentiment analysis platform may utilize the transaction information to determine correlations with the interim scores (e.g., the necessity score, the abstract score, the ethics score, the industry score, the demands for service score, the supply for service score, the vendor score, the innovation score, the adaptability score, the execution score, and/or the like). In some implementations, the sentiment analysis platform may utilize a correlation clustering method, a Pearson's product-moment coefficient method, an Anscombe's quartet method, a Spearman's rank-correlation coefficient method, and/or the like in order to determine the correlations between the transaction information and the interim scores. [0060] A clustering method may include partitioning data points into groups based on their similarity, and the correlation clustering method may include clustering a set of objects into an optimum number of clusters without specifying that number in advance. Given a collection of paired (x, y) variables, the Pearson's product-moment coefficient method produces a value, between −1 and +1, that quantifies a strength of dependence between the variables x and y. A value of +1 means that all of the (x, y) points lie exactly on a line with positive slope, a value of −1 means that all of the points lie exactly on a line with negative slope, and a value of 0 means that there is no relationship between the two variables. The Anscombe's quartet method utilizes four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. The Spearman's rank-correlation coefficient method provides a nonparametric measure of rank correlation (e.g., a statistical dependence between a ranking of two variables), and assesses how well a relationship between two variables can be described using a monotonic function.“; ¶ 88 – “Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 320 executing software instructions stored by a non-transitory computer-readable medium, such as memory 330 and/or storage component 340. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.”);
a score logic configured to compute intent scores for labelled data using a trained statistical model stored in non-transitory memory (¶ 36 – “In some implementations, the artificial intelligence techniques may include a sentiment analysis model that utilizes multiple artificial analysis techniques. In some implementations, the sentiment analysis model may include a model that uses natural language processing, text analysis, and machine learning to systematically identify, extract, quantify, and study affective states and subjective information.”; ¶¶ 59-60 – “[0059] As shown in FIG. 1F, and by reference number 155, the sentiment analysis platform may utilize the transaction information to determine correlations with the interim scores (e.g., the necessity score, the abstract score, the ethics score, the industry score, the demands for service score, the supply for service score, the vendor score, the innovation score, the adaptability score, the execution score, and/or the like). In some implementations, the sentiment analysis platform may utilize a correlation clustering method, a Pearson's product-moment coefficient method, an Anscombe's quartet method, a Spearman's rank-correlation coefficient method, and/or the like in order to determine the correlations between the transaction information and the interim scores. [0060] A clustering method may include partitioning data points into groups based on their similarity, and the correlation clustering method may include clustering a set of objects into an optimum number of clusters without specifying that number in advance. Given a collection of paired (x, y) variables, the Pearson's product-moment coefficient method produces a value, between −1 and +1, that quantifies a strength of dependence between the variables x and y. A value of +1 means that all of the (x, y) points lie exactly on a line with positive slope, a value of −1 means that all of the points lie exactly on a line with negative slope, and a value of 0 means that there is no relationship between the two variables. The Anscombe's quartet method utilizes four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. The Spearman's rank-correlation coefficient method provides a nonparametric measure of rank correlation (e.g., a statistical dependence between a ranking of two variables), and assesses how well a relationship between two variables can be described using a monotonic function.“; ¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶ 88 – “Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 320 executing software instructions stored by a non-transitory computer-readable medium, such as memory 330 and/or storage component 340. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.”).
While Jezewski strongly suggests that the disclosed operations may be performed substantially simultaneously (i.e., in parallel) since the various models may be applied in conjunction with one another, including to classify and extract data (as seen in ¶¶ 36-46 of Jezewski), Jezewski does not explicitly disclose:
wherein the data aggregation logic routes the collected text data in parallel to the classification logic and the information extraction logic;
wherein outputs of the score logic and the intent identification logic are supplied to a feedback mechanism;
wherein the feedback mechanism uses the outputs from the score logic and the intent identification logic with a weighted balance between the outputs of the intent identification logic and the score logic.
Biessmann verifies item attributes in an artificial intelligence environment and explains how the extraction and classification may be performed in parallel, as seen in the following excerpt:
The classifier training module 150, the feature extractor module 152, and/or the classifier module 154 can operate in parallel and for multiple users at the same time. For example, the verification of attributes listed in an item description may be requested from unique user devices 102 for different listings and the components of the attribute verification system 104 can verify the listed attributes simultaneously or nearly simultaneously for the unique user devices 102 in “real-time,” where “real-time” may be based on the perspective of the user. The generation and verification of attributes may be considered to occur in real time if, for example, the delay is sufficiently short (e.g., less than a few seconds) such that the user typically would not notice a processing delay. Real-time may also be based on the following: the attribute verification system 104 can verify listed attributes for a first user at the same time or at nearly the same time (e.g., within a couple seconds) as a verification of listed attributes is performed for a second user; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously as instantaneously as possible, limited by processing resources, available memory, network bandwidth conditions, and/or the like; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously based on a time it takes the hardware components of the attribute verification system 104 to process data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously immediately as data is received (instead of storing, buffering, caching, or persisting data as it is received and processing the data later on); the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by transforming data without intentional delay, given the processing limitations of the attribute verification system 104 and other systems, like the user devices 102, and the time required to accurately receive and/or transmit the data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by processing or transforming data fast enough to keep up with an input data stream; etc. (Biessmann: col. 12: 13-51)
Furthermore, Biessmann uses feedback to update the training, thereby confirming which suggested attributes are correct and incorrect (Biessmann: col 3: 5-24; col. 5: 41-58; col. 9: 26 – col. 10: 3; col. 14: 7-14).
While Biessmann does not explicitly link the classification and extraction to an environment in which an audience sentiment is evaluated, Huang sheds some additional light on the benefits of performing operations related to extraction (like sentiment analysis) simultaneously with classification in the area of user sentiment analysis, as described in the following excerpts of Huang:
[0002] Sentiment and topic analysis have a wide application in business marketing and customer care applications to assist in evaluating and understanding brand perception and customer requirements based on, for example, data gathered from millions of online posts such as social media, forums, and blogs. For example, when promoting a new policy/product, a company may monitor electronically posted customer comments regarding a particular policy/product so that the company can respond properly and address criticisms and issues in a timely manner. Hence, online monitoring of current sentiment trend and topics related to, for example, a preset product and brand name is important for modern marketing…
[0011] The aforementioned aspects and other objectives and advantages can now be achieved as described herein. Methods, systems and processor-readable media for simultaneous sentiment analysis and topic classification with multiple labels are disclosed herein. A sentiment and topic associated with a post can be classified at similar time and a result can be incorporated to predict a feature so that a label of two tasks can promote and reinforce each other iteratively. A feature extraction and selection can be performed on both tasks of sentiment and topic classification. A multi-task multi-label classification model can be trained for each task with maximum entropy utilizing multiple labels to ascertain data indicative of and/or derived from an extra label and to manage with class ambiguities. Each task has a separate classification model with different predicting features and they can be trained collectively which allows flexibility in model construction. Such multi-task multi-label (MTML) classification model produces a probabilistic result and the classes can be ranked by the probabilistic result and the post can be classified with the multi-label.
Regarding the weighted balance details, Givental uses hybrid machine learning to detect anomalies using feedback from the outputs of various learning models of an ensemble to assign weights to the various models based on whether or not each respective model has outputted a correct result (Givental: ¶¶ 52, 68).
As discussed above, Jezewski uses machine learning and semantic analysis to glean user sentiment. The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Jezewski:
wherein the data aggregation logic routes the collected text data in parallel to the classification logic and the information extraction logic;
wherein outputs of the score logic and the intent identification logic are supplied to a feedback mechanism;
wherein the feedback mechanism uses the outputs from the score logic and the intent identification logic with a weighted balance between the outputs of the intent identification logic and the score logic
in order to minimize delay in the sentiment analysis (as suggested in Biessmann: col. 12: 13-51), so that the various dimensions of gathered information may be used to reinforce each other (as suggested in ¶ 11 of Huang), and to improve the accuracy of the models (as suggested in Biessmann: col 3: 5-24; col. 5: 41-58; col. 9: 26 – col. 10: 3; col. 14: 7-14), including improvement in the overall accuracy of the ensemble of learning models (as suggested in ¶ 24 of Givental).
[Claim 3] Jezewski discloses wherein the score logic adds probability to the at least one assigned label, wherein the probability indicates how likely each labelled data belongs to the at least one assigned label (¶¶ 43-45).
[Claim 6] Jezewski discloses wherein output of the intent identification logic couples to input of the classification logic so that the extracted information without clearly identified intent is sent to the classification logic (¶¶ 36-46 – The various models may be applied in conjunction with one another, including to classify and extract data. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40)).
[Claim 7] Jezewski discloses wherein the intent identification logic couples to the feedback so that the extracted information with clearly identified intent is sent to the feedback (¶¶ 36-46 – The various models may be applied in conjunction with one another, including to classify and extract data; ¶¶ 43-63, 111-112, 120 – A probabilistic classifier model may be used for sentiment analysis. Weights may be applied as part of training models in regard to interim scores, opinions, complaints, predictions, labeling features, identifying correlations among the various data and scores, etc. Feedback may refer to the feedback from customers and/or to feedback used to train the models).
[Claim 8] A computer-implemented method executed by a hardware controller operating a text analysis application to analyze text data and social media posts to acquire accurate measure of audience interest level including business target features (¶¶ 11-12, 120), the method comprising:
collecting text data associated with at least one business target feature including social media posts (¶¶ 15-20 – Indications of user sentiment toward companies may be gathered; ¶ 12 – “As shown in FIG. 1A, example implementation 100 may include a client device, comment sources, and a sentiment analysis platform. Assume that a user utilizes the client device to access one or more applications provided by the comment sources. In some implementations, the comment sources may include sources that provide social media applications, blog applications, chat room applications, message board applications, ratings system applications, and/or the like. As further shown in FIG. 1A, the user may utilize the client device to provide complaint information, about an entity (e.g., company A), to the one or more applications provided by the comment sources. For example, the complaint information may include a complaint indicating that the user is switching from company A to company B, a complaint indicating a negative statement about company A, a complaint indicating invalidation of a product or service of company A, and/or the like.”; ¶ 120 – “Some implementations described herein may provide a sentiment analysis platform that utilizes artificial intelligence to make a prediction about an entity based on user sentiment and transaction history. For example, the sentiment analysis platform may consider sentiments of users who have opinions, complaints, and predictions about the entity, and transactions conducted by users with the entities in order to predict a future stock price of the entity. The sentiment analysis platform may receive the opinions, the complaints, and the predictions of the users, about the entity, from social media sources, and may receive transaction information associated with the users and the entity from financial institutions. The sentiment analysis platform may determine correlations between the transaction information and the opinions, the complaints, and the predictions, in order to apply weights to the opinions, the complaints, and the predictions. The sentiment analysis platform may generate a prediction about the future stock price of the entity based on the opinions, the complaints, the predictions, the transaction information, and the correlations between the transaction information and the opinions, the complaints, and the predictions.” Comments made on/via social media and/or in association with a social media source are examples of social media posts.);
extracting information from the text data including metadata, actions, and entities with associated connections using tools that identify a role or a set of features for each word (¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”);
identifying intent actions based on the extracted information that includes related entities by aggregating a general idea or action toward an object (¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶ 120 – AI sentiment analysis platform.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”);
assigning at least one label using a trained classification model stored in non-transitory memory (¶ 36 – “In some implementations, the artificial intelligence techniques may include a sentiment analysis model that utilizes multiple artificial analysis techniques. In some implementations, the sentiment analysis model may include a model that uses natural language processing, text analysis, and machine learning to systematically identify, extract, quantify, and study affective states and subjective information.”; ¶¶ 59-60 – “[0059] As shown in FIG. 1F, and by reference number 155, the sentiment analysis platform may utilize the transaction information to determine correlations with the interim scores (e.g., the necessity score, the abstract score, the ethics score, the industry score, the demands for service score, the supply for service score, the vendor score, the innovation score, the adaptability score, the execution score, and/or the like). In some implementations, the sentiment analysis platform may utilize a correlation clustering method, a Pearson's product-moment coefficient method, an Anscombe's quartet method, a Spearman's rank-correlation coefficient method, and/or the like in order to determine the correlations between the transaction information and the interim scores. [0060] A clustering method may include partitioning data points into groups based on their similarity, and the correlation clustering method may include clustering a set of objects into an optimum number of clusters without specifying that number in advance. Given a collection of paired (x, y) variables, the Pearson's product-moment coefficient method produces a value, between −1 and +1, that quantifies a strength of dependence between the variables x and y. A value of +1 means that all of the (x, y) points lie exactly on a line with positive slope, a value of −1 means that all of the points lie exactly on a line with negative slope, and a value of 0 means that there is no relationship between the two variables. The Anscombe's quartet method utilizes four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. The Spearman's rank-correlation coefficient method provides a nonparametric measure of rank correlation (e.g., a statistical dependence between a ranking of two variables), and assesses how well a relationship between two variables can be described using a monotonic function.“; ¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶ 88 – “Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 320 executing software instructions stored by a non-transitory computer-readable medium, such as memory 330 and/or storage component 340. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.”);
computing intent scores using a trained scoring model stored in non-transitory memory (¶ 36 – “In some implementations, the artificial intelligence techniques may include a sentiment analysis model that utilizes multiple artificial analysis techniques. In some implementations, the sentiment analysis model may include a model that uses natural language processing, text analysis, and machine learning to systematically identify, extract, quantify, and study affective states and subjective information.”; ¶¶ 59-60 – “[0059] As shown in FIG. 1F, and by reference number 155, the sentiment analysis platform may utilize the transaction information to determine correlations with the interim scores (e.g., the necessity score, the abstract score, the ethics score, the industry score, the demands for service score, the supply for service score, the vendor score, the innovation score, the adaptability score, the execution score, and/or the like). In some implementations, the sentiment analysis platform may utilize a correlation clustering method, a Pearson's product-moment coefficient method, an Anscombe's quartet method, a Spearman's rank-correlation coefficient method, and/or the like in order to determine the correlations between the transaction information and the interim scores. [0060] A clustering method may include partitioning data points into groups based on their similarity, and the correlation clustering method may include clustering a set of objects into an optimum number of clusters without specifying that number in advance. Given a collection of paired (x, y) variables, the Pearson's product-moment coefficient method produces a value, between −1 and +1, that quantifies a strength of dependence between the variables x and y. A value of +1 means that all of the (x, y) points lie exactly on a line with positive slope, a value of −1 means that all of the points lie exactly on a line with negative slope, and a value of 0 means that there is no relationship between the two variables. The Anscombe's quartet method utilizes four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. The Spearman's rank-correlation coefficient method provides a nonparametric measure of rank correlation (e.g., a statistical dependence between a ranking of two variables), and assesses how well a relationship between two variables can be described using a monotonic function.“; ¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶ 88 – “Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 320 executing software instructions stored by a non-transitory computer-readable medium, such as memory 330 and/or storage component 340. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.”);
filtering and recognizing related input data based on intent criteria using the extracted information (¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶ 42 – “In some implementations, the feature selection method may select subsets of relevant features (e.g., variables or predictors), from the historical information, the complaint information, the opinion information, and the prediction information, for use in the sentiment analysis model. In some implementations, the feature selection method may include a wrapper method (e.g., that uses a predictive model to score feature subsets), a filter method (e.g., that uses a proxy measure instead of an error rate to score a feature subset), an embedded method (e.g., that performs feature selection as part of construction of the sentiment analysis mode), and/or the like.”);
providing aggregated data about each business target feature as a feedback regarding the intent (¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶ 52 – “In some implementations, the industry score calculated by the sentiment analysis platform may provide an indication of aggregate opinion information about an industry in which company A operates, an indication of whether the industry is dying (e.g., based on previously-identified industries that were subject to automation), and/or the like.”; ¶ 54 – “In some implementations, the supply for services score calculated by the sentiment analysis platform may provide an indication of aggregate complaint information associated with product stocking by company A, product delivery time by company A, product quality issues of company A, and/or the like, which may indicate supply problems for company A, and/or the like.”; ¶ 55 – “In some implementations, the vendor score calculated by the sentiment analysis platform may provide an indication of qualities associated with third party vendors, partners, contractors, and/or the like used by company A, and/or the like.”).
While Jezewski strongly suggests that the disclosed operations may be performed substantially simultaneously (i.e., in parallel) since the various models may be applied in conjunction with one another, including to classify and extract data (as seen in ¶¶ 36-46 of Jezewski), Jezewski does not explicitly disclose:
applying the collected text data in parallel to both a classification logic and an information extraction logic;
supplying outputs of intent identification and scoring operations to a feedback mechanism;
using, by the feedback mechanism, outputs from the intent identification and scoring operations with a weighted balance between the outputs of the intent identification and scoring operations.
Biessmann verifies item attributes in an artificial intelligence environment and explains how the extraction and classification may be performed in parallel, as seen in the following excerpt:
The classifier training module 150, the feature extractor module 152, and/or the classifier module 154 can operate in parallel and for multiple users at the same time. For example, the verification of attributes listed in an item description may be requested from unique user devices 102 for different listings and the components of the attribute verification system 104 can verify the listed attributes simultaneously or nearly simultaneously for the unique user devices 102 in “real-time,” where “real-time” may be based on the perspective of the user. The generation and verification of attributes may be considered to occur in real time if, for example, the delay is sufficiently short (e.g., less than a few seconds) such that the user typically would not notice a processing delay. Real-time may also be based on the following: the attribute verification system 104 can verify listed attributes for a first user at the same time or at nearly the same time (e.g., within a couple seconds) as a verification of listed attributes is performed for a second user; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously as instantaneously as possible, limited by processing resources, available memory, network bandwidth conditions, and/or the like; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously based on a time it takes the hardware components of the attribute verification system 104 to process data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously immediately as data is received (instead of storing, buffering, caching, or persisting data as it is received and processing the data later on); the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by transforming data without intentional delay, given the processing limitations of the attribute verification system 104 and other systems, like the user devices 102, and the time required to accurately receive and/or transmit the data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by processing or transforming data fast enough to keep up with an input data stream; etc. (Biessmann: col. 12: 13-51)
Furthermore, Biessmann uses feedback to update the training, thereby confirming which suggested attributes are correct and incorrect (Biessmann: col 3: 5-24; col. 5: 41-58; col. 9: 26 – col. 10: 3; col. 14: 7-14).
While Biessmann does not explicitly link the classification and extraction to an environment in which an audience sentiment is evaluated, Huang sheds some additional light on the benefits of performing operations related to extraction (like sentiment analysis) simultaneously with classification in the area of user sentiment analysis, as described in the following excerpts of Huang:
[0002] Sentiment and topic analysis have a wide application in business marketing and customer care applications to assist in evaluating and understanding brand perception and customer requirements based on, for example, data gathered from millions of online posts such as social media, forums, and blogs. For example, when promoting a new policy/product, a company may monitor electronically posted customer comments regarding a particular policy/product so that the company can respond properly and address criticisms and issues in a timely manner. Hence, online monitoring of current sentiment trend and topics related to, for example, a preset product and brand name is important for modern marketing…
[0011] The aforementioned aspects and other objectives and advantages can now be achieved as described herein. Methods, systems and processor-readable media for simultaneous sentiment analysis and topic classification with multiple labels are disclosed herein. A sentiment and topic associated with a post can be classified at similar time and a result can be incorporated to predict a feature so that a label of two tasks can promote and reinforce each other iteratively. A feature extraction and selection can be performed on both tasks of sentiment and topic classification. A multi-task multi-label classification model can be trained for each task with maximum entropy utilizing multiple labels to ascertain data indicative of and/or derived from an extra label and to manage with class ambiguities. Each task has a separate classification model with different predicting features and they can be trained collectively which allows flexibility in model construction. Such multi-task multi-label (MTML) classification model produces a probabilistic result and the classes can be ranked by the probabilistic result and the post can be classified with the multi-label.
Regarding the weighted balance details, Givental uses hybrid machine learning to detect anomalies using feedback from the outputs of various learning models of an ensemble to assign weights to the various models based on whether or not each respective model has outputted a correct result (Givental: ¶¶ 52, 68).
As discussed above, Jezewski uses machine learning and semantic analysis to glean user sentiment. The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Jezewski:
applying the collected text data in parallel to both a classification logic and an information extraction logic;
supplying outputs of intent identification and scoring operations to a feedback mechanism;
using, by the feedback mechanism, outputs from the intent identification and scoring operations with a weighted balance between the outputs of the intent identification and scoring operations
in order to minimize delay in the sentiment analysis (as suggested in Biessmann: col. 12: 13-51), so that the various dimensions of gathered information may be used to reinforce each other (as suggested in ¶ 11 of Huang), and to improve the accuracy of the models (as suggested in Biessmann: col 3: 5-24; col. 5: 41-58; col. 9: 26 – col. 10: 3; col. 14: 7-14), including improvement in the overall accuracy of the ensemble of learning models (as suggested in ¶ 24 of Givental).
[Claim 10] Jezewski discloses wherein intent is identified by aggregating general idea or action toward an object (¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; ¶¶ 54, 57 – Complaint and opinion information may be aggregated).
[Claim 11] Jezewski discloses assigning at least one label to each data of the collected text data using a trained classification logic (¶¶ 44-48).
[Claim 12] Jezewski discloses scoring each labelled data based on training and assign intent based on the at least one assigned label using a score logic (¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.).
[Claim 14] Jezewski discloses wherein extracting information is performed by an information extraction logic (¶¶ 15-20).
[Claim 15] While Jezewski strongly suggests that the disclosed operations may be performed substantially simultaneously (i.e., in parallel) since the various models may be applied in conjunction with one another, including to classify and extract data (as seen in ¶¶ 36-46 of Jezewski), Jezewski does not explicitly perform the step of applying the collected text data in parallel to both the classification logic and the information extraction logic. Biessmann verifies item attributes in an artificial intelligence environment and explains how the extraction and classification may be performed in parallel, as seen in the following excerpt:
The classifier training module 150, the feature extractor module 152, and/or the classifier module 154 can operate in parallel and for multiple users at the same time. For example, the verification of attributes listed in an item description may be requested from unique user devices 102 for different listings and the components of the attribute verification system 104 can verify the listed attributes simultaneously or nearly simultaneously for the unique user devices 102 in “real-time,” where “real-time” may be based on the perspective of the user. The generation and verification of attributes may be considered to occur in real time if, for example, the delay is sufficiently short (e.g., less than a few seconds) such that the user typically would not notice a processing delay. Real-time may also be based on the following: the attribute verification system 104 can verify listed attributes for a first user at the same time or at nearly the same time (e.g., within a couple seconds) as a verification of listed attributes is performed for a second user; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously as instantaneously as possible, limited by processing resources, available memory, network bandwidth conditions, and/or the like; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously based on a time it takes the hardware components of the attribute verification system 104 to process data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously immediately as data is received (instead of storing, buffering, caching, or persisting data as it is received and processing the data later on); the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by transforming data without intentional delay, given the processing limitations of the attribute verification system 104 and other systems, like the user devices 102, and the time required to accurately receive and/or transmit the data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by processing or transforming data fast enough to keep up with an input data stream; etc. (Biessmann: col. 12: 13-51)
While Biessmann does not explicitly link the classification and extraction to an environment in which an audience sentiment is evaluated, Huang sheds some additional light on the benefits of performing operations related to extraction (like sentiment analysis) simultaneously with classification in the area of user sentiment analysis, as described in the following excerpts of Huang:
[0002] Sentiment and topic analysis have a wide application in business marketing and customer care applications to assist in evaluating and understanding brand perception and customer requirements based on, for example, data gathered from millions of online posts such as social media, forums, and blogs. For example, when promoting a new policy/product, a company may monitor electronically posted customer comments regarding a particular policy/product so that the company can respond properly and address criticisms and issues in a timely manner. Hence, online monitoring of current sentiment trend and topics related to, for example, a preset product and brand name is important for modern marketing…
[0011] The aforementioned aspects and other objectives and advantages can now be achieved as described herein. Methods, systems and processor-readable media for simultaneous sentiment analysis and topic classification with multiple labels are disclosed herein. A sentiment and topic associated with a post can be classified at similar time and a result can be incorporated to predict a feature so that a label of two tasks can promote and reinforce each other iteratively. A feature extraction and selection can be performed on both tasks of sentiment and topic classification. A multi-task multi-label classification model can be trained for each task with maximum entropy utilizing multiple labels to ascertain data indicative of and/or derived from an extra label and to manage with class ambiguities. Each task has a separate classification model with different predicting features and they can be trained collectively which allows flexibility in model construction. Such multi-task multi-label (MTML) classification model produces a probabilistic result and the classes can be ranked by the probabilistic result and the post can be classified with the multi-label.
As discussed above, Jezewski uses machine learning and semantic analysis to glean user sentiment. The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Jezewski to perform the step of applying the collected text data in parallel to both the classification logic and the information extraction logic in order to minimize delay in the sentiment analysis (as suggested in Biessmann: col. 12: 13-51) and so that the various dimensions of gathered information may be used to reinforce each other (as suggested in ¶ 11 of Huang).
[Claim 16] Jezewski discloses sending the extracted information with clearly identified intent to the feedback (¶¶ 36-46 – The various models may be applied in conjunction with one another, including to classify and extract data; ¶¶ 43-63, 111-112, 120 – A probabilistic classifier model may be used for sentiment analysis. Weights may be applied as part of training models in regard to interim scores, opinions, complaints, predictions, labeling features, identifying correlations among the various data and scores, etc. Feedback may refer to the feedback from customers and/or to feedback used to train the models; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.); and
sending the extracted information without clearly identified intent is sent to the classification logic (¶¶ 36-46 – The various models may be applied in conjunction with one another, including to classify and extract data. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40)).
[Claim 17] A non-transitory computer-readable storage medium storing instructions which, when executed by a hardware controller operating a text analysis applications cause the hardware controller (¶¶ 11-12, 88; ¶ 120 – AI sentiment analysis platform) to perform operations comprising:
collecting text data associated with business target features including social media posts (¶¶ 15-20 – Indications of user sentiment toward companies may be gathered; ¶ 12 – “As shown in FIG. 1A, example implementation 100 may include a client device, comment sources, and a sentiment analysis platform. Assume that a user utilizes the client device to access one or more applications provided by the comment sources. In some implementations, the comment sources may include sources that provide social media applications, blog applications, chat room applications, message board applications, ratings system applications, and/or the like. As further shown in FIG. 1A, the user may utilize the client device to provide complaint information, about an entity (e.g., company A), to the one or more applications provided by the comment sources. For example, the complaint information may include a complaint indicating that the user is switching from company A to company B, a complaint indicating a negative statement about company A, a complaint indicating invalidation of a product or service of company A, and/or the like.”; ¶ 120 – “Some implementations described herein may provide a sentiment analysis platform that utilizes artificial intelligence to make a prediction about an entity based on user sentiment and transaction history. For example, the sentiment analysis platform may consider sentiments of users who have opinions, complaints, and predictions about the entity, and transactions conducted by users with the entities in order to predict a future stock price of the entity. The sentiment analysis platform may receive the opinions, the complaints, and the predictions of the users, about the entity, from social media sources, and may receive transaction information associated with the users and the entity from financial institutions. The sentiment analysis platform may determine correlations between the transaction information and the opinions, the complaints, and the predictions, in order to apply weights to the opinions, the complaints, and the predictions. The sentiment analysis platform may generate a prediction about the future stock price of the entity based on the opinions, the complaints, the predictions, the transaction information, and the correlations between the transaction information and the opinions, the complaints, and the predictions.” Comments made on/via social media and/or in association with a social media source are examples of social media posts.);
extracting information including metadata, actions, and entities with associated connections using tools that identify a role or a set of features for each word (¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”);
identifying intent actions based on the extracted information that includes related entities by aggregating a general idea or action toward an object (¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶ 120 – AI sentiment analysis platform.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”);
assigning labels using a trained classification model stored in non-transitory memory (¶ 36 – “In some implementations, the artificial intelligence techniques may include a sentiment analysis model that utilizes multiple artificial analysis techniques. In some implementations, the sentiment analysis model may include a model that uses natural language processing, text analysis, and machine learning to systematically identify, extract, quantify, and study affective states and subjective information.”; ¶¶ 59-60 – “[0059] As shown in FIG. 1F, and by reference number 155, the sentiment analysis platform may utilize the transaction information to determine correlations with the interim scores (e.g., the necessity score, the abstract score, the ethics score, the industry score, the demands for service score, the supply for service score, the vendor score, the innovation score, the adaptability score, the execution score, and/or the like). In some implementations, the sentiment analysis platform may utilize a correlation clustering method, a Pearson's product-moment coefficient method, an Anscombe's quartet method, a Spearman's rank-correlation coefficient method, and/or the like in order to determine the correlations between the transaction information and the interim scores. [0060] A clustering method may include partitioning data points into groups based on their similarity, and the correlation clustering method may include clustering a set of objects into an optimum number of clusters without specifying that number in advance. Given a collection of paired (x, y) variables, the Pearson's product-moment coefficient method produces a value, between −1 and +1, that quantifies a strength of dependence between the variables x and y. A value of +1 means that all of the (x, y) points lie exactly on a line with positive slope, a value of −1 means that all of the points lie exactly on a line with negative slope, and a value of 0 means that there is no relationship between the two variables. The Anscombe's quartet method utilizes four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. The Spearman's rank-correlation coefficient method provides a nonparametric measure of rank correlation (e.g., a statistical dependence between a ranking of two variables), and assesses how well a relationship between two variables can be described using a monotonic function.“; ¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶ 88 – “Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 320 executing software instructions stored by a non-transitory computer-readable medium, such as memory 330 and/or storage component 340. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.”);
computing intent scores using a trained scoring model stored in non-transitory memory (¶ 36 – “In some implementations, the artificial intelligence techniques may include a sentiment analysis model that utilizes multiple artificial analysis techniques. In some implementations, the sentiment analysis model may include a model that uses natural language processing, text analysis, and machine learning to systematically identify, extract, quantify, and study affective states and subjective information.”; ¶¶ 59-60 – “[0059] As shown in FIG. 1F, and by reference number 155, the sentiment analysis platform may utilize the transaction information to determine correlations with the interim scores (e.g., the necessity score, the abstract score, the ethics score, the industry score, the demands for service score, the supply for service score, the vendor score, the innovation score, the adaptability score, the execution score, and/or the like). In some implementations, the sentiment analysis platform may utilize a correlation clustering method, a Pearson's product-moment coefficient method, an Anscombe's quartet method, a Spearman's rank-correlation coefficient method, and/or the like in order to determine the correlations between the transaction information and the interim scores. [0060] A clustering method may include partitioning data points into groups based on their similarity, and the correlation clustering method may include clustering a set of objects into an optimum number of clusters without specifying that number in advance. Given a collection of paired (x, y) variables, the Pearson's product-moment coefficient method produces a value, between −1 and +1, that quantifies a strength of dependence between the variables x and y. A value of +1 means that all of the (x, y) points lie exactly on a line with positive slope, a value of −1 means that all of the points lie exactly on a line with negative slope, and a value of 0 means that there is no relationship between the two variables. The Anscombe's quartet method utilizes four datasets that have nearly identical simple descriptive statistics, yet appear very different when graphed. The Spearman's rank-correlation coefficient method provides a nonparametric measure of rank correlation (e.g., a statistical dependence between a ranking of two variables), and assesses how well a relationship between two variables can be described using a monotonic function.“; ¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶ 88 – “Device 300 may perform one or more processes described herein. Device 300 may perform these processes based on processor 320 executing software instructions stored by a non-transitory computer-readable medium, such as memory 330 and/or storage component 340. A computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.”);
filtering and recognizing related input data based on intent criteria using the extracted information (¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶ 42 – “In some implementations, the feature selection method may select subsets of relevant features (e.g., variables or predictors), from the historical information, the complaint information, the opinion information, and the prediction information, for use in the sentiment analysis model. In some implementations, the feature selection method may include a wrapper method (e.g., that uses a predictive model to score feature subsets), a filter method (e.g., that uses a proxy measure instead of an error rate to score a feature subset), an embedded method (e.g., that performs feature selection as part of construction of the sentiment analysis mode), and/or the like.”);
providing aggregated data about each business target feature as a feedback regarding the intent (¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski implements its sentiment analysis using machine learning and/or natural language processing (¶ 36), which means that text data and related scoring data will be classified (i.e., labeled) and trained; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) Identifying noun phrases and verb phrases is an example of identifying syntactic roles.; The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.; ¶¶ 36, 39, 92, 113-118 – Information related to an intent is extracted and identified.; ¶¶ 38-40 – Text, i.e., metadata, is classified to identify entities and sentiment based on opinion information and in light of the different domains and respective set of features for different domains.; ¶¶ 15-20 – Indications of user sentiment toward companies may be gathered. This demonstrates entity relationships.; ¶ 46 – “In some implementations, the sentiment analysis platform may utilize a maximum entropy classifier model. A maximum entropy classifier model may convert labeled feature sets to vectors using encoding. The encoded vector may then be used to calculate weights for each feature, which may then be combined to determine a most likely label for a feature set.”; ¶ 52 – “In some implementations, the industry score calculated by the sentiment analysis platform may provide an indication of aggregate opinion information about an industry in which company A operates, an indication of whether the industry is dying (e.g., based on previously-identified industries that were subject to automation), and/or the like.”; ¶ 54 – “In some implementations, the supply for services score calculated by the sentiment analysis platform may provide an indication of aggregate complaint information associated with product stocking by company A, product delivery time by company A, product quality issues of company A, and/or the like, which may indicate supply problems for company A, and/or the like.”; ¶ 55 – “In some implementations, the vendor score calculated by the sentiment analysis platform may provide an indication of qualities associated with third party vendors, partners, contractors, and/or the like used by company A, and/or the like.”).
While Jezewski strongly suggests that the disclosed operations may be performed substantially simultaneously (i.e., in parallel) since the various models may be applied in conjunction with one another, including to classify and extract data (as seen in ¶¶ 36-46 of Jezewski), Jezewski does not explicitly disclose:
applying the collected text data in parallel to both a classification logic and an information extraction logic;
supplying outputs to a feedback mechanism;
using, by the feedback mechanism, outputs from the intent identification and scoring operations with a weighted balance between the outputs of the intent identification and scoring operations.
Biessmann verifies item attributes in an artificial intelligence environment and explains how the extraction and classification may be performed in parallel, as seen in the following excerpt:
The classifier training module 150, the feature extractor module 152, and/or the classifier module 154 can operate in parallel and for multiple users at the same time. For example, the verification of attributes listed in an item description may be requested from unique user devices 102 for different listings and the components of the attribute verification system 104 can verify the listed attributes simultaneously or nearly simultaneously for the unique user devices 102 in “real-time,” where “real-time” may be based on the perspective of the user. The generation and verification of attributes may be considered to occur in real time if, for example, the delay is sufficiently short (e.g., less than a few seconds) such that the user typically would not notice a processing delay. Real-time may also be based on the following: the attribute verification system 104 can verify listed attributes for a first user at the same time or at nearly the same time (e.g., within a couple seconds) as a verification of listed attributes is performed for a second user; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously as instantaneously as possible, limited by processing resources, available memory, network bandwidth conditions, and/or the like; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously based on a time it takes the hardware components of the attribute verification system 104 to process data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously immediately as data is received (instead of storing, buffering, caching, or persisting data as it is received and processing the data later on); the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by transforming data without intentional delay, given the processing limitations of the attribute verification system 104 and other systems, like the user devices 102, and the time required to accurately receive and/or transmit the data; the attribute verification system 104 can verify listed attributes simultaneously or nearly simultaneously by processing or transforming data fast enough to keep up with an input data stream; etc. (Biessmann: col. 12: 13-51)
Furthermore, Biessmann uses feedback to update the training, thereby confirming which suggested attributes are correct and incorrect (Biessmann: col 3: 5-24; col. 5: 41-58; col. 9: 26 – col. 10: 3; col. 14: 7-14).
While Biessmann does not explicitly link the classification and extraction to an environment in which an audience sentiment is evaluated, Huang sheds some additional light on the benefits of performing operations related to extraction (like sentiment analysis) simultaneously with classification in the area of user sentiment analysis, as described in the following excerpts of Huang:
[0002] Sentiment and topic analysis have a wide application in business marketing and customer care applications to assist in evaluating and understanding brand perception and customer requirements based on, for example, data gathered from millions of online posts such as social media, forums, and blogs. For example, when promoting a new policy/product, a company may monitor electronically posted customer comments regarding a particular policy/product so that the company can respond properly and address criticisms and issues in a timely manner. Hence, online monitoring of current sentiment trend and topics related to, for example, a preset product and brand name is important for modern marketing…
[0011] The aforementioned aspects and other objectives and advantages can now be achieved as described herein. Methods, systems and processor-readable media for simultaneous sentiment analysis and topic classification with multiple labels are disclosed herein. A sentiment and topic associated with a post can be classified at similar time and a result can be incorporated to predict a feature so that a label of two tasks can promote and reinforce each other iteratively. A feature extraction and selection can be performed on both tasks of sentiment and topic classification. A multi-task multi-label classification model can be trained for each task with maximum entropy utilizing multiple labels to ascertain data indicative of and/or derived from an extra label and to manage with class ambiguities. Each task has a separate classification model with different predicting features and they can be trained collectively which allows flexibility in model construction. Such multi-task multi-label (MTML) classification model produces a probabilistic result and the classes can be ranked by the probabilistic result and the post can be classified with the multi-label.
Regarding the weighted balance details, Givental uses hybrid machine learning to detect anomalies using feedback from the outputs of various learning models of an ensemble to assign weights to the various models based on whether or not each respective model has outputted a correct result (Givental: ¶¶ 52, 68).
As discussed above, Jezewski uses machine learning and semantic analysis to glean user sentiment. The Examiner submits that it would have been obvious to one of ordinary skill in the art before the effective filing date of Applicant’s invention to modify Jezewski:
applying the collected text data in parallel to both a classification logic and an information extraction logic;
supplying outputs to a feedback mechanism;
using, by the feedback mechanism, outputs from the intent identification and scoring operations with a weighted balance between the outputs of the intent identification and scoring operations
in order to minimize delay in the sentiment analysis (as suggested in Biessmann: col. 12: 13-51), so that the various dimensions of gathered information may be used to reinforce each other (as suggested in ¶ 11 of Huang), and to improve the accuracy of the models (as suggested in Biessmann: col 3: 5-24; col. 5: 41-58; col. 9: 26 – col. 10: 3; col. 14: 7-14), including improvement in the overall accuracy of the ensemble of learning models (as suggested in ¶ 24 of Givental).
[Claim 18] Jezewski discloses executable instructions that cause the computer to assign at least one label to each data of the collected text data (¶¶ 44-48).
[Claim 19] Jezewski discloses executable instructions that cause the computer to score each labelled data based on training and assign intent based on the at least one assigned label (¶¶ 43-45; ¶¶ 13, 116 – The nature of the opinions, which is related to sentiment analysis, can include an assessment of actions, such as a desire to switch from one company to another or an action to be taken in regard to an entity; Jezewski provides relevant details of the disclosed classification and label processing in ¶¶ 38-47. For example, Jezewski explains, “In some implementations, the sentiment analysis model may utilize natural language processing methods with the historical information, the complaint information, the opinion information, and the prediction information in order to make the historical information, the complaint information, the opinion information, and the prediction information analyzable. For example, the sentiment analysis model may use natural language processing to derive meaning from natural language input stemming from the historical information, the complaint information, the opinion information, and the prediction information. For example, the sentiment analysis model may utilize deep parsing to break sentences down into noun phrases and verb phrases and then determine associated prepositional phrases. In this way, the sentiment analysis model may determine how entities relate to each other and navigate through unstructured text.” (Jezewski: ¶ 37) The classifier may provide additional context for the sentiment analysis, as explained in the following excerpt: “In some implementations, the classification method may categorize the historical information, the complaint information, the opinion information, and the prediction information into different domains (e.g., markets, economy, industry, technology, and/or the like). The classification method may be used since there may be a different set of features for different domains and thus, each domain may have a different classifier. For example, a news article in the technology domain may be positive news for company A but may be negative news for company B, great news about company A may be slightly bad news for company B, who is a competitor (e.g., an vice versa), and/or the like. Thus, if the sentiment analysis model knows competitor information associated with competitors for each entity, then when the sentiment analysis model identifies information that is good (or bad) for an entity, the sentiment analysis model may determine that the information is bad (or good) for the competitors of the entity.” (Jezewski: ¶ 40). The fact that certain news in one domain may be good news and that the same news may be bad news in another domain is an example of scoring (e.g., as “good” or “bad”) based on intent.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SUSANNA M DIAZ whose telephone number is (571)272-6733. The examiner can normally be reached M-F, 8 am-4:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached at (571) 270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SUSANNA M. DIAZ/
Primary Examiner
Art Unit 3625A
Prosecution Insights