Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on July 20, 2026, in which claims 1-18 and 25-30 have been amended. No claims have been newly added or cancelled.
Claims 1-18 and 25-30 are currently pending.
Response to Arguments
With regards to the rejections of claims 1-18 and 25-30 under 35 U.S.C. 103 as unpatentable over Huffman et al. (US Publication No. 2022/0115033), in view of Hada et al. ("Codewords Detection in Microblogs Focusing on Differences in Word Use Between Two Corpora"), further in view of Manchanda et al. (US Publication No. 2021/0182709), Applicant’s arguments that the claims as amended overcome the rejections are persuasive, however the point is moot in view of a new grounds of rejection, necessitated by Applicant’s amendments to the claims, as detailed below.
With regards to the rejections of claims 1-18 and 25-30 under 35 U.S.C. 101 as directed towards abstract ideas, Applicant’s arguments that the claims as amended overcome the rejections are persuasive. As amended, the independent claims recite techniques for transforming unlabeled user feedback on performance issues into labeled feedback to train a neural network, which incorporates any recited abstract ideas into a practical solution at Step 2A, Prong Two of the Subject Matter Eligibility Test.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11, 12, and 15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 11,
Claim 11 recites the limitation wherein the one or more processors are further to process the unannotated user feedback using the classifier network to identify a classification for the unannotated user feedback. The term “the unannotated user feedback” does not have antecedent basis, as no “unannotated user feedback” is earlier recited in claim 11, or in any parent claims. Thus, the scope of the claim is indefinite. For examination purposes, the limitation will be interpreted as reading “wherein the one or more processors are further to process unannotated user feedback using the classifier network to identify a classification for the unannotated user feedback”.
In reference to dependent claim 12, claim 12 does not cure the deficiencies noted in the rejection of claim 11. Therefore, claim 12 is rejected under the same rationale as claim 11.
Regarding claim 15,
Claim 15 recites the limitation wherein to identify the one performance issue, the one or more circuits are to identify one of the respective embeddings having a highest similarity score, above at least a minimum similarity threshold, with respect to the embedding. The term “the one or more circuits” does not have antecedent basis, as no “one or more circuits” is earlier recited in claim 15, or in any parent claims. Thus, the scope of the claim is indefinite. For examination purposes, the limitation will be interpreted as reading “wherein to identify the one performance issue, one or more circuits are to identify one of the respective embeddings having a highest similarity score, above at least a minimum similarity threshold, with respect to the embedding”.
Prior Art
The following references are used for prior art claim rejections:
McIlroy et al. “Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews”, hereinafter McIlroy
Deng et al. “Semi-Supervised Learning Approach to Discover Enterprise User Insights from Feedback and Support”, hereinafter Deng
Wang et al. “Weak Supervision for Fake News Detection via Reinforcement Learning”, hereinafter Wang
Cavallari (U.S. Patent Application Publication No. 2022/0075961), hereinafter Cavallari
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-18 and 25-30 are rejected under 35 U.S.C. 103 as being unpatentable over McIlroy, in view of Deng, further in view of Wang, further in view of Cavallari.
Regarding claim 1,
McIlroy teaches:
one or more circuits to use one or more [neural networks] to generate information about a computer program based, at least in part on: ((McIlroy Pg. 2) “Such mobile app user reviews (from this point forward, we refer to the description section of a mobile app user review as simply a user review) contain valuable information…developers receive timely feedback about issues related to their apps e.g. bugs, feature requests, and any other issue”, (McIlroy Pg. 9) “we conclude that the user reviews of the Google Play Store and Apple App Store contain a substantial amount of multi-labelled data. Hence, we wish to explore how well we can automatically label user reviews using machine learning models that are tailored specifically to multi-labelled data”, McIlroy does not teach the use of neural networks)
generating respective embeddings ((McIlroy Pg. 11) “Frequency Building: We transform the text of the user reviews into an array of word frequencies. The word frequency format is readable by machine learning algorithms. We use the String To Word Vector filter, available in MEKA…The MEKA filter forms a M×N matrix of M user reviews and N word attributes. MiNj represents the frequency of a word”, Applicant’s specification discloses [0049] “In at least one embodiment, these embeddings can take form of machine-readable vectors that convert words and phrases into numerical values”, therefore arrays/vectors representing user feedback corresponds to embeddings) corresponding to different instances of performance issues of the computer program (McIlroy Pg. 7, Table 4 lists several issues, including several types of performance issues such as crashing, network problem, resource heavy, etc.) based, at least in part, on different respective samples of labeled ((McIlroy Pg. 10) “The input to the data preparation step is the manually-labelled data from RQ1. The output is a list of processed user reviews”) feedback describing the different instances of the performance issues; ((McIlroy Pg. 7) “Our manual labelling process identified issue types from app reviews via manual inspection”)
PNG
media_image1.png
366
592
media_image1.png
Greyscale
obtaining an instance of unlabeled user feedback for the computer program; ((McIlroy Pg. 20) “there are 210, 89, and 237 one-star and two-star user reviews for the ‘AccuWeather’, ‘WeatherBug’, and ‘The Weather Channel’ apps respectively. We run our multi-labelling approach on the reviews of these selected apps”, collected reviews that are not previously labelled correspond to instances of unlabeled user feedback)
comparing an embedding ((McIlroy Pg. 11) “Frequency Building: We transform the text of the user reviews into an array of word frequencies”) of the instance of unlabeled user feedback… ((McIlroy Pg. 20) “We collected 6,532 user reviews from three apps from the Google Play Store in the weather category. We preprocess and retain only one-star and two-star user reviews for a total of 536”) …to identify [one] performance issue of the different instances of performance issues of the computer program; (McIlroy Pg. 20, Fig. 6 shows identification of instances of performance issues for three computer programs in unlabeled user feedback, McIlroy teaches multi-labelling of issues in reviews rather than identification of singular issues)
PNG
media_image2.png
266
597
media_image2.png
Greyscale
Deng teaches the following further limitations that McIlroy does not teach:
one or more circuits to use one or more neural networks ((Deng Pg. 3) “BERT is essentially a Transformer Encoder stack which consists of multiple encoders of self-attention layer and feed-forward neural network layer”) to generate information about a computer program based, at least in part on: ((Deng Pg. 5) “Take the feedback for Microsoft Outlook as an example, we web-crawl from the customer support site to retrieve a domain knowledge library of the potential issues in all perspectives: Calendar, manage and organize, email, customize, contacts, tasks and so on. Each issue is linked to some self-help content. For instance, we have feedback ‘I want to know how to convert a meeting request to an appointment’, ‘Calendar’ is predicted as one of the main topics from the BERT-based Multiclassification model”, Microsoft Outlook is a computer program)
At the time of filing, one of ordinary skill in the art would have motivation to combine McIlroy and Deng by taking the method for generating information about computer programs using machine learning models, based on generating embeddings of issues, including several types of performance issues, from labelled feedback, and obtaining unlabeled feedback to compare and identify performance issues in, taught by McIlroy, and using at least one neural network to generate the information on the computer program, taught by Deng, as simple substitution of the other machine learning models used in McIlroy for the neural network used in Deng would predictably also generate information on a computer program based on input data. Such a combination would be obvious.
Wang teaches the following further limitations that neither McIlroy nor Deng teach:
adding the instance of unlabeled user feedback to the respective sample of labeled feedback of the one [performance] issue; ((Wang Pg. 2) “given a small set of labeled fake news samples together with users’ feedback towards these news articles, we can train an annotator based on the feedback, which can then be used to automatically assign weak labels for those unlabeled news articles simply based on the user feedback they received”, McIlroy and Deng but not Wang teach performance issues)
and updating the one or more neural networks using the respective samples of labeled feedback, including the added instance of unlabeled feedback, ((Wang Pg. 4) “The objective of the data selector is to automatically select high-quality samples from those with weak labels obtained from the annotator”, (Wang Pg. 5) “the goal of the action is to retain the samples that can bring improvement to fake news detection…Since the designed data selector can choose some high-quality samples from {x(k)1, x(k)2, ..., x(k)B}, the fake news detection model will be retrained using the retained data samples”) to predict a [performance] issue to include in the information generated using the one or more neural networks ((Wang Pg. 8) “The fake news detector then predicts the label of all the news articles by a model trained on the enhanced training set generated by both annotator and reinforced selector”, McIlroy and Deng but not Wang teach performance issues)
At the time of filing, one of ordinary skill in the art would have motivation to combine McIlroy, Deng, and Wang by taking the method for generating information about computer programs using a neural network, based on generating embeddings of performance issues from labelled feedback, and obtaining unlabeled feedback to compare and identify performance issues in, jointly taught by McIlroy and Deng, and subsequently adding the unlabeled user feedback as labeled user feedback and updating the neural network with the expanded labeled feedback, taught by Wang, as it is known in the art that training machine learning models, such as neural networks, on larger and more diverse datasets causes the models to become more accurate, and additionally manual labeling of data samples is expensive and time-consuming, therefore means to automatically label data and update a neural network using the data provides the predictable benefit of increasing the accuracy of the neural network at a relatively low cost. Such a combination would be obvious.
Cavallari teaches the following further limitations that neither McIlroy, nor Deng, nor Wang explicitly teach:
A processor, comprising: ((Cavallari [0064]) “Computing device 910 may be used to implement various portions of this disclosure…computing device 910 includes processing unit 950”)
comparing an embedding of the instance of unlabeled user feedback ((Cavallari [0012]) “Unlabeled content may include any of various forms of user-generated content for which relevant labels are not yet known or have not yet been generated”) with the respective embeddings corresponding to the different instances [of performance issues of the computer program] to identify one [performance issue] of the different instances [of performance issues of the computer program;] ((Cavallari [0014]) “Once unlabeled content has been embedded into vector form, this vector is directly compared to an index of labeled vectors generated from a database of labeled content for propagating a label”, (Cavallari [0020]) “assignment module 140 may determine a matching vector in set 132 that is the most similar to unlabeled vector 122 and assign the label of this matching vector to vector 122”, McIlroy and Deng but not Cavallari teach user feedback on performance issues of a computer program)
At the time of filing, one of ordinary skill in the art would have motivation to combine McIlroy, Deng, Wang, and Cavallari by taking the method for generating information about computer programs using a neural network, based on generating embeddings of performance issues from labelled feedback, obtaining unlabeled feedback to compare and identify performance issues in, and labeling the feedback to use it to update a neural network, jointly taught by McIlroy, Deng, and Wang, and identifying a classification in the unlabeled feedback via comparison of an embedding of the unlabeled feedback with embeddings from labeled feedback, taught by Cavallari, as Cavallari teaches: (Cavallari [0021]) “The disclosed information retrieval techniques automatically infer labels for unlabeled content based on the semantic meaning of labeled content by comparing labeled and unlabeled content in the vector space. This may advantageously provide a greater amount of labeled content which may be used for training machine learning classifiers” and (Cavallari [0022]) “Using a semantic-aware vector space, the disclosed IR system is able to more accurately label user-generated content relative to traditional techniques”, that is, comparison of embeddings of semantic meaning of user content provides more accurate automatic labels, which then can be used to train machine learning models. Such a combination would be obvious.
Regarding claim 2,
McIlroy, Deng, Wang, and Cavallari jointly teach The processor of claim 1, wherein the comparing the embedding of the unlabeled user feedback comprises:
Deng further teaches:
calculating similarity scores between the embedding and respective embeddings corresponding to different instances of performance issues ((Deng Pg. 5) “In the second stage, the cosine similarity score between the Sentence-BERT embedding vector of this clustered sub-topic and the Sentence-BERT embedding vector of each issue in the domain knowledge library will be computed”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the processor jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 2, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 3,
McIlroy, Deng, Wang, and Cavallari jointly teach The processor of claim 2,
Deng further teaches:
wherein to identify the one performance issue, the one or more circuits are to identify one of the respective embeddings having a highest similarity score, above at least a minimum similarity threshold, with respect to the embedding ((Deng Pg. 5) “In the second stage, the cosine similarity score between the Sentence-BERT embedding vector of this clustered sub-topic and the Sentence-BERT embedding vector of each issue in the domain knowledge library will be computed. The top 10 issue candidates with the highest score above some threshold are generated”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the processor jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 3, claim 2. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 4,
McIlroy, Deng, Wang, and Cavallari jointly teach The processor of claim 1,
Deng further teaches:
wherein the one or more neural networks include a classifier network trained to identify one of the different instances of performance issues (Deng Pg. 4, Fig. 4 shows sentences indicating issues with email rankings and email search) according to input feedback ((Deng Pg. 3) “The first component – BERT-based multiclassification model aims at predicting the main topics given the input feedback among the hundreds of targeted topic classes”)
PNG
media_image3.png
577
455
media_image3.png
Greyscale
At the time of filing, one of ordinary skill in the art would have motivation to combine the processor jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 4, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 5,
McIlroy, Deng, Wang, and Cavallari jointly teach The processor of claim 4,
Deng further teaches:
wherein the information about the computer program ((Deng Pgs. 2-3) “Most cloud-based services and mobile applications have feedback from different channels, such as store reviews including Windows store reviews, Mac store reviews, iTunes store reviews and Google Play reviews, customer problem descriptions, help content and product feedback, product sentiment including NPS (Net Promoter Score) and surveys”) is based at least in part upon a classification of the classifier ((Deng Pg. 3) “The first component – BERT-based multiclassification model aims at predicting the main topics given the input feedback among the hundreds of targeted topic classes”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the processor jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 5, claim 4. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 6,
McIlroy, Deng, Wang, and Cavallari jointly teach The processor of claim 5,
Wang further teaches:
wherein the updating updates the classifier ((Wang Pg. 5) “Since the designed data selector can choose some high-quality samples from {x(k)1, x(k)2, ..., x(k)B}, the fake news detection model will be retrained using the retained data samples”) network ((Wang Pg. 4) “Fake news detection model is a neural network, which consists of a textual feature extractor and a fully-connected layer, namely Fake-fc, with corresponding activation functions”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the processor jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 6, claim 5. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claims 7-10,
Claims 7-10 recite a system comprising at least one processor for performing the function of the processor of claims 1-4, respectively. Specifically, claim 7 recites: A system comprising: one or more processors to [perform the function of the processor of claim 1]. Cavallari teaches (Cavallari [0064]) “Computing device 910 may be used to implement various portions of this disclosure…computing device 910 includes processing unit 950”.
All other limitations in claims 7-10 are substantially the same as those in claims 1-4, or broader, therefore the same rationale for rejection applies.
Regarding claim 11,
McIlroy, Deng, Wang, and Cavallari jointly teach The system of claim 10,
McIlroy further teaches:
wherein the one or more processors are further to process the unannotated user feedback [using the classifier network] to identify a classification for the unannotated user feedback ((McIlroy Pg. 20) “there are 210, 89, and 237 one-star and two-star user reviews for the ‘AccuWeather’, ‘WeatherBug’, and ‘The Weather Channel’ apps respectively. We run our multi-labelling approach on the reviews of these selected apps”, collected reviews that are not previously labelled correspond to unannotated user feedback, Deng and Wang but not McIlroy teach a classifier network)
Deng further teaches:
wherein the information about the computer program ((Deng Pgs. 2-3) “Most cloud-based services and mobile applications have feedback from different channels, such as store reviews including Windows store reviews, Mac store reviews, iTunes store reviews and Google Play reviews, customer problem descriptions, help content and product feedback, product sentiment including NPS (Net Promoter Score) and surveys”) is based at least in part upon a classification of the classifier ((Deng Pg. 3) “The first component – BERT-based multiclassification model aims at predicting the main topics given the input feedback among the hundreds of targeted topic classes”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the system jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 11, claim 10. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 12,
McIlroy, Deng, Wang, and Cavallari jointly teach The system of claim 11,
Wang further teaches:
wherein the updating updates the classifier ((Wang Pg. 5) “Since the designed data selector can choose some high-quality samples from {x(k)1, x(k)2, ..., x(k)B}, the fake news detection model will be retrained using the retained data samples”) network ((Wang Pg. 4) “Fake news detection model is a neural network, which consists of a textual feature extractor and a fully-connected layer, namely Fake-fc, with corresponding activation functions”)
At the time of filing, one of ordinary skill in the art would have motivation to combine the processor jointly taught by McIlroy, Deng, Wang, and Cavallari for the parent claim of claim 12, claim 11. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claims 13-18,
Claims 13-18 recite a method for performing the function of the processor of claims 1-6, respectively. All other limitations in claims 13-18 are substantially the same as those in claims 1-6, or broader, therefore the same rationale for rejection applies.
Regarding claims 25-30,
Claims 25-30 recite an information generation system comprising at least one processor for performing the function of the processor of claims 1-6, respectively. Specifically, claim 25 recites: An information generation system, comprising: one or more processors to [perform the function of the processor of claim 1]. Cavallari teaches (Cavallari [0064]) “Computing device 910 may be used to implement various portions of this disclosure…computing device 910 includes processing unit 950”.
All other limitations in claims 25-30 are substantially the same as those in claims 1-6, or broader, therefore the same rationale for rejection applies.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mekala et al. “Classifying User Requirements from Online Feedback in Small Dataset Environments using Deep Learning” teaches use of deep learning methods to classify the usefulness of sentences as feedback in user reviews for apps.
Zheng et al. “iFeedback: Exploiting User Feedback for Real-time Issue Detection in Large-Scale Online Service Systems” teaches a method for real-time issue detection in online service systems using user feedback.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTOR A NAULT whose telephone number is (703) 756-5745. The examiner can normally be reached M - F, 12 - 8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.A.N./Examiner, Art Unit 2124
/MIRANDA M HUANG/ Supervisory Patent Examiner, Art Unit 2124
