DETAILED ACTION
Claims 1-20 of the instant application are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections
Claims 2 and 12 objected to because of the following informalities: the limitation of “the respective user” should read “respective user”.
Appropriate correction is required.
Claim 8 objected to because of the following informalities: the limitation of “the beginning and the end of each set of text data” should read “a beginning and an end of each set of text data”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. More specifically directed to the abstract idea grouping of: mental process.
The independent claim(s) 1 and 11 recite(s):
1. A personalized natural language processing system comprising:
at least one processor, communicatively coupled to non-volatile memory storing a natural language processing (NLP) model personalized for use by multiple users and instructions that, when executed by the processor, cause the processor to:
receive a plurality of sets of text data, each set of text data corresponding to a user of a plurality of users;
append a predetermined user-specific token to each of the plurality of sets of text data;
process the plurality of sets of text data using the NLP model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data; and
output the personalized predictions or classifications.
11. A personalized natural language processing method, comprising:
receiving a plurality of sets of text data, each set of text data corresponding to a user of a plurality of users;
appending predetermined user-specific tokens to each of the plurality of sets of text data;
processing the plurality of sets of text data using a natural language processing (NLP) model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data from each of the plurality of users; and
outputting the personalized predictions or classifications.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving multiple texts (i.e., written on paper) from different users (i.e., other humans);
Adding words/tokens specific to each user of the received multiple texts to a list (e.g., segmentations of text);
Using a predetermined set of rules (i.e., natural language processing / sentiment analysis model) to analyze the segmented data; and
Writing down a classification or determination associated with each user.
This judicial exception is not integrated into a practical application because for example: claim 1 recites “at least one processor” and “non-volatile memory”. As an example, in [0018] of the as filed specification, it is disclosed: “The personalized NLP system 10 comprises a computing device 12 including a processor 14, volatile memory 16, an input/output module 18, and non-volatile memory 24 storing a NLP application 26 including a tokenizer 30, a user specifier 34, and a multi-user personalized NLP model 38.” Therefore, a general-purpose computer or computing device is described and mainly used as an application thereof. Accordingly, these additional elements do not integrate the abstract idea into a practical idea because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is listed as a general computing device as noted. The claim is not patent eligible.
With respect to claims 2 and 12, the claim(s) recite:
2 and 12. The personalized natural language processing system/method of claims 1/11, wherein each set of text data is appended with the predetermined user-specific token that corresponds to the respective user associated with that set of text data.
This reads on a human (e.g., mentally and/or using pen and paper):
Adding words/tokens specific to each user of the received multiple texts to a list (e.g., segmentations of text)
No additional limitations are present.
With respect to claims 3 and 13, the claim(s) recite:
3 and 13. The personalized natural language processing system/method of claims 1/11, wherein each set of text data is tokenized prior to being appended with the predetermined user specific token.
This reads on a human (e.g., mentally and/or using pen and paper):
Segmenting these texts by each user, including words/portions of words
No additional limitations are present.
With respect to claims 4 and 14, the claim(s) recite:
4 and 14. The personalized natural language processing system/method of claims 1/11, wherein
the NLP model is a text classification model; and
the personalized classifications are personalized text classifications for each of the plurality of users.
This reads on a human (e.g., mentally and/or using pen and paper):
wherein the predetermined set of rules used (i.e., natural language processing / sentiment analysis model) to analyze the segmented data is used for text classification; and
the classifications are personalized text classifications for the users.
No additional limitations are present.
With respect to claims 5 and 15, the claim(s) recite:
5 and 15. The personalized natural language processing system/method of claims 1/11, wherein
the NLP model is a text prediction model; and
the personalized classifications are personalized text predictions for each of the plurality of users.
This reads on a human (e.g., mentally and/or using pen and paper):
wherein the predetermined set of rules used (i.e., natural language processing / sentiment analysis model) to analyze the segmented data is used for text prediction; and
the classifications are personalized text predictions for the users.
No additional limitations are present.
With respect to claims 6 and 16, the claim(s) recite:
6 and 16. The personalized natural language processing system/method of claims 1/11, wherein the predetermined user-specific tokens include at least one of consecutive numbers, usernames, random sequences of digits, random sequences of tokens with non-alphanumeric characters, or random sequences of all available tokens in a tokenizer vocabulary.
This reads on a human (e.g., mentally and/or using pen and paper):
wherein tokens specific to each user of the received multiple texts include numbers, usernames, non-alphanumeric characters, etc.
No additional limitations are present.
With respect to claims 7 and 17, the claim(s) recite:
7. The personalized natural language processing system of claim 1, wherein
the processor is configured to train the NLP model using the plurality of sets of text data with the appended predetermined user-specific tokens, and
the training of the NLP model includes minimizing cross-entropy loss for classification.
17. The personalized natural language processing method of claim 11, further comprising training the NLP model using the plurality of sets of text data with the appended predetermined user-specific tokens.
This reads on a human (e.g., mentally and/or using pen and paper):
adding or considering additional steps (i.e., training the NLP model) are performed using the segmented text with the added tokens specific to the users.
No additional limitations are present.
With respect to claims 8 and 18, the claim(s) recite:
8. The personalized natural language processing system of claim 1, wherein the predetermined user-specific tokens are appended to the beginning and the end of each set of text data.
18. The personalized natural language processing method of claim 11, wherein the predetermined user-specific tokens are appended to a beginning and an end of each set of text data.
This reads on a human (e.g., mentally and/or using pen and paper):
Adding tokens specific to each user of the received multiple texts to the segmentations by writing down the words/tokens at the beginning and/or end of the segmented data
No additional limitations are present.
With respect to claim 9, the claim(s) recite:
9. The personalized natural language processing system of claim 1, wherein lengths of the predetermined user-specific tokens do not exceed a predetermined number of tokens.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the length of the predetermined tokens do not exceed a predefined number.
No additional limitations are present.
With respect to claim 10, the claim(s) recite:
10. The personalized natural language processing system of claim 1, wherein the NLP model is a transformer sequence classifier, and user embedding parameters of the predetermined user-specific tokens are tied to embedding parameters of the transformer sequence classifier.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the predetermined set of rules used (i.e., natural language processing / sentiment analysis model) consists of specific rules according to e.g., transformer sequence classifier, sequence-to-sequence, etc.
No additional limitations are present.
With respect to claim 19, the claim(s) recite:
19. The personalized natural language processing method of claim 11, wherein
each of the plurality of sets of text data contain utterances from a corresponding user of the plurality of users,
the outputting includes outputting personalized classifications for each of the plurality of users based upon the utterances of each user, and
the personalized classifications include a plurality of sentiment labels.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein each text data corresponds to utterances from a corresponding user/human;
Using a predetermined set of rules (i.e., natural language processing / sentiment analysis model) to analyze the segmented data and write down a classification or determination associated with each user;
Wherein the classification includes a plurality of sentiment labels (e.g., positive, negative, neutral).
No additional limitations are present.
With respect to claim 20, the claim(s) recite:
20. The personalized natural language processing method of claim 19, wherein the sentiment labels include at least a positive sentiment, a neutral sentiment, and a negative sentiment.
This reads on a human (e.g., mentally and/or using pen and paper):
wherein the predetermined set of rules used (i.e., natural language processing / sentiment analysis model) to analyze the segmented data include labeling them as negative, positive or neutral.
No additional limitations are present.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-18 of U.S. Patent No. US 12182511 B2. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the issued patent anticipate the claims of the instant application.
Please see below for pertinent mappings of the instant application in comparison to the issued patent.
Table 1 shows the overall claim mapping comparing equivalence between claims from instant application and issued patent.
Table 2 show the limitations of the independent claim 1 of the instant application when compared with independent claim of the issued patent, respectively, wherein the underlined portions indicate the main differences between instant application and issued patent.
Table 1: Overall claim mapping comparing Instant Application and Issued Patent.
Instant Application
Issued Patent
US 12182511 B2
1* and 11*
1 and 10*
2 and 12
1 and 10*
3 and 14
1 and 10*
4 and 14
2 and 11
5 and 15
3 and 12
6 and 16
4 and 13
7 and 17
5 and 14
8 and 18
6 and 15
9
7
10
8 and 16
19
10* and 18
20
9 and 17-18
Note: * denotes an independent claim
Table 2: Independent claim mapping (comparing each of the limitations)
Instant Application
Issued Patent
US 12182511 B2
Independent claim 1:
Independent claim 1:
1. A personalized natural language processing system comprising:
1. A personalized natural language processing system comprising:
at least one processor, communicatively coupled to non-volatile memory storing a natural language processing (NLP) model personalized for use by multiple users and instructions that, when executed by the processor, cause the processor to:
at least one processor, communicatively coupled to non-volatile memory storing a natural language processing (NLP) model personalized for use by multiple users and instructions that, when executed by the processor, cause the processor to:
receive a plurality of sets of text data, each set of text data corresponding to a user of a plurality of users;
receive or retrieve a plurality of sets of raw text data from a plurality of users, respectively;
tokenize the plurality of sets of raw text data to generate a plurality of sets of tokenized text data for the plurality of users, respectively, each set of tokenized text data including a sequence of tokens corresponding to the raw text data, the tokens at least identifying distinct words or portions of words in the raw text data;
append a predetermined user-specific token to each of the plurality of sets of text data;
append predetermined user-specific tokens to the plurality of sets of tokenized text data from the plurality of users, respectively, to generate a plurality of user-specific token sets, each predetermined user-specific token corresponding to one user of the plurality of users;
process the plurality of sets of text data using the NLP model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data; and
process the plurality of user-specific token sets using the NLP model in accordance with the appended predetermined user-specific tokens to predict a personalized classification for each of the plurality of user-specific token sets; and
output the personalized predictions or classifications.
output the personalized classifications of the plurality of user-specific token sets, wherein the NLP model is trained using a training data set comprising multiple tuples of training data, each tuple including user identifier data that identifies a particular user, raw text data from the particular user, and ground truth classification data, the trained NLP model being configured to perform personalized text classification and/or text prediction tasks for each user of the multiple users, and during processing of the plurality of user-specific token sets, embeddings are produced for each token in each set of the plurality of user-specific token sets, and attention weights are computed between each token in each user-specific token set using the embeddings for each token.
Note: Main differences between instant application and issued patent are underlined.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6, 8-9, 11-16, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sinkar et al. (US 20220210021 A1) further in view of Li et al. ((2019). Towards Personalized Review Summarization via User-Aware Sequence Network. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6690-6697. https://doi.org/10.1609/aaai.v33i01.33016690).
As to independent claim 1, Sinkar et al. teaches:
1. A personalized natural language processing system (see ¶ [0027]: “…In some embodiments, NLU processing 210 may be customized for a given user, for a given set of users, and/or based on contextual features.”) comprising:
at least one processor, communicatively coupled to non-volatile memory storing a natural language processing (NLP) model personalized for use by multiple users and instructions that, when executed by the processor (see ¶ [0027] citation as in limitation above and further ¶ [0066]: “In some embodiments, the methods may be implemented in one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information). The processing devices may include one or more devices executing some or all of the operations of the methods in response to instructions stored electronically on an electronic storage medium. The processing devices may include one or more devices configured through hardware, firmware, and/or software to be specifically designed for execution of one or more of the operations of the methods.” and ¶ [0082]: “In an operation 804, natural language processing, including sentiment analysis, may be performed to the set of communications…”), cause the processor to:
receive a plurality of sets of text data, each set of text data corresponding to a user of a plurality of users (see ¶ [0019]: “…The communications may include text communications, audio communications, video communications, or other communication formats, or combinations thereof. In some embodiments, the communications may be between one or more users, users and a service, users and an agent of a service (e.g., a human agent, an automated agent, etc.), and the like. NLP subsystem 112 may transform the communications into a format processable by other components of system 100, determine an intention of a user associated with the communications, cause an action to be performed based on the communications, or perform other functions.” and ¶ [0082]: “In an operation 804, natural language processing, including sentiment analysis, may be performed to the set of communications. In some embodiments, each communication in the set of communications may be segmented, and the utterances included within those communications may be parsed. The list of utterances in each communication may then have STT processing 204 performed to generate text data representing each utterance...”);
process the plurality of sets of text data using the NLP model (see ¶ [0026] citation as in limitation above and further ¶ [0082]: “In an operation 804, natural language processing, including sentiment analysis, may be performed to the set of communications. In some embodiments, each communication in the set of communications may be segmented, and the utterances included within those communications may be parsed. The list of utterances in each communication may then have STT processing 204 performed to generate text data representing each utterance. In some embodiments, sentiment analysis processing 212 may be performed to the text data to determine a polarity of each utterance. For example, a determination may be made as to whether an utterance is positive, negative, or neutral. In some embodiments, words, phrases, and other features of the utterances may be analyzed to identify common traits between the utterances which could have caused the communication suppression flag to be assigned to the user account. In some embodiments, operation 804 may be performed by a subsystem that is the same or similar to trigger generation subsystem 118.”); and
output the personalized predictions or classifications (see ¶ [0026 and 0082] citations as in limitations above and further ¶ [0082]: “…In some embodiments, sentiment analysis processing 212 may be performed to the text data to determine a polarity of each utterance. For example, a determination may be made as to whether an utterance is positive, negative, or neutral…”).
However, Sinkar et al. does not explicitly teach, but Li et al. does teach:
append a predetermined user-specific token to each of the plurality of sets of text data (see ¶ 2 of Section Introduction (page 6690): “…This paper addresses personalization issues of review summarization1, which have not been discussed in previous research. Given a review, different users may care about different contents according to their own experiences or thoughts. Figure 1 illustrates the motivation with a hotel review sample. Bob may travel on business and he cares about location and room more than price, while John may travel on a tight budget and care about price more. What’s more, different users have their own writing styles. Alice often summarizes reviews with the words which can explicitly express her emotions, such as “love” or “hate”, while Bob and John don’t do that”, Figure 1: “Alice – Summary: “clean and comfortable rooms, i love!!!””, ¶ 1-2 of Section User-aware Sequence Network (page 6691): “It is obvious that different users may care about different content of a review and have different word-using habits. Therefore we encode user information into encoder and decoder modules to model these different characteristics to perform personalized review summarization. Specifically, we consider user from two views as follows: (1) user embedding (we embed user u as vector u and add u into our models) , (2) user-specific vocabulary memory, which is composed of K most user-specific words {Uk} K k=1 from u’s previous reviews and summaries. After embedding each word in {Uk} K k=1 into vector {Uk} K k=1 using embedding matrix Ev, we can get the user-specific vocabulary memory U for user u. To build {Uk} K k=1, we first merge all reviews and summaries posted by u into a document. Then we compute tf-idf scores for each word appears in the document, and we finally select top-K words for u. Using tf-idf scores means we do not include too general terms that many users commonly use, because they are not helpful for considering u.” and Figure 4 along with ¶ 1 of Section Case Study (pages 6695-6696): “First, although the review describes UserB’s attitudes on room, food, service, and location, the reference only contains room and location. This shows UserB cares these two aspects more. Actually, we observe all reviews posted by UserB. There are 40 reviews with summaries posted by UserB, more than 80% these reviews and summaries contain UserB’s attitude on these two aspects. Existing methods without modeling user information (S2S+Att and PGN) cannot capture UserB’s preference on these two aspects, which results in these methods generate words (such as the “staff”) about service. While our personalized model can mine such preference and only generate words about location and room. Second, the word “comfortable” is hard to generate, because it does not appear in the review. However, we find that it appears in UserB-specific vocabulary. After merging the vocabulary, USN can generate the word accurately.”);
process the plurality of sets of text data using the NLP model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data (see Table 5 (image and caption, page 6695): “Table 5: User-based selective gate visualization of a input review. The important words are selected from the input review, such as “impressed”, “staff”, “bed” and “perfect”. The output summary of our model is “excellent service , comfy be” and the true summary is “excellent service , very comfortable bed”.”, ¶ 2 of Section User-based Selective Gate Visualization (page 6695): “From the gold summary given by UserA, we can find UserA may care about “service” and “bed” more and the important words found by our user-based selective mechanism are “impressed”, “staff”, “bed” and “perfect”, which also reflects UserA’s experience on these two aspects. It shows our personalized model can mine the important information for users.”, and Figure 4 along with ¶ 1 of Section Case Study (pages 6695-6696) citation as in limitation above: “... Second, the word “comfortable” is hard to generate, because it does not appear in the review. However, we find that it appears in UserB-specific vocabulary. After merging the vocabulary, USN can generate the word accurately.”)
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of append a predetermined user-specific token to each of the plurality of sets of text data and process the plurality of sets of text data using the NLP model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
As to independent claim 11, Sinkar et al. further teaches:
11. A personalized natural language processing method (see ¶ [0027]: “…In some embodiments, NLU processing 210 may be customized for a given user, for a given set of users, and/or based on contextual features.”), comprising:
receiving a plurality of sets of text data, each set of text data corresponding to a user of a plurality of users (see ¶ [0019 and 0082] citations as in claim 1, above.);
processing the plurality of sets of text data using a natural language processing (NLP) model (see ¶ [0026 and 0082] citations as in claim 1, above.); and
outputting the personalized predictions or classifications (see ¶ [0026 and 0082] citations as in claim 1, above.).
Li et al. further teaches:
appending predetermined user-specific tokens to each of the plurality of sets of text data (see ¶ 2 of Section Introduction (page 6690), ¶ 1-2 of Section User-aware Sequence Network (page 6691), and Figure 4 along with ¶ 1 of Section Case Study (pages 6695-6696) citations as in claim 1, above.);
processing the plurality of sets of text data using a natural language processing (NLP) model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data from each of the plurality of users (see Table 5 (image and caption, page 6695), ¶ 2 of Section User-based Selective Gate Visualization (page 6695), and Figure 4 along with ¶ 1 of Section Case Study (pages 6695-6696) citations as in claim 1, above.);
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of append a predetermined user-specific token to each of the plurality of sets of text data and process the plurality of sets of text data using the NLP model in accordance with the appended predetermined user-specific tokens to make a personalized prediction or classification for each of the plurality of sets of text data which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claims 2 and 12, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1 and 11, above.
Li et al. further teaches:
2 and 12. The personalized natural language processing system/method of claims 1/11, wherein each set of text data is appended with the predetermined user-specific token that corresponds to the respective user associated with that set of text data (see Table 5 (image and caption, page 6695): “Table 5: User-based selective gate visualization of a input review. The important words are selected from the input review, such as “impressed”, “staff”, “bed” and “perfect”. The output summary of our model is “excellent service , comfy be” and the true summary is “excellent service , very comfortable bed”.”, ¶ 2 of Section User-based Selective Gate Visualization (page 6695): “From the gold summary given by UserA, we can find UserA may care about “service” and “bed” more and the important words found by our user-based selective mechanism are “impressed”, “staff”, “bed” and “perfect”, which also reflects UserA’s experience on these two aspects. It shows our personalized model can mine the important information for users.”, and Figure 4 along with ¶ 1 of Section Case Study (pages 6695-6696) citation as in claim 1 above: “... Second, the word “comfortable” is hard to generate, because it does not appear in the review. However, we find that it appears in UserB-specific vocabulary. After merging the vocabulary, USN can generate the word accurately.”).
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of wherein each set of text data is appended with the predetermined user-specific token that corresponds to the respective user associated with that set of text data which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claims 3 and 13, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1 and 11, above.
Li et al. further teaches:
3 and 13. The personalized natural language processing system/method of claims 1/11, wherein each set of text data is tokenized prior to being appended with the predetermined user specific token (see ¶ 2 of Section Introduction (page 6690), ¶ 1-2 of Section User-aware Sequence Network (page 6691), and Figure 4 along with ¶ 1 of Section Case Study (pages 6695-6696) citations as in claim 1, above and further Table 5: visualization of input review – word-level).
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of wherein each set of text data is tokenized prior to being appended with the predetermined user specific token which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claims 4 and 14, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1 and 11, above.
Sinkar et al. further teaches:
4 and 14. The personalized natural language processing system/method of claims 1/11, wherein the NLP model is a text classification model (see ¶ [0029]: “To determine an emotional state of a user from the text communications, sentiment analysis processing 212 may be configured to classify each utterance of each text communication as being one of a set of possible polarities, such as positive, negative, or neutral.”); and
the personalized classifications are personalized text classifications for each of the plurality of users (see ¶ [0029] citation as in limitation above and further ¶ [0026]: “Each text communication may be input to communication parser 208, which may be configured to parse that text communication's utterances. In some embodiments, communication parser 208 extracts the utterances from the text communication and organizes the utterances into an ordered list of utterances, which can also be referred to as a sequence of utterance. The ordered list may indicate which utterance was temporally first, second, and so on. The ordered list may also include contextual information related to the corresponding text utterance. For example, if communications 202 are between a user of a service and an agent of the service, some utterances may be from the user, whereas other may be from the agent. Communication parser 208 may be configured to track or determine the utterances' producers as the communication progresses and generate the ordered list of utterances with the contextual information.”).
Regarding claim 5 and 15, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1 and 11, above.
Li et al. further teaches:
5 and 15. The personalized natural language processing system/method of claims 1/11, wherein the NLP model is a text prediction model (see ¶ 8 of Introduction: “…we propose a User-aware Decoder to consider different writing styles of users. It incorporates user embedding and user-specific vocabulary memory into word prediction module to generate personalized summaries.”); and
the personalized classifications are personalized text predictions for each of the plurality of users (see ¶ 8 of Introduction citation as in limitation above.).
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of wherein the NLP model is a text prediction model and the personalized classifications are personalized text predictions for each of the plurality of users which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claim 6 and 16, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1 and 11, above.
Li et al. further teaches:
6 and 16. The personalized natural language processing system/method of claims 1/11,
wherein the predetermined user-specific tokens include at least one of consecutive numbers, usernames, random sequences of digits, random sequences of tokens with non-alphanumeric characters, or random sequences of all available tokens in a tokenizer vocabulary (see Figure 1 (Alice: Summary: clean and comfortable rooms, i love [i.e., random sequences of tokens] !!! [i.e., non-alphanumeric characters]) and ¶ of Introduction: “…Figure 1 illustrates the motivation with a hotel review sample. Bob may travel on business and he cares about location and room more than price, while John may travel on a tight budget and care about price more. What’s more, different users have their own writing styles. Alice often summarizes reviews with the words which can explicitly express her emotions, such as “love” or “hate”, while Bob and John don’t do that.”).
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of wherein the predetermined user-specific tokens include at least one of consecutive numbers, usernames, random sequences of digits, random sequences of tokens with non-alphanumeric characters, or random sequences of all available tokens in a tokenizer vocabulary which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claims 8 and 18, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1 and 11, above.
Li et al. further teaches:
8 and 18. The personalized natural language processing system/method of claims 1/11,
wherein the predetermined user-specific tokens are appended to the/a beginning and the/an end of each set of text data (see Figure 1 (Alice: Summary: clean and comfortable rooms, i love !!! [i.e., end]) and ¶ of Introduction: “…Figure 1 illustrates the motivation with a hotel review sample. Bob may travel on business and he cares about location and room more than price, while John may travel on a tight budget and care about price more. What’s more, different users have their own writing styles. Alice often summarizes reviews with the words which can explicitly express her emotions, such as “love” or “hate”, while Bob and John don’t do that.”).
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of wherein the predetermined user-specific tokens are appended to the/a beginning and the/an end of each set of text data which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claim 9, Sinkar et al. in combination with Li et al. teaches the limitations as in claim 1, above.
Li et al. further teaches:
9. The personalized natural language processing system of claim 1,
wherein lengths of the predetermined user-specific tokens do not exceed a predetermined number of tokens (see ¶ Problem Formulation: “Suppose we have a corpus D with m user-review-summary triples, and each triple contains a review x, a summary y and a user u who posts x and summarizes x to y. Review x consists of n words as fx1; x2; :::; xng, where xi 2 Vs and Vs is the source vocabulary. Summary [i.e., comprising the predetermined user-specific tokens] y consists of l ≤ n words as fy1; y2; :::; ylg, where yi 2 Vt and Vt is the target vocabulary. Personalized review summarization aims to generate summary y from review x by attending to u’s characteristics on summarizing reviews.”).
Sinkar et al. and Li et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. to incorporate the teachings of Li et al. of wherein lengths of the predetermined user-specific tokens do not exceed a predetermined number of tokens which provides the benefit of achieving state-of-the-art performance on personalized review summarization (abstract of Li et al.).
Regarding claim 19, Sinkar et al. in combination with Li et al. teaches the limitations as in claim 11, above.
Sinkar et al. further teaches:
19. The personalized natural language processing method of claim 11,
wherein each of the plurality of sets of text data contain utterances from a corresponding user of the plurality of users (see ¶ [0026 and 0082] citations as in claim 11 above and further ¶ [0082]: “In an operation 804, natural language processing, including sentiment analysis, may be performed to the set of communications. In some embodiments, each communication in the set of communications may be segmented, and the utterances included within those communications may be parsed. The list of utterances in each communication may then have STT processing 204 performed to generate text data representing each utterance...”),
the outputting includes outputting personalized classifications for each of the plurality of users based upon the utterances of each user (see ¶ [0026 and 0082] citations as in claim 11 above and further ¶ [0082]: “…In some embodiments, sentiment analysis processing 212 may be performed to the text data to determine a polarity of each utterance. For example, a determination may be made as to whether an utterance is positive, negative, or neutral…”), and
the personalized classifications include a plurality of sentiment labels (see ¶ [0026 and 0082] citations as in claim 11 above and further ¶ [0082]: “…For example, a determination may be made as to whether an utterance is positive, negative, or neutral…”).
Regarding claim 20, Sinkar et al. in combination with Li et al. teaches the limitations as in claim 11, above.
Sinkar et al. further teaches:
20. The personalized natural language processing method of claim 19,
wherein the sentiment labels include at least a positive sentiment, a neutral sentiment, and a negative sentiment (see ¶ [0026 and 0082] citations as in claim 11 above and further ¶ [0082]: “…In some embodiments, sentiment analysis processing 212 may be performed to the text data to determine a polarity of each utterance. For example, a determination may be made as to whether an utterance is positive, negative, or neutral…”).
Claims 7 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sinkar et al. (US 20220210021 A1) further in view of Li et al. ((2019). Towards Personalized Review Summarization via User-Aware Sequence Network. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6690-6697. https://doi.org/10.1609/aaai.v33i01.33016690) as applied to claims 1 and 11, above, and further in view of Wu et al. (US 20230222358 A1).
Regarding claim 7, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 1, above.
However, Sinkar et al. in combination with Li et al. do not explicitly teach, but Wu et al. does teach:
7. The personalized natural language processing system of claim 1,
wherein the processor is configured to train the NLP model using the plurality of sets of text data with the appended predetermined user-specific tokens (see ¶ [0083]: “In implementations, the keyword classification model 618 is initially trained with keyword and non-keywords from historic data of a customer environment. In implementations, when user feedback 606 indicates that the one or more groups of data records provided to the user are not similar, the server 404 splits and transforms the text description of the data records (event description) into tokens using natural language process (NLP) data processing methods. Features are then extracted from the tokens. Using the technology of feature engineering, the server 404 creates character level features such as numbers of specific characters in a token for each single token. These features are used in the keyword classification model 618 to determine whether a word in a data record description is a potential keyword. In embodiments, every time customers provide the server 404 with feedback, the keyword classification model 618 processes the data records at issue to predict which word in the data records description is a potential keyword, and saves the potential keyword in a database with an assigned weight. After multiple feedback processes, the weight for a certain potential keyword will reach a threshold value, at which time the server 404 considers the potential keyword to be an actual keyword. The server 404 then adds the actual keyword to training data used for further training of the keyword classification model 618.”), and
the training of the NLP model includes minimizing cross-entropy loss for classification (see ¶ [0083] citation as in limitation above, above and further ¶ [0112]: “The cross-entropy loss function set forth above can measure the performance of the data classification model during training, where the lower the loss the better the model is. The variable y.sub.i is the label of data, wherein y.sub.i=(y.sub.i1,y.sub.i2, . . . ,y.sub.iM) and y.sub.ic is the c.sup.th component of y.sub.i. The variable p.sub.i stands for predicted probability based on our model, wherein p.sub.i=(p.sub.i1, p.sub.i2, . . . , p.sub.iM) and p.sub.ic is the c.sup.th component of p.sub.i.Math.L.sub.i=−Σ.sub.c=1.sup.My.sub.ic log (p.sub.ic) and M represents the number of categories. Accordingly, embodiments of the invention optimize and update parameters of the data classification model through minimizing the loss function.”).
Sinkar et al. and Li et al. and Wu et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. in combination with Li et al. to incorporate the teachings of Wu et al. of wherein the processor is configured to train the NLP model using the plurality of sets of text data with the appended predetermined user-specific tokens, and the training of the NLP model includes minimizing cross-entropy loss for classification which provides the benefit of optimizing and updating parameters of the data classification model ([0112] of Wu et al.).
Regarding claim 17, Sinkar et al. in combination with Li et al. teaches the limitations as in claims 11, above.
However, Sinkar et al. in combination with Li et al. do not explicitly teach, but Wu et al. does teach:
17. The personalized natural language processing method of claim 11, further comprising training the NLP model using the plurality of sets of text data with the appended predetermined user-specific tokens (see ¶ [0083] citation as in claim 7 above: “In implementations, the keyword classification model 618 is initially trained with keyword and non-keywords from historic data of a customer environment. In implementations, when user feedback 606 indicates that the one or more groups of data records provided to the user are not similar, the server 404 splits and transforms the text description of the data records (event description) into tokens using natural language process (NLP) data processing methods. Features are then extracted from the tokens. Using the technology of feature engineering, the server 404 creates character level features such as numbers of specific characters in a token for each single token. These features are used in the keyword classification model 618 to determine whether a word in a data record description is a potential keyword. In embodiments, every time customers provide the server 404 with feedback, the keyword classification model 618 processes the data records at issue to predict which word in the data records description is a potential keyword, and saves the potential keyword in a database with an assigned weight. After multiple feedback processes, the weight for a certain potential keyword will reach a threshold value, at which time the server 404 considers the potential keyword to be an actual keyword. The server 404 then adds the actual keyword to training data used for further training of the keyword classification model 618.”).
Sinkar et al. and Li et al. and Wu et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. in combination with Li et al. to incorporate the teachings of Wu et al. of training the NLP model using the plurality of sets of text data with the appended predetermined user-specific tokens which provides the benefit of optimizing and updating parameters of the data classification model ([0112] of Wu et al.).
Claim 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sinkar et al. (US 20220210021 A1) further in view of Li et al. ((2019). Towards Personalized Review Summarization via User-Aware Sequence Network. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6690-6697. https://doi.org/10.1609/aaai.v33i01.33016690) as applied to claims 1 and 11, above, and further in view of Zhong et al. ("UserAdapter: Few-shot user learning in sentiment analysis." Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021. https://aclanthology.org/2021.findings-acl.129.pdf).
Regarding claim 10, Sinkar et al. in combination with Li et al. teaches the limitations as in claim 1, above.
However, Sinkar et al. in combination with Li et al. do not explicitly teach, but Zhong et al. does teach:
10. The personalized natural language processing system of claim 1,
wherein the NLP model is a transformer sequence classifier, and user embedding parameters of the predetermined user-specific tokens are tied to embedding parameters of the transformer sequence classifier (see ¶ 3.1 Model: “Specifically, UserAdapter adds a trainable user-specific vector u
θ
∈
Rd for each user, where d denotes its dimension. For each input x, we prepend a trainable user-specific vector 110 to the input embeddings E = Embeddings(x), which is taken as the input of a Transformer-based encoder. Then we produce the last hidden vector H of the user-aware sequential vectors: H= Transformer
∅
([u
θ
; E) (1) where [;] denotes concatenation. The final hidden vector H is taken for classification: p(x) = classifier
∅
(H) (2) where classifier is two linear layers followed by a softmax layer and p(x) is the predicted score for classes. The parameters
∅
include the parameters of the Transformer and the classifier. During the few-shot learning stage, dominant parameters
∅
are fixed and only user-specific parameters
θ
are learned.”).
Sinkar et al. and Li et al. and Zhong et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Sinkar et al. in combination with Li et al. to incorporate the teachings of Zhong et al. of wherein the NLP model is a transformer sequence classifier, and user embedding parameters of the predetermined user-specific tokens are tied to embedding parameters of the transformer sequence classifier which provides the benefit of only optimizing and storing the user-specific vector for each new user (3.1 Model of Zhong et al.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Keisha Y Castillo-Torres whose telephone number is (571)272-3975. The examiner can normally be reached Monday - Friday, 9:00 am - 4:00 pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Keisha Y. Castillo-Torres
Examiner
Art Unit 2659
/Keisha Y. Castillo-Torres/Examiner, Art Unit 2659
Prosecution Insights