Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Note: The claims are not directed towards patent ineligible subject matter under 35 U.S.C. 101
Step 1: IS THE CLAIM DIRECTED TO A PROCESS, MACHINE, MANUFACTURE OR COMPOSITION OF MATTER?
Yes
Step 2A.1: IS THE CLAIM DIRECTED TO A LAW OF NATURE, A NATURAL PHENOMENON (PRODUCT OF NATURE) OR AN ABSTRACT IDEA?
No
Step 2A.2: DOES THE CLAIM RECITE ADDITIONAL ELEMENTS THAT INTEGRATE THE JUDICIAL EXCEPTION INTO A PRACTICAL APPLICATION?
Yes, if the claims are alternatively construed to be abstract in step 2A1. The claims seek to improve generation of prompts accuracy with minimal interactions supported by the specification, and reflected by the claims e.g. in spec: 0051, 0063 In other words, the claims enable the invention to improve at least the generation of prompts by refining them through multiple rounds prior to displaying them to the user resulting in a minimal number of interactions.
Supported by the following:
In Finjan Inc. v. Blue Coat Systems, Inc., 879 F.3d 1299, 125 USPQ2d 1282 (Fed. Cir. 2018), the claimed invention was a method of virus scanning that scans an application program, generates a security profile identifying any potentially suspicious code in the program, and links the security profile to the application program. 879 F.3d at 1303-04, 125 USPQ2d at 1285-86. The Federal Circuit noted that the recited virus screening was an abstract idea, and that merely performing virus screening on a computer does not render the claim eligible. 879 F.3d at 1304, 125 USPQ2d at 1286. The court then continued with its analysis under part one of the Alice/Mayo test by reviewing the patent’s specification, which described the claimed security profile as identifying both hostile and potentially hostile operations. The court noted that the security profile thus enables the invention to protect the user against both previously unknown viruses and “obfuscated code,” as compared to traditional virus scanning, which only recognized the presence of previously-identified viruses. The security profile also enables more flexible virus filtering and greater user customization. 879 F.3d at 1304, 125 USPQ2d at 1286. The court identified these benefits as improving computer functionality, and verified that the claims recite additional elements (e.g., specific steps of using the security profile in a particular way) that reflect this improvement. Accordingly, the court held the claims eligible as not being directed to the recited abstract idea. 879 F.3d at 1304-05, 125 USPQ2d at 1286-87. This analysis is equivalent to the Office’s analysis of determining that the additional elements integrate the judicial exception into a practical application at Step 2A Prong Two, and thus that the claims were not directed to the judicial exception (Step 2A: NO).
Examples of claims that improve technology and are not directed to a judicial exception include: Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1339, 118 USPQ2d 1684, 1691-92 (Fed. Cir. 2016) (claims to a self-referential table for a computer database were directed to an improvement in computer capabilities and not directed to an abstract idea); McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1315, 120 USPQ2d 1091, 1102-03 (Fed. Cir. 2016) (claims to automatic lip synchronization and facial expression animation were directed to an improvement in computer-related technology and not directed to an abstract idea); Visual Memory LLC v. NVIDIA Corp., 867 F.3d 1253,1259-60, 123 USPQ2d 1712, 1717 (Fed. Cir. 2017) (claims to an enhanced computer memory system were directed to an improvement in computer capabilities and not an abstract idea); Finjan Inc. v. Blue Coat Systems, Inc., 879 F.3d 1299, 125 USPQ2d 1282 (Fed. Cir. 2018) (claims to virus scanning were found to be an improvement in computer technology and not directed to an abstract idea); SRI Int’l, Inc. v. Cisco Systems, Inc., 930 F.3d 1295, 1303 (Fed. Cir. 2019) (claims to detecting suspicious activity by using network monitors and analyzing network packets were found to be an improvement in computer network technology and not directed to an abstract idea). Additional examples are provided in MPEP § 2106.05(a).
Regarding the December 5th 2025 Memo in light of September 26, 2025 Appeals Review Panel Decision in Ex parte Desjardins, Appeal 2024-000567 for Application 16/319,040, in deciding if a recited abstract idea does or does not direct the entire claim to an abstract idea, when a claim is considered as a whole:
Paragraph 21 of the Specification, which the Appellant cites, identifies improvements in training the machine learning model itself. Of course, such an assertion in the Specification alone is insufficient to support a patent eligibility determination, absent a subsequent determination that the claim itself reflects the disclosed improvement. See MPEP § 2106.05(a) (citing Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1316 (Fed. Cir. 2016)). Here, however, we are persuaded that the claims reflect such an improvement. For example, one improvement identified in the 8 Appeal2024-000567 Application 16/319,040 Specification is to "effectively learn new tasks in succession whilst protecting knowledge about previous tasks." Spec. ,r 21. The Specification also recites that the claimed improvement allows artificial intelligence (AI) systems to "us[e] less of their storage capacity" and enables "reduced system complexity." Id. When evaluating the claim as a whole, we discern at least the following limitation of independent claim 1 that reflects the improvement: "adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task." We are persuaded that constitutes an improvement to how the machine learning model itself operates, and not, for example, the identified mathematical calculation. Under a charitable view, the overbroad reasoning of the original panel below is perhaps understandable given the confusing nature of existing § 101 jurisprudence, but troubling, because this case highlights what is at stake. Categorically excluding AI innovations from patent protection in the United States jeopardizes America's leadership in this critical emerging technology. Yet, under the panel's reasoning, many AI innovations are potentially unpatentable-even if they are adequately described and nonobvious-because the panel essentially equated any machine learning with an unpatentable "algorithm" and the remaining additional elements as "generic computer components," without adequate explanation. Dec. 24. Examiners and panels should not evaluate claims at such a high level of generality.
Specifically, Ex Parte Desjardins explained the following:
Enfish ranks among the Federal Circuit's leading cases on the eligibility of technological improvements. In particular, Enfish recognized that “[m]uch of the advancement made in computer technology consists of improvements to software that, by their very nature, may not be defined by particular physical features but rather by logical structures and processes.” 822 F.3d at 1339. Moreover, because “[s]oftware can make non-abstract improvements to computer technology, just as hardware improvements can,” the Federal Circuit held that the eligibility determinations should turn on whether “the claims are directed to an improvement to computer functionality versus being directed to an abstract idea.” Id. at 1336. (Desjardins, page 8).
Further in Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB September 26, 2025, Appeals Review Panel Decision) (precedential), the claimed invention was a method of training a machine learning model on a series of tasks. The Appeals Review Panel (ARP) overall credited benefits including reduced storage, reduced system complexity and streamlining, and preservation of performance attributes associated with earlier tasks during subsequent computational tasks as technological improvements that were disclosed in the patent application specification. Specifically, the ARP upheld the Step 2A Prong One finding that the claims recited an abstract idea (i.e., mathematical concept). In Step 2A Prong Two, the ARP then determined that the specification identified improvements as to how the machine learning model itself operates, including training a machine learning model to learn new tasks while protecting knowledge about previous tasks to overcome the problem of “catastrophic forgetting” encountered in continual learning systems. Importantly, the ARP evaluated the claims as a whole in discerning at least the limitation “adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task” reflected the improvement disclosed in the specification. Accordingly, the claims as a whole integrated what would otherwise be a judicial exception instead into a practical application at Step 2A Prong Two, and therefore the claims were
The claim itself does not need to explicitly recite the improvement described in the specification (e.g., “thereby increasing the bandwidth of the channel”). See, e.g., Ex Parte Desjardins, Appeal No. 2024-000567 (PTAB September 26, 2025, Appeals Review Panel Decision) (precedential), in which the specification identified the improvement to machine learning technology by explaining how the machine learning model is trained to learn new tasks while protecting knowledge about previous tasks to overcome the problem of “catastrophic forgetting,” and that the claims reflected the improvement identified in the specification. Indeed, enumerated improvements identified in the Desjardins specification included disclosures of the effective learning of new tasks in succession in connection with specifically protecting knowledge concerning previously accomplished tasks; allowing the system to reduce use of storage capacity; and the enablement of reduced complexity in the system. Such improvements were tantamount to how the machine learning model itself would function in operation and therefore not subsumed in the identified mathematical calculation.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Claims 1, 11, and 16 with dependent claims thereof, are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1, 11, and 16 and any dependent claims thereof of U.S. Patent No. 12190864. Although the conflicting claims are not identical, they are not patentably distinct from each other because said claims of the instant application includes all of the features of said claims of U.S. Patent No. 12190864. It would have been obvious to one of ordinary skill in the art to omit the step of using an interest probability distribution, as the claims revolve around necessitated probabilistic operations, In re Karlson 136 USPQ 184 (1963): "Omission of an element and its function is an obvious expedient if the remaining elements perform the same functions as before"
Present invention US Patent 12190864
1. A computer-implemented method for training a conversational recommendation system for generating an output, having a high play-probability, based on a minimal number of iterations of conversation, comprising: generating, by at least one computer processor, a pseudo-user neural network model corresponding to a pseudo-user profile; training, using the pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on an estimated state of the pseudo-user neural network model; selecting, by the prompt-decision engine, an interest prompt based on the estimated state of the pseudo-user neural network model and the selected interest-exploration strategy; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on a pseudo-user response generated by the pseudo-user neural network model; and updating, using a reinforcement-learning method, the recommendation policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having the high play-probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system.
2. The computer-implemented method of claim 1, further comprising: terminating the iterative learning process if the pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
3. The computer-implemented method of claim 1, wherein the recommendation policy corresponds to an interest-exploration policy and a prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
4. The computer-implemented method of claim 1, wherein the pseudo-user neural network model is based on at least one interest probability distribution corresponding to a pseudo-user profile, and wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
5. The computer-implemented method of claim 1, wherein the pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
6. The computer-implemented method of claim 5, wherein the updating the reward function further comprises: incrementing the reward function by a predetermined value if the pseudo-user response comprises accepting to play the recommended media content corresponding to the selected interest prompt; decrementing the reward function by a first value if the pseudo-user response comprises quitting the conversation session with the conversational recommendation system; and decrementing the reward function by a second value if the pseudo-user response comprises generating the further pseudo-user response.
7. The computer-implemented method of claim 1, wherein the selecting the interest-exploration strategy further comprises: extracting a current interest from the pseudo-user interaction history using named entity recognition; and performing an interest prediction based on the current interest.
8. The computer-implemented method of claim 1, wherein the selecting the interest-exploration strategy further comprises: selecting the interest-exploration strategy from a plurality of candidate interest-exploration strategies, including one or more of the following: exploration via an area target, exploration via a point target, exploration via a filtered target, exploration via a popular target, and exploration via a similar target.
9. The computer-implemented method of claim 1, wherein a response generated by the pseudo-user neural network model is processed by an automatic speech recognition module and a natural language understanding module before being received by the interest exploration engine.
10. The computer-implemented method of claim 1, wherein an output of the prompt decision engine corresponding to the selected interest prompt is processed by a large language model and a text to speech module before being received by the pseudo-user neural network model.
11. A system, comprising: one or more memories; and at least one processor each coupled to at least one of the memories and configured to perform operations comprising: generating a pseudo-user neural network model corresponding to a pseudo-user profile; training, using the pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on an estimated state of the pseudo-user neural network model; selecting, by the prompt-decision engine, an interest prompt based on the estimated state of the pseudo-user neural network model and the selected interest-exploration strategy; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the a pseudo-user response generated by the pseudo-user neural network model; and updating, using a reinforcement-learning method, the recommendation policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system.
12. The system of claim 11, further comprising: terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
13. The system of claim 11, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
14. The system of claim 11, wherein the pseudo-user neural network model is based on at least one interest probability distribution corresponding to a pseudo-user profile, and wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
15. The system of claim 11, wherein the pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
16. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising: generating a pseudo-user neural network model corresponding to a pseudo-user profile; training, using the pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on an estimated state of the pseudo-user neural network model; selecting, by the prompt-decision engine, an interest prompt based on the estimated state of the pseudo-user neural network model and the selected interest-exploration strategy; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on a pseudo-user response generated by the pseudo-user neural network model; and updating, using a reinforcement-learning method, the recommendation policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system.
17. The non-transitory computer-readable medium of claim 16, further comprising: terminating the iterative learning process if the pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
18. The non-transitory computer-readable medium of claim 16, wherein the recommendation policy corresponds to an interest-exploration policy and a prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
19. The non-transitory computer-readable medium of claim 16, wherein the pseudo-user neural network model is based on at least one interest probability distribution corresponding to a pseudo-user profile, and wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
20. The non-transitory computer-readable medium of claim 16, wherein the pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
1. (Currently Amended) A computer-implemented method for training a conversational recommendation system for generating an output, having a high play-probability, based on a minimal number of iterations of conversation, comprising:generating, by at least one computer processor, a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile;training, using the probabilistic pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including:selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history;selecting, by the prompt-decision engine, an interest prompt based on a prompt- decision policy and the selected interest-exploration strategy;generating, by the probabilistic pseudo-user neural network model, another pseudo- user response based on the selected interest prompt;updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; andupdating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having the high play-probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system.
2. (Original) The method of claim 1, further comprising:terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
3. (Original) The method of claim 1, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
4. (Original) The method of claim 1, wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
5. (Original) The method of claim 1, wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
6. (Original) The method of claim 5, wherein the updating the reward function further comprises:incrementing the reward function by a predetermined value if the another pseudo-user response comprises accepting to play the recommended media content corresponding to the selected interest prompt;decrementing the reward function by a first value if the another pseudo-user response comprises quitting the conversation session with the conversational recommendation system; anddecrementing the reward function by a second value if the another pseudo-user response comprises generating the further pseudo-user response.
7. (Original) The method of claim 1, wherein the selecting the interest-exploration strategy further comprises:extracting a current interest from the pseudo-user interaction history using named entity recognition; andperforming an interest prediction based on the current interest.
8. (Original) The method of claim 1, wherein the selecting the interest-exploration strategy further comprises:selecting the interest-exploration strategy from a plurality of candidate interest-exploration strategies, including one or more of the following: exploration via an area target, exploration via a point target, exploration via a filtered target, exploration via a popular target, and exploration via a similar target.
9. (Currently Amended) The method of claim 1, wherein a response generated by the probabilistic pseudo-user neural network model is processed by an automatic speech recognition module and a natural language understanding module before being received by the interest exploration engine.
10. (Currently Amended) The method of claim 1, wherein an output of the prompt decision engine corresponding to the selected interest prompt is processed by a large language model and a text to speech module before being received by the probabilistic pseudo-user neural network model.
11. (Currently Amended) A system, comprising:one or more memories; andat least one processor each coupled to at least one of the memories and configured to perform operations comprising:generating a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile; training, using the probabilistic pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including:selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history;selecting, by the prompt-decision engine, an interest prompt based on a prompt- decision policy and the selected interest-exploration strategy;generating, by the probabilistic pseudo-user neural network model, another pseudo- user response based on the selected interest prompt;updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; andupdating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system.
12. (Currently Amended) The system of claim 11, the operations further comprising:terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
13. (Original) The system of claim 11, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
14. (Original) The system of claim 11, wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
15. (Original) The system of claim 11, wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
16. (Currently Amended) A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:generating, by at least one computer processor, a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile;training, using the probabilistic pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including:selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history;selecting, by the prompt-decision engine, an interest prompt based on a prompt- decision policy and the selected interest-exploration strategy;generating, by the probabilistic pseudo-user neural network model, another pseudo- user response based on the selected interest prompt;updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system.
17. (Currently Amended) The non-transitory computer-readable medium of claim 16,the operations further comprising:terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
18. (Original) The non-transitory computer-readable medium of claim 16, wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
19. (Original) The non-transitory computer-readable medium of claim 16, wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
20. (Original) The non-transitory computer-readable medium of claim 16, wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
Allowable Subject Matter
Claims 1-20 are allowed.
The following is an examiner’s statement of reasons for allowance:
After a careful review of the complex claims as a whole, the examiner believes that the prior art taken alone or in combination fails to teach the claims as a whole such training a conversational recommendation system for generating an output, having a high play-probability, based on a minimal number of iterations of conversation, comprising: generating, by at least one computer processor, a pseudo-user neural network model corresponding to a pseudo-user profile; training, using the pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including: selecting, by the interest-exploration engine, an interest-exploration strategy based on an estimated state of the pseudo-user neural network model; selecting, by the prompt-decision engine, an interest prompt based on the estimated state of the pseudo-user neural network model and the selected interest-exploration strategy; updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on a pseudo-user response generated by the pseudo-user neural network model; and updating, using a reinforcement-learning method, the recommendation policy based on at least the updated reward function; and generating, using the trained conversational recommendation system, a real-time recommendation having the high play-probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system.
The above claims are deemed allowable given the complex nature of the claims as a whole including interest-exploration, prompt-decision, reward functions, with training, selecting, generating, updating, and subsequent generating with pseudo-user operations tied in and amongst thereof as a whole as precisely claimed. The specificity of the claim limitations taken together provides strong reason the closest prior art. For instance, a closest prior art group consists of reward function concepts for user interaction, reinforcement learning, neural networks, general policy maps, and user sentiment extraction. Another group consists of user-profile based reward functions with an explicitly policy-decision model, where a model is analogous to an engine for operational purposes. Other prior art groups consist of preventing alias accounts, test vs live mode, pre-training, and minimal learning or optimized learning. Further art suggests ASR based learning for searching media with updatable models similar to that of for example but not limited to Amazon, Google, and Apple, simply for learning model-based searching of media e.g. movies/music. Under BRI, the prior art taken together, disregarding a piece-wise combination for limitations amounts to reinforcement learning using media selection as a basis for ASR/conversational learning and rewards. However, and under BRI, the specificity of the claim’s operations cannot feasibly produce a suggest combination of prior art that would read upon the claims as a whole, since the art’s updating operations and series of engines, prompts, pseudo-user responses, strategies, and reward function steps are tied together in an express arrangement distinct from the prior art. At best the art combination produces enhanced policy-decision-focused reinforcement learning using media selection as a basis for ASR/conversational learning and rewards. Therefore, the prior art fails to teach or suggest the complex claim limitations as a whole as precisely claimed.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230376697 A1 Chow; Yinlam et al.
Reinforcement learning, user satisfaction, reward
US 20180329998 A1 THOMSON; Blaise et al.
Reinforcement learning, user profile, policy, reward
US 11157488 B2 Feuz; Sandro et al.
Reinforcement learning, reward, searching
US 11181988 B1 Bellegarda; Jerome R. et al.
Reward, predicting user intent
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL COLUCCI whose telephone number is (571)270-1847. The examiner can normally be reached on M-F 9 AM - 7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571)272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL COLUCCI/Primary Examiner, Art Unit 2655 (571)-270-1847
Examiner FAX: (571)-270-2847
Michael.Colucci@uspto.gov
Prosecution Insights