DETAILED ACTION
This office action is made final. Claims 1-19 are pending. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Applicants amended claims 1, 5, 9, and 15; and added new claims 17-19.
Response to Amendment
The previously pending rejection to claims 1-16, under 35 USC 101 (Alice), will be maintained.
The previously pending rejection to claims 9 and 15, under 35 USC 101 (software per se), have been withdrawn in light of amendment.
Response to Arguments
Applicant’s arguments received on date 09/16/2026 have been fully considered, but they are not persuasive. Moreover, any new grounds of rejection have been necessitated by Applicant's amendments to the claims.
Response to Arguments under 35 USC 101:
Applicant asserts that “Step 2A, Prong One - Claim 1 Is Not Directed to a Judicial Exception.” Examiner respectfully disagrees.
Pursuant to 2019 Revised Patent Subject Matter Eligibility Guidance, in order to determine whether a claim is directed to an abstract idea, under Step 2A, we first (1) determine whether the claims recite limitations, individually or in combination, that fall within the enumerated subject matter groupings of abstract ideas (mathematical concepts, certain methods of organizing human activity, or mental processes), and (2) determine whether any additional elements beyond the recited abstract idea, individually and as an ordered combination, integrate the judicial exception into a practical application. 84 Fed. Reg. 52, 54-55. Next, if a claim (1) recites an abstract idea and (2) does not integrate that exception into a practical application, in order to determine whether the claim recites an “inventive concept,” under Step 2B, we then determine whether the claim recites any of the additional elements beyond the recited abstract idea, individually and in combination, are significantly more than the abstract idea itself. 84 Fed. Reg. 56.
Here, under the first prong of Step 2A, the claims (claim 1, and similarly claims 9 and 15) recite “receive predictions generated together with associated explanations for the predictions; modify the received predictions and/or associated explanations under consideration of predefined requirements without modifying; and transfer the modified predictions and associated explanations.” Therefore, contrary to Applicant’s assertions, the claims are directed to mental processes.
Applicant asserts that “Step 2A, Prong Two - Claim 1 Integrates Any Alleged Exception into a Practical Application.” Examiner respectfully disagrees.
As discussed above, under the second prong of Step 2A, we determine whether any additional elements beyond the recited abstract idea, individually and as an ordered combination, integrate the judicial exception into a practical application. 84 Fed. Reg. 52, 54-55.
Here, under the second prong of Step 2A, the only additional elements beyond the recited abstract idea of claim 1, and similarly claims 9 and 15, are the recitations of “a background system and an operator environment comprising one or more operator systems, each of the systems comprising one or more hardware processors and an artificial intelligence (Al) model, which, given an input, produces a prediction and/or explanation output, and a communication adapter comprising one or more hardware processors implemented to act as middle-ware between the background system and the operator environment, wherein the communication adapter is configured to: receive predictions generated by the background system together with associated explanations for the predictions; modify the received predictions and/or associated explanations under consideration of predefined requirements without modifying the AI model of the background system; and transfer the modified predictions and associated explanations to the one or more operator systems of the operator environment are carried out by at least one or more hardware processors,” and these additional elements, individually and in combination, are nothing more than computing elements recited at high level of generality implementing the abstract idea on a computer (i.e. apply it), and thus, are no more than applying the abstract idea with generic computer components. Accordingly, contrary to Applicant’s assertions, the judicial exception is not integrated into a practical application under the second prong of Step 2A.
Applicant asserts that “Step 2B - The Claim Elements Amount to Significantly More Than Any Alleged Exception.” Examiner respectfully disagrees.
The MPEP discusses that "the second part of the Alice/Mayo test [(Step 2B)] is often referred to as a search for an inventive concept," and "an 'inventive concept' is furnished by an element or combination of elements that is recited in the claim in addition to (beyond) the judicial exception, and is sufficient to ensure that the claim as a whole amounts to significantly more than the judicial exception itself." MPEP 2106.05 (emphasis added). Further, the MPEP goes on to describe "Step 2B asks: Does the claim recite additional elements that amount to significantly more than the judicial exception? Examiners should answer this question by first identifying whether there are any additional elements (features/limitations/steps) recited in the claim beyond the judicial exception(s), and then evaluating those additional elements individually and in combination to determine whether they contribute an inventive concept (i.e., amount to significantly more than the judicial exception(s)).” MPEP 2106.05 (emphasis added).
The search for an inventive concept under § 101 is distinct from demonstrating novel and non-obviousness. See SAP America Inc. v. Investpic, LLC, No. 2017-2081, slip op. at 2-3 (Fed Cir. May 15, 2018) (citing Synopsys, Inc. v. Mentor Graphics Corp., 839 F.3d 1138, 1151 (Fed. Cir. 2016); Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1315 (Fed. Cir. 2016). Even novel and newly discovered judicial exceptions are still exceptions, despite their novelty. July 2015 Update, p. 3; see SAP America at 2. In Step 2B, “[w]hat is needed is an inventive concept in the non-abstract application realm.” SAP America at 11. As discussed in SAP America, no matter how much of an advance the claims recite, when “the advance lies entirely in the realm of abstract ideas, with no plausibly alleged innovation in the non-abstract application realm,” “[a]n advance of that nature is ineligible for patenting.” Id. at 3.
Here, under Step 2B, the only additional elements beyond the recited abstract idea of claim 1, and similarly claims 9 and 15, are the recitations of “a background system and an operator environment comprising one or more operator systems, each of the systems comprising one or more hardware processors and an artificial intelligence (Al) model, which, given an input, produces a prediction and/or explanation output, and a communication adapter comprising one or more hardware processors implemented to act as middle-ware between the background system and the operator environment, wherein the communication adapter is configured to: receive predictions generated by the background system together with associated explanations for the predictions; modify the received predictions and/or associated explanations under consideration of predefined requirements without modifying the AI model of the background system; and transfer the modified predictions and associated explanations to the one or more operator systems of the operator environment are carried out by at least one or more hardware processors,” and these additional elements, individually and in combination, are nothing more than computing elements recited at high level of generality implementing the abstract idea on a computer (i.e. apply it), and thus, are no more than applying the abstract idea with generic computer components. Accordingly, contrary to Applicant’s assertions, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception under Step 2B.
Response to Arguments under 35 USC 103:
Applicant's arguments with respect to the claim rejections have been considered, but are moot in view of the new ground(s) of rejection set forth below in this Office action. The art rejection has been updated to address these amendments.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 19 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 19 recites, “an indication of which input the one or more operator systems prefer, wherein the communication adapter is further configured to modify the received predictions and/or the associated explanations based on the indication.” As best understood, it appears that there is no support for the underlined recitation in the original disclosure of the present application for this limitation. Moreover, none of the drawings shows this particular feature. Therefore, this limitation of claim 19 is considered to be new matter. Appropriate correction is required.
Specification
The amendment filed on 09/16/2026 is objected to under 35 U.S.C. 132(a) because it introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment shall introduce new matter into the disclosure of the invention. The added material which is not supported by the original disclosure is as follows:
1) Applicant’s introduces new matter into the specification such as “The method has applications including, but not limited to, use cases in machine learning (ML) and artificial intelligence (AI), for medical/ healthcare, to optimize predictions or support decision making.” The added material which is not supported by the original disclosure as filed. The Examiner concludes that the Applicants' disclosure fails to sufficiently disclose possession at the time of the invention.
2) Applicant’s introduces new matter into the abstract such as “The method has applications including, but not limited to, use cases in machine learning (ML) and artificial intelligence (Al), for medical/ healthcare, to optimize predictions or support decision making.” The added material which is not supported by the original disclosure as filed. The Examiner concludes that the Applicants' disclosure fails to sufficiently disclose possession at the time of the invention.
3) Applicant’s introduces new matter into the disclosure paragraph [0013] and abstract. The added material which is not supported by the original disclosure as filed. The Examiner concludes that the Applicants' disclosure fails to sufficiently disclose possession at the time of the invention.
Applicant is required to cancel the new matter in the reply to this Office Action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Specifically, claims 1-19 are directed to an abstract idea without additional elements amounting to significantly more than the abstract idea.
With respect to Step 2A Prong One of the framework, claims 1, 9, and 15 recite an abstract idea. Claims 1, 9, and 15 include “receive predictions generated together with associated explanations for the predictions; modify the received predictions and/or associated explanations under consideration of predefined requirements without modifying; and transfer the modified predictions and associated explanations”.
The limitations above recite an abstract idea under Step 2A Prong One. More particularly, the elements above recite mental processes-concepts performed in the human mind (including an observation, evaluation, judgment, opinion) because the elements describe a process for producing a prediction. As a result, claims 1, 9, and 15 recite an abstract idea under Step 2A Prong One.
Claims 2-8, 10-14, and 16 further describe the process for producing a prediction. As a result, claims 2-8, 10-14, and 16 recite an abstract idea under Step 2A Prong One for the same reasons as stated above with respect to claims 1, 9, and 15.
With respect to Step 2A Prong Two of the framework, claims 1, 9, and 15 do not include additional elements that integrate the abstract idea into a practical application. Claims 1, 9, and 15 include additional elements that do not recite an abstract idea under Step 2A Prong One. The additional elements of claims 1, 9, and 15 include one or more hardware processors, a middle-ware, a multi-agent communication system, a background system, operator systems, Artificial Intelligence, a communication adapter. When considered in view of the claim as a whole, the additional elements do not integrate the abstract idea into a practical application because the additional computing elements are generic computing elements that are merely used as a tool to perform the recited abstract idea. As a result, claims 1, 9, and 15 do not include additional elements that integrate the abstract idea into a practical application under Step 2A Prong Two.
Claim 3 does not include any additional elements beyond those recited with respect to claims 1, 9, and 15. As a result, claim 3 does not include additional elements that integrate the abstract idea into a practical application under Step 2A Prong Two for the same reasons as stated above with respect to claims 1, 9, and 15.
Claims 2, 4-8, 10-14, and 16-19 include additional elements that do not recite an abstract idea under Step 2A Prong One. The additional elements of claims 2, 4-8, 10-14, and 16-19 include a background system, a communication adapter, operator systems, a reranker component, a filter component, a regulator component, a neural network, a reinforcement learning algorithm, and an explainable AI. When considered in view of the claims as a whole, the additional elements do not integrate the abstract idea into a practical application because the additional computing elements do no more than generally link the use of the recited abstract idea to a particular technological environment. As a result, claims 2, 4-8, 10-14, and 16-19 do not include additional elements that integrate the abstract idea into a practical application under Step 2A Prong Two.
With respect to Step 2B of the framework, claims 1, 9, and 15 do not include additional elements amounting to significantly more than the abstract idea. As noted above, claims 1, 9, and 15 include additional elements that do not recite an abstract idea under Step 2A Prong One. The additional elements of claims 1, 9, and 15 include a middle-ware, a multi-agent communication system, a background system, operator systems, Artificial Intelligence, a communication adapter. The additional elements do not amount to significantly more than the abstract idea because the additional computing elements are generic computing elements that are merely used as a tool to perform the recited abstract idea. Further, looking at the additional elements as an ordered combination adds nothing that is not already present when considering the additional elements individually. As a result, independent claims 1, 9, and 15 do not include additional elements that amount to significantly more than the abstract idea under Step 2B.
Claim 3 does not include any additional elements beyond those recited with respect to claims 1, 9, and 15. As a result, claim 3 does not include additional elements that amount to significantly more than the abstract idea under Step 2B for the same reasons as stated above with respect to claims 1, 9, and 15.
Claims 2, 4-8, 10-14, and 16-19 include additional elements that do not recite an abstract idea under Step 2A Prong One. The additional elements of claims 2, 4-8, 10-14, and 16-19 include a background system, a communication adapter, operator systems, a reranker component, a filter component, a regulator component, a neural network, a reinforcement learning algorithm, and an explainable AI. The additional elements do not amount to significantly more than the abstract idea because the additional computing elements do no more than generally link the use of the recited abstract idea to a particular technological environment. Further, looking at the additional elements as an ordered combination adds nothing that is not already present when considering the additional elements individually. As a result, claims 2, 4-8, 10-14, and 16-19 do not include additional elements that amount to significantly more than the abstract idea under Step 2B.
Therefore, the claims are directed to an abstract idea without additional elements amounting to significantly more than the abstract idea. Accordingly, claims 1-19 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-15, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US Pat No. 11,715,042) (hereinafter Liu et al.) in view of Hwang et al. (US Pub No. 2021/0027136) (hereinafter Hwang et al.).
Regarding claims 1, 9, and 15, Liu in view of Hwang discloses a method of supporting communication between a background system and an operator environment comprising one or more operator systems, wherein each of the systems comprise an artificial intelligence model, which, given an input, produces a prediction and/or explanation output (see Liu, column 8, lines 66-67 & column 9, lines 1-16, wherein an application programming interfaces (API) or other communication channels……APIs, one or more web services, one or more content sources, one or more networks, or any other suitable components, e.g., that servers may communicate with a computing system; column 14, lines 13-20, wherein implementations may be handled by multiple agents associated with multiple domains. In particular embodiments, the agents may comprise first-party agents 250 and third-party agents 255. In particular embodiments, first-party agents 250 may comprise internal agents that are accessible and controllable by the assistant system 140 (e.g. agents associated with services provided by the online social network (Messenger, Instagram)); column 40, lines 24-25, wherein artificial neural network ("ANN") 1200; and column 2, lines 57-62, wherein (1) explain reinforcement-learning models by extracting a set of representative dialog policy rules with sequential pattern mining, (2) compare the rules learned from reinforcement learning and other supervised models, and finally (3) refine the reinforcement-learning models by incorporating human knowledge through user simulation), the method comprising:
receiving, by a communication adapter implemented to act as (network) between the background system and the operator environment, predictions generated by the background system together with associated explanations for the predictions (see Liu, column 16, lines 36-58, wherein the assistant xbot 215 may interact with a proactive inference layer 280 without receiving a user input. The proactive inference layer 280 may infer user interests and preferences based on the user profile that is retrieved from the user context engine 225. The proactive inference layer 280 may further communicate with proactive agents 285 regarding the inference. The proactive agents 285 may execute proactive tasks based on the inference…… Therefore, the proactive agent 285 may execute the proactive task in a personalized and context-aware manner. The proactive inference layer may infer that the user likes the band Maroon 5 and the proactive agent 285 may generate a recommendation of Maroon 5's new song/album to the user; column 30, lines 14-17, wherein these are the patterns that correlate with a high probability of user clicking on the displayed suggestion according to the training data, or according to the GBDT and RL model predictions; and column 6, lines 22-24, wherein generate a Hyper Text Transfer Protocol (HTTP) request and communicate the HTTP request to server));
modifying, by the communication adapter, the received predictions and/or associated explanations under consideration of predefined requirements without modifying the AI model of the background system (see Liu, column 28, lines 10-67, wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……The refining may comprise the following process. The assistant system 140 may first determine, for each of the plurality of rules, a quality measurement. The assistant system 140 may then delete one or more rules of the plurality of rules if the quality measurements for the one or more rules do not satisfy a threshold quality measurement…..aim to refine the models to produce the desired prediction…..; and column 24, lines 51-53, wherein behavior of an ML-based policy may be refined by adding or removing (i.e. fine-tuning step is removed from training) all training dialogs that correspond to a small set of handpicked dialog sequences); and
transferring, by the communication adapter, the modified predictions and associated explanations to the one or more operator systems of the operator environment (see Liu, column 45, lines 32-34, wherein a first user may transmit a message to a second user via an application related to the online social network (e.g., a messaging app ); column 28, lines 10-67 wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……The refining may comprise the following process. The assistant system 140 may first determine, for each of the plurality of rules, a quality measurement. The assistant system 140 may then delete one or more rules of the plurality of rules if the quality measurements for the one or more rules do not satisfy a threshold quality measurement…..aim to refine the models to produce the desired prediction. This is achieved by leveraging a user simulator disclosed herein to generate simulated user dialogs based on the patterns where the unrefined RL model may predict wrongly).
Liu et al. fails to explicitly disclose receiving, by a communication adapter implemented to act as middle-ware between the background system and the operator environment.
Analogous art Hwang discloses receiving, by a communication adapter implemented to act as middle-ware between the background system and the operator environment, predictions generated by the background system together with associated explanations for the predictions (see Hwang, para [0045], wherein the monitoring component 104 can monitor one or more events associated with a server, middleware, an application and/or another component of the artificial intelligence system. In another example, the monitoring component 104 can monitor one or more events associated with different artificial intelligence subsystems of the artificial intelligence system; para [0062], wherein a set of predictions, and/or a set of results….The output data 714 can include learned data associated with learning, classifications, recommendations, predictions and/or inferences determined by an artificial intelligence model).
Liu directed to a system for training a target machine-learning model iteratively by accessing training data of content objects. Hwang directed to facilitating feedback loop learning between artificial intelligence systems. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Liu, regarding the Interpretability of Deep Reinforcement Learning Models In Assistant Systems, to have included receiving, by a communication adapter implemented to act as middle-ware between the background system and the operator environment, predictions generated by the background system together with associated explanations for the predictions because both inventions teach improving processing efficiency. Further, the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Regarding claims 9 and 15 recite additional features such as one or more hardware processors and one or more hardware processors (see Liu, Fig. 13).
Regarding claim 2, Liu in view of Hwang discloses the method according to claim 1, further comprising an initialization step comprising:
defining a first space of possible predictions and defining a second space of possible explanations providable by the background system (see Liu, column 28, lines 10-67, wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……The top n decision sequence patterns are defined as decision sequences D such that…..is ranked among top n. By comparing the common patterns extracted from the RL and GBDT models, the embodiments disclosed herein gain insights into how differently these models behave and allow using human knowledge to further improve the models with augmented training data……. analyzing extracted dialog policies from the RL model, sometimes patterns that are against the desired outcome may be observed. For examples where the model predictions are apparently against common sense, the embodiments disclosed herein aim to refine the models to produce the desired prediction. This is achieved by leveraging a user simulator disclosed herein to generate simulated user dialogs based on the patterns where the unrefined RL model may predict wrongly).
Regarding claim 3, Liu in view of Hwang discloses the method according to claim 2, wherein the first space of possible predictions and the second space of possible explanations comprise a set of labels or a sequence of a set of labels (see Liu, column 30, lines 16-67, wherein referring to Fig. 5 shows the "-4, -3, -2, -1, current" labels mark the number of the tum prior to the message in consideration. The "AW, SW" letters indicate the intents of each turn, as predicted by the pretrained NLU models).
Regarding claims 4, Liu in view of Hwang discloses the method according to claim 1, wherein the predefined requirements considered by the communication adapter to modify the received predictions and/or associated explanations comprise needs of the operator environment concerning which explanations are most suitable for the operator environment and/or regulations imposed by an outside source (see Liu, column 28, lines 10-67, wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……The refining may comprise the following process. The assistant system 140 may first determine, for each of the plurality of rules, a quality measurement. The assistant system 140 may then delete one or more rules of the plurality of rules if the quality measurements for the one or more rules do not satisfy a threshold quality measurement…..aim to refine the models to produce the desired prediction. This is achieved by leveraging a user simulator disclosed herein to generate simulated user dialogs based on the patterns where the unrefined RL model may predict wrongly).
Regarding claim 5, Liu in view of Hwang discloses the method according to claim 1, further comprising:
updating the communication adapter based on the feedback without modifying the AI model of the background system (see Liu, column 17, lines 18-19, wherein the ranking may be alternatively based on a ranking model that is trained based on user feedback data; and column 24, lines 51-53, wherein behavior of an ML-based policy may be refined by adding or removing (i.e. fine-tuning is removed from training) all training dialogs that correspond to a small set of handpicked dialog sequences).
Liu et al. fails to explicitly disclose requesting, for outputs of the communication adapter, feedback from the one or more operator systems of the operator environment; and updating the communication adapter based on the feedback.
Analogous art Hwang discloses requesting, for outputs of the communication adapter, feedback from the one or more operator systems of the operator environment; and updating the communication adapter based on the feedback (see Hwang, para [0045], wherein the monitoring component 104 can monitor one or more events associated with a server, middleware, an application and/or another component of the artificial intelligence system……; para [0063], wherein the data analytics component 102 can perform a machine learning process 710 based on user feedback data 716 provided by a user device 708 and/or system status data 718 provided by an automation system 706. The user device 708 can be configured to interact with a user (e.g., a user identity) and/or generate the user feedback data 716; and para [0058], wherein verify that dependent systems within the artificial intelligence system have a corresponding interface and/or trigger a change request to update the dependent systems. For example, uniformity of data and/or updates among dependent systems within the artificial intelligence system can be verified).
One of ordinary skill in the art would have recognized that applying the known technique of Hwang would have yielded predictable results and resulted in an improved system for the same reasons as stated above with respect to claim 1.
Regarding claims 6 and 11, Liu in view of Hwang discloses the method according to claim 1, further comprising:
examining, by a reranker component of the communication adapter, a set of explanations received from the background system and updating the an ordering of the explanations (see Liu, wherein column 15, lines 52-57, wherein the first task may comprise a translation task, in which the input to the natural-language generator 271 may be translated to concepts. The second task may comprise a selection task, in which relevant concepts may be selected among those resulted from the translation task based on the user model; and column 21, lines 40-57, wherein dialog intent resolution 236 and dialog state update/ranker 237. In particular embodiments, the dialog intent resolution 236 may resolve the user intent associated with the current dialog session based on dialog history between the user and the assistant system 140. The dialog intent resolution 236 may map intents determined by the NLU module 220 to different dialog intents. The dialog intent resolution 236 may further rank dialog intents based on signals from the NLU module 220, the entity resolution module 240, and dialog history between the user and the assistant system 140. In particular embodiments, the dialog state update/ranker 237 may update/rank the dialog state of the current dialog session. As an example and not by way of limitation, the dialog state update/ranker 237 may update the dialog state as "completed" if the dialog session is over. As another example and not by way of limitation, the dialog state update/ranker 237 may rank the dialog state based on a priority associated with it).
Regarding claims 7 and 13, Liu in view of Hwang discloses the method according to claim 6, further comprising:
receiving, by a filter component of the communication adapter, the explanations with updated ordering from the reranker component (see Liu, wherein column 16, lines 61-63, wherein the generation may be based on a straightforward backend query using deterministic filters to retrieve the candidate entities from a structured data store; and column 21, lines 40-57, wherein dialog intent resolution 236 and dialog state update/ranker 237. In particular embodiments, the dialog intent resolution 236 may resolve the user intent associated with the current dialog session based on dialog history between the user and the assistant system 140. The dialog intent resolution 236 may map intents determined by the NLU module 220 to different dialog intents. The dialog intent resolution 236 may further rank dialog intents based on signals from the NLU module 220, the entity resolution module 240, and dialog history between the user and the assistant system 140. In particular embodiments, the dialog state update/ranker 237 may update/rank the dialog state of the current dialog session. As an example and not by way of limitation, the dialog state update/ranker 237 may update the dialog state as "completed" if the dialog session is over. As another example and not by way of limitation, the dialog state update/ranker 237 may rank the dialog state based on a priority associated with it); and
selecting, by the filter component, a predefined or configurable number of the top-ranked explanations according to the updated ordering, and passing on the selected explanations to the operator environment (see Liu, wherein column 21, lines 40-57 & column 30, lines 24-30, wherein the top dialog policies extracted from training data (i.e., rated or highest-confidence) and RL model intuitively make sense-users are likely to click the suggestion presented at the most recent tum, if the conversation roughly follows an ask-when 412-reply-when 416-suggest-when 414-acknowledgement 418 sequence, or other variations of this sequence; and column 17, lines 14-57, wherein the assistant system 140 may then calculate similarity scores ( e.g., based on cosine similarity) between the feature vector representing the user's interest and the feature vectors representing the candidate entities. The ranking may be alternatively based on a ranking model that is trained based on user feedback data……The proactive scheduler may determine an actual time to send the recommended candidate entities to the user based on the priority associated with the task and other relevant factors (e.g., clicks and impressions of the recommended candidate entities). In particular embodiments, the proactive scheduler may then send the recommended candidate entities with the determined actual time to an asynchronous tier……….If the user is engaged in an ongoing conversation and the priority of the task of recommendation is high, the dialog engine 235 may initiate a new dialog session with the user in which the selected candidate entities may be presented).
Regarding claim 8, Liu in view of Hwang discloses the method according to claim 1, wherein a same background system is used for multiple operator systems of the operator environment (see Liu, column 8, lines 66-67 & column 9, lines 1-16, wherein an application programming interfaces (API) or other communication channels (i.e., multiple users can access different resources of a computer at the same time)……APIs, one or more web services, one or more content sources, one or more networks, or any other suitable components, e.g., that servers may communicate with a computing system; column 14, lines 13-20, wherein implementations may be handled by multiple agents associated with multiple domains. In particular embodiments, the agents may comprise first-party agents 250 and third-party agents 255. In particular embodiments, first-party agents 250 may comprise internal agents that are accessible and controllable by the assistant system 140 (e.g. agents associated with services provided by the online social network (Messenger, Instagram)).
Regarding claim 10, Liu in view of Hwang discloses the system according to claim 9, wherein the communication adapter comprises a regulator component configured to update predictions and explanations received from the background system in such a way that the predictions and explanations comply with regulations defined by an outside source (see Liu, column 13, lines 45-46, wherein the privacy check module 245 may use an authorization/privacy server to enforce privacy policies; column 28, lines 10-67, wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……The refining may comprise the following process. The assistant system 140 may first determine, for each of the plurality of rules, a quality measurement. The assistant system 140 may then delete one or more rules of the plurality of rules if the quality measurements for the one or more rules do not satisfy a threshold quality measurement…..aim to refine the models to produce the desired prediction. This is achieved by leveraging a user simulator disclosed herein to generate simulated user dialogs based on the patterns where the unrefined RL model may predict wrongly; and column 10, lines 30-32, wherein authorization servers may be used to enforce one or more privacy settings of the users of the social-networking system 160).
Regarding claim 12, Liu in view of Hwang discloses the system according to claim 11, wherein the reranker component is implemented as a neural network that is updated using a reinforcement-learning algorithm based on feedback from the operator environment (see Liu, column 17, lines 14-57, wherein the assistant system 140 may then calculate similarity scores ( e.g., based on cosine similarity) between the feature vector representing the user's interest and the feature vectors representing the candidate entities. The ranking may be alternatively based on a ranking model that is trained based on user feedback data; column 27, lines 4-7, wherein consists of a neural network value function estimator that is trained on offline logs of conversations that were collected in the presence of a rule-based suggestion triggering policy; and column 3, lines 47-51, wherein refining the reinforcement-learning model by incorporating human knowledge through user simulation, based on which the reinforcement-learning model is re-trained, thereby resulting in improved debuggability).
Regarding claim 14, Liu in view of Hwang discloses the system according to claim 9, wherein the background system providing the predictions together with corresponding explanations is configured to apply a knowledge base representations learning mechanism together with an explainable AI mechanism (see Liu, column 28, lines 10-67, wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……The top n decision sequence patterns are defined as decision sequences D such that…..is ranked among top n. By comparing the common patterns extracted from the RL and GBDT models, the embodiments disclosed herein gain insights into how differently these models behave and allow using human knowledge to further improve the models with augmented training data……. analyzing extracted dialog policies from the RL model, sometimes patterns that are against the desired outcome may be observed. For examples where the model predictions are apparently against common sense, the embodiments disclosed herein aim to refine the models to produce the desired prediction. This is achieved by leveraging a user simulator disclosed herein to generate simulated user dialogs based on the patterns where the unrefined RL model may predict wrongly; column 23, lines 21-26, wherein explain reinforcement-learning models by extracting a set of representative dialog policy rules with sequential pattern mining, (2) compare the rules learned from reinforcement learning and other supervised models, and finally (3) refine the reinforcement-learning models by incorporating human knowledge through user simulation; and column 27, lines 33-39, wherein extract patterns from the reinforcement-learning model may be an effective solution for addressing the technical challenge of improving the interpretability of a reinforcement-learning model, as human-understandable rules (e.g., dialog policies) may be further extracted based on the patterns, thereby resulting in improved interpretability).
Regarding claim 17, Liu in view of Hwang discloses the method according to claim 7, wherein the predefined or configurable number of the top-ranked explanations are learned and updated (see Liu, column 28, lines 39-45, wherein defined as decision sequences D such that count is ranked among top n. By comparing the common patterns extracted from the RL and GBDT models, the embodiments disclosed herein gain insights into how differently these models behave and allow using human knowledge to further improve the models with augmented training data).
Liu et al. fails to explicitly disclose by the communication adapter based on feedback from the operator environment.
Analogous art Hwang discloses updated by the communication adapter based on feedback from the operator environment (see Hwang, para [0045], wherein the monitoring component 104 can monitor one or more events associated with a server, middleware, an application and/or another component of the artificial intelligence system……; para [0063], wherein the data analytics component 102 can perform a machine learning process 710 based on user feedback data 716 provided by a user device 708 and/or system status data 718 provided by an automation system 706. The user device 708 can be configured to interact with a user (e.g., a user identity) and/or generate the user feedback data 716; and para [0058], wherein verify that dependent systems within the artificial intelligence system have a corresponding interface and/or trigger a change request to update the dependent systems. For example, uniformity of data and/or updates among dependent systems within the artificial intelligence system can be verified).
One of ordinary skill in the art would have recognized that applying the known technique of Hwang would have yielded predictable results and resulted in an improved system for the same reasons as stated above with respect to claim 1.
Regarding claim 19, Liu in view of Hwang discloses the communication adapter according to claim 15, further comprising memory configured to store, for the one or more operator systems of the operator environment, an indication of which input the one or more operator systems prefer (see Liu, column 48, lines 34-35, wherein memory for storing instructions for processor 1302 to execute or data for processor 1302; column 9, lines 53-54, wherein user-interface module, user-profile store, connection store, third-party content store; and column 23, lines 44-46, wherein the plurality of state-indications may comprise one or more user-intents, one or more system actions, and one or more user actions),
wherein the communication adapter is further configured to modify the received predictions and/or the associated explanations based on the indication (see Liu, column 28, lines 60-64, wherein gain explanatory insights into the model behaviors, and potentially identify how the models may be improved……the model predictions are apparently against common sense, the embodiments disclosed herein aim to refine the models to produce the desired prediction.…..; and column 23, lines 44-46, wherein the plurality of state-indications may comprise one or more user-intents, one or more system actions, and one or more user actions).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US Pat No. 11,715,042) (hereinafter Liu et al.) in view of Hwang et al. (US Pub No. 2021/0027136) (hereinafter Hwang et al.), and further in view of Lawrence, C., Sztyler, T. & Niepert, M. et al. (Explaining Neural Matrix Factorization with Gradient Rollback) arXiv:2010.05516 [cs, stat] (2020).
Regarding claim 16, Liu in view of Hwang discloses the system according to claim 14, wherein the explainable AI mechanism, as set forth above with claim 14.
Liu et al. and Hwang et al. combined fail to explicitly disclose a gradient rollback mechanism.
Analogous art Lawrence discloses a gradient rollback mechanism (see Lawrence, abstract, wherein gradient rollback provides faithful explanations for knowledge base completion and recommender datasets).
Liu directed to a system for training a target machine-learning model iteratively by accessing training data of content objects. Lawrence directed to explaining the predictions of neural black-box models. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Liu, regarding the Interpretability of Deep Reinforcement Learning Models In Assistant Systems, to have included a gradient rollback mechanism because both inventions teach improving processing efficiency. Further, the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US Pat No. 11,715,042) (hereinafter Liu et al.) in view of Hwang et al. (US Pub No. 2021/0027136) (hereinafter Hwang et al.), and further in view of Dhesi et al. (US Pub No. 2021/0019694) (hereinafter Dhesi et al.).
Regarding claim 18, Liu in view of Hwang discloses the method according to claim 1.
Liu et al. fails to explicitly disclose updating the communication adapter based on feedback from the operator environment.
Analogous art Hwang discloses updating the communication adapter based on feedback from the operator environment (see Hwang, abstract, wherein feedback loop learning between artificial intelligence systems are provided. In one example, a system includes a monitoring component and a machine learning component; para [0045], wherein the monitoring component 104 can monitor one or more events associated with a server, middleware, an application and/or another component of the artificial intelligence system……; para [0063], wherein the data analytics component 102 can perform a machine learning process 710 based on user feedback data 716 provided by a user device 708 and/or system status data 718 provided by an automation system 706. The user device 708 can be configured to interact with a user (e.g., a user identity) and/or generate the user feedback data 716; and para [0058], wherein verify that dependent systems within the artificial intelligence system have a corresponding interface and/or trigger a change request to update the dependent systems. For example, uniformity of data and/or updates among dependent systems within the artificial intelligence system can be verified).
One of ordinary skill in the art would have recognized that applying the known technique of Hwang would have yielded predictable results and resulted in an improved system for the same reasons as stated above with respect to claim 1.
Liu et al. and Hwang et al. combined fail to explicitly disclose quantifying how helpful the modified predictions and associated explanations were for the one or more operator systems, the feedback comprising a numerical score within a predefined range.
Analogous art Dhesi discloses quantifying how helpful the modified predictions and associated explanations were for the one or more operator systems, the feedback comprising a numerical score within a predefined range (see Dhesi, para [0091], wherein aggregate, and/or otherwise analyzed or processed, and may be applied to the calculation of the client satisfaction score so as to either increase or decrease the likelihood that a particular merchant 32 will be allocated to a client's order in the future based on if the rating is positive or negative; paras [0025]-[0029], wherein
the optimization of merchant-client relationships may incorporate feedback, implicit or explicit, received from customers about merchants from which they have received orders. The feedback data may take the form of starred ratings, numerical ratings, one or more survey fields, or any other structured value. Additionally or alternatively, the feedback may take the form of unstructured data such as freeform text reviews that are analyzed by any known natural language processor to obtain feedback on the client's experience, and the party (e.g., restaurant, courier, retailer, etc.) to whom the feedback is directed. Client feedback can be used by the fulfillment engine to optimize future orders from the client. In some embodiments, the feedback may be used to make changes to existing orders that have not been delivered…..may generate scores (e.g., numerical scores) based on one or more other types of client feedback (e.g., written feedback, survey, thumbs up/down, likes, indications of whether the client would order again, etc.)).
Liu directed to a system for training a target machine-learning model iteratively by accessing training data of content objects. Dhesi directed to an optimal allocation of client orders. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Liu, regarding the Interpretability of Deep Reinforcement Learning Models In Assistant Systems, to have included quantifying how helpful the modified predictions and associated explanations were for the one or more operator systems, the feedback comprising a numerical score within a predefined range because both inventions teach improving processing efficiency. Further, the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAFIZ A KASSIM whose telephone number is (571)272-8534. The examiner can normally be reached 9:00 - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on 571-272-6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAFIZ A KASSIM/Primary Examiner, Art Unit 3623 10/14/2026
