DETAILED ACTION
This action is responsive to the Application/amendment filed on 01/31/2024. Claims 1-20 are pending in the case. Claims 1, 10, and 18 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 01/31/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 are rejected under 35 U.S.C. 101 as the claims are directed towards judicially recognized exception(s) without significantly more.
Regarding claim 1:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites generate a user state representation including an implicit user state representation and an explicit user state representation based on the set of features and session data for at least one session associated with the user which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation and judgement to observe user/session facts and inferring values based on data associated with a user. See 2106.04.(a)(2).III.C.
The claim recites generate an action reward value for each of a plurality of candidate actions based on the user state representation which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation to select a value based on data. See 2106.04.(a)(2).III.C.
Subject Matter Eligibility Analysis Step 2A Prong 2:
a non-transitory memory recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f))
a processor communicatively coupled to the non-transitory memory wherein the processor is configured to read a set of instructions recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f))
receive a request for an interface including a set of features representative of a user associated with the request which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g)
generate an interface including at least one interface element representative of a candidate action having a highest action reward value recites creating an interface to display candidate actions based on results is Insignificant Extra-Solution Activity (see MPEP §2106.05(g)). Alternatively, a limitation based on “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) and (b) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional element (c) of obtaining a network input is well understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
Additional element (d) recites a well understood and conventional practice recites creating an interface to display candidate actions based on results quoted from Deep Reinforcement Learning for Page-wise Recommendations (Page 2, Col. 1, Paragraph 2, “Conventional RL methods could recommend a set of items each time, for instance, DQN can recommend a set of items with highest Q-values”). Alternatively, a limitation based on “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016) can recite a mental process.
The additional element(s) (a) (b) (c) and (d) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 2:
The rejection of claim 1 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim does not contain elements that would warrant a Step 2A Prong 1 analysis.
Subject Matter Eligibility Analysis Step 2A Prong 2:
wherein the user state representation is generated by a trained personalized representation model recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f))
including a first portion configured to generate the implicit user state representation and a second portion configured to generate the explicit user state representation specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional elements (b) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely links the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)).
The additional element(s) (a) and (b) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 3:
The rejection of claim 2 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim does not contain elements that would warrant a Step 2A Prong 1 analysis.
Subject Matter Eligibility Analysis Step 2A Prong 2:
wherein the first portion of the trained personalized representation model comprises a reinforced coupled recurrent network specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely links the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)).
The additional element(s) (a) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 4:
The rejection of claim 3 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim does not contain elements that would warrant a Step 2A Prong 1 analysis.
Subject Matter Eligibility Analysis Step 2A Prong 2:
wherein the reinforced coupled recurrent network comprises a plurality of coupled recurrent units including a plurality of gates specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely links the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)).
The additional element(s) (a in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 5:
The rejection of claim 2 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim does not contain elements that would warrant a Step 2A Prong 1 analysis.
Subject Matter Eligibility Analysis Step 2A Prong 2:
wherein the second portion of the trained personalized representation model comprises at least one fully-connected network specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely links the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)).
The additional element(s) (a) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 6:
The rejection of claim 1 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim does not contain elements that would warrant a Step 2A Prong 1 analysis.
Subject Matter Eligibility Analysis Step 2A Prong 2:
wherein the action reward value for each of the plurality of candidate actions is generated based on residual network framework recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2B:
Additional elements (a) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
The additional element(s) (a) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 7:
The rejection of claim 6 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites wherein the action reward value for each of the plurality of candidate actions is generated based on a difference between an actual action taken by a user and a predicted action based on the residual network framework which is an abstract idea (Mathematical Calculations (see MPEP 2106.04(a)(2)(I)(C))).
Subject Matter Eligibility Analysis Step 2A Prong 2:
The claim does not contain elements that would warrant a Step 2A Prong 2 analysis.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include any additional element, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 8:
The rejection of claim 1 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites wherein the action reward value for each of the plurality of candidate actions includes a lifetime value factor and a loss value which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation to select a value based on data with further clarification on value. See 2106.04.(a)(2).III.C.
Subject Matter Eligibility Analysis Step 2A Prong 2:
The claim does not contain elements that would warrant a Step 2A Prong 2 analysis.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include any additional element, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 9:
The rejection of claim 1 is incorporated and further claim recites further additional
elements/limitations:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites wherein user state representation is generated based on a user representation including a triplet comprising historical user features, a current action, and a current response which, under the broadest reasonable interpretation, covers performance of the limitation in the mind with physical aid. The limitations encompasses using evaluation and judgement to observe user/session facts and inferring values in the form of a tuple based on data associated with a user. See 2106.04.(a)(2).III.C.
Subject Matter Eligibility Analysis Step 2A Prong 2:
The claim does not contain elements that would warrant a Step 2A Prong 2 analysis.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include any additional element, when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites generating a user state representation including an implicit user state representation and an explicit user state representation based on the set of features and a user representation including a triplet comprising historical user features, a current action, and a current response which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation and judgement to observe user/session facts and inferring values in the form of a tuple based on data associated with a user. See 2106.04.(a)(2).III.C.
The claim recites generating an action reward value for each of a plurality of candidate actions based on the user state representation which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation to select a value based on data. See 2106.04.(a)(2).III.C.
Subject Matter Eligibility Analysis Step 2A Prong 2:
receiving a request for an interface including a set of features representative of a user associated with the request which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g)
generating an interface including at least one interface element representative of a candidate action having a highest action reward value recites creating an interface to display candidate actions based on results is Insignificant Extra-Solution Activity (see MPEP §2106.05(g)). Alternatively, a limitation based on “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).
Subject Matter Eligibility Analysis Step 2B:
Additional element (a) of obtaining a network input is well understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
Additional element (b) recites a well understood and conventional practice recites creating an interface to display candidate actions based on results quoted from Deep Reinforcement Learning for Page-wise Recommendations (Page 2, Col. 1, Paragraph 2, “Conventional RL methods could recommend a set of items each time, for instance, DQN can recommend a set of items with highest Q-values”). Alternatively, a limitation based on “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016) can recite a mental process.
The additional element(s) (a) and (b) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 11:
The rejection of claim 10 is incorporated and further claim recites further additional elements/limitations:
Claim 11 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 2 found in claim 11.
Regarding claims 12:
The rejection of claim 11 is incorporated and further claim recites further additional elements/limitations:
Claim 12 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 3 found in claim 12.
Regarding claim 13:
The rejection of claim 12 is incorporated and further claim recites further additional elements/limitations:
Claim 13 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 4 found in claim 13.
Regarding claim 14:
The rejection of claim 11 is incorporated and further claim recites further additional elements/limitations:
Claim 14 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 5 found in claim 14.
Regarding claim 15:
The rejection of claim 10 is incorporated and further claim recites further additional elements/limitations:
Claim 15 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 6 found in claim 15.
Regarding claim 16:
The rejection of claim 15 is incorporated and further claim recites further additional elements/limitations:
Claim 16 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 7 found in claim 16.
Regarding claim 17:
The rejection of claim 10 is incorporated and further claim recites further additional elements/limitations:
Claim 17 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 8 found in claim 17.
Regarding claim 18:
Subject Matter Eligibility Analysis Step 2A Prong 1:
The claim recites generating a user state representation including an implicit user state representation and an explicit user state representation based on the set of features and session data for at least one session associated with the user which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation and judgement to observe user/session facts and inferring values based on data associated with a user. See 2106.04.(a)(2).III.C.
The claim recites generating an action reward value for each of a plurality of candidate actions based on the user state representation which, under the broadest reasonable interpretation, covers performance of the limitation in the mind. The limitations encompasses using evaluation to select a value based on data. See 2106.04.(a)(2).III.C.
Subject Matter Eligibility Analysis Step 2A Prong 2:
receiving a request for an interface including a set of features representative of a user associated with the request which amount to mere extra solution activity of obtaining and/or gathering data over a network, see MPEP §2106.05(g)
wherein the user state representation is generated by a trained personalized representation model recites a generic computer on which to perform the abstract idea, e.g. "apply it on a computer" (see MPEP 2106.05(f))
including a first portion configured to generate the implicit user state representation and a second portion configured to generate the explicit user state representation specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
wherein the first portion of the trained personalized representation model comprises a reinforced coupled recurrent network specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
wherein the second portion of the trained personalized representation model comprises at least one fully-connected network specifies linking the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)))
generating an interface including at least one interface element representative of a candidate action having a highest action reward value recites creating an interface to display candidate actions based on results is Insignificant Extra-Solution Activity (see MPEP §2106.05(g)). Alternatively, a limitation based on “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016).
Subject Matter Eligibility Analysis Step 2B:
Additional element (a) of obtaining a network input is well understood, routine, and conventional activity of “transmitting or receiving data over a network" (see MPEP 2106.05(d)(II)(i) using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362)
Additional elements (b) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation amount to no more than mere instructions to apply the exception using a generic computer component. Please see MPEP §2106.05(f).
Additional elements (c) (d) and (e) do not integrate the abstract idea into a practical application nor do the additional limitation provide significantly more than the abstract idea because the limitation merely links the use of a judicial exception to a particular technological environment or field of use(see MPEP 2106.05(h)).
Additional element (f) recites a well understood and conventional practice recites creating an interface to display candidate actions based on results quoted from Deep Reinforcement Learning for Page-wise Recommendations (Page 2, Col. 1, Paragraph 2, “Conventional RL methods could recommend a set of items each time, for instance, DQN can recommend a set of items with highest Q-values”). Alternatively, a limitation based on “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016) can recite a mental process.
The additional element(s) (a) (b) (c) (d) (e) and (f) in the claim do/does not include any additional elements , when considered separately and in combination, that amount to an integration of the judicial exception into a practical application, nor significantly more than the judicial exception for the reasons set forth in step 2A prong 2 analysis above. The claim is not patent eligible.
Regarding claim 19:
The rejection of claim 18 is incorporated and further claim recites further additional elements/limitations:
Claim 19 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 6 found in claim 19.
Regarding claim 20:
The rejection of claim 18 is incorporated and further claim recites further additional elements/limitations:
Claim 20 is rejected under that same 101 claim analysis due to the substantially similarity of the limitations and additional elements of claim 8 found in claim 20.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 6, 8-13, 15 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al(“Personalized next-best action recommendation with multi-party interaction learning for automated decision-making” henceforth known as Cao) in view of Bandyopadhyay et al(US20250272115A1, “Dynamic personalized banking user interface”, henceforth known as Bandyopadhyay).
Regarding claim 1:
Cao discloses generate a user state representation including an implicit user state representation and an explicit user state representation (Cao, Page 8, Figure 3, “simp indicates the learned data-driven implicit features, sexp refers to the transformed domain-driven explicit features, st is the resultant state vector representation for the client c” where the client’s implicit features(simp) and explicit features(sexp) feeding into the resultant client state representation st corresponds to generating a user state representation) based on the set of features and session data for at least one session associated with the user(Cao, Page 8-9, Paragraph 3, “We learn the personalized client representation using a coupled recurrent network (CRN, Fig 3). Given a client tuple Ct =< Dt, At-1, Ot > the decision action ai ∈ At−1 and the set of client responses Ot,i ∈ Ot at each prior time step i are sequentially fed into the CRN. Initially, the client response’s hidden state is extracted by a fully connected network from the client’s relatively stable personal information. An embedding layer transforms actions described by categorical values (e.g., sending a message to a debtor) to numerical vectors. CRN embeds the client behaviors and personal information as a vector simp, which describes the hidden state of each client at time t in terms of a data-driven implicit feature since simp is purely generated based on the client’s observable data and its characteristics by the deep network” where the implicit features used to generate the client state representation being based on actions of the client corresponds with generating a user state representation… based on the set of features and session data for at least one session associated with the user as client actions and behaviors are captured as a time-ordered interaction sequence represented as Ct =< Dt, At-1, Ot > with Ot being the sequence of the client’s responses/behaviors)
Cao discloses generate an action reward value for each of a plurality of candidate actions based on the user state representation(Cao, Page 12, Paragraph 2, “Given a client state representation, it efficiently predicts the reward values for different actions” where predicting the reward values for different actions of a given client state representation corresponds to generating an action reward value for each of a plurality of candidate actions based on the user state representation)
Cao does not explicitly disclose, however Bandyopadhyay discloses thee following limitations:
a non-transitory memory; a processor communicatively coupled to the non-transitory memory, wherein the processor is configured to read a set of instructions
receive a request for an interface including a set of features representative of a user associated with the request
generate an interface including at least one interface element representative of a candidate action having a highest action reward value
Bandyopadhyay discloses a non-transitory memory; a processor communicatively coupled to the non-transitory memory, wherein the processor is configured to read a set of instructions(Bandyopadhyay, [0046]-[0048], “Modules are tangible entities (e.g., hardware) capable of performing specified operations when operating. A module includes hardware. In an example, the hardware may be specifically configured to carry out a specific operation (e.g., hardwired)…In an example…a computer readable medium containing instructions, where the instructions configure the execution units to carry out a specific operation when in operation… Machine (e.g., computer system) 600 may include a hardware processor 602 (e.g., a central processing unit (CPU),”)
Bandyopadhyay discloses receive a request for an interface including a set of features representative of a user associated with the request(Bandyopadhyay, [0054], “receiving data corresponding to a user pathway interaction by a user at a user interface; personalizing, using reinforcement learning, a trained model to the user based on the data to generate a personalized reinforcement learning model; receiving an indication that the user has accessed the user interface or requested access to the user interface; dynamically generating the user interface using the personalized reinforcement learning model including at least one of replacing a user interface component” where receiving an indication the user has requested access to a user interface corresponds to receive a request for an interface including a set of features representative of a user associated with the request as the interface is requested with user data and the interface is personalized for the user using a reinforcement model to change the interface)
Bandyopadhyay discloses generate an interface including at least one interface element representative of a candidate action having a highest action reward value(Bandyopadhyay, [0023], “A model (e.g., a machine learning model) may be used to generate a personalized user interface based on user interactions...The reinforcement learning model may use an input to be customized to a user…The reinforcement learning model may use a reward in training, such as minimizing clicks (e.g., fewer clicks is a higher reward)” where generating a personalized user interface based on fewest clicks and where the fewest clicks are the highest reward corresponds to generating an interface including one interface element representative of a candidate action having a highest action reward value(See also Bandyopadhyay, [0054], “…dynamically generating the user interface using the personalized reinforcement learning model including at least one of replacing a user interface component…and outputting the dynamically generated user interface for display on a user device”))
Regarding claim 2:
The rejection of claim 1 with Cao-Bandyopadhyay prior art is incorporated and further:
Cao further discloses wherein the user state representation is generated by a trained personalized representation model including a first portion configured to generate the implicit user state representation and a second portion configured to generate the explicit user state representation(Cao, Page 8, Figure 3, where the top portion of Figure 3 generates an implicit user state representation and the bottom portion of Figure 3 generates an explicit user state representation. See the below annotation for clarity)
PNG
media_image1.png
635
1289
media_image1.png
Greyscale
Regarding claim 3:
The rejection of claim 2 with Cao-Bandyopadhyay prior art is incorporated and further:
Cao further discloses wherein the first portion of the trained personalized representation model comprises a reinforced coupled recurrent network(Cao, Page 17, Paragraph 4, “The proposed reinforced coupled recurrent network (CRN) provides a general neural multi-sequence interaction learning solution”)
Regarding claim 4:
The rejection of claim 3 with Cao-Bandyopadhyay prior art is incorporated and further:
Cao further discloses wherein the reinforced coupled recurrent network comprises a plurality of coupled recurrent units including a plurality of gates(Cao, Page 18, Paragraph 1, “The CRN incorporated with coupled recurrent units (CRU)”)
Regarding claim 6:
The rejection of claim 1 with Cao-Bandyopadhyay prior art is incorporated and further:
Cao further discloses wherein the action reward value for each of the plurality of candidate actions is generated based on residual network framework(Cao, Page 11, Paragraph 2, “We further measure the reward of each decision action on a client state using a reward prediction module (Fig 6), which is built on a residual network”)
Regarding claim 8:
The rejection of claim 1 with Cao-Bandyopadhyay prior art is incorporated and further:
Cao further discloses wherein the action reward value for each of the plurality of candidate actions includes a lifetime value factor and a loss value(Cao, Page 14, Paragraph 4, “…To reflect action imbalance in the loss function…the loss value on the client is multiplied by wic for backward gradient propagation” where wic corresponds to a lifetime value factor as it is a weighting factor based on the frequency of an action over the lifetime of the historical data and multiplying the loss value on the client by wic corresponds to the action reward value for each of the plurality of candidate actions includes a lifetime value factor and a loss value as the resulting loss value and weighting factor being multiplied for back gradient propagation is the action reward value as it changes how the example’s error influences learning for that action)
Regarding claim 9:
The rejection of claim 1 with Cao-Bandyopadhyay prior art is incorporated and further:
Cao further discloses wherein user state representation is generated based on a user representation including a triplet(Cao, Page 5, Paragraph 3, “Without loss of generality, we assume a client cover time t can be described by a three-element tuple Ct =< Dt, At-1, Ot >” where the tuple Ct corresponds to a triplet) comprising historical user features, a current action, and a current response(Cao, Page 5, Paragraph 3, “Without loss of generality, we assume a client cover time t can be described by a three-element tuple Ct =< Dt, At-1, Ot >…where Dt…refers to a set of client’s relatively stable information… At−1…refers to a sequence of t−1 past actions and Ot…refers to a sequence of client responses… Further, after taking action ai, a reward value.. measures the effectiveness of ai on the client’s next responses Ot,i+1.” where Dt being stable information such as demographics are historical/profile user features at time t corresponds to the historical user features, At-1 being the sequence of past actions to time t-1 with ai being decision actions corresponds to current action and Ot being client responses with client’s next responses Ot,i+1 corresponds to client response)
Regarding claim 10:
Cao discloses generating a user state representation including an implicit user state representation and an explicit user state representation(Cao, Page 8, Figure 3, “simp indicates the learned data-driven implicit features, sexp refers to the transformed domain-driven explicit features, st is the resultant state vector representation for the client c” where the client’s implicit features(simp) and explicit features(sexp) feeding into the resultant client state representation st corresponds to generating a user state representation) based on the set of features and a user representation(Cao, Page 8-9, Paragraph 3, “We learn the personalized client representation using a coupled recurrent network (CRN, Fig 3). Given a client tuple Ct =< Dt, At-1, Ot > the decision action ai ∈ At−1 and the set of client responses Ot,i ∈ Ot at each prior time step i are sequentially fed into the CRN. Initially, the client response’s hidden state is extracted by a fully connected network from the client’s relatively stable personal information. An embedding layer transforms actions described by categorical values (e.g., sending a message to a debtor) to numerical vectors. CRN embeds the client behaviors and personal information as a vector simp, which describes the hidden state of each client at time t in terms of a data-driven implicit feature since simp is purely generated based on the client’s observable data and its characteristics by the deep network” where the implicit features used to generate the client state representation being based on actions of the client corresponds with generating a user state representation… based on the set of features as client actions and behaviors are captured as a time-ordered interaction sequence represented as Ct =< Dt, At-1, Ot > with Ot being the sequence of the client’s responses/behaviors) including a triplet comprising historical user features, a current action, and a current response comprising historical user features, a current action, and a current response(Cao, Page 5, Paragraph 3, “Without loss of generality, we assume a client cover time t can be described by a three-element tuple Ct =< Dt, At-1, Ot >…where Dt…refers to a set of client’s relatively stable information… At−1…refers to a sequence of t−1 past actions and Ot…refers to a sequence of client responses… Further, after taking action ai, a reward value.. measures the effectiveness of ai on the client’s next responses Ot,i+1.” where Dt being stable information such as demographics are historical/profile user features at time t corresponds to the historical user features, At-1 being the sequence of past actions to time t-1 with ai being decision actions corresponds to current action and Ot being client responses with client’s next responses Ot,i+1 corresponds to client response)
Cao discloses generating an action reward value for each of a plurality of candidate actions based on the user state representation(Cao, Page 12, Paragraph 2, “Given a client state representation, it efficiently predicts the reward values for different actions” where predicting the reward values for different actions of a given client state representation corresponds to generating an action reward value for each of a plurality of candidate actions based on the user state representation)
Cao does not explicitly disclose the following:
receiving a request for an interface including a set of features representative of a user associated with the request
generating an interface including at least one interface element representative of a candidate action having a highest action reward value
Bandyopadhyay discloses receiving a request for an interface including a set of features representative of a user associated with the request(Bandyopadhyay, [0054], “receiving data corresponding to a user pathway interaction by a user at a user interface; personalizing, using reinforcement learning, a trained model to the user based on the data to generate a personalized reinforcement learning model; receiving an indication that the user has accessed the user interface or requested access to the user interface; dynamically generating the user interface using the personalized reinforcement learning model including at least one of replacing a user interface component” where receiving an indication the user has requested access to a user interface corresponds to receive a request for an interface including a set of features representative of a user associated with the request as the interface is requested with user data and the interface is personalized for the user using a reinforcement model to change the interface)
Bandyopadhyay discloses generating an interface including at least one interface element representative of a candidate action having a highest action reward value (Bandyopadhyay, [0023], “A model (e.g., a machine learning model) may be used to generate a personalized user interface based on user interactions...The reinforcement learning model may use an input to be customized to a user…The reinforcement learning model may use a reward in training, such as minimizing clicks (e.g., fewer clicks is a higher reward)” where generating a personalized user interface based on fewest clicks and where the fewest clicks are the highest reward corresponds to generating an interface including one interface element representative of a candidate action having a highest action reward value(See also Bandyopadhyay, [0054], “…dynamically generating the user interface using the personalized reinforcement learning model including at least one of replacing a user interface component…and outputting the dynamically generated user interface for display on a user device”))
Regarding claim 11:
The rejection of claim 10 is incorporated in claim 11 with the claim also being rejected under the same rationale as set forth in the rejection of claim 2.
Regarding claims 12:
The rejection of claim 11 is incorporated in claim 12 with the claim also being rejected under the same rationale as set forth in the rejection of claim 3.
Regarding claim 13:
The rejection of claim 12 is incorporated in claim 13 with the claim also being rejected under the same rationale as set forth in the rejection of claim 4.
Regarding claim 15:
The rejection of claim 10 is incorporated in claim 15 with the claim also being rejected under the same rationale as set forth in the rejection of claim 6.
Regarding claim 17:
The rejection of claim 10 is incorporated in claim 17 with the claim also being rejected under the same rationale as set forth in the rejection of claim 8.
Claim(s) 5, 14 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al(“Personalized next-best action recommendation with multi-party interaction learning for automated decision-making” henceforth known as Cao) in view of Bandyopadhyay et al(US20250272115A1, “Dynamic personalized banking user interface”, henceforth known as Bandyopadhyay) and in further view of Shi et al(“Efficient Tree Policy with Attention-Based State Representation for Interactive Recommendation” henceforth known as Shi).
Regarding claim 5:
The rejection of claim 2 with Cao-Bandyopadhyay prior art is incorporated and further
Shi discloses wherein the second portion of the trained personalized representation model comprises at least one fully-connected network (Shi, Page 5, Figure 2 and Paragraph 3, “In the feature embedding layer, the input user features and item features are encoded by a fully connected layer to obtain the feature embeddings” where the user features being encoded generates a user state representation with explicit user features and feature embeddings using fully connected layers )
References Cao-Bandyopadhyay and Shi are analogous art because they are from the same field of endeavor of using reinforcement learning that selects action/items for users/clients over time.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Cao-Bandyopadhyay and Shi before him or her, to modify the explicit extraction of Cao-Bandyopadhyay to include the fully connected layers of Shi because as fully connected layers help convert inputs into a comparable feature embedding for AI handling in the next layer. The suggestion/motivation for doing so would have been Shi, Page 5, Paragraph 3, “The input layer receives the feature vectors from items and the users’ feedback, and then preprocessed them by a fully connected layer.”
Regarding claim 14:
The rejection of claim 11 is incorporated in claim 14 with the claim also being rejected under the same rationale as set forth in the rejection of claim 5.
Regarding claim 18:
Cao discloses generating a user state representation including an implicit user state representation and an explicit user state representation(Cao, Page 8, Figure 3, “simp indicates the learned data-driven implicit features, sexp refers to the transformed domain-driven explicit features, st is the resultant state vector representation for the client c” where the client’s implicit features(simp) and explicit features(sexp) feeding into the resultant client state representation st corresponds to generating a user state representation) based on the set of features and session data for at least one session associated with the user (Cao, Page 8-9, Paragraph 3, “We learn the personalized client representation using a coupled recurrent network (CRN, Fig 3). Given a client tuple Ct =< Dt, At-1, Ot > the decision action ai ∈ At−1 and the set of client responses Ot,i ∈ Ot at each prior time step i are sequentially fed into the CRN. Initially, the client response’s hidden state is extracted by a fully connected network from the client’s relatively stable personal information. An embedding layer transforms actions described by categorical values (e.g., sending a message to a debtor) to numerical vectors. CRN embeds the client behaviors and personal information as a vector simp, which describes the hidden state of each client at time t in terms of a data-driven implicit feature since simp is purely generated based on the client’s observable data and its characteristics by the deep network” where the implicit features used to generate the client state representation being based on actions of the client corresponds with generating a user state representation… based on the set of features and session data for at least one session associated with the user as client actions and behaviors are captured as a time-ordered interaction sequence represented as Ct =< Dt, At-1, Ot > with Ot being the sequence of the client’s responses/behaviors) wherein the user state representation is generated by a trained personalized representation model including a first portion configured to generate the implicit user state representation and a second portion configured to generate the explicit user state representation (Cao, Page 8, Figure 3, where the top portion of Figure 3 generates an implicit user state representation and the bottom portion of Figure 3 generates an explicit user state representation. See the below annotation for clarity)
PNG
media_image1.png
635
1289
media_image1.png
Greyscale
Cao discloses wherein the first portion of the trained personalized representation model comprises a reinforced coupled recurrent network (Cao, Page 17, Paragraph 4, “The proposed reinforced coupled recurrent network (CRN) provides a general neural multi-sequence interaction learning solution”)
Cao discloses generating an action reward value for each of a plurality of candidate actions based on the user state representation(Cao, Page 12, Paragraph 2, “Given a client state representation, it efficiently predicts the reward values for different actions” where predicting the reward values for different actions of a given client state representation corresponds to generating an action reward value for each of a plurality of candidate actions based on the user state representation)
Cao does not explicitly disclose the following limitations:
receiving a request for an interface including a set of features representative of a user associated with the request
generating an interface including at least one interface element representative of a candidate action having a highest action reward value
wherein the second portion of the trained personalized representation model comprises at least one fully-connected network
Bandyopadhyay discloses receive a request for an interface including a set of features representative of a user associated with the request(Bandyopadhyay, [0054], “receiving data corresponding to a user pathway interaction by a user at a user interface; personalizing, using reinforcement learning, a trained model to the user based on the data to generate a personalized reinforcement learning model; receiving an indication that the user has accessed the user interface or requested access to the user interface; dynamically generating the user interface using the personalized reinforcement learning model including at least one of replacing a user interface component” where receiving an indication the user has requested access to a user interface corresponds to receive a request for an interface including a set of features representative of a user associated with the request as the interface is requested with user data and the interface is personalized for the user using a reinforcement model to change the interface)
Bandyopadhyay discloses generate an interface including at least one interface element representative of a candidate action having a highest action reward value(Bandyopadhyay, [0023], “A model (e.g., a machine learning model) may be used to generate a personalized user interface based on user interactions...The reinforcement learning model may use an input to be customized to a user…The reinforcement learning model may use a reward in training, such as minimizing clicks (e.g., fewer clicks is a higher reward)” where generating a personalized user interface based on fewest clicks and where the fewest clicks are the highest reward corresponds to generating an interface including one interface element representative of a candidate action having a highest action reward value(See also Bandyopadhyay, [0054], “…dynamically generating the user interface using the personalized reinforcement learning model including at least one of replacing a user interface component…and outputting the dynamically generated user interface for display on a user device”))
Cao-Bandyopadhyay does not explicitly disclose the following limitations:
Shi discloses wherein the second portion of the trained personalized representation model comprises at least one fully-connected network(Shi, Page 5, Figure 2 and Paragraph 3, “In the feature embedding layer, the input user features and item features are encoded by a fully connected layer to obtain the feature embeddings” where the user features being encoded generates a user state representation with explicit user features and feature embeddings using fully connected layers)
References Cao-Bandyopadhyay and Shi are analogous art because they are from the same field of endeavor of using reinforcement learning that selects action/items for users/clients over time.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Cao-Bandyopadhyay and Shi before him or her, to modify the explicit extraction of Cao-Bandyopadhyay to include the fully connected layers of Shi because as fully connected layers help convert inputs into a comparable feature embedding for AI handling in the next layer. The suggestion/motivation for doing so would have been Shi, Page 5, Paragraph 3, “The input layer receives the feature vectors from items and the users’ feedback, and then preprocessed them by a fully connected layer.”
Regarding claim 19:
The rejection of claim 18 is incorporated in claim 19 with the claim also being rejected under the same rationale as set forth in the rejection of claim 6.
Regarding claim 20:
The rejection of claim 18 is incorporated in claim 20 with the claim also being rejected under the same rationale as set forth in the rejection of claim 8.
Claim(s) 7 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao et al(“Personalized next-best action recommendation with multi-party interaction learning for automated decision-making” henceforth known as Cao) in view of Bandyopadhyay et al(US20250272115A1, “Dynamic personalized banking user interface”, henceforth known as Bandyopadhyay) and in further view of Pathakota et al(“DCT: Dual Channel Training of Action Embeddings for Reinforcement Learning with Large Discrete Action Spaces” henceforth known as Pathakota).
Regarding claim 7:
The rejection of claim 6 with Cao-Bandyopadhyay prior art is incorporated and further:
Pathakota discloses wherein the action reward value for each of the plurality of candidate actions is generated based on a difference between an actual action taken by a user and a predicted action based on the residual network framework(Pathakota, Page 12, Paragraph 5, “The reward is determined by the cosine similarity between the predicted action (DDPG output) and the actual purchased item embedding vector obtained from the encoder”)
References Cao-Bandyopadhyay and Pathakota are analogous art because they are from the same field of endeavor of using reinforcement learning for action selection in recommendation systems and learned representations of actions/states
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Cao-Bandyopadhyay and Pathakota before him or her, to modify the reward calculation of Cao-Bandyopadhyay to include the difference of actions taken and predicted as stated in Pathakota because to avoid overlapping embeddings and preserve next-state/action-effect information. The suggestion/motivation for doing so would have been Pathakota, Page 4, Paragaphs 3-4, “Given the lack of a loss based on the distinction between embeddings of similar actions, there is a good chance of overlap between action embeddings…We assume that actions a1 and a3 correspond to similar transitions (perhaps in different parts of the state space) while a2 is very distinct…the regions for a1 and a3 could overlap (there is no intrinsic incentive for distinct embeddings), leading to difficulty during decoding. Unless they have some separation boundary, the decoder f cannot map those continuous values to the original discrete action space.”
Regarding claim 16:
The rejection of claim 15 is incorporated in claim 16 with the claim also being rejected under the same rationale as set forth in the rejection of claim 7.
Relevant Art:
While not used in the rejection the following are was found to be relevant:
Xiangyu Zhao, “Deep Reinforcement Learning for List-wise Recommendations” as it discusses Markov Decision Process with deep reinforcement learning for recommendation with tuples/triples
Xiangyu Zhao, “Deep Reinforcement Learning for Page-wise Recommendations” as it discusses Markov Decision Process with deep reinforcement learning for recommendation with tuples/triples
Jan Peters, “Reinforcement Learning by Reward-weighted Regression for Operational Space Control” as it discusses reward-weighted regression with reinforcement learning
Yufei Feng, “Deep Session Interest Network for Click-Through Rate Prediction” as it discloses click through rate prediction with recommenders
Tianqi He, “DMBIN: A DualMulti-behavior Interest Network for Click-Through Rate Prediction via Contrastive Learning” as it discloses recommenders with implicit and explicit states
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES JEFFREY JONES JR whose telephone number is (703)756-1414. The examiner can normally be reached Monday - Friday 8:00 - 5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.J.J./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights