DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application filed on 07/23//2021. Claims 1-20 are pending in the application and have been considered below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
For Step 1, the claim is a method so it does recite a statutory category of invention.
For Step 2A, Prong 1:
The claim recites the limitation of “predicting, by the agent, an expected reward for each respective string of a plurality of strings associated with the first transaction file based on a policy of the agent, wherein the policy is determined based on a context comprising at least an attribute of the entity” The predicting imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the predicting step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “determining a first string based on a highest expected reward.” The determining imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the determining step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “updating the policy of the agent based on the response to the first string.” The updating imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the updating step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
For Step 2A, Prong 2, the claim recites additional elements:
; receiving a first transaction file associated with an entity; providing, to an environment, the first string; receiving a response to the first string, wherein the response comprises an actual reward
The recited "receiving a first transaction file associated with an entity” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The “providing, to an environment, the first string” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The recited "receiving a first transaction file associated with an entity” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B
Under the Subject Matter Eligibility, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B.
Here the "receiving a first transaction file associated with an entity” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Here the " providing (i.e. transmitting), to an environment, the first string” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Here the "receiving a first transaction file associated with an entity” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 2:
Claim 2, which incorporates the rejection of claim 1, recites further limitations such as “
generating, based on the policy of the agent, the plurality of strings associated with the first transaction file” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 3:
Claim 3, which incorporates the rejection of claim 1, recites further limitations such as
“updating the context based on the actual reward, wherein updating the policy of the agent based on the response to the first string comprises updating the policy of the agent based on the updated context” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 4:
Claim 4, which incorporates the rejection of claim 1, recites further limitations such as
“the expected reward for each string is based on a likelihood that a payment for a transaction associated with the first transaction file will be received” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 5:
Claim 5, which incorporates the rejection of claim 1, recites further limitations such as
“each string of the plurality of strings is associated with a topic of a plurality of
topics, and
the method further comprises ranking each respective topic in the plurality of topics
based on an expected reward of the string associated with the respective topic” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 6:
Claim 6, which incorporates the rejection of claim 1 recites further limitations such as
“each attribute of a set of attributes is based on transaction data related to at least
one of the entity, a payor associated with the first transaction file, or the first transaction
file, wherein the set of attributes includes the attribute of the entity, and the attribute of the entity comprises one of:
an amount of business of the entity;
a maximum amount of a transaction associated with the first transaction file;
a minimum amount of a transaction associated with the first transaction file;
an industry associated with the entity;
a location of the entity;
a time associated with the first transaction file;
a payment history of the payor associated with the first transaction file;
an email address of the payor associated with the entity;
a number of customers associated with the entity; or
a number of transaction files per customer associated with the entity” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 7:
Claim 7, which incorporates the rejection of claim 1 recites further limitations such as
“an attribute of a payor associated with the first transaction file” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 8:
For Step 1, the claim is a processing system so it does recite a statutory category of invention.
For Step 2A, Prong 1:
The claim recites the limitation of “predict, by the agent, an expected reward for each respective string of a plurality of strings associated with the first transaction file based on a policy of the agent, wherein the policy is determined based on a context comprising at least an attribute of the entity” The predict imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the predicting step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “determine a first string based on a highest expected reward.” The determine imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the determine step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “update the policy of the agent based on the response to the first string.” The updating imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the update step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
For Step 2A, Prong 2, the claim recites additional elements: memory; processor;
receive a first transaction file associated with an entity; provide, to an environment, the first string; predicting, by the agent, an expected reward for each respective string of a plurality
of strings associated with the first transaction file based on a policy of the agent, wherein the policy is determined based on a context comprising at least an attribute of the entity; provide, to an environment, the first string; receive a response to the first string, wherein the response comprises an actual reward
The processor is recited at a high level of generality, i.e., as a generic processor performing a generic computer function of processing data. This generic processor limitation is no more than mere instructions to apply the exception using a generic computer component. (MPEP 2106.05(f)).
The memory is a generic computer component to apply an abstract idea under 2106.05(f).
The recited "receive a first transaction file associated with an entity” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The “provide, to an environment, the first string” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The recited "receive a first transaction file associated with an entity” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
Step 2B
The additional elements of “processor and memory” do not amount to significantly more for the reasons set forth in step 2A above.
Additionally, under the Subject Matter Eligibility (SME), a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B.
Additionally, under the Subject Matter Eligibility, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B.
Here the "receive a first transaction file associated with an entity” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Here the "provide (i.e. transmitting), to an environment, the first string” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Here the "receive a first transaction file associated with an entity” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “processor and memory” to perform the claim steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 9:
Claim 9, which incorporates the rejection of claim 8, recites further limitations such as “
generate, based on the policy of the agent, the plurality of strings associated with the first transaction file” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 10:
Claim 10, which incorporates the rejection of claim 8, recites further limitations such as
“update the context based on the actual reward, wherein update the policy of the agent based on the response to the first string comprises update the policy of the agent based on the updated context” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 11:
Claim 11 which incorporates the rejection of claim 8, recites further limitations such as
“the expected reward for each string is based on a likelihood that a payment for a transaction associated with the first transaction file will be received” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 12:
Claim 12, which incorporates the rejection of claim 8, recites further limitations such as
“each string of the plurality of strings is associated with a topic of a plurality of
topics, and
the method further comprises ranking each respective topic in the plurality of topics
based on an expected reward of the string associated with the respective topic” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 13:
For Step 1, the claim is a method so it does recite a statutory category of invention.
For Step 2A, Prong 1:
The claim recites the limitation of “generating a first set of strings based on the first transaction file.” The generating imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the generating step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “determining an expected reward for each respective string of the first set of strings.” The determining imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the determining step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “determining a first string based on a highest expected reward.” The determining imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the determining step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “updating the policy of the agent based on the response to the first string.” The updating imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the updating step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
The claim recites the limitation of “generating a context of a set of attributes associated with the entity and the first transaction file based on the expected reward for each respective string, the actual reward for each respective string, and the first string with the highest actual reward.” The generating imitation, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. That is nothing in the claim precludes the generating step from practically being performed in the human mind. This limitation is a mental process. (MPEP 2106.04(a)(2)(III)(C)).
For Step 2A, Prong 2, the claim recites additional elements:
receiving a first transaction file associated with an entity; providing, to an environment, the first string; receiving a response to the first string, wherein the response comprises an actual reward; and training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input.
The recited "receiving a first transaction file associated with an entity” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The recited "receiving a first transaction file associated with an entity” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The recited “training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input” is a generic training recitation that may amount to a generic computer component to apply an abstract idea under MPEP 2106.05(f).
Step 2B
The additional element of “training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input” does not amount to significantly more for the reasons set forth in step 2A above.
Additionally, under the Subject Matter Eligibility, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B.
Here the "receiving a first transaction file associated with an entity” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
Here the "receiving a first transaction file associated with an entity” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input” to perform the claim steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 14:
Claim 2, which incorporates the rejection of claim 1, recites further limitations such as “
ranking each string in the first set of strings based on the expected reward for each respective string” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 15:
Claim 15, which incorporates the rejection of claim 1, recites further limitations such as
“updating the context of the set of attributes based on the transaction data” that are part of the abstract idea.
The claim recites additional elements:
receiving transaction data of the entity based on the string of the second set of strings and “training the agent to choose a third string of a third set of strings for a third
transaction file based on the updated context when receiving the third transaction file as
input.”
The recited "receiving transaction data of the entity based on the string of the second set of strings” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
The recited “training the agent to choose a third string of a third set of strings for a third
transaction file based on the updated context when receiving the third transaction file as
input” is a generic training recitation that may amount to a generic computer component to apply an abstract idea under MPEP 2106.05(f).
Step 2B
The additional element of “training the agent to choose a third string of a third set of strings for a third transaction file based on the updated context when receiving the third transaction file as input” do not amount to significantly more for the reasons set forth in step 2A above.
Additionally, under the Subject Matter Eligibility, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B.
Here the "receiving transaction data of the entity based on the string of the second set of strings” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of “training the agent to choose a third string of a third set of strings for a third transaction file based on the updated context when receiving the third transaction file as input” to perform the claim steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 16:
Claim 16, which incorporates the rejection of claim 13 recites further limitations such as
“each attribute of a set of attributes is based on transaction data related to at least
one of the entity, a payor associated with the first transaction file, or the first transaction
file, wherein the set of attributes includes the attribute of the entity, and the attribute of the entity comprises one of:
an amount of business of the entity;
a maximum amount of a transaction associated with the first transaction file;
a minimum amount of a transaction associated with the first transaction file;
an industry associated with the entity;
a location of the entity;
a time associated with the first transaction file;
a payment history of the payor associated with the first transaction file;
an email address of the payor associated with the entity;
a number of customers associated with the entity; or
a number of transaction files per customer associated with the entity” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 17:
Claim 17, which incorporates the rejection of claim 13, recites further limitations such as
“the first transaction file is an invoice associated with the entity and a payor; and
each string in the first set of strings is a personalized subject string for the invoice” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 18:
Claim 18, which incorporates the rejection of claim 13, recites an additional element:
receiving the actual reward for each respective string of the first set of strings is further based on an action of a payor associated with the first transaction file.
The recited "receiving the actual reward for each respective string of the first set of strings is further based on an action of a payor associated with the first transaction file” step is a form of insignificant extra-solution activity. See MPEP 2106.05(g).
Under the Subject Matter Eligibility, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B.
Here the "receiving the actual reward for each respective string of the first set of strings is further based on an action of a payor associated with the first transaction file” step was considered to be extra-solution activity in Step 2A, and thus it is reevaluated in Step 2B to determine if it is more than what is well-understood, routine, conventional activity in the field. The addition of insignificant extra-solution activity does not amount to an inventive concept, particularly when the activity is well-understood or conventional (MPEP 2106.05(d)). This appears to be well-understood, routine, conventional as evidenced by MPEP 2106.05(d)(II)(i).
i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 19:
Claim 19, which incorporates the rejection of claim 13, recites further limitations such as
“each respective string is associated with a topic related to a likelihood that the entity will receive a payment associated with the first transaction file” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Regarding Claim 20:
Claim 20, which incorporates the rejection of claim 13, recites further limitations such as
“the expected reward for each respective string is associated with a type of a plurality
of types; and a value of the expected reward for each respective string is based on the type of the plurality of types” that are part of the abstract idea.
There are no additional elements recited in this claim that amount to an integration of the judicial exception into a practical application or significantly more than the judicial exception. Therefore, the claim is not eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng).
As to claim 1, Wu teaches a method of updating a policy of an agent, comprising:
receiving a first transaction file associated with an entity (paragraphs [0044]-[0045]
“an entity that receives transaction authorization requests from merchants or other entities; [0076] and [0094] “receive historical transaction data associated with one or more historical payment transactions involving the account of the user; [0099] “a first transaction related action from the set of transaction related actions”);
predicting, by the agent, an expected reward for each respective string of a plurality of strings associated with the first transaction file based on a policy of the agent, wherein the policy is determined based on a context comprising at least an attribute of the entity (paragraphs [0044], “an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution; [0045], “ an entity ( e.g., a merchant service provider, a payment gateway service provider, a payment facilitator, a payment facilitator that contracts with an acquirer, a payment aggregator, and/or the like);” [0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string; [0099]-[0100], “fraud detection system 102 may select a first transaction related action based on an output provided by an agent action machine learning model…The prediction may include data associated with a transaction related action. In some non-limiting embodiments or aspects, the agent action machine learning model may provide a prediction including the data associated with the transaction related action, where the transaction related action is determined to maximize future fraudulent reward amounts of the agent”; [0104] “generate the fraudulent reward amount for the agent based on fraud detection system 102 determining that the prediction is associated with a determination that the fraudulent payment transaction associated with the transaction data is a fraudulent payment transaction or a non-fraudulent payment transaction. For example, fraud detection system 102 may determine the fraudulent reward amount based on subtracting the prediction from one (e.g., subtracting 1 from 1 where the prediction indicates that the fraudulent payment transaction is a fraudulent payment transaction or subtracting 0 from 1 where the prediction indicates that the fraudulent payment transaction is a non-fraudulent payment transaction”; [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string);
determining a first string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 as first string) based on a highest expected reward (paragraph [0017], a sequence of transactions related actions from the plurality of sequences of transaction
related actions that is associated with a maximum reward amount of a plurality of fraudulent reward amounts; wherein Examiner interprets the sequence of transaction to include a first string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 as first string) and the maximum reward as a highest expected reward);
providing, to an environment, the first-string (paragraph [0017], a sequence of transaction related actions from the plurality of sequences of transaction related actions that is associated with a maximum reward amount of a plurality of fraudulent reward amounts; [0052]- [0053] As illustrated in FIG. 1, environment 100 includes fraud detection system 102, transaction service provider system 104, user device 106, merchant system 108, issuer system 110, and communication network 112; wherein Examiner interprets the sequence of transaction in the environment 100 included in the fraud detection system 102 to supply the first-string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 as first string)).
However, Wu teaches the first string but fails to explicitly teach:
receiving a response to the first string, wherein the response comprises an actual
reward; and
updating the policy of the agent based on the response to the first string.
Zheng, in combination with Wu, teaches:
receiving a response to the first string (Wu: first string), wherein the response comprises an actual reward (paragraph [0006] “This specification generally describes a reinforcement learning system that selects actions to be performed by a reinforcement learning agent interacting with an environment. In order for the agent to interact with the environment, the system receives data characterizing the current state of the environment and selects an action to be performed by the agent in response to the received data. Data characterizing a state of the environment will be referred to in this specification as an observation;” [0016] and [0023], “the reinforcement learning system is configured to receive observations, i.e., data, characterizing a current state of the environment, and in response to select actions to be performed by an agent interacting with an environment, e.g., to perform a task. In response to an action the agent may also receive a reward, that is an external or extrinsic reward, e.g., characterizing performance of a task”); and
updating the policy of the agent based on the response to the first string (Wu: first string) (paragraphs [0006]- [0007], “updating the agent's policy based upon the intrinsic reward values generated by the intrinsic reward system; updating the intrinsic reward system based upon an extrinsic reward value obtained based upon the task being performed by the agent; and re-initializing the agent's policy when an expiration criterion associated with the agent is met; [0010] Updating the agent's policy may comprise repeatedly updating the agent's policy for a plurality of updates on a same task and wherein updating the intrinsic reward system follows the repeated updating of the agent's policy).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu to add a policy update to the system of Wu, as taught by Zheng above. The modification would have been obvious because one of ordinary skill would be motivated to generate intrinsic reward values for the agent based upon the actions taken, as suggested by Zheng, ([0007]).
As to claim 8, Wu discloses a processing system (paragraph [0045] payment processing system), comprising:
a memory storing executable instructions (paragraph [0061] memory); and
a processor configured to execute the executable instructions (paragraph [0061] processor) and cause the processing system to:
receive a first transaction file associated with an entity (paragraphs [0044], “an entity that receives transaction authorization requests from merchants or other entities; [0076] and [0094] “receive historical transaction data associated with one or more historical payment transactions involving the account of the user; [0099] “a first transaction related action from the set of transaction related actions”);
predict, by an agent, an expected reward for each respective string of a plurality of strings associated with the first transaction file based on a policy of the agent, wherein the policy is determined based on a context comprising at least an attribute of the entity (paragraphs [0044]-[0045], “an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution; [0045], “ an entity ( e.g., a merchant service provider, a payment gateway service provider, a payment facilitator, a payment facilitator that contracts with an acquirer, a payment aggregator, and/or the like);” );” [0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string; [0099], “fraud detection system 102 may select a first transaction related action based on an output provided by an agent action machine learning model…The prediction may include data associated with a transaction related action. In some non-limiting embodiments or aspects, the agent action machine learning model may provide a prediction including the data associated with the transaction related action, where the transaction related action is determined to maximize future fraudulent reward amounts of the agent”; [0104] “generate the fraudulent reward amount for the agent based on fraud detection system 102 determining that the prediction is associated with a determination that the fraudulent payment transaction associated with the transaction data is a fraudulent payment transaction or a non-fraudulent payment transaction. For example, fraud detection system 102 may determine the fraudulent reward amount based on subtracting the prediction from one (e.g., subtracting 1 from 1 where the prediction indicates that the fraudulent payment transaction is a fraudulent payment transaction or subtracting 0 from 1 where the prediction indicates that the fraudulent payment transaction is a non-fraudulent payment transaction.”); [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string);
determine a first string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 as first string) based on a highest expected reward (paragraph [0017], a sequence of transactions related actions from the plurality of sequences of transaction
related actions that is associated with a maximum reward amount of a plurality of fraudulent reward amounts; wherein Examiner interprets the sequence of transaction to include a first string and the maximum reward as a highest expected reward);
provide, to an environment, the first string (paragraph [0017], a sequence of transaction related actions from the plurality of sequences of transaction related actions that is associated with a maximum reward amount of a plurality of fraudulent reward amounts; [0052]- [0053] As illustrated in FIG. 1, environment 100 includes fraud detection system 102, transaction service provider system 104, user device 106, merchant system 108, issuer system 110, and communication network 112; wherein Examiner interprets the sequence of transaction in the environment 100 included in the fraud detection system 102 to supply the first-string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 as first string).
However, Wu teaches the first string but fails to explicitly teach:
receive a response to the first string, wherein the response comprises an actual
reward; and
update the policy of the agent based on the response to the first string.
Zheng, in combination with Wu, teaches:
receive a response to the first string (Wu: first string), wherein the response comprises an actual reward (paragraph [0006] “This specification generally describes a reinforcement learning system that selects actions to be performed by a reinforcement learning agent interacting with an environment. In order for the agent to interact with the environment,
the system receives data characterizing the current state of the environment and selects an action to be performed by the agent in response to the received data. Data characterizing a state of the environment will be referred to in this specification as an observation;” wherein Examiner interprets the observation to include a first string; [0016] and [0023], “the reinforcement learning system is configured to receive observations, i.e., data, characterizing a current state of the environment, and in response to select actions to be performed by an agent interacting with an environment, e.g., to perform a task. In response to an action the agent may also receive a reward, that is an external or extrinsic reward, e.g., characterizing performance of a task”); and
update the policy of the agent based on the response to the first string (Wu: first string) (paragraphs [0006]- [0007], “updating the agent's policy based upon the intrinsic
reward values generated by the intrinsic reward system; updating the intrinsic reward system based upon an extrinsic reward value obtained based upon the task being performed by the agent; and re-initializing the agent's policy when an expiration criterion associated with the agent is met; [0010] Updating the agent's policy may comprise repeatedly updating the agent's policy for a plurality of updates on a same task and wherein updating the intrinsic reward system follows the repeated updating of the agent's policy;)
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu to add a policy update to the system of Wu, as taught by Zheng above. The modification would have been obvious because one of ordinary skill would be motivated to generate intrinsic reward values for the agent based upon the actions taken, as suggested by Zheng, ([0007]).
Claims 2 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng), and further in view of
Kapoor et al. (US 9063978 B1, hereinafter referred to as Kapoor).
As to claim 2, which incorporates the rejection of claim1, Wu and Zheng teach strings but fail to explicitly teach:
generating, based on the policy of the agent, the plurality of strings associated with the first transaction file.
Kapoor, in combination with Wu and Zheng, teaches:
generating, based on the policy of the agent, the plurality of strings associated with the first transaction file (col. 11, lines 38-47, wherein Examiner interprets “String Generator component 440, which may, in one implementation, be configured to generate strings in association with various inputs and/or stored data” to teach the limitation).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add strings generator to the system of Wu and Zheng, as taught by Kapoor above. The modification would have been obvious because one of ordinary skill would be motivated to generate strings matching data queries, as suggested by Kapoor, (col. 11, lines 38-47)
As to claim 9, which incorporates the rejection of claim1, Wu and Zheng fail to explicitly teach wherein the processor is further configured to cause the processing system to generate, based on the policy of the agent, the plurality of strings associated with the first transaction file.
Kapoor, in combination with Wu and Zheng, teaches wherein the processor is further configured to cause the processing system to generate, based on the policy of the agent, the plurality of strings associated with the first transaction file (col. 11, lines 38-47, “String Generator component 440, which may, in one implementation, be configured to generate strings in association with various inputs and/or stored data”).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add strings generator to the system of Wu and Zheng, as taught by Kapoor above. The modification would have been obvious because one of ordinary skill would be motivated to generate strings matching data queries, as suggested by Kapoor, (col. 11, lines 38-47).
Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng), and further in view of ACHTENBERG et al. (US 2022/0004456 A1, hereinafter referred to as ACHTENBERG)
As to claim 3, which incorporates the rejection of claim1, Wu and Zheng fail to explicitly teach: updating the context based on the actual reward, wherein updating the policy of the agent based on the response to the first string comprises updating the policy of the agent based on the updated context.
ACHTENBERG, in combination with Wu and Zheng, teaches:
updating the context based on the actual reward, wherein updating the policy of the agent based on the response to the first string comprises updating the policy of the agent based on the updated context (paragraph [0034] The agent 210 selects an action based on the state of the environment 220 and maximizing accumulative rewards. The selected action impacts the current reward and future rewards. The agent 210 takes actions which in turn changes the state of the environment 220 and provides a reward. If the sum of the immediate reward plus the next state's expected total reward is high, the agent 210 would learn to prefer more of the same action in this state.
Otherwise, other actions may be chosen in the future. The Q-learning process is configured to converge to the agent's optimal policy and update the Q-table values accordingly. After convergence to the optimal policy, the converged action the agent selects in each state is the action with the maximal Q value in the Q-table. Convergence to the optimal policy may be achieved by offline training in the lab during production or on-the-fly in the field).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add a policy update of the agent based on the updated context to the system of Wu and Zheng, as taught by ACHTENBERG above. The modification would have been obvious because one of ordinary skill would be motivated to converge to the agent's optimal policy and update the Q-table values, as suggested by ACHTENBERG, (paragraph [0034]).
As to claim 10, which incorporates the rejection of claim 8, Wu and Zheng fail to explicitly teach
wherein the processor is further configured to cause the processing system to: update the context based on the actual reward, wherein updating the policy of the agent based on the response to the first string comprises updating the policy of the agent based on the updated context.
ACHTENBERG, in combination with Wu and Zheng, teaches wherein the processor is further configured to cause the processing system to: update the context based on the actual reward, wherein updating the policy of the agent based on the response to the first string comprises updating the policy of the agent based on the updated context (paragraph [0034] The agent 210 selects an action based on the state of the environment 220 and maximizing accumulative rewards. The selected action impacts the current reward and future rewards. The agent 210 takes actions which in turn changes the state of the environment 220 and provides a reward. If the sum of the immediate reward plus the next state's expected total reward is high, the agent 210 would learn to prefer more of the same action in this state.
Otherwise, other actions may be chosen in the future. The Q-learning process is configured to converge to the agent's optimal policy and update the Q-table values accordingly. After convergence to the optimal policy, the converged action the agent selects in each state is the action with the maximal Q value in the Q-table. Convergence to the optimal policy may be achieved by offline training in the lab during production or on-the-fly in the field).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add a policy update of the agent based on the updated context to the system of Wu and Zheng, as taught by ACHTENBERG above. The modification would have been obvious because one of ordinary skill would be motivated to converge to the agent's optimal policy and update the Q-table values, as suggested by ACHTENBERG, (paragraph [0034]).
Claims 4 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng), and further in view of VARMA et al. (US 2022/0036326 A1, hereinafter referred to as VARMA).
As to claim 4, which incorporates the rejection of claim1, Wu and Zheng fail to explicitly teach wherein the expected reward for each string is based on a likelihood that a payment for a transaction associated with the first transaction file will be received.
VARMA, in combination with Wu and Zheng, teaches wherein the expected reward for each string is based on a likelihood that a payment for a transaction associated with the first transaction file will be received (paragraphs [0008] The reward or reminder of the stored value is provided based on a determination, with the aid of a computer processor of a computer system, of the likelihood of the payer to conduct a transaction with the merchant among one or merchants at or in proximity to a geolocation of the payer).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add a policy update of the agent based on the updated context to the system of Wu and Zheng, as taught by VARMA, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a current context of the payer based on a current time or location data from a device of the payer, as suggested by VARMA (paragraph [0008]).
As to claim 11, which incorporates the rejection of claim 8, Wu and Zheng fail to explicitly teach wherein the expected reward for each string is based on a likelihood that a payment for a transaction associated with the first transaction file will be received.
VARMA, in combination with Wu and Zheng, teaches wherein the expected reward for each string is based on a likelihood that a payment for a transaction associated with the first transaction file will be received (paragraphs [0008] The reward or reminder of the stored value is provided based on a determination, with the aid of a computer processor of a computer system, of the likelihood of the payer to conduct a transaction with the merchant among one or merchants at or in proximity to a geolocation of the payer).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add a policy update of the agent based on the updated context to the system of Wu and Zheng, as taught by VARMA, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a current context of the payer based on a current time or location data from a device of the payer, as suggested by VARMA (paragraph [0008]).
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng), and further in view of Podgorny et al. (US 2017/0124184 A1, hereinafter referred to as Podgorny).
As to claim 5, which incorporates the rejection of claim1, Wu and Zheng fail to explicitly teach wherein: each string of the plurality of strings is associated with a topic of a plurality of topics, and the method further comprises ranking each respective topic in the plurality of topics based on an expected reward of the string associated with the respective topic.
Podgorny, in combination with Wu and Zheng, teaches wherein: each string of the plurality of strings is associated with a topic of a plurality of topics, and the method further comprises ranking each respective topic in the plurality of topics based on an expected reward of the string associated with the respective topic (paragraphs [0088], determines a most relevant or a highest relevant topic for the search terms by ranking, sorting, and/or comparing the topic relevance scores 715, 716 for each of the topics 709, 712, according to one embodiment. The customer support system 611 determines that the topic with the highest topic relevance score is the highest relevant topic to the search query terms 703, 704, 705, 706, according to one embodiment. Accordingly, because the topic relevance score for topic 709 (e.g., 0.192) is less than the topic relevance score 716 for the topic 712 (e.g., 0.2886), the customer support system 611 determines that the relevant topic for the search query terms 703, 704, 705, 706 is the topic 712-"change/amend", according to one embodiment).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add a topic ranking to the system of Wu and Zheng, as taught by Podgorny, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a most relevant or a highest relevant topic for the search terms, as suggested by Podgorny (paragraph [0088]).
As to claim 12, which incorporates the rejection of claim 8, Wu and Zheng fail to explicitly teach wherein each string of the plurality of strings is associated with a topic of a plurality of topics, and wherein the processor is further configured to cause the processing system to rank each respective topic in the plurality of topics based on an expected reward of the string associated with the respective topic.
Podgorny, in combination with Wu and Zheng, teaches wherein each string of the plurality of strings is associated with a topic of a plurality of topics, and wherein the processor is further configured to cause the processing system to rank each respective topic in the plurality of topics based on an expected reward of the string associated with the respective topic (paragraphs [0088], determines a most relevant or a highest relevant topic for the search terms by ranking, sorting, and/or comparing the topic relevance scores 715, 716 for each of the topics 709, 712, according to one embodiment. The customer support system 611 determines that the topic with the highest topic relevance score is the highest relevant topic to the search query terms 703, 704, 705, 706, according to one embodiment. Accordingly, because the topic relevance score for topic 709 (e.g., 0.192) is less than the topic relevance score 716 for the topic 712 (e.g., 0.2886), the customer support system 611 determines that the relevant topic for the search query terms 703, 704, 705, 706 is the topic 712-"change/amend", according to one embodiment).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add a topic ranking to the system of Wu and Zheng, as taught by Podgorny, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a most relevant or a highest relevant topic for the search terms, as suggested by Podgorny (paragraph [0088]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng), and further in view of Ibrahim (US 2010/0106585 A1, hereinafter referred to as Ibrahim).
As to claim 6, which incorporates the rejection of claim1, Wu and Zheng fail to explicitly teach wherein: each attribute of a set of attributes is based on transaction data related to at least one of the entity, a payor associated with the first transaction file, or the first transaction file, wherein the set of attributes includes the attribute of the entity, and
the attribute of the entity comprises one of:
an amount of business of the entity;
a maximum amount of a transaction associated with the first transaction file;
a minimum amount of a transaction associated with the first transaction file;
an industry associated with the entity;
a location of the entity;
a time associated with the first transaction file;
a payment history of the payor associated with the first transaction file;
an email address of the payor associated with the entity;
a number of customers associated with the entity; or
a number of transaction files per customer associated with the entity.
Ibrahim, in combination with Wu and Zheng, teaches wherein each attribute of a set of attributes is based on transaction data related to at least one of the entity, a payor associated with the first transaction file, or the first transaction file, wherein the set of attributes includes the attribute of the entity (paragraphs [0043] In one embodiment, payment information may be divided or parsed into separate data (e.g. attributes); [0086] In one embodiment, payment information ( or the attributes derived from the payment information) is used in a calculation or to increment a counter or balance), and the attribute of the entity comprises one of:
an amount of business of the entity;
a maximum amount of a transaction associated with the first transaction file;
a minimum amount of a transaction associated with the first transaction file;
an industry associated with the entity; a location of the entity;
a time associated with the first transaction file (paragraph [0062], the timing of a payment, time period for a purchase; [0090], a product held by the consumer, a purchase (e.g. of a specific product or at a particular merchant), the timing of a
payment, time period for a purchase);
a payment history of the payor associated with the first transaction file (paragraphs [0068], payment history; [0069], overall transaction history, payment history; an email address of the payor associated with the entity (paragraphs [0060] The statement may be provided to consumer 105 online (e.g., email, accessible from
a link, a customized uniform resource locator (URL), accessible at a website, sent to a PDA, etc.); [0096] For instance, the consumer may be sent an email with the incentive information and provide a customized URL that, if selected by the consumer, signs the consumer up for the incentive);’
a number of customers associated with the entity; or
a number of transaction files per customer associated with the entity.
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add an email address of the payor associated with the entity to the system of Wu and Zheng, as taught by Ibrahim, above. The modification would have been obvious because one of ordinary skill would be motivated to send an incentive information and provide a customized URL, as suggested by Ibrahim (paragraph [0096]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Zheng et al . (US 2021/0089910 A1, hereinafter referred to as Zheng), and further in view of Unsal et al. (US 10,163,082 B1, hereinafter referred to as Unsal).
As to claim 7, which incorporates the rejection of claim1, Wu and Zheng fail to explicitly teach wherein the context further comprises an attribute of a payor associated with the first transaction file.
Unsal, in combination with Wu and Zheng, teaches wherein the context further comprises an attribute of a payor associated with the first transaction file (col. 11, lines 14-27, the payment collection records are analyzed to extract a payor attribute of the payors, a payee attribute of the payees, and a transaction attribute of the eCommerce actions. As noted above, the payee attribute may include one or more of an industry category, a business location/address, a store name, a store location, a number of stores, a number of employees, or other characteristics of a business entity. In addition, the payor attribute may include one or more of a payor category (e.g., individual or business), a payor address, or other demographic information of the payor. The transaction attribute may include the invoice issue date, invoice amount, payment due date, payment completion date, partial payment status, overdue payment status, etc.).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu and Zheng to add an attribute of a payor to the system of Wu and Zheng, as taught by Unsal, above. The modification would have been obvious because one of ordinary skill would be motivated to analyze payment collection records to extract a payor attribute of the payors, as suggested by Unsal (col. 11, lines 14-27).
Claims 13, 15, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Kapoor et al. (US 9,659,062 B1, herein after referred to as Kapoor062), and further in view of VASHISHT et al. (US 2022/0261875 A1, herein after referred to as VASHISHT).
As to claim 13, Wu teaches a method, comprising:
receiving a first transaction file associated with an entity (paragraphs [0044]-[0045]
“an entity that receives transaction authorization requests from merchants or other entities; [0076] and [0094] “receive historical transaction data associated with one or more historical payment transactions involving the account of the user; [0099] “a first transaction related action from the set of transaction related actions”);
[generating a first set of strings based on the first transaction file];
determining an expected reward for each respective string of the first set of strings (paragraphs [0104] “generate the fraudulent reward amount for the agent based on fraud detection system 102 determining that the prediction is associated with a determination that the fraudulent payment transaction associated with the transaction data is a fraudulent payment transaction or a non-fraudulent payment transaction. For example, fraud detection system 102 may determine the fraudulent reward amount based on subtracting the prediction from one (e.g., subtracting 1 from 1 where the prediction indicates that the fraudulent payment transaction is a fraudulent payment transaction or subtracting 0 from 1 where the prediction indicates that the fraudulent payment transaction is a non-fraudulent payment transaction); [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction”);
receiving an actual reward for each respective string of the first set of strings (paragraph [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string);
determining a first string of the first set of strings with a highest actual reward (paragraph [0017], a sequence of transactions related actions from the plurality of sequences of transaction related actions that is associated with a maximum reward amount of a plurality of fraudulent reward amounts; wherein Examiner interprets the sequence of transaction to include a first string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 as first string) and the maximum reward as a highest expected reward);
generating a context of a set of attributes associated with the entity and the first transaction file based on the expected reward for each respective string, the actual reward for each respective string, and the first string with the highest actual reward (paragraphs [0104] “generate the fraudulent reward amount for the agent based on fraud detection system 102 determining that the prediction is associated with a determination that the fraudulent payment transaction associated with the transaction data is a fraudulent payment transaction or a non-fraudulent payment transaction. For example, fraud detection system 102 may determine the fraudulent reward amount based on subtracting the prediction from one (e.g., subtracting 1 from 1 where the prediction indicates that the fraudulent payment transaction is a fraudulent payment transaction or subtracting 0 from 1 where the prediction indicates that the fraudulent payment transaction is a non-fraudulent payment transaction”; [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string)..
However, Wu fails to explicitly teach:
generating a first set of strings based on the first transaction file; and
training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input.
Kapoor062, in combination with Wu, teaches:
generating a first set of strings based on the first transaction file (col. 13, lines 41-51, wherein Examiner interprets “String Generator component 440, which may, in one implementation, be configured to generate strings in association with various inputs and/or stored data” to teach the limitation).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the system of Wu to add strings generator to the system of Wu, as taught by Kapoor062 above. The modification would have been obvious because one of ordinary skill would be motivated to generate strings matching data queries, as suggested by Kapoor062, (col. 13, lines 41-51).
However, Wu and Kapoor062 fail to explicitly teach:
training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input.
VASHISHT, in combination with Wu and Kapoor062, teaches:
training an agent to choose a string of a second set of strings for a second transaction file based on the context of the set of attributes when receiving the second transaction file as input (paragraphs [0072], select all past transaction-level
data (i.e., payment authorization request and payment authorization response messages of past payment transactions) associated with the issuer 108 and/or the merchant 104, for training the RL agent 222…The transaction-level data associated with the issuer 108 or the merchant 104 includes a number of declined/approved/fraud transactions such that the RL agent 222 learns the apt representation of the transaction-level data associated with the issuer 108 or the merchant 104; [0077]-[078] The RL agent 222 implements a machine learning model (for example, a deep reinforcement learning model). The RL agent 222 is trained using the payment transaction attributes associated with the past payment transactions with declined/approved authorization responses; [0079]-[0081] During the training process, the RL agent 222 is configured to define state space and action space of the deep reinforcement learning model. The state space represents the payment transaction attributes associated with a payment
transaction and authorizing components applied to the payment transaction.).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the combination system of Wu and Kapoor062 to add “training an agent” to the combination system of Wu and Kapoor062, as taught by VASHISHT above. The modification would have been obvious because one of ordinary skill would be motivated to have an agent trained based on the payment transaction attributes predict what authorizing components (i.e., products) should be applied to a particular payment transaction in real-time., as suggested by VASHISHT, ([0081]).
As to claim 15, which incorporates the rejection of claim 13, Wu teaches:
receiving transaction data of the entity based on the string of the second set of strings (paragraphs [0076] and [0094] “receive historical transaction data associated with one or more historical payment transactions involving the account of the user; [0099] “a first transaction related action from the set of transaction related actions”); [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as a set of strings).
However, Wu and Kapoor062 fail to explicitly teach:
updating the context of the set of attributes based on the transaction data; and
training the agent to choose a third string of a third set of strings for a third transaction file based on the updated context when receiving the third transaction file as input.
VASHISHT, in combination with Wu and Kapoor062 teaches:
updating the context of the set of attributes based on the transaction data (paragraphs [0140]-[0141] At 704, the server system 200 extracts payment transaction attributes from the historical transaction data. In particular, the server system 200 extracts various data elements present in each payment transaction from the historical
transaction data and perform data sanitization process (interpreted by Examiner as updating); and
training the agent to choose a third string of a third set of strings for a third transaction file based on the updated context when receiving the third transaction file as input (paragraphs [0072], select all past transaction-level
data (i.e., payment authorization request and payment authorization response messages of past payment transactions) associated with the issuer 108 and/or the merchant 104, for training the RL agent 222…The transaction-level data associated with the issuer 108 or the merchant 104 includes a number of declined/approved/fraud transactions such that the RL agent 222 learns the apt representation of the transaction-level data associated with the issuer 108 or the merchant 104; [0077]-[078] The RL agent 222 implements a machine learning model (for example, a deep reinforcement learning model). The RL agent 222 is trained using the payment transaction attributes associated with the past payment transactions with declined/approved authorization responses; [0079]-[0081] During the training process, the RL agent 222 is configured to define state space and action space of the deep reinforcement learning model. The state space represents the payment transaction attributes associated with a payment
transaction and authorizing components applied to the payment transaction.).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the combination system of Wu and Kapoor062 to add “training an agent” to the combination system of Wu and Kapoor062, as taught by VASHISHT above. The modification would have been obvious because one of ordinary skill would be motivated to have an agent trained based on the payment transaction attributes predict what authorizing components (i.e., products) should be applied to a particular payment transaction in real-time., as suggested by VASHISHT, ([0081]).
As to claim 18, which incorporates the rejection of claim 13, Wu teaches wherein receiving the actual reward for each respective string of the first set of strings is further based on an action of a payor associated with the first transaction file (paragraphs [0076] and [0094] “receive historical transaction data associated with one or more historical payment transactions involving the account of the user; [0099] “a first transaction related action from the set of transaction related actions”); [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string).
As to claim 20, which incorporates the rejection of claim 13, Wu teaches wherein the expected reward for each respective string is associated with a type of a plurality of types; and value of the expected reward for each respective string is based on the type of the plurality of types (paragraphs [0076] and [0094] “receive historical transaction data associated with one or more historical payment transactions involving the account of the user; [0099] “a first transaction related action from the set of transaction related actions”); [0105] “predicted fraudulent reward amount associated with a fraudulent payment transaction…Multiple agents can also be deployed together with different reward or policy functions to optimize the aggregated R, over all the agents;” wherein Examiner interprets the predicted fraudulent reward amount associated with a fraudulent payment transaction of an entity to include the expected reward for each respective string ([0095], FIG. 4B, wherein Examiner interprets Merchant 1 through Merchant 7, in “[Merchant 1, $4.88]” through [Merchant 7, $54.19], as respective string).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Kapoor et al. (US 9,659,062 B1, herein after referred to as Kapoor062), and further in view of VASHISHT et al. (US 2022/0261875 A1, herein after referred to as VASHISHT), and Podgorny et al. (US 2017/0124184 A1, hereinafter referred to as Podgorny).
As to claim 14, which incorporates the rejection of claim 13, Wu, Kapoor062 and VASHISHT fail to explicitly teach:
ranking each string in the first set of strings based on the expected reward for each respective string.
Podgorny, in combination with Wu, Kapoor062 and VASHISHT, teaches:
ranking each string in the first set of strings based on the expected reward for each respective string (paragraphs [0088], determines a most relevant or a highest relevant topic for the search terms by ranking, sorting, and/or comparing the topic relevance scores 715, 716 for each of the topics 709, 712, according to one embodiment. The customer support system 611 determines that the topic with the highest topic relevance score is the highest relevant topic to the search query terms 703, 704, 705, 706, according to one embodiment. Accordingly, because the topic relevance score for topic 709 (e.g., 0.192) is less than the topic relevance score 716 for the topic 712 (e.g., 0.2886), the customer support system 611 determines that the relevant topic for the search query terms 703, 704, 705, 706 is the topic 712-"change/amend", according to one embodiment).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the combination system of Wu, Kapoor062 and VASHISHT to add a topic ranking to the system of combination system of Wu, Kapoor062 and VASHISHT, as taught by Podgorny, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a most relevant or a highest relevant topic for the search terms, as suggested by Podgorny (paragraph [0088]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Kapoor et al. (US 9,659,062 B1, herein after referred to as Kapoor062), and further in view of VASHISHT et al. (US 2022/0261875 A1, herein after referred to as VASHISHT), and Ibrahim (US 2010/0106585 A1, hereinafter referred to as Ibrahim).
As to claim 16, which incorporates the rejection of claim 13, Wu, Kapoor062 and VASHISHT fail to explicitly teach wherein each attribute of a set of attributes is based on transaction data related to at least one of the entity, a payor associated with the first transaction file, or the first transaction file, and wherein the set of attributes comprises at least one of:
an amount of business of the entity;
a maximum amount of a transaction associated with the first transaction file;
a minimum amount of a transaction associated with the first transaction file;
an industry associated with the entity; a location of the entity;
a time associated with the first transaction file;
a payment history of the payor associated with the first transaction file;
an email address of the payor associated with the entity;
a number of customers associated with the entity; or
a number of transaction files per customer associated with the entity.
Ibrahim, in combination with Wu, Kapoor062 and VASHISHT, teaches:
wherein each attribute of a set of attributes is based on transaction data related to at least one of the entity, a payor associated with the first transaction file, or the first transaction file (paragraphs [0043] In one embodiment, payment information may be divided or parsed into separate data (e.g. attributes); [0086] In one embodiment, payment information (or the attributes derived from the payment information), and wherein the set of attributes comprises at least one of:
an amount of business of the entity;
a maximum amount of a transaction associated with the first transaction file;
a minimum amount of a transaction associated with the first transaction file;
an industry associated with the entity; a location of the entity;
a time associated with the first transaction file (paragraph [0062], the timing of a payment, time period for a purchase; [0090], a product held by the consumer, a purchase (e.g. of a specific product or at a particular merchant), the timing of a
payment, time period for a purchase);
a payment history of the payor associated with the first transaction file (paragraphs [0068], payment history; [0069], overall transaction history, payment history;
an email address of the payor associated with the entity (paragraphs [0060] The statement may be provided to consumer 105 online (e.g., email, accessible from
a link, a customized uniform resource locator (URL), accessible at a website, sent to a PDA, etc.); [0096] For instance, the consumer may be sent an email with the incentive information and provide a customized URL that, if selected by the consumer, signs the consumer up for the incentive);
an email address of the payor associated with the entity;
a number of customers associated with the entity; or
a number of transaction files per customer associated with the entity.
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the combination system of Wu, Kapoor062 and VASHISHT to add an email address of the payor associated with the entity to the combination system of Wu, Kapoor062 and VASHISHT, as taught by Ibrahim, above. The modification would have been obvious because one of ordinary skill would be motivated to send an incentive information and provide a customized URL, as suggested by Ibrahim (paragraph [0096]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Kapoor et al. (US 9,659,062 B1, herein after referred to as Kapoor062), and further in view of VASHISHT et al. (US 2022/0261875 A1, herein after referred to as VASHISHT), and Kapoor et al. (US 9,063,978 B1, herein after referred to as Kapoor).
As to claim 17, which incorporates the rejection of claim 13, Wu, Kapoor062 and VASHISHT fail to explicitly teach wherein: the first transaction file is an invoice associated with the entity and a payor; and each string in the first set of strings is a personalized subject string for the invoice.
Kapoor, in combination with Wu, Kapoor062 and VASHISHT, teaches wherein: the first transaction file is an invoice associated with the entity and a payor; and each string in the first set of strings is a personalized subject string for the invoice (col. 37, lines 21-29 A transaction may be paying a bill, depositing received payment, placing an order, receiving an order, receiving an approval to pay the invoice, paying salary to an employee, reimbursing an employee, paying taxes, receiving a tax refund, purchasing investments, selling investment, signing a contract, and/or the like. In one embodiment, an indication of a transaction may be received by having (e.g., a human) operator enter the transaction. For example, the operator may receive a paper invoice statement…: col. 38, lines 29-37 At 2917, transaction characteristics describing the transaction are identified. Examples of transaction characteristics may include transaction type, transaction frequency, transaction amount, transaction date, department code, program code, entity, payee, payor, location, country, business type, team, business category, business subcategory, employee id, employee title, payment method, bank account number, bank routing number, SWIFT code, IBAN, investment instrument, investor, and/or the like….)
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the combination system of Wu, Kapoor062 and VASHISHT to add an invoice to the combination system of Wu, Kapoor062 and VASHISHT, as taught by Kapoor, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a transaction characteristic, as suggested by Kapoor (col. 38, lines 29-37).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2021/0103926 A1, hereinafter referred to as Wu), in view of Kapoor et al. (US 9,659,062 B1, herein after referred to as Kapoor062), and further in view of VASHISHT et al. (US 2022/0261875 A1, herein after referred to as VASHISHT), and VARMA et al. (US 2022/0036326 A1, hereinafter referred to as VARMA).
As to claim 19, which incorporates the rejection of claim13, Wu, Kapoor062 and VASHISHT fail to explicitly teach wherein each respective string is associated with a topic related to a likelihood that the entity will receive a payment associated with the first transaction file.
VARMA, in combination with Wu, Kapoor062 and VASHISHT, teaches wherein each respective string is associated with a topic related to a likelihood that the entity will receive a payment associated with the first transaction file (paragraphs [0008] The reward or reminder of the stored value is provided based on a determination, with the aid of a computer processor of a computer system, of the likelihood of the payer to conduct a transaction with the merchant among one or merchants at or in proximity to a geolocation of the payer).
It would have been obvious to one of ordinary skill in the art before the effective filing of
the claimed invention to modify the combination system of Wu, Kapoor062 and VASHISHT to add a policy update of the agent based on the updated context to the combination system of Wu, Kapoor062 and VASHISHT, as taught by VARMA, above. The modification would have been obvious because one of ordinary skill would be motivated to determine a current context of the payer based on a current time or location data from a device of the payer, as suggested by VARMA (paragraph [0008]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Patents and patent related publications are cited in the Notice of References Cited (Form PTO-892) attached to this action to further show the state of the art with respect to the invention.
Kapoor et al. (US 9,659,062 B1) teaches a method for facilitating plan of purse-based global benefits e.g. health insurance, involves creating benefits package offered by employer for employee and associated with employee identifier if evaluated business rules are satisfied.
Hunter (US 2023/0195429 A1) teaches a tangible non-transitory machine-readable medium for facilitating directed graph-manipulation of program i.e. stand-alone program in domain-specific execution environments, has set of instructions for determining and storing outcome program state in memory by computer system based on action values.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABABACAR SECK whose telephone number is (571)-270-7146. The examiner can normally be reached Monday-Friday 8:00 A.M.-6:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lamardo Viker can be reached on 571-270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABABACAR SECK/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights