Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-10, 17-19, and 35 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by He (W. He, T. -Y. Lee, J. van Baar, K. Wittenburg and H. -W. Shen, "DynamicsExplorer: Visual Analytics for Robot Control Tasks involving Dynamics and LSTM-based Control Policies," 2020 IEEE Pacific Visualization Symposium (PacificVis), Tianjin, China, 2020, pp. 36-45, doi: 10.1109/PacificVis48177.2020.7127).
Regarding claim 1, He teaches a computer-implemented method for evaluating a recurrent neural network (RNN)-based deep learning model, comprising:
generating, with at least one processor, a first graphical user interface (GUI) (see Figure 1, which is a GUI which contains multiple sub-GUIs. The first GUI can be (b1) which presents hidden states of an LSTM and the status of the environment as a heat map) based on model data generated by a RNN-based deep learning model (§2, “The latent vectors…are then fed into a long short-term memory (LSTM) layer” – an LSTM layer is a subtype of an RNN, therefore is RNN-based), the model data comprising
a plurality of events associated with a plurality of states in an environment (§2, “A deep RL model is trained to play the game by mapping states of the game to actions”, emphasis added),
the first GUI comprising a line chart visually representing a timeline for the plurality of events in relation to at least one parameter value based on the plurality of events and the plurality of states (see Figure 1, d2. See §5.3 and Figure 6, where d2 is a sub-trajectory of a time series and therefore can be considered a line chart which represents a timeline);
generating, with the at least one processor, a second GUI comprising a point chart visually representing multi-dimensional intermediate data having greater than two dimensions as a two-dimensional projection, each point of the point chart representing a time step and at least one event from the time step based on multi-dimensional intermediate data between transformations in the RNN-based deep learning model that connect at least one state of the plurality of states to at least one event of the plurality of events (see Figure 1, b2 , which shows the trajectory along the maze. The trajectory is a “point chart” because it is chart made from points. It “visually represents a two-dimensional projection of the multi-dimensional intermediate data” because it is a visual representation in two dimensions of the LSTM network function, which is multi-dimensional. Each point represents a time step because each point represents a subsequent decision/action, where the action is based on the multi-dimensional data between transformations from one hidden state to another in the LSTM which ultimately connect states to events. Along with position, a dimension of action is portrayed, see §5.2.2 “Trajectory View”, “We use the color of the trajectory to encode additional information such as the action the model takes at each frame.”); and
perturbing, with the at least one processor, the environment at a time step based on user interaction with at least one of the first GUI and the second GUI (see §6.1.2, where a user can see the behavior of the system via the first and/or second GUI and make a constraint to fix issues, i.e. a perturbation, to change system operation).
Regarding claim 2, He teaches all of the limitations of claim 1, further comprising:
generating, with the at least one processor, a third GUI based on at least one hidden state and/or cell state of the RNN-based deep learning model (Figure 1, b3 which is a heatmap of hidden state activations).
Regarding claim 3, He teaches all of the limitations of claim 2, further comprising:
identifying, with the at least one processor, a hidden state and/or cell state from the at least one hidden state and/or cell state impacting an event of the plurality of events (Figure 1, b3 which is a heatmap of hidden state activations and b4 shows associated actions/events).
Regarding claim 4, He teaches all of the limitations of claim 2, wherein
generating the third GUI comprises generating a visual representation of contrasted distributions over different subsets of a plurality of steps (Figure 1, b3 shows a visual representation of contrasted distributions (see individual rows) over different subsets of a plurality of steps (each step 79-100 can be a subset of the plurality of shown steps)).
Regarding claim 5, He teaches all of the limitations of claim 2, wherein
generating the third GUI comprises generating at least two rows including a plurality of visual representations (Figure 1, b3 has at least two rows, each with a visual representation at steps 79-100), each visual of the plurality of visual representations visualizing a hidden state and/or a cell state for one dimension of a plurality of dimensions of the model (Figure 1, b3 is a representation of hidden states at each step of 79-100), wherein a first row of the at least two rows visually represents static global information of the plurality of steps (Figure 1, row 1 of b3 is a representation of hidden states and the label of “hidden states” for each of the plurality of steps is a piece of static global information), and wherein a second row of the at least two rows visually represents local information of a single step associated with the first GUI (Figure 1, b3 is a snippet of b1).
Regarding claim 6, He teaches all of the limitations of claim 1, wherein
the first GUI and second GUI comprise different windows within the same primary GUI (see Figure 1, b1 and b2 are in different windows within the entire GUI).
Regarding claim 7, He teaches all of the limitations of claim 1, further comprising:
updating, with the at least one processor, at least one of the first GUI and the second GUI based on output resulting from perturbing the environment (see §6.1.2, where perturbation leads to additional testing and GUI output).
Regarding claim 8, He teaches all of the limitations of claim 1, further comprising:
determining at least one predicted rule generated by the RNN-based deep learning model based on data underlying the first and/or second GUI (see §6.1.2, where the system is used to identify a “predicted rule” based on underlying data which states that “…overshooting only happens when the ball starts from the top right corner of the maze”), wherein
the at least one predicted rule comprises at least one condition and at least one predicted response to the at least one condition (see §6.1.2, where the system is used to identify a “predicted rule” based on underlying data which states that “…overshooting only happens when the ball starts from the top right corner of the maze”); and
verifying that the RNN-based deep learning model implements the at least one predicted rule by: perturbing the environment based on the at least one condition; and analyzing a response of the RNN-based deep learning model to the perturbation based on the at least one predicted response (see §6.1.2, where the system is used to identify a “predicted rule” based on underlying data which states that “…overshooting only happens when the ball starts from the top right corner of the maze” and subsequently that the rule was implemented by the RNN because perturbing and retraining eliminates the overshooting, i.e. the retraining proves the rule for the previous version of the RNN).
Regarding claim 9, He teaches all of the limitations of claim 1, further comprising:
training, with the at least one processor, a second deep learning model using the RNN-based deep learning model (see §6.1.2 where retraining occurs).
Regarding claim 10, He teaches all of the limitations of claim 1, wherein
the environment comprises a simulator performing a simulated event, wherein the plurality of events and the plurality of states are associated with the simulated event (§6, “ball-in-maze task”, see §2).
Regarding claim 17, He teaches all of the limitations of claim 1, wherein
the parameter value represents at least one effect of a plurality of effects or at least one state of the plurality of states (see Figure 1, b1. See §5.2.3 where the heatmap represents activations of the LSTM hidden states, i.e. at least one parameter value. Activations can be interpreted as representing effects or states or both);
Regarding claim 18, He teaches all of the limitations of claim 1, wherein
the RNN-based deep learning model is based on the plurality of states, the plurality of events, and a plurality of rewards associated with at least one state of the plurality of states and/or at least one event of the plurality of events (see §2, the system is a reinforcement learning system, i.e. the RNN-based model uses states, actions, and rewards associated therewith to train).
Regarding claims 19 and 35, He is implemented on a computer and when implemented covers the system and computer program product of claims 19 and 35. For example, see §6, “Training comprised four million iterations on the simulator, with a fixed setting of the dynamics-related parameters decided by the expert…Hence, we tested the model with the combinations of 12 initial ball locations, placed at regular intervals in the outermost ring, and 10 different friction values.” A processor, a data storage device with model data representing events and associated states, and a computer readable medium having program instructions that when executed by the processor performs the method of claim 1 are all present because all such components are necessary for such a computer based simulation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 11-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over He (W. He, T. -Y. Lee, J. van Baar, K. Wittenburg and H. -W. Shen, "DynamicsExplorer: Visual Analytics for Robot Control Tasks involving Dynamics and LSTM-based Control Policies," 2020 IEEE Pacific Visualization Symposium (PacificVis), Tianjin, China, 2020, pp. 36-45, doi: 10.1109/PacificVis48177.2020.7127) in view of Wang (CN109767225A).
Regarding claim 11, He teaches all of the limitations of claim 10, but does not teach wherein the simulated event comprises a simulated electronic payment fraud determination event.
Wang teaches wherein the simulated event comprises a simulated electronic payment fraud determination event (“The invention claims a network payment based fraud detection method of self-learning sliding time window in reinforcement learning”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the invention of He to a simulated electronic payment fraud determination event as the simulated event in order to detect electronic payment fraud.
Regarding claim 12, He as modified teaches all of the limitations of claim 11, but does not teach wherein perturbing the environment comprises submitting a simulated electronic payment transaction to the RNN-based deep learning model.
Wang teaches wherein perturbing the environment comprises submitting a simulated electronic payment transaction to the RNN-based deep learning model (“in order to adapt the change of environment further comprises the following steps: according to the preset period according to the feedback…re-selecting sliding time window”) in order to adapt to changes in the transaction environment.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to, in He as modified, perturb the system via submitting a simulated electronic payment transaction to the RNN-based deep learning model in order to adapt the model to environmental changes.
Regarding claim 13, He teaches all of the limitations of claim 1, but does not teach wherein the environment comprises an electronic payment processing network, wherein the plurality of events comprise a plurality of transactions associated with transaction data, and wherein each state of the plurality of states comprises at least one of the following: a plurality of fraud determinations, a plurality of charge-backs, a plurality of cross-border transactions, or any combination thereof.
Wang teaches wherein the environment comprises an electronic payment processing network (“The invention claims a network payment based fraud detection method of self-learning sliding time window in reinforcement learning”), wherein the plurality of events comprise a plurality of transactions associated with transaction data (“the tag refers to the history whether the transaction record is marked fraudulent transaction is a transaction record in the specific time section”), and wherein each state of the plurality of states comprises at least one of the following: a plurality of fraud determinations (“the tag refers to the history whether the transaction record is marked fraudulent transaction is a transaction record in the specific time section”), a plurality of charge-backs, a plurality of cross-border transactions, or any combination thereof.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the method of He in an electronic payment processing network such that the plurality of events comprise a plurality of transactions associated with transaction data, and wherein each state of the plurality of states comprises at least one of the following: a plurality of fraud determinations, a plurality of charge-backs, a plurality of cross-border transactions, or any combination thereof in order to detect fraudulent payments in a payment system.
Regarding claim 14, He as modified teaches all of the limitations of claim 13, wherein the model data is generated based on historical transaction data (see Wang - “the tag refers to the history whether the transaction record is marked fraudulent transaction is a transaction record in the specific time section” – He as modified utilizes historical transaction data, i.e. a transaction record, for training and analyzing), further comprising: extracting at least one rule generated by the RNN-based deep learning model; and applying the at least one rule to future transactions by a transaction processing system (In general, learning to detect fraud can be considered “extracting at least one rule” and “applying the at least one rule” because such a model’s purpose is to be trained to find “rules” for recognizing fraudulent transactions and to apply such “rules” to future cases in order to detect new fraudulent transactions).
Regarding claim 15, He as modified teaches all of the limitations of claim 13, further comprising: integrating the RNN-based deep learning model with a transaction processing system processing new transaction data associated with a new transaction by: evaluating the new transaction data with the RNN-based deep learning model; and determining the state of the new transaction with the RNN-based deep learning model (In general, in He as modified, learning to detect fraud can be considered “evaluating new transaction data” and “determining the state of the new transaction” because such a model’s purpose is to be trained find fraudulent transactions and to apply such analysis to future cases in order to detect new fraudulent transactions).
Regarding claim 16, He as modified teaches all of the limitations of claim 15, further comprising: denying the new transaction in response to determining the state of the new transaction to be a fraudulent state (see Wang – “intercepting the fraudulent transaction and protecting fund security field of user and enterprise technical support”).
Response to Arguments
Applicant’s remarks filed 03/04/2026 have been fully considered.
Applicant notes that b2 of He does not represent a line chart. However, as explained herein, d2 can be considered a line chart as claimed. See §5.3 and Figure 6, where d2 is a sub-trajectory of a time series and therefore can be considered a line chart which represents a timeline.
As noted herein, b2 shows data which has more than two dimensions (at least space, time, and action) encoded in two dimensions (the flat projection in b2) see §5.2.2 “Trajectory View”, “We use the color of the trajectory to encode additional information such as the action the model takes at each frame.”
In another interpretation, b2 can be considered a representation of the LSTM hidden states. These states are more than two dimensional per He and therefore b2 is a representation of more-than-two dimensional data in two dimensions (a flat projection).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCHYLER S SANKS whose telephone number is (571)272-6125. The examiner can normally be reached 06:30 - 15:30 Central Time, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCHYLER S SANKS/ Primary Examiner, Art Unit 2129
Prosecution Insights