DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a Non-Final Office Action in response to application 18/300,344 entitled "SYSTEMS AND METHODS FOR OUTLIER DETECTION USING UNSUPERVISED MACHINE LEARNING MODELS TRAINED ON OVERSAMPLED DATA" filed on March 17, 2025, with claims 1-15 and 18-22 pending.
Status of Claims
Claims 1-4, 6-11, 14-15, and 18-20 have been amended and are hereby entered.
Claims 16-17 are cancelled.
Claims 21-22 are new
Claims 1-15 and 18-22 are pending and have been examined.
Response to Amendment
The amendment filed March 17, 2025, has been entered. Claims 1-15 and 18-22 remain pending in the application. Applicant’s amendments to the Specification, Drawings, and/or Claims have been noted in response to the Final Office Action mailed December 30, 2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on April 9, 2025 and June 23, 2025, are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the Examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 and 18-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Please see MPEP 2106 for additional information regarding Patent Subject Matter Eligibility Guidance.
Claims 1-15 and 18-22 are directed to a system, method/process, machine/apparatus, or composition of matter, which are/is one of the statutory categories of invention. (Step 1: YES).
The claimed invention is directed to an abstract idea without significantly more.
Independent Claim 1 recites:
“A system of improving access control to resources based on outlier detection using a … model that is … via…, based on an automatically triggered update routine, using oversampled …data of an initially-detected outlier, the system comprising:
comprising: in response to a first … model's prediction indicating that a first event of a plurality of events indicating …attempting to access an electronically accessible resource belongs to an outlier category, triggering an update routine for a second … model comprising a plurality of component models … to identify sub-categories of the outlier category, the update routine comprising:
generating a plurality of synthetic outlier events by oversampling the first event for use in updating the second …. model during an outlier detection process;
augmenting a set of outlier-event … data, including a plurality of prior outlier events previously detected by the first … model, with the plurality of synthetic outlier events, such that an amount of the augmented set of outlier-event … data corresponds to an amount of regular-event … data used to … the first … model;
updating the second … model via …. using the augmented set of outlier-event … data, during the outlier detection process but prior to providing the first event as input to the second … model, to generate an output indicating a respective multi-class result that events belong to, the multi-class result indicating a sub-category of a plurality of sub-categories of the outlier category;
receiving an event data set of the first event;
inputting the event data set of the first event into the second …model to generate a multi-class result indicating which sub-category of the plurality of sub-categories of the outlier category the first event belongs, wherein the second …. model is … via the …. using the augmented set of outlier-event … data;
generating, based on the multi-class result, a recommendation indicating an action to be performed based on the multi-class result for the first event;
and in response to the action indicating to suspend access to the electronically accessible resource, suspending the ….access to the electronically accessible resource.”
These limitations clearly relate to managing transactions/interactions between consumer/client, merchant, and/or financial institution. These limitations, under their broadest reasonable interpretation, cover performance of the limitation as certain methods of organizing human activity. Specific instances include instructing to “generating… for output a recommended subset …wherein the recommended subset comprises events determined to belong to the outlier category”.
The Specification defines an event as:
[036] an event may include a specific occurrence, incident, or situation …events may include a transaction, an offer relating to a (tangible or intangible) product or service, an acceptance of an offer, a series of transactions/offers/acceptances/occurrences within a time period that share at least one common character (e.g., a series of transactions involving a same user, a same merchant, a same credit card, a same account (banking, email, etc.), a same buyer, a same seller, a same geographic location, a same stock, a same class of products, a same company, a same financial institute, or the like
[038] events are credit card transactions
As such, the limitation recites a fundamental economic principles or practice and/or commercial or legal interactions. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation as a fundamental economic, commercial, or financial action, principle, or practice then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A-Prong 1: YES. The claims recite an abstract idea).
This judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements of:
[one or more processors and non-transitory, computer-readable media storing instructions that, when executed by the one or more processors, cause operations][client device's][ a client device]:
merely applying computer processing, storage, and networking technology as tools to perform an abstract idea
[machine learning][unsupervised machine learning][training/train]:
merely applying machine learning technology as tools to perform an abstract idea
are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer components and/or electronic processes. For example, the Applicant’s Specification reads:
[020] system 100 may include mobile device 122 and user terminal 124....mobile device 122 and user terminal 124 may be any computing device, including, but not limited to, a laptop computer, a tablet computer, a handheld computer, and other computer equipment (e.g., a server)...and/or mobile devices.
[021] The control circuitry may comprise any suitable processing, storage, and/or I/O circuitry.
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea and are at a high level of generality. Therefore, Claim 1 is directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
Accordingly, these additional elements, do not change the outcome of the analysis, when considered separately and as an ordered combination. The independent claim further define the abstract idea that is present and hence are abstract for the reasons presented above. The independent claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the independent claim is directed to an abstract idea. Thus, the claim is not patent eligible. (Step 2B: NO. The claims do not provide significantly more)
Independent Claim 2 recites:
“method of improving access control to resources based on outlier detection using a …model that is…, based on an automatically triggered update routine, using oversampled training data of an initially-detected outlier, the method comprising:
in connection with a first … model's prediction indicating that an event associated with a client device attempting to access an electronically accessible resource belongs to an outlier category, triggering an update routine for a second …. model trained to determine sub-categories of the outlier category, the update routine comprising:
generating a plurality of synthetic outlier events based on oversampling the event;
augmenting a set of outlier-event … data, including a plurality of prior outlier events, with the plurality of synthetic outlier events, such that an amount of the augmented set of outlier-event … data corresponds to an amount of regular-event … data used to … the first …. model;
updating the second …. model, …, using the augmented set of outlier-event training data to generate outputs indicating a respective multi-class result that events belong to, the multi-class result indicating a sub-category of a plurality of sub-categories of the an outlier category;
receiving an event data set of the event;
inputting the event data set of the event to the second … model to generate a multi-class result indicating which sub-category of the plurality of sub-categories of the outlier category the event belongs, wherein the second …. model is …. using the augmented set of outlier-event … data;
generating, based on the multi-class result, a recommendation indicating an action to be performed based on the multi-class result;
and in response to the action indicating to suspend access to the electronically accessible resource, suspending the access to the electronically accessible resource.”
These limitations clearly relate to managing transactions/interactions between consumer/client, merchant, and/or financial institution. These limitations, under their broadest reasonable interpretation, cover performance of the limitation as certain methods of organizing human activity. Specific instances include instructing to “generating… for output an identifier of the event”.
The Specification defines an event as:
[036] an event may include a specific occurrence, incident, or situation …events may include a transaction, an offer relating to a (tangible or intangible) product or service, an acceptance of an offer, a series of transactions/offers/acceptances/occurrences within a time period that share at least one common character (e.g., a series of transactions involving a same user, a same merchant, a same credit card, a same account (banking, email, etc.), a same buyer, a same seller, a same geographic location, a same stock, a same class of products, a same company, a same financial institute, or the like
[038] events are credit card transactions
As such, the limitation recites a fundamental economic principles or practice and/or commercial or legal interactions. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation as a fundamental economic, commercial, or financial action, principle, or practice then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A-Prong 1: YES. The claims recite an abstract idea).
This judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements of:
[client device's]:
merely applying computer processing, storage, and networking technology as tools to perform an abstract idea
[machine learning] [trained via unsupervised machine learning][training] [train] [via unsupervised machine learning]:
merely applying machine learning technology as tools to perform an abstract idea
are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer components and/or electronic processes. For example, the Applicant’s Specification reads:
[020] system 100 may include mobile device 122 and user terminal 124....mobile device 122 and user terminal 124 may be any computing device, including, but not limited to, a laptop computer, a tablet computer, a handheld computer, and other computer equipment (e.g., a server)...and/or mobile devices.
[021] The control circuitry may comprise any suitable processing, storage, and/or I/O circuitry.
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea and are at a high level of generality. Therefore, Claim 2 is directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
Accordingly, these additional elements, do not change the outcome of the analysis, when considered separately and as an ordered combination. The independent claim further define the abstract idea that is present and hence are abstract for the reasons presented above. The independent claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the independent claim is directed to an abstract idea. Thus, the claim is not patent eligible. (Step 2B: NO. The claims do not provide significantly more)
Dependent Claims recite additional elements.
This judicial exception is not integrated into a practical application. In particular, the recited additional elements of
Claim 3:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 4:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 5:
“user interface”: merely applying computer processing, networking, and display technologies as a tool to perform an abstract idea
Claim 6:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 7:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 8:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 9:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 10:
“machine learning”, “trained”: merely applying machine learning technology as tools to perform an abstract idea
Claim 11:
“machine learning”: merely applying machine learning technology as tools to perform an abstract idea
Claim 12: (none found: does not include additional elements and merely narrows the abstract idea)
Claim 13:
“training an unsupervised machine learning”, “training”: merely applying machine learning technology as tools to perform an abstract idea
Claim 14:
“machine learning”, “training”: merely applying machine learning technology as tools to perform an abstract idea
Claim 15:
“using control circuitry”: merely applying computer processing, networking, and display technologies as a tool to perform an abstract idea
“machine learning”, “trained”: merely applying machine learning technology as tools to perform an abstract idea
are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer components and/or electronic processes. For example, the Applicant’s Specification reads:
[020] system 100 may include mobile device 122 and user terminal 124....mobile device 122 and user terminal 124 may be any computing device, including, but not limited to, a laptop computer, a tablet computer, a handheld computer, and other computer equipment (e.g., a server)...and/or mobile devices.
[021] The control circuitry may comprise any suitable processing, storage, and/or I/O circuitry.
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea and are at a high level of generality. Therefore, the claim is directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, do not change the outcome of the analysis, when considered separately and as an ordered combination. Dependent claims further define the abstract idea that is present in their respective independent claims and hence are abstract for the reasons presented above. The dependent claims do not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the dependent claims are directed to an abstract idea. Thus, the dependent claims are not patent eligible. (Step 2B: NO. The claims do not provide significantly more)
Independent Claim 18 recites:
“A … comprising: in connection with a first … model's prediction indicating that an event associated with an accessing of an electronically accessible resource belongs to an outlier category, triggering an update routine for a second … model trained to determine a sub-category of the outlier category, the update routine comprising:
generating a plurality of synthetic outlier events based on oversampling the event;
augmenting a set of outlier-event … data, including a plurality of prior outlier events, with the plurality of synthetic outlier events;
updating the second …. model, …, using the augmented set of outlier-event … data, to generate outputs indicating a respective multi-class result that events belong to, the multi-class result indicating a sub-category of a plurality of sub-categories of the outlier category;
inputting an event data set of the event to the second … model to generate a multi-class result indicating which sub-category of the plurality of sub-categories of the outlier category the event belongs, wherein the second … model is … … on the augmented set of outlier-event training data;
generating based on the multi-class result, a recommendation indicating an action to be performed based on the multi-class result;
and in response to the action indicating to suspend access to the electronically accessible resource, suspending the access to the electronically accessible resource.”
These limitations clearly relate to managing transactions/interactions between consumer/client, merchant, and/or financial institution. These limitations, under their broadest reasonable interpretation, cover performance of the limitation as certain methods of organizing human activity. Specific instances include instructing to “generating for output an identifier of the event”.
The Specification defines an event as:
[036] an event may include a specific occurrence, incident, or situation …events may include a transaction, an offer relating to a (tangible or intangible) product or service, an acceptance of an offer, a series of transactions/offers/acceptances/occurrences within a time period that share at least one common character (e.g., a series of transactions involving a same user, a same merchant, a same credit card, a same account (banking, email, etc.), a same buyer, a same seller, a same geographic location, a same stock, a same class of products, a same company, a same financial institute, or the like
[038] events are credit card transactions
As such, the limitation recites a fundamental economic principles or practice and/or commercial or legal interactions. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation as a fundamental economic, commercial, or financial action, principle, or practice then it falls within the “Certain Methods of Organizing Human Activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A-Prong 1: YES. The claims recite an abstract idea).
This judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements of:
[non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause operations]:
merely applying computer processing, storage, and networking technology as tools to perform an abstract idea
[machine learning] [via unsupervised machine learning][trained via the unsupervised machine learning] [training] [machine learning]:
merely applying machine learning technology as tools to perform an abstract idea
are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer components and/or electronic processes. For example, the Applicant’s Specification reads:
[020] system 100 may include mobile device 122 and user terminal 124....mobile device 122 and user terminal 124 may be any computing device, including, but not limited to, a laptop computer, a tablet computer, a handheld computer, and other computer equipment (e.g., a server)...and/or mobile devices.
[021] The control circuitry may comprise any suitable processing, storage, and/or I/O circuitry.
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea and are at a high level of generality. Therefore, Claim 18 is directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
Accordingly, these additional elements, do not change the outcome of the analysis, when considered separately and as an ordered combination. The independent claim further define the abstract idea that is present and hence are abstract for the reasons presented above. The independent claim does not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the independent claim is directed to an abstract idea. Thus, the claim is not patent eligible. (Step 2B: NO. The claims do not provide significantly more)
Dependent Claims recite additional elements.
This judicial exception is not integrated into a practical application. In particular, the recited additional elements of
Claim 19:
“non-transitory computer-readable media”: merely applying computer processing, networking, and display technologies as a tool to perform an abstract idea
“machine learning”, “trained”: merely applying machine learning technology as tools to perform an abstract idea
Claim 20:
“non-transitory computer-readable media”: merely applying computer processing, networking, and display technologies as a tool to perform an abstract idea
“machine learning”, “support vector machine (SVM)”: merely applying machine learning technology as tools to perform an abstract idea
Claim 21:
“non-transitory computer-readable media”: merely applying computer processing, networking, and display technologies as a tool to perform an abstract idea
“machine learning”, “majority expert”, “trained”, “training”: merely applying machine learning technology as tools to perform an abstract idea
Claim 22:
“non-transitory computer-readable media”: merely applying computer processing, networking, and display technologies as a tool to perform an abstract idea
“machine learning”, “training”: merely applying machine learning technology as tools to perform an abstract idea
are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer components and/or electronic processes. For example, the Applicant’s Specification reads:
[020] system 100 may include mobile device 122 and user terminal 124....mobile device 122 and user terminal 124 may be any computing device, including, but not limited to, a laptop computer, a tablet computer, a handheld computer, and other computer equipment (e.g., a server)...and/or mobile devices.
[021] The control circuitry may comprise any suitable processing, storage, and/or I/O circuitry.
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea and are at a high level of generality. Therefore, the claim is directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements merely add instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, see MPEP 2106.05(f). Accordingly, these additional elements, do not change the outcome of the analysis, when considered separately and as an ordered combination. Dependent claims further define the abstract idea that is present in their respective independent claims and hence are abstract for the reasons presented above. The dependent claims do not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the dependent claims are directed to an abstract idea. Thus, the dependent claims are not patent eligible. (Step 2B: NO. The claims do not provide significantly more)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 10-14, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wong (“TRAINING A MACHINE LEARNING SYSTEM FOR TRANSACTION DATA PROCESSING”, U.S. Publication Number: 20230118240 A1), in view of Saeed (“NETWORK ANOMALY DETECTION”, U.S. Publication Number: 20230125203 A1).
Regarding Claim 1,
Wong teaches,
A system of improving access control to resources based on outlier detection using a machine learning model that is trained via unsupervised machine learning, based on an automatically triggered update routine,
(Wong [0143] There are a number of comparative approaches to detecting anomalies in data sets. These include unsupervised outlier detection, a Synthetic Minority Over-sampling Technique (SMOTE), the “Snorkel” system
Wong [0006] data that is siloed or partitioned based on access security
Wong [0002] applying machine learning systems to transaction data.)
using oversampled
(Wong [0143] There are a number of comparative approaches to detecting anomalies in data sets. These include...a Synthetic Minority Over-sampling Technique (SMOTE))
training data of an initially-detected outlier,
(Wong [0144] In unsupervised outlier detection,...where the features are input to an anomaly detection system, which is configured to identify features that...are outliers relative to the overall data distribution of those features.
Wong [0061] the model initialisation operation may comprise loading a defined machine learning model and parameters that instantiate the defined machine learning model
Wong [Abstract] obtaining a training set of data samples)
the system comprising: one or more processors and non-transitory, computer-readable media storing instructions that, when executed by the one or more processors, cause operations
(Wong [0033] memory references to a function initiated via a method call, where the function comprises computer program code that is executed by one or more processors; in a hardware implementation, an interface may comprise a wired interconnect)
comprising: in response to a first machine learning model's prediction indicating that a first event of a plurality of events indicating a client device attempting to access an electronically accessible resource
(Wong [0040] neural network architectures ...may be trained using an approach called backpropagation. During backpropagation, the neural network layers that make up each neural network architecture are initialized .... and then used to make a prediction using a set of input data from a training set
Wong [0149] help determine whether a malicious third party has gained access to payment data
Wong [0032] transaction data may be used broadly to refer actions taken with respect to one or more electronic devices.)
generating a plurality of synthetic outlier events
(Wong [0116] the partitioned data samples 1030 are passed to a synthetic data generation stage 1040. During the synthetic data generation stage 1040, a set of synthetic data samples 1050 is generated)
by oversampling
(Wong [0143] There are a number of comparative approaches to detecting anomalies in data sets. These include...a Synthetic Minority Over-sampling Technique (SMOTE))
the first event for use in updating the second machine learning model during an outlier detection process;
(Wong [0150] The output of the machine learning system may be used as is, or provide the basis for a further pipeline of processing (e.g., as an input feature to another machine learning system trained to predict other output data).
Wong [0025] an indication of whether a transaction or entity is “normal” or “anomalous” ... that is then useable to prevent fraud ....machine learning systems may apply machine learning models that are updated as more transaction data is obtained, e.g. that are constantly trained based on new data)
augmenting a set of outlier-event training data, including a plurality of prior outlier events previously detected by the first machine learning model, with the plurality of synthetic outlier events, such that an amount of the augmented set of outlier-event training data corresponds to an amount of regular-event training data used to train the first machine learning model;
(Wong [0116] The synthetic data samples 1050 thus comprise mixed pairs 1052, 1054 from the first and second set of features 1032, 1034.
Wong [0117] Following the synthetic data generation stage 1040, the original partitioned data 1030 and the synthetic data samples 1050 are passed to a data labelling stage 1060. ... The augmented data set 1070 may then be used to train a binary classifier that implements the machine learning system of previous examples.
Wong [0125] At block 1208, synthetic data samples are generated by combining features from the two feature sets that respectively relate to two different ones of the set of uniquely identifiable entities. For example, each data sample may be directly ...or indirectly...associated with a particular uniquely identifiable entity
Wong [0128] the observable features are derived from a function of transaction data within a predefined temporal window for the current transaction. For example, this may comprise a defined time range (such as within 24 hours) and/or a defined number of transactions (e.g., the last 3 actions). )
updating the second machine learning model via unsupervised machine learning using the augmented set of outlier-event training data, during the outlier detection process but prior to providing the first event as input to the second machine learning model
(Wong [0150] data augmentation method for training that synthesises binary labels for unlabelled historical training data. ...Once trained on this augmented dataset, the machine learning system may be applied to new data samples {C*, O*} to determine if an anomaly is present. The output of the machine learning system may be used as is, or provide the basis for a further pipeline of processing (e.g., as an input feature to another machine learning system trained to predict other output data).
Wong [0061] the machine learning system 508 may perform any defined pre-processing prior to application of the machine learning model)
receiving an event data set of the first event;
(Wong [0044] The machine learning server 150 implements a machine learning system 160 for the processing of transaction data. The machine learning system 160 is arranged to receive input data 162 and to map this to output data 1...the machine learning system 160 receives at least transaction data associated with the particular transaction)
generating, based on the multi-class result a recommendation indicating an action to be performed based on the multi-class result for the first event
(Wong [0044] determine whether the transaction is to be authorised (i.e., approved) or declined.)
and in response to the action indicating to suspend access to the electronically accessible resource, suspending the client device's access to the electronically accessible resource.
(Wong [0042] a set of client devices 110 that are configured to initiate a transaction.
Wong [0032] transaction data may be used broadly to refer actions taken with respect to one or more electronic devices.
Wong [0139] This may comprise generating control data to control whether at least one transaction within the transaction data is...denied based on the value output... a threshold may be applied to the output...and values higher than the threshold (representing an “anomaly”) may be declined)
Wong does not explicitly teach belongs to an outlier category, triggering an update routine for a second machine learning model comprising a plurality of component models trained to identify sub-categories of the outlier category, the update routine comprising; to generate an output indicating a respective multi-class result that events belong to, the multi-class result indicating a sub-category of a plurality of sub-categories of the outlier category; inputting the event data set of the first event into the second machine learning model to generate a multi-class result indicating which sub-category of the plurality of sub-categories of the outlier category the first event belongs, wherein the second machine learning model is trained via the unsupervised machine learning using the augmented set of outlier-event training data;
Saeed explicitly teaches,
belongs to an outlier category,
(Saeed [0025] an anomaly classifier 240, a known anomaly model 250, an unknown anomaly analyser 260
Saeed [0026] detect new types of previously unknown anomalies involving those computer devices. When a new type of anomaly is detected
Saeed [0047] indicating the classification of the known type of anomaly)
triggering an update routine for a second machine learning model comprising a plurality of component models
(Saeed [0009] receiving an indication confirming that the network traffic associated with the cluster is anomalous such that the updated second model represents the characteristics of the network traffic)
trained to identify sub-categories of the outlier category, the update routine comprising;
(Saeed [0046] the known anomaly model represents the characteristics of network traffic which are associated with known types of anomalies involving the set of devices....can be used to train a machine learning model to classify network traffic having the same or similar characteristics.
Saeed [0056] new type of anomaly (i.e. that associated with the cluster) to a set of training data for the known anomaly model 250 and retraining the model 250.
Saeed [0025] an anomaly classifier 240, a known anomaly model 250, an unknown anomaly analyser)
to generate an output indicating a respective multi-class result that events belong to, the multi-class result indicating a sub-category of a plurality of sub-categories of the outlier category;
(Saeed [0005] anomaly-based intrusion detection systems work by comparing a device's behaviour to the “normal” behaviour that is expected for the device. Any deviations from the device's “normal” behaviour are then considered to be anomalies that could indicate an attack involving that device
Saeed [0025] an anomaly classifier 240, a known anomaly model 250, an unknown anomaly analyser)
inputting the event data set of the first event into the second machine learning model to generate a multi-class result indicating which sub-category of the plurality of sub-categories of the outlier category the first event belongs,
(Saeed [Claim 3] obtaining a second model representing the characteristics of network traffic associated with known types of anomalies involving the set of devices, wherein analysing the network traffic using the model to identify anomalous network traffic....comprises using the second model to identify anomalous network traffic that is not associated with a known type of anomaly)
wherein the second machine learning model is trained via the unsupervised machine learning using the augmented set of outlier-event training data;
(Saeed [0041] the system 200 can use any suitable machine-learning technique to train the model ... One suitable technique for learning the normal behaviour model 230 is the hierarchical density-based spatial clustering of applications with noise (HDBSCAN) technique. HDBSCAN is an unsupervised density based algorithm
Saeed [0046] the method 300 obtains the known anomaly model 250.)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the outlier classification teachings of Saeed for “for detecting anomalies in a computer network …to identify anomalous network traffic associated with the set of devices. ” (Saeed [Abstract]). The modification would have been obvious, because it is merely applying a known technique (i.e. outlier classification) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. “provides an indication that either (i) the network traffic associated with a cluster relates to a new type of anomaly involving the set of devices or (ii) that no new types of anomaly are present” Saeed [Abstract])
Claim 2 is rejected on the same basis as Claim 1.
Regarding Claim 3,
Wong and Saeed teach the outlier detection of Claim 2 as described earlier.
Wong teaches,
for each data set of a plurality of respective events,
(Wong [0005] many thousands of transactions need to be processed every second)
inputting each event data set into the second machine learning model; and
(Wong [0066] The machine learning system 600 comprises a first processing stage 603 and a second processing stage
Wong [0150] The output of the machine learning system may be used as is, or provide the basis for a further pipeline of processing (e.g., as an input feature to another machine learning system trained to predict other output data).)
generating an identifier for a respective event of the plurality of respective events based on a respective output from the second machine learning model, wherein the identifier indicates whether the event belongs to the outlier category.
(Wong [0025] an indication of whether a transaction or entity is “normal” or “anomalous”
Wong [0055] a value of “0.2” may be a common output for a “normal” event and a value of “0.8” may be seen as being over a threshold for a typical “anomalous” or fraudulent event.
Wong [0139] a threshold may be applied to the output of the supervised machine learning system and values higher than the threshold (representing an “anomaly”) may be declined while those below the threshold may be approved (representing “normal” actions))
Regarding Claim 4,
Wong and Saeed teach the outlier detection of Claim 3 as described earlier.
Wong teaches,
generating, based on the respective outputs from the second machine learning model, for output a recommended subset among the plurality of respective events, wherein the recommended subset comprises events determined to belong to the outlier category by the second machine learning model.
(Wong [0139] a threshold may be applied to the output of the supervised machine learning system and values higher than the threshold (representing an “anomaly”) may be declined while those below the threshold may be approved (representing “normal” actions))
Regarding Claim 10,
Wong and Saeed teach the outlier detection of Claim 2 as described earlier.
Wong does not teach wherein the second machine learning model comprises a plurality of component models each of which is trained to identify a respective sub-category of the plurality of sub-categories of the outlier category.
Saeed teaches,
wherein the second machine learning model comprises a plurality of component models each of which is trained to identify a respective sub-category of the plurality of sub-categories of the outlier category.
(Saeed [0046] the known anomaly model represents the characteristics of network traffic which are associated with known types of anomalies involving the set of devices....can be used to train a machine learning model to classify network traffic having the same or similar characteristics.
Saeed [0056] new type of anomaly (i.e. that associated with the cluster) to a set of training data for the known anomaly model 250 and retraining the model 250.
Saeed [0025] an anomaly classifier 240, a known anomaly model 250, an unknown anomaly analyser)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the outlier classification teachings of Saeed for “for detecting anomalies in a computer network …to identify anomalous network traffic associated with the set of devices. ” (Saeed [Abstract]). The modification would have been obvious, because it is merely applying a known technique (i.e. outlier classification) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. “provides an indication that either (i) the network traffic associated with a cluster relates to a new type of anomaly involving the set of devices or (ii) that no new types of anomaly are present” Saeed [Abstract])
Regarding Claim 11,
Wong and Saeed teach the outlier detection of Claim 10 as described earlier.
Wong does not teach wherein the second machine learning model generates the multi-class result based on one or more outputs of the plurality of component models indicating to which one or more sub-categories of the plurality of sub-categories of the outlier category the event belongs.
Saeed teaches,
wherein the second machine learning model generates the multi-class result based on one or more outputs of the plurality of component models indicating to which one or more sub-categories of the plurality of sub-categories of the outlier category the event belongs.
(Saeed [0025] an anomaly classifier 240, a known anomaly model 250, an unknown anomaly analyser
[0046] the known anomaly model represents the characteristics of network traffic which are associated with known types of anomalies involving the set of devices. For example, various attacks or modes of failure may already be known
Saeed [0050] the method 300 clusters the unknown anomalous network traffic into clusters of network traffic that share similar characteristics.)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the outlier classification teachings of Saeed for “for detecting anomalies in a computer network …to identify anomalous network traffic associated with the set of devices. ” (Saeed [Abstract]). The modification would have been obvious, because it is merely applying a known technique (i.e. outlier classification) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. “provides an indication that either (i) the network traffic associated with a cluster relates to a new type of anomaly involving the set of devices or (ii) that no new types of anomaly are present” Saeed [Abstract])
Regarding Claim 12,
Wong and Saeed teach the outlier detection of Claim 11 as described earlier.
Wong teaches,
selecting, from a plurality of candidate reactions, a reaction the action to be performed based on the multi-class result
(Wong [0044] determine whether the transaction is to be authorised (i.e., approved) or declined.)
Regarding Claim 13,
Wong and Saeed teach the outlier detection of Claim 11 as described earlier.
Wong teaches,
obtaining at least one of the plurality of component models by training an unsupervised machine learning model using at least a portion of oversampled training data,
(Wong [0143] There are a number of comparative approaches to detecting anomalies in data sets. These include unsupervised outlier detection, a Synthetic Minority Over-sampling Technique (SMOTE), the “Snorkel” system)
wherein the portion of the oversampled training data comprise synthetic outlier events deemed to belong to the sub-category of the outlier category that corresponds to the at least one component model.
(Wong [0116] the partitioned data samples 1030 are passed to a synthetic data generation stage 1040. During the synthetic data generation stage 1040, a set of synthetic data samples 1050 is generated
Wong [0116] The synthetic data samples 1050 thus comprise mixed pairs 1052, 1054 from the first and second set of features 1032, 1034.
Wong [0125] At block 1208, synthetic data samples are generated by combining features from the two feature sets that respectively relate to two different ones of the set of uniquely identifiable entities. For example, each data sample may be directly ...or indirectly...associated with a particular uniquely identifiable entity
Wong [0138] as application of a threshold to output a binary label of “anomaly” or “not an anomaly”.)
Regarding Claim 14,
Wong and Saeed teach the outlier detection of Claim 2 as described earlier.
Wong teaches,
(i) a first prior outlier event detected by the second machine learning model;
(Wong [0150] The output of the machine learning system may be used as is, or provide the basis for a further pipeline of processing (e.g., as an input feature to another machine learning system trained to predict other output data).)
(ii) a second prior outlier event detected …. is trained using event data sets of sample regular events of a regular category that include activities deemed to be no or low risk to the electronically accessible resource;
(Wong [0055] In this case, a value of “0” may represent a transaction that matches normal patterns of activity for one or more of a user, merchant and issuing bank... a value of “0.2” may be a common output for a “normal” event)
(iii) a third prior outlier event determined by a user; or (iv) a synthetic outlier event determined based on the first prior outlier event the second prior outlier event, or the third prior outlier event.,
(Wong [0063] based on feedback from a user (e.g., when the user reports fraud or indicates that a payment card or account was compromised from a certain date).)
Claim 18 is rejected on the same basis as Claim 1.
Regarding Claim 19,
Wong and Saeed teach the outlier detection of Claim 18 as described earlier.
Wong teaches,
and wherein the instructions that, when executed by the one or more processors, cause operations further comprising selecting, from a plurality of candidate reactions the action to be performed based on the multi-class result
(Wong [0025] an indication of whether a transaction or entity is “normal” or “anomalous” ... that is then useable to prevent fraud ....machine learning systems may apply machine learning models that are updated as more transaction data is obtained, e.g. that are constantly trained based on new data
Wong [0044] determine whether the transaction is to be authorised (i.e., approved) or declined.)
Wong does not explicitly teach the second machine learning model comprises a plurality of component models each of which is trained to identify a respective sub-category of the plurality of sub-categories of the outlier category;
Saeed explicitly teaches,
the second machine learning model comprises a plurality of component models each of which is trained to identify a respective sub-category of the plurality of sub-categories of the outlier category;
(Saeed [0009] receiving an indication confirming that the network traffic associated with the cluster is anomalous such that the updated second model represents the characteristics of the network traffic
Saeed [0026] detect new types of previously unknown anomalies involving those computer devices. When a new type of anomaly is detected
Saeed [0047] indicating the classification of the known type of anomaly)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the outlier classification teachings of Saeed for “for detecting anomalies in a computer network …to identify anomalous network traffic associated with the set of devices. ” (Saeed [Abstract]). The modification would have been obvious, because it is merely applying a known technique (i.e. outlier classification) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. “provides an indication that either (i) the network traffic associated with a cluster relates to a new type of anomaly involving the set of devices or (ii) that no new types of anomaly are present” Saeed [Abstract])
Claims 5 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wong and Saeed in view of Cella (“TRANSACTION PLATFORMS WHERE SYSTEMS INCLUDE SETS OF OTHER SYSTEMS”, U.S. Publication Number: 20230206329 A1).
Regarding Claim 5,
Wong and Saeed teach the outlier detection of Claim 4 as described earlier.
Wong teaches,
generating for output, on a user interface, a second recommendation to a user, wherein the second recommendation indicates that the event be further reviewed; providing for output, on the user interface, the event data set of the event or access to the event data set;
(Wong [0033] The term “interface” is used herein to refer to any physical and/or logical interface that allows for one or more of data input and data output.
Wong [0063] The transaction data may be saved along with one or more of the output of the machine learning system 508 (e.g., the scalar fraud or anomaly probability) and a final result of the transaction...a transaction may only be labelled as anomalous via later review by an analyst...or based on feedback from a user)
and based on the user selection, generating for output, on the user interface, the confirmed or modified identifier of the event.
(Wong [0063] based on feedback from a user (e.g., when the user reports fraud or indicates that a payment card or account was compromised from a certain date). In these cases, ground truth labels.)
Wong does not teach generating for output, on the user interface, options for a user to select from, the options including at least one of confirming the identifier or modifying the identifier; receiving, via the user interface, a user selection of one of the options.
Cella teaches,
generating for output, on the user interface, options for a user to select from, the options including at least one of confirming the identifier or modifying the identifier;
(Cella [0129] transactions including ones involving... machine learning, artificial intelligence and including neural net systems trained for pattern recognition, for classification of one or more parameters, characteristics, or phenomena
Cella [0258] a display interface
Cella [2312] present a result of the first automated portion to a human for review and validation, and can perform a second automated portion of the workflow based on the review and validation of the result of the first automated portion based on a result of the review and validation by the human.
Cella [0404] data determinations (e.g., accuracy, timing, amount of data), and/or actuator confirmation capability) of components of the service)
receiving, via the user interface, a user selection of one of the options
(Cella [0860] information may be displayed, for example, in response to a user interaction...(e.g., via user selection of the element or user selection ...))
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the user interface option selections of Cella where “information may be displayed, for example, in response to a user interaction” (Cella [0860]). The modification would have been obvious, because it is merely applying a known technique (i.e user interface option selections) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because “validation of the result of the … automated portion based on a result of the review and validation by the human” Cella [2312])
Regarding Claim 20,
Wong and Saeed teach the outlier detection of Claim 18 as described earlier.
Wong does not teach wherein the second machine learning model comprises a one-class support vector machine (SVM), an isolation forests model, a robust covariance model, or a local outlier factor model.
Cella teaches,
wherein the second machine learning model comprises a one-class support vector machine (SVM), an isolation forests model, a robust covariance model, or a local outlier factor model.
(Cella [0592] RBF networks may use kernel methods such as support vector machines (SVM)
Cella [0545] a machine learning system that has one... classifiers
Cella [0673] one or more leaves representing one... class labels
Cella [2172] covariance matrix of the data)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the support vector machine of Cella where “use kernel methods such as support vector machines (SVM)” (Cella [0592]). The modification would have been obvious, because it is merely applying a known technique (i.e support vector machine) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because “perform high-level determination to output a feature vector” Cella [3319])
Claims 6 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Wong, Saeed, and Cella in view of Patel (“SYSTEMS AND METHODS FOR PREDICTION OF AUTOMOTIVE WARRANTY FRAUD”, U.S. Publication Number: 20190213605 A1).
Regarding Claim 6,
Wong, Saeed, and Cella teach the outlier detection of Claim 5 as described earlier.
Wong teaches,
determining at least one of: a first count of events of the plurality of respective events;
(Wong [0067] prior transactions may be selected from a predefined time range ...and/or a predefined number of transactions.)
Wong does not teach a second count of events of the plurality of respective events that are determined to belong to the outlier category by the second machine learning model; or a third count of events of the plurality of respective events that are determined to belong to the outlier category by the second machine learning model but determined to not belong to the outlier category by the user; assessing performance of the second machine learning model based on at least two of the first count, the second count, or the third count; and generating, for output, a report regarding the performance of the second machine learning model.
Patel teaches,
a second count of events of the plurality of respective events that are determined to belong to the outlier category by the second machine learning model; or a third count of events of the plurality of respective events that are determined to belong to the outlier category by the second machine learning model but determined to not belong to the outlier category by the user;
(Patel [Claim 7] based on a predictive fraud detection model generated by one or more machine learning techniques
Patel [0006] Model Validation in identifying fraud claim in out of sample data by using Confusion Matrix
Patel [Figure 13]
PNG
media_image1.png
448
530
media_image1.png
Greyscale
Patel [0129] Model performance ...are given by confusion matrix
Patel [FIG. 19A] “FN - Fraudulent to Non-Fraudulent”
PNG
media_image2.png
358
542
media_image2.png
Greyscale
)
assessing performance of the second machine learning model based on at least two of the first count, the second count, or the third count; and
(Patel [0006] by using Confusion Matrix; and/or incorporating smart statistical models that discover, learn and predict fraud claims
Patel [Figure 13]
PNG
media_image1.png
448
530
media_image1.png
Greyscale
)
generating, for output, a report regarding the performance of the second machine learning model.
(Patel [0045] may include displaying a message on a screen, reproducing a sound via a speaker, or other appropriate output to alert the operator... alerting the operator to the determination that fraud is unlikely by displaying a message or other appropriate output.)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the model performance validation teachings of Patel “to classify data into a first classification or a second classification.” ("data cleaning is to perform outlier detection" Patel [0053] and "Model performance ...are given by confusion matrix " Patel [0129]). The modification would have been obvious, because it is merely applying a known technique (i.e. model performance validation) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because " for training and validation of models." Patel [0031])
Regarding Claim 7,
Wong, Saeed, Cella, and Patel teach the outlier detection of Claim 6 as described earlier.
Wong does not teach wherein assessing performance of the second machine learning model based on at least two of the first count, the second count, or the third count comprises: generating a confusion matrix based on the first count, the second count, and the third count; and assessing the performance of the second machine learning model based on the confusion matrix.
Patel teaches,
wherein assessing performance of the second machine learning model based on at least two of the first count, the second count, or the third count comprises: generating a confusion matrix based on the first count, the second count, and the third count; and assessing the performance of the second machine learning model based on the confusion matrix.
(Patel [Claim 7] based on a predictive fraud detection model generated by one or more machine learning techniques
Patel [0006] Model Validation in identifying fraud claim in out of sample data by using Confusion Matrix
Patel [Figure 13]
PNG
media_image1.png
448
530
media_image1.png
Greyscale
Patel [0129] Model performance ...are given by confusion matrix)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the model performance validation teachings of Patel “to classify data into a first classification or a second classification.” ("data cleaning is to perform outlier detection" Patel [0053] and "Model performance ...are given by confusion matrix " Patel [0129]). The modification would have been obvious, because it is merely applying a known technique (i.e. model performance validation) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because " for training and validation of models." Patel [0031])
Claims 8, 9, 15, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Wong, Saeed, Cella, and Patel in view of Li (“CONFIGURING MACHINE LEARNING MODEL THRESHOLDS IN MODELS USING IMBALANCED DATA SETS”, U.S. Patent Number: US 11526606 B1).
Regarding Claim 8,
Wong, Saeed, Cella, and Patel teach the outlier detection of Claim 6 as described earlier.
Wong does not teach wherein assessing performance of the second machine learning model based on at least two of the first count, the second count, or the third count comprises: assessing performance of the second machine learning model based on a first ratio of the third count to the first count or a second ratio of the third count to the second count.
Patel teaches,
wherein assessing performance of the second machine learning model based on at least two of the first count, the second count, or the third count
(Patel [Figure 13]
PNG
media_image1.png
448
530
media_image1.png
Greyscale
Patel [0129] Model performance ...are given by confusion matrix)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the model performance validation teachings of Patel “to classify data into a first classification or a second classification.” ("data cleaning is to perform outlier detection" Patel [0053] and "Model performance ...are given by confusion matrix " Patel [0129]). The modification would have been obvious, because it is merely applying a known technique (i.e. model performance validation) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because " for training and validation of models." Patel [0031])
Patel does not teach comprises: assessing performance of the second machine learning model based on a first ratio of the third count to the first count or a second ratio of the third count to the second count.
Li teaches,
comprises: assessing performance of the second machine learning model based on a first ratio of the third count to the first count or a second ratio of the third count to the second count.
(Li [Col 3, Lines 33-40] identifying a threshold between the majority class and the minority class in the machine learning model... the ratio between true positive classifications and false positive classifications, and recall, or the ratio between true positive classifications and the sum of true positive and false negative classifications)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the classification ratio teachings of Li for “the ratio between true positive classifications and false positive classifications.” (Li [Col 3, Lines 33-40]). The modification would have been obvious, because it is merely applying a known technique (i.e. classification ratio) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because “selection of the wrong threshold value may negatively impact the accuracy of the machine learning model, and thus may result in a substantial numbers of false positives (e.g., classifying negative events, such as malicious activity or fraudulent transactions, as positive events) and/or false negatives (e.g., classifying positive events, such as normal activity or legitimate transactions, as negative events)” Li [Col 1, Line 63 to Col 2, Line 4])
Regarding Claim 9,
Wong, Saeed, Cella, Patel and Li teach the outlier detection of Claim 8 as described earlier.
Wong does not teach updating the second machine learning model based on the performance.
Saeed teaches,
updating the second machine learning model
(Saeed [0009] receiving an indication confirming that the network traffic associated with the cluster is anomalous such that the updated second model represents the characteristics of the network traffic)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the outlier classification teachings of Saeed for “for detecting anomalies in a computer network …to identify anomalous network traffic associated with the set of devices. ” (Saeed [Abstract]). The modification would have been obvious, because it is merely applying a known technique (i.e. outlier classification) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. “provides an indication that either (i) the network traffic associated with a cluster relates to a new type of anomaly involving the set of devices or (ii) that no new types of anomaly are present” Saeed [Abstract])
Saeed does not teach based on the performance.
Patel teaches,
based on the performance
(Patel [0129] Model performance ...are given by confusion matrix)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the model performance validation teachings of Patel “to classify data into a first classification or a second classification.” ("data cleaning is to perform outlier detection" Patel [0053] and "Model performance ...are given by confusion matrix " Patel [0129]). The modification would have been obvious, because it is merely applying a known technique (i.e. model performance validation) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because " for training and validation of models." Patel [0031])
Regarding Claim 15,
Wong and Saeed teach the outlier detection of Claim 2 as described earlier.
Wong teaches,
using the regular-event training data indicating data sets of sample regular events of a regular category that include activities deemed to be no or low risk to the electronically accessible resource,
(Wong [0055] In this case, a value of “0” may represent a transaction that matches normal patterns of activity for one or more of a user, merchant and issuing bank... a value of “0.2” may be a common output for a “normal” event)
to predict whether the event belongs to the outlier category.
(Wong [0044] the output of the machine learning system 160 may comprise a label, alert or other indication of fraud, or general malicious or anomalous activity.)
Wong does not teach , wherein (i) the second machine learning model is a minority expert model (ii) the first machine learning model is a majority expert model trained; (iii) the majority expert model is trained.
Li teaches,
wherein (i) the second machine learning model is a minority expert model
(Li [Col 3, Lines 16-21] a training data set of inputs for training such models may include a higher proportion of inputs ...having a second classification (also referred to as a “minority class”).)
(ii) the first machine learning model is a majority expert model trained;
(Li [Col 3, Lines 16-19] a training data set of inputs for training such models may include a higher proportion of inputs having a first classification (also referred to as a “majority class”)
Li [Col 3, Lines 17-17] a training data set of inputs for training such models)
(iii) the majority expert model is trained.
(Li [Col 3, Lines 16-19] a training data set of inputs for training such models may include a higher proportion of inputs having a first classification (also referred to as a “majority class”)
Li [Col 3, Lines 17-17] a training data set of inputs for training such models)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the minority/majority expert models of Li having “a training data set of inputs for ...having a second classification (also referred to as a “minority class”).” (Li [Col 3, Lines 33-40]). The modification would have been obvious, because it is merely applying a known technique (i.e. minority/majority expert models) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because “determine whether input data and the devices from which the input data is obtained is operating normally or abnormally” Li [Col 1, Lines 21-23])
Regarding Claim 21,
Wong and Saeed teach the outlier detection of Claim 18 as described earlier.
Wong teaches,
that include activities deemed to be no or low risk to electronically accessible resources
(Wong [0055] In this case, a value of “0” may represent a transaction that matches normal patterns of activity for one or more of a user, merchant and issuing bank... a value of “0.2” may be a common output for a “normal” event)
Wong does not teach the wherein the first machine learning model is a majority expert model trained using regular-event training data indicating data sets of sample regular events of a regular category
Li teaches,
wherein the first machine learning model is a majority expert model trained
(Li [Col 3, Lines 16-19] a training data set of inputs for training such models may include a higher proportion of inputs having a first classification (also referred to as a “majority class”)
Li [Col 3, Lines 17-17] a training data set of inputs for training such models)
using regular-event training data indicating data sets of sample regular events of a regular category
(Li [Col 1, Lines 21-23] anomaly detection systems used to determine whether input data and the devices from which the input data is obtained is operating normally or abnormally
Li [Col 3, Lines 30-31] normal activity may correspond to the majority class)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Wong to incorporate the minority/majority expert models of Li having “a training data set of inputs for ...having a second classification (also referred to as a “minority class”).” (Li [Col 3, Lines 33-40]). The modification would have been obvious, because it is merely applying a known technique (i.e. minority/majority expert models) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because “determine whether input data and the devices from which the input data is obtained is operating normally or abnormally” Li [Col 1, Lines 21-23])
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Wong, Saeed, Cella, Patel and Li in view of Metzler (“FACILITY SURVEILLANCE SYSTEMS AND METHODS”, U.S. Publication Number: US 20220005332 A1).
Regarding Claim 22,
Wong, Saeed, Cella, Patel and Li teach the outlier detection of Claim 21 as described earlier.
Wong does not teach wherein an amount of the augmented set of outlier-event training data corresponds to an amount of regular-event training data used to train the first machine learning model.
Metzler teaches,
wherein an amount of the augmented set of outlier-event training data corresponds to an amount of regular-event training data used to train the first machine learning model.
(Metzler [0485] For example, such a synthetic generation of virtual samples allows generating an equal amount of training data for the abnormal state of security events as for those of the normal state.
Metzler [0090] the state detection functionality comprises using at least one machine learning algorithm for training a state detection model and/or the at least one machine learning algorithm is provided in the first computing unit
Metzler [0146] the algorithmic models are improved and gradually adapted over time)
It is prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the outlier detection of Cella to incorporate the augmented data equaling regular data teachings of Metzler where “a synthetic generation of virtual samples allows generating an equal amount of training data for the abnormal state of security events as for those of the normal state " Metzler [0485]). The modification would have been obvious, because it is merely applying a known technique (i.e. augmented data equaling regular data) to a known concept (i.e. outlier detection) ready for improvement to yield predictable result (i.e. because "In contrast thereto, synthetic samples according to this aspect of the invention can be freely generated in various realizations of the abnormal state to help the decision maker, automatic state detection or criticality-classification means with its decision." Metzler [0485])
Response to Remarks
Applicant's arguments filed on March 17, 2025, have been fully considered and Examiner’s remarks to Applicant’s amendments follow.
Response Remarks on Claim Rejections - 35 USC § 112
Applicant's amendments rectify the previous rejections under 35 USC § 112.
The rejection under 35 USC § 112 is lifted.
Response Remarks on Claim Rejections - 35 USC 101
The Applicant states:
“The claims are patent- eligible because they provide a technical improvement as provided for in Example 48 of the USPTO's July 2024 Subject Matter Eligibility Guidance ("PEG") Examples…. According to the PEG, claim 2 of Example 48 is patent-eligible because the example disclosure describes a technical problem (e.g., "devices that capture audio cannot properly distinguish different speech sources belonging to the same class..."), an improvement that solves the problem (e.g., "the disclosure states that this invention offers an improvement over existing speech-separation methods by ..."), and the features of example claim 2 describe that improvement… this pending application are patent-eligible for similar reasons"
Examiner responds:
Claim 2 of Example 48 is patent eligible due to specific claim language:
“… (f) synthesizing speech waveforms from the masked clusters, wherein each speech waveform corresponds to a different source sn; (g) combining the speech waveforms to generate a mixed speech signal x' by stitching together the speech waveforms corresponding to the different sources sn, excluding the speech waveform from a target source ss such that the mixed speech signal x' includes speech waveforms from the different sources sn and excludes the speech waveform from the target source ss;”
Step (f) requires conversion into separate speech signals in the time domain. Synthesizing speech waveforms from a cluster of numbers is not a process that can be practically performed in the human mind.
Step (f) is not a method of organizing human activity because it does not fall within the enumerated sub-groupings of fundamental economic principles or practices, commercial or legal interactions, and managing personal behavior and relationships or interactions between people.
Similarly, step (g) combines the speech waveforms to generate a mixed speech signal by stitching together speech waveforms corresponding to the different sources excluding the speech signal from a target source.
Moreover, MPEP 2106.04(a)(2), subsection I states generating a mixed speech signal such that it includes speech signals from different sources and excludes the speech signal from a target source is not a process that can be practically performed in a human mind. Therefore, step (g) is neither a mathematical concept nor a mental process. In addition, step (g) is not a method of organizing human activity because it does not fall within the enumerated sub-groupings.
Ultimately, the instant application in no way mirrors Claim 2 of Example 48.
Outlier detection, minority/majority sub-population classification, oversampling, synthetic data generation, access control, and model updating all express abstract ideas. They all amount to gathering, sharing, and manipulation of data which expresses an Abstract Idea [Intellectual Ventures I v. Capital One Fin. Corp., 850 F.3d 1332, 121 USPQ2d 1940 (Fed. Cir. 2017) “collecting, displaying, and manipulating data” was considered part of the abstract idea], and Selecting A Particular Data Source or Type Of Data To Be Manipulated [Selecting information, based on types of information and availability of information in a power-grid environment, for collection, analysis and display, Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354-55, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016)]
In the absence of unexpected results, changes or alteration of sequence do not make for a patentable invention, see Ex parte Rubin, 128 USPQ 440 (Bd. App. 1959) ; In re Burhans, 154 F.2d 690, 69 USPQ 330 (CCPA 1946); In re Gibson, 39 F.2d 975, 5 USPQ 230 (CCPA 1930)
The rejection under 35 USC § 101 remains.
Response Remarks on Claim Rejections - 35 USC § 103
Applicant's amendments required the application of new/additional prior art.
Applicant’s remarks regarding the rejection made under 35 USC § 103 are rendered moot by the introduction of new prior art.
Therefore, the rejection under 35 USC § 103 remains.
Prior Art Cited But Not Applied
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Japkowicz (“SUPERVISED VERSUS UNSUPERVISED BINARY-LEARNING BY FEEDFORWARD NEURAL NETWORKS”, 2001) teaches, “Binary classification is typically achieved by supervised learning methods. Nevertheless, it is also possible using unsupervised schemes. This paper describes a connectionist unsupervised approach to binary classification and compares its performance to that of its supervised counterpart.”
Ben Fadhel (“METHOD AND APPARATUS FOR AUGMENTED DATA ANOMALY DETECTION”, U.S. Publication Number: 20210287071 A1) proposes data anomaly detection method and apparatus in which a deep neural network is trained on baseline data. Sequences of statistics of each layer of the deep neural network are saved, processed and used to train an LSTM autoencoder across a variety of reconstruction error thresholds, and a preferred threshold is selected for an optimized autoencoder. In an Inference mode, a data sample is presented to the autoencoder; the reconstruction error is calculated and compared to the threshold. If it is above the threshold, then the data sample is an out-of-distribution sample, and the sample is tagged as anomalous.
Dods (“NEURAL NETWORK CLASSIFIERS FOR BLOCK CHAIN DATA STRUCTURES”, U.S. Patent Number: 10873456 B1) provides a neural network enabled interface server and blockchain interface establishing a blockchain network implementing event detection, tracking and management for rule based compliance, with significant implications for anomaly detection, resolution and safety and compliance reporting.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHINEDU EKECHUKWU whose telephone number is (571)272-4493. The examiner can normally be reached on Mon-Fri 9 AM ET to 3:30 PM ET.
Examiner interviews are available via telephone and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Christine Behncke, can be reached on (571) 272-8103. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov.
Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHINEDU U. EKECHUKWU/Examiner, Art Unit 3695
/CHRISTINE M Tran/Supervisory Patent Examiner, Art Unit 3695
Prosecution Insights