DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The following is a Non-Final Office Action in response to applicant’s filing on
February 10, 2026. Claims 11, 20 and 30 were canceled. Claims 1, 4, 6, 10, 12, 14, 23, 25, 27-29 were amended. Claim 31 is newly added. Claims 1-10, 12-19, 21-29 and 31 are pending, of which claims 1, 14, 23 and 25 are in independent form.
Response to Arguments
Applicant’s amendment regarding claim 4 obviates the rejection under 35 USC § 112 (b), therefore the rejection is withdrawn.
Response to Arguments
Applicant’s arguments with respect to claim(s) are rejected, under 35 USC 103(a), have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter.
On Pages 12-13 of remarks, Applicant argues that “Baracaldo-Angel fails to disclose or suggest at least "associate a legitimacy score with one or more of a plurality of data signatures, wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value; and send the legitimacy score associated with the one or more of the plurality of data signatures to a network entity," as recited in independent claim 1.
Applicant, Further argues “Baracaldo-Angel fails to disclose or suggest "receive a first wireless transmission that includes a configuration from a network node assigning at least one data signature to be reported by the UE with a dataset for a machine learning (ML) model,”, as recited in independent claim 23.
Applicant’s arguments, with respect to the rejection(s) of claim(s) 1 and 23 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Fong et al. (US 2022/0400383 A1).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10, 12-19, 21-29, and 31 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Analysis
Step 1 (Statutory Categories) — 2019 PEG pq. 53
Claims 1-10, 12-19, 21-29, and 31 are directed to the statutory categories of invention.
Step 2A, Prong 1 (Do the claims recite an abstract idea?) — 2019 PEG pq. 54
Claim 25 recites the following types of subject matter that are judicial exceptions: Abstract idea — mental processes:
“A method of wireless communication at a network node, comprising: obtaining a plurality of datasets for training a machine learning (ML) model from at least one user equipment (UE) in wireless communication with the network node, each dataset including a set of metrics collected by a corresponding UE from the at least one UE; assigning at least one data signature associated with a source to each dataset of the plurality of datasets; associating a legitimacy score with one or more of a plurality of data signatures, wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value; and sending the legitimacy score associated with the one or more of the plurality of data signatures to a network entity.”.
The claim recites:
obtaining a plurality of datasets (collecting the information)
assigning at least one data signature (organizing and labeling the information)
associating a legitimacy score (analyzing information)
identifying a first data signature (evaluating and making determination)
sending the legitimacy score (reporting results)
These steps constitute mental process, mathematical concepts, and method of organizing human activity. Such operations can be performed mentally or with pen and paper and therefore fall within categories of abstract ideas.
Accordingly, claim 25 is directed to an abstract idea.
Step 2A, Prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?) - 2019 PEG pq. 54
Claim 25 does not integrate the abstract idea into a practical application.
Although the claim recites a “network node”, “user equipment (UE)”, and “network entity”, these elements merely represent generic computer components performing generic computer functions.
These elements are recited at a high level of generality and are used only as tool to perform the abstract idea. The claim does not improve the functioning of a computer. Instead, the claim merely uses generic computing components to data evaluation and classification. Limiting the method to “wireless communication” merely confines the abstract idea to a practical application. Further, the step of “sending the legitimacy score” is merely extra-solution activity and does not limit the abstract idea.
Therefore, claim 25 is directed to an abstract idea and is not integrated into a practical application under Step 2A.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?) - 2019 PEG pq. 56
Claim 25 does not include an inventive concept.
The additional elements (“network node”, “user equipment (UE)”, and “network entity”) are routine and conventional in the field. The claim merely applies generic component in wireless systems to perform the abstract idea. The claim therefore amounts to: applying an abstract idea using generic computer components which does not constitute significantly more.
Accordingly, under Step 2B of the PEG, the claim 25 is not patent eligible.
Step 2A, Prong 1 (Do the claims recite an abstract idea?) — 2019 PEG pq. 54
Claim 26 depends from claim 25 and further recites the following types of subject matter that are judicial exceptions: Abstract idea — mental processes:
“wherein the at least one data signature indicates … time or location of data collection, a first identifier (ID) of the corresponding UE associated with a corresponding dataset, a second ID of the network node, a power class of the corresponding UE, a vendor of the corresponding UE, a component of the corresponding UE, age of the corresponding UE, or a model or a type of a sensor that collected a metric.”.
Claim 26 merely specifies that the “data signature” includes (timer or location of data collection, identifiers of the UE and network node, UE characteristics (power, class, vendor, component, age) or sensor/model information). The additional limitations correspond to categorizing data. Such activities fall within mental process. Specifying such information does not remove the claim from the abstract idea.
Accordingly, claim 26 is directed to an abstract idea.
Step 2A, Prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?) - 2019 PEG pq. 54
Claim 26 does not integrate the abstract idea into a practical application.
Although the claim recites a “timer or location of data collection, identifiers of the UE and network node, UE characteristics (power, class, vendor, component, age) or sensor/model information”, these categories does not impose any meaningful technological limitation. Therefore, the claim does not improve wireless communication.
These elements are recited at a high level of generality and are used only as tool to perform the abstract idea. The claim does not improve the functioning of a computer. Instead, the claim merely uses generic computing components to network, node, UE. Limiting the method to “wireless communication” merely confines the abstract idea to a practical application.
Therefore, claim 26 is directed to an abstract idea and is not integrated into a practical application under Step 2A.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?) - 2019 PEG pq. 56
Claim 26 does not include an inventive concept.
The additional elements “timer or location of data collection, identifiers of the UE and network node, UE characteristics (power, class, vendor, component, age) or sensor/model information” are routine and conventional in the field. The claim merely applies generic component in wireless systems to perform the abstract idea. The claim therefore amounts to: applying an abstract idea using generic computer components which does not constitute significantly more.
Accordingly, under Step 2B of the PEG, the claim 26 is not patent eligible.
Step 2A, Prong 1 (Do the claims recite an abstract idea?) — 2019 PEG pq. 54
Claim 27 depends from claim 25 and further recites the following types of subject matter that are judicial exceptions: Abstract idea — mental processes:
“identifying the first data signature associated with the corrupted dataset; and filtering out at least one dataset associated with the first data signature from the plurality of datasets for training the ML model based on the first data signature being associated with the corrupted dataset.”.
identifying a data signature
filtering out datasets
These additional steps constitute mental process. Such operations can be performed mentally and therefore fall within categories of abstract ideas.
Accordingly, claim 27 is directed to an abstract idea.
Step 2A, Prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?) - 2019 PEG pq. 54
Claim 27 does not integrate the abstract idea into a practical application.
Although the claim recites a “identifying a data signature and filtering out datasets”, these steps do not improve a technological process.
These elements are recited at a high level of generality and are used only as tool to perform the abstract idea. The claim does not improve the functioning of a computer. Instead, the claim merely uses generic computing wireless system (network, node, UE). Limiting the method to “wireless communication” merely confines the abstract idea to a practical application. Further, filtering dataset is aa routine data processing step. Therefore, claim 27 is directed to an abstract idea and is not integrated into a practical application under Step 2A.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?) - 2019 PEG pq. 56
Claim 27 does not include an inventive concept.
The additional elements (“identifying a data signature associated with a corrupted dataset, filtering out datasets…”) are routine and conventional in the field. The claim merely applies generic component in wireless systems to perform the abstract idea. The claim therefore amounts to: applying an abstract idea using generic computer components which does not constitute significantly more.
Accordingly, under Step 2B of the PEG, the claim 27 is not patent eligible.
Step 2A, Prong 1 (Do the claims recite an abstract idea?) — 2019 PEG pq. 54
Claim 28 depends from claim 27 and further recites the following types of subject matter that are judicial exceptions: Abstract idea — mental processes:
“training a first ML model using a first dataset of the plurality of datasets; and applying a trusted testing dataset to the first ML model and a trusted ML model, the trusted ML model being trained using a trusted training dataset, wherein the first dataset is identified as the corrupted dataset based on a performance difference between the first ML model and the trusted ML model being greater than the threshold value”.
Claim 28 merely specifies that “training a model using data, testing and comparing the model, and determining whether data is corrupted…”. The additional limitations correspond to decision making. Such activities fall within mental process and mathematical concepts. Specifying such information does not remove the claim from the abstract idea.
Accordingly, claim 28 is directed to an abstract idea.
Step 2A, Prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?) - 2019 PEG pq. 54
Claim 28 does not integrate the abstract idea into a practical application.
Although the claim recites a “training a model using data, testing and comparing the model, and determining whether data is corrupted…”., these categories does not improve ML techniques. Therefore, the claim does not improve wireless communication.
These elements are recited at a high level of generality and are used only as tool to perform the abstract idea. The claim does not improve the functioning of a computer. Instead, the claim merely uses generic computing components to network, node, UE. Limiting the method to “wireless communication” merely confines the abstract idea to a practical application.
Therefore, claim 28 is directed to an abstract idea and is not integrated into a practical application under Step 2A.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?) - 2019 PEG pq. 56
Claim 28 does not include an inventive concept.
The additional elements “training an ML model, applying testing datasets, comparing model, threshold-based determination” are routine and conventional in machine learning training. The claim merely applies generic component in model training to perform the abstract idea. The claim therefore amounts to: applying an abstract idea using generic computer components which does not constitute significantly more.
Accordingly, under Step 2B of the PEG, the claim 28 is not patent eligible.
Step 2A, Prong 1 (Do the claims recite an abstract idea?) — 2019 PEG pq. 54
Claim 29 depends from claim 27 and further recites the following types of subject matter that are judicial exceptions: Abstract idea — mental processes:
“generating a plurality of dataset groups from the plurality of datasets based on the plurality of data signatures, each dataset groups being associated with one data signature of the plurality of data signatures; and training a plurality of ML models using a plurality of dataset group combinations, each dataset group combination including more than one dataset groups of the plurality of dataset groups, wherein the first data signature is identified as being associated with the corrupted dataset based on a first subset of ML models trained using a second subset of dataset group combination including a first dataset group associated with the first data signature having lower performances than the plurality of ML models other than the first subset of the ML models”.
Claim 29 merely specifies that “generating a plurality of dataset, training a plurality of ML models, and identifying…”. The additional limitations correspond to decision making. Such activities fall within mental process and mathematical concepts. Specifying such information does not remove the claim from the abstract idea.
Accordingly, claim 29 is directed to an abstract idea.
Step 2A, Prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?) - 2019 PEG pq. 54
Claim 29 does not integrate the abstract idea into a practical application.
Although the claim recites a “generating a plurality of dataset, training a plurality of ML models, and identifying…”., these categories does not improve ML techniques. Therefore, the claim does not improve wireless communication.
These elements are recited at a high level of generality and are used only as tool to perform the abstract idea. The claim does not improve the generic machine learning usage. Instead, the claim merely uses generic computing components to network, node, UE. Limiting the method to “wireless communication” merely confines the abstract idea to a practical application.
Therefore, claim 29 is directed to an abstract idea and is not integrated into a practical application under Step 2A.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?) - 2019 PEG pq. 56
Claim 29 does not include an inventive concept.
The additional elements “dataset partitioning, model training, comparing, and anomaly detection” are routine and conventional in machine learning training. The claim merely applies generic component in model training to perform the abstract idea. The claim therefore amounts to: applying an abstract idea using generic computer components which does not constitute significantly more.
Accordingly, under Step 2B of the PEG, the claim 29 is not patent eligible.
Claim 1 recites the additional limitations “memory… least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 2 is dependent on claim 1. Claim 2 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 3 is dependent on claim 1. Claim 3 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 4 is dependent on claim 1. Claim 4 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 5 is dependent on claim 1. Claim 5 recites additional limitation “at least one data signature”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 6 is dependent on claim 1. Claim 6 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 7 is dependent on claim 6. Claim 7 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 8 is dependent on claim 6. Claim 8 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 9 is dependent on claim 8. Claim 9 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 10 is dependent on claim 6. Claim 10 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 12 is dependent on claim 6. Claim 12 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 13 is dependent on claim 1. Claim 13 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 14 recites the additional limitations “memory… least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 15 is dependent on claim 14. Claim 15 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 16 is dependent on claim 14. Claim 16 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 17 is dependent on claim 14. Claim 17 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 18 is dependent on claim 17. Claim 18 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 19 is dependent on claim 14. Claim 19 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 21 is dependent on claim 14. Claim 21 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 22 is dependent on claim 14. Claim 22 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 23 recites the additional limitations “memory… least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 24 is dependent on claim 23. Claim 24 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Claim 31 is dependent on claim 1. Claim 31 recites additional limitation “at least one processor”, which in Step 2A, Prong 2, the limitations are merely elaborating on the abstract idea, by further specifying an additional limitation at a high-level of generality, therefore, does not amount to significantly more than the abstract idea.
Therefore, claims 1-10, 12-19, 21-29, and 31 are rejected under 35 USC § 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10, 12-14, 16-19, 21, 23-29, and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Baracaldo-Angel et al. (US 2020/0019821 A1), hereinafter Baracaldo-Angel in view of Fong et al. (US 2022/0400383 A1), hereinafter Fong.
Regarding claim 1, Baracaldo-Angel discloses an apparatus for a wireless communication at a first network node, comprising:
a memory (Baracaldo-Angel, Fig. 2); and
at least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor is configured to (Baracaldo-Angel, Para. 0072, some combination of hardware and software logic that is accessible by the processor and configured to cause the processor to perform some functionality upon execution by the processor. Software logic may be stored on local and/or remote memory of any memory type, as known in the art):
obtain a plurality of datasets for training a machine learning (ML) model from at least one user equipment (UE) in wireless communication with the network node (Baracaldo-Angel, Fig. 3, and Fig. 10, and Paras 0006-0007, receiving a dataset and associated provenance data, wherein the dataset comprises a plurality of untrusted data points and excludes trusted data points, wherein each untrusted data point is associated with one or more provenance signatures of the provenance data; randomly assigning a first portion of the untrusted data points to a training dataset), each dataset including a set of metrics collected by a corresponding UE from the at least one UE (Baracaldo-Angel, Fig. 3, and Para. 0030, this may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system); assign at least one data signature associated with a source of each dataset of the plurality of datasets (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.” The set of collected data points sharing a provenance signature is called the “data segment” of this signature); associate a legitimacy score with one or more of a plurality of data signatures (Fong, Para. 0045, in order to determine the trust score 230, the controller of the safe case compares the first set of data 210 and the second set of data 220, and determines deviations and/or correlations of values in the second set of data 220 from values in the first set of data 210. The controller of the safe case then assigns the trust score 230 as a function of the deviations and correlations), wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value (Fong, Para. 0046, the first set of data 210 may include biometric readings of the user of the user equipment, e.g., gaits, heart rhythm, plus rate, etc. Signatures, patterns, and/or traits of the biometric reading may show the user at a particular location) and (Fong, Fig. 5A and Para. 0078, various embodiments of the alert triggering are described below with reference to FIG. 5B. On the other hand, if the safe case determines that the trust score is not below the threshold (“No” branch from the decision block 560), as represented by block 580, the method 500 includes deriving trusted location data from one or more of the first set of data and the second set of data, and allowing access to the trusted location data by one or more trusted applications); and send the legitimacy score associated with the one or more of the plurality of data signatures to a network entity (Fong, Fig. 5B and Para. 0080, the safe case transmits, via the remote communication device, the trust score to a secure server).
Baracaldo-Angel and Fong are both considered to be analogous to the claim invention because they are in the same field of training the machine learning models to detect the vulnerabilities from the inaccurate data. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filling date of the claimed invention to have modified Baracaldo-Angel to incorporate the teachings of Fong to include associate a legitimacy score with one or more of a plurality of data signatures (Fong, Para. 0045), wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value (Fong, Para. 0046) and (Fong, Fig. 5A and Para. 0078); and send the legitimacy score associated with the one or more of the plurality of data signatures to a network entity (Fong, Fig. 5B and Para. 0080). Doing so would aid to provide location validity by validating the location data with its own data and generates the trust score. As such, the safe case provides data integrity and confidentiality (e.g., using secure storage and/or a secure communication channel to cloud)(Fong, Para. 0012).
Regarding claim 2, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, wherein the at least one processor is further configured to: configure, prior to obtaining the plurality of datasets, the corresponding UE to associate the at least one data signature with reported data from the corresponding UE (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.” The set of collected data points sharing a provenance signature is called the “data segment” of this signature).
Regarding claim 3, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, wherein to assign the at least one data signature associated with the source of each dataset, the at least one processor is configured to: add the at least one data signature to each obtained dataset (Baracaldo-Angel, Para. 0044, the provenance data provide a mechanism to link data in a training set to data with the same provenance signature in a test or evaluation set).
Regarding claim 4, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, wherein the at least one processor is further configured to: request at least one different network node to assign the at least one data signature to datasets obtained by the at least one different network node (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.” The set of collected data points sharing a provenance signature is called the “data segment” of this signature).
Regarding claim 5, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, wherein each of the at least one data signature indicates at least one of: time or location of data collection, a first identifier (ID) of the corresponding UE associated with a corresponding dataset, a second ID of the network node, a power class of the corresponding UE, a vendor of the corresponding UE, a component of the corresponding UE, age of the corresponding UE, or a model or a type of a sensor that collected a metric (Baracaldo-Angel, Para. 0030, provenance data” refers to the lineage or data associated with a data point and shows the operations that led to its creation, origin and manipulation. This may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system. Hence, provenance data are considered “trusted.” In preferred approaches, provenance data are associated with corresponding data as metadata) and (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.”).
Regarding claim 6, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, wherein the at least one processor is further configured to: identify the first data signature associated with the corrupted dataset (Baracaldo-Angel, Para. 0125, where a dataset and associated provenance data are received, e.g. at a computer. The dataset includes a plurality of untrusted data points and excludes trusted data points. Moreover, each untrusted data point is associated with one or more provenance signatures of the provenance data) and (Baracaldo-Angel, Para. 0125, in operation 1108, the untrusted data points of the training dataset are grouped into a plurality of groups each characterized by a different one of the provenance signatures. The untrusted data points of each group are characterized by a same one of the provenance signatures.); and filter out at least one dataset associated with the first data signature from the plurality of datasets for training the ML model based on the first data signature being associated with the corrupted dataset (Baracaldo-Angel, Para. 0129, designating as poisonous all data point(s) in the training dataset and all data point(s) in the full evaluation dataset that have a same provenance signature as: (1) the group of untrusted data points excluded from the second training dataset; and (2) the data points removed from the full evaluation dataset based on the fact that those data points share the same provenance signature with the one or more data points of the second training dataset).
Regarding claim 7, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 6, wherein to identify the first data signature associated with the corrupted dataset, the at least one processor is further configured to: receive, from a core network, an indication that the first data signature is associated with the corrupted dataset (Baracaldo-Angel, Para. 0089, E serves as a tunable parameter to determine how large the performance decrease should be to conclude a segment of data points is poisonous. Methodologies for computing E are discussed above regarding the calibration procedure and thresholds generated thereby. Algorithm 1 returns a set of tuples containing data points that are suspected of being poisonous, associated provenance signatures and corresponding expected loss in performance if the suspect data points are not filtered (i.e. removed from the untrusted dataset)) and (Baracaldo-Angel, Para. 0142, accuracy and F1-score of a provenance-based defense against poison attacks in a fully untrusted data environment (FIG. 9)).
Regarding claim 8, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 6, wherein the at least one processor is further configured to: train a first ML model using a first dataset of the plurality of datasets (Baracaldo-Angel, Para. 0085, one segment of data is randomly removed from the untrusted dataset 302 and one segment of legitimate data is selected at random from the trusted dataset 310); and apply a trusted testing dataset to the first ML model and a trusted ML model, the trusted ML model being trained using a trusted training dataset (Baracaldo-Angel, Para. 0083, the classifier trained with the particular segment (unfiltered model 308) on the trusted test dataset 310, the particular segment is considered poisoned and removed from the untrusted dataset 302, ultimately yielding a filtered dataset 312 excluding poison data points from the untrusted dataset), wherein the first dataset is identified as the corrupted dataset based on a performance difference between the first ML model and the trusted ML model being greater than a first threshold value (Baracaldo-Angel, Para. 0087, a model is then trained on the randomly selected segments plus the segment being evaluated, while another model is trained only on the randomly selected segments. Performance of the two models is compared, and in at least one approach the procedure may be repeated several times in order to account for natural variance in the results. If the average change in performance is greater than the threshold value, the segment is deemed poisonous and filtered from the dataset).
Regarding claim 9, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 8, wherein the first dataset is identified as being associated with the corrupted dataset based on a distribution or statistical property of the first dataset differing by more than a second threshold value from the trusted training dataset (Baracaldo-Angel, Para. 0086, tuning this parameter using a cross-validation set is also an option, in some approaches. If the user is able to model the distribution of performance change in the calibration trials, conducting statistical tests of the hypothesis that an untrusted segment is legitimate allows adjusting the threshold value according to the modeled distribution and a p-value).
Regarding claim 10, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 6, wherein the at least one processor is further configured to: generate a plurality of dataset groups from the plurality of datasets based on a plurality of data signatures, each dataset groups being associated with one data signature of the plurality of data signatures (Baracaldo-Angel, Para. 0051, generating a new evaluation dataset by removing, from the full evaluation dataset, any data points sharing a provenance signature with one or more data points of the second training dataset; applying each of the complete prediction model and the filtered prediction model to the new evaluation dataset); and train a plurality of ML models using a plurality of dataset group combinations, each dataset group combination including more than one dataset groups of the plurality of dataset groups (Baracaldo-Angel, Para. 0051, recombining the training dataset and full evaluation dataset after removing the one or more poisonous data points therefrom so as to create a final filtered training set; and training a final prediction model using the final filtered training set), wherein the first data signature is identified as being associated with the corrupted dataset based on a first subset of ML models trained using a second subset of dataset group combination including a first dataset group associated with the first data signature having lower performances than the plurality of ML models other than the first subset of the ML models (Baracaldo-Angel, Para. 0089, data is segmented by provenance signature and for each data segment two models are trained: a Modelunfiltered and a Modelfiltered. In line 7, the difference in performance between the Modelunfiltered and the Modelfiltered models is computed. The performance of each model is evaluated using trusted dataset DT. Different performance metrics can be used for this purpose, including but not limited to F1-measure and accuracy. In line 8, E serves as a tunable parameter to determine how large the performance decrease should be to conclude a segment of data points is poisonous).
Regarding claim 12, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 6, wherein the at least one processor is further configured to: filter the least one dataset associated with the first data signature from the plurality of datasets for training the ML model based on a use case of the ML model (Baracaldo-Angel, Para. 0007, the method still further includes removing the one or more poisonous data points from the training dataset and the full evaluation dataset; recombining the training dataset and full evaluation dataset after removing the one or more poisonous data points therefrom so as to create a final filtered training set; and training a final prediction model using the final filtered training set).
Regarding claim 13, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, further comprising a transceiver coupled to the at least one processor, wherein the at least one processor is further configured to: transmit the plurality of datasets assigned with the at least one data signature to a core network (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.” The set of collected data points sharing a provenance signature is called the “data segment” of this signature).
Regarding claim 14, Baracaldo-Angel discloses an apparatus for wireless communication at a core network, comprising: memory; and at least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor is configured to: obtain a plurality of datasets for training a machine learning (ML) model from at least one network node (Baracaldo-Angel, Fig. 3, and Fig. 10, and Paras 0006-0007, receiving a dataset and associated provenance data, wherein the dataset comprises a plurality of untrusted data points and excludes trusted data points, wherein each untrusted data point is associated with one or more provenance signatures of the provenance data; randomly assigning a first portion of the untrusted data points to a training dataset), each dataset being assigned with a corresponding data signature and each dataset including a set of metrics collected by a user equipment (UE) served by the at least one network node (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.” The set of collected data points sharing a provenance signature is called the “data segment” of this signature) and (Baracaldo-Angel, Fig. 3, and Para. 0030, this may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system); and filter out at least one dataset associated with the first data signature from the plurality of datasets for training the ML model based on the first data signature being associated with the corrupted dataset (Baracaldo-Angel, Para. 0129, designating as poisonous all data point(s) in the training dataset and all data point(s) in the full evaluation dataset that have a same provenance signature as: (1) the group of untrusted data points excluded from the second training dataset; and (2) the data points removed from the full evaluation dataset based on the fact that those data points share the same provenance signature with the one or more data points of the second training dataset).
Baracaldo-Angel does not explicitly disclose receive, from a network node, a legitimacy score associated with a first data signature: identify first data signature that is associated with a corrupted dataset; wherein the first data signature is identified as being associated with the corrupted dataset based on the legitimacy score being lower than a threshold value.
However, Fong teaches receive, from a network node, a legitimacy score associated with a first data signature (Fong, Para. 0045, in order to determine the trust score 230, the controller of the safe case compares the first set of data 210 and the second set of data 220, and determines deviations and/or correlations of values in the second set of data 220 from values in the first set of data 210. The controller of the safe case then assigns the trust score 230 as a function of the deviations and correlations): identify first data signature that is associated with a corrupted dataset (Fong, Para. 0046, the first set of data 210 may include biometric readings of the user of the user equipment, e.g., gaits, heart rhythm, plus rate, etc. Signatures, patterns, and/or traits of the biometric reading may show the user at a particular location) and (Fong, Fig. 5A and Para. 0078, various embodiments of the alert triggering are described below with reference to FIG. 5B. On the other hand, if the safe case determines that the trust score is not below the threshold (“No” branch from the decision block 560), as represented by block 580, the method 500 includes deriving trusted location data from one or more of the first set of data and the second set of data, and allowing access to the trusted location data by one or more trusted applications); wherein the first data signature is identified as being associated with the corrupted dataset based on the legitimacy score being lower than a threshold value (Fong, Para. 0046, the first set of data 210 may include biometric readings of the user of the user equipment, e.g., gaits, heart rhythm, plus rate, etc. Signatures, patterns, and/or traits of the biometric reading may show the user at a particular location) and (Fong, Fig. 5A and Para. 0078, various embodiments of the alert triggering are described below with reference to FIG. 5B. On the other hand, if the safe case determines that the trust score is not below the threshold (“No” branch from the decision block 560), as represented by block 580, the method 500 includes deriving trusted location data from one or more of the first set of data and the second set of data, and allowing access to the trusted location data by one or more trusted applications);
Baracaldo-Angel and Fong are both considered to be analogous to the claim invention because they are in the same field of training the machine learning models to detect the vulnerabilities from the inaccurate data. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filling date of the claimed invention to have modified Baracaldo-Angel to incorporate the teachings of Fong to include receive, from a network node, a legitimacy score associated with a first data signature (Fong, Para. 0045): identify first data signature that is associated with a corrupted dataset (Fong, Para. 0046) and (Fong, Fig. 5A and Para. 0078); wherein the first data signature is identified as being associated with the corrupted dataset based on the legitimacy score being lower than a threshold value (Fong, Para. 0046) and (Fong, Fig. 5A and Para. 0078). Doing so would aid to provide location validity by validating the location data with its own data and generates the trust score. As such, the safe case provides data integrity and confidentiality (e.g., using secure storage and/or a secure communication channel to cloud)(Fong, Para. 0012).
Regarding claim 16, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 14, wherein the corresponding data signature comprises at least one of: time or location of data collection, a first identifier (ID) of a corresponding UE that is a source of a corresponding dataset, a second ID of a network node associated with the corresponding UE, a power class of the corresponding UE, a vendor of the corresponding UE, a component of the corresponding UE, age of the corresponding UE, or a model or a type of a sensor that collected a metric (Baracaldo-Angel, Para. 0030, provenance data” refers to the lineage or data associated with a data point and shows the operations that led to its creation, origin and manipulation. This may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system. Hence, provenance data are considered “trusted.” In preferred approaches, provenance data are associated with corresponding data as metadata) and (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.”).
Regarding claim 17, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 14, wherein the at least one processor is further configured to: train a first ML model using a first dataset of the plurality of datasets (Baracaldo-Angel, Para. 0085, one segment of data is randomly removed from the untrusted dataset 302 and one segment of legitimate data is selected at random from the trusted dataset 310); and apply a trusted testing dataset to the first ML model and a trusted ML model, the trusted ML model being trained using a trusted training dataset (Baracaldo-Angel, Para. 0083, the classifier trained with the particular segment (unfiltered model 308) on the trusted test dataset 310, the particular segment is considered poisoned and removed from the untrusted dataset 302, ultimately yielding a filtered dataset 312 excluding poison data points from the untrusted dataset), wherein the first dataset is identified as the corrupted dataset based on a performance difference between the first ML model and the trusted ML model being greater than a first threshold value (Baracaldo-Angel, Para. 0087, a model is then trained on the randomly selected segments plus the segment being evaluated, while another model is trained only on the randomly selected segments. Performance of the two models is compared, and in at least one approach the procedure may be repeated several times in order to account for natural variance in the results. If the average change in performance is greater than the threshold value, the segment is deemed poisonous and filtered from the dataset).
Regarding claim 18, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 17, wherein the first dataset is identified as being associated with the corrupted dataset based on a distribution or statistical property of the first dataset differing by more than a second threshold value from the trusted training dataset (Baracaldo-Angel, Para. 0086, tuning this parameter using a cross-validation set is also an option, in some approaches. If the user is able to model the distribution of performance change in the calibration trials, conducting statistical tests of the hypothesis that an untrusted segment is legitimate allows adjusting the threshold value according to the modeled distribution and a p-value).
Regarding claim 19, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 14, wherein the at least one processor is further configured to: generate a plurality of dataset groups from the plurality of datasets based on a plurality of data signatures (Baracaldo-Angel, Para. 0051, generating a new evaluation dataset by removing, from the full evaluation dataset, any data points sharing a provenance signature with one or more data points of the second training dataset; applying each of the complete prediction model and the filtered prediction model to the new evaluation dataset), each dataset groups being associated with one data signature of the plurality of data signatures (Baracaldo-Angel, Para. 0051); and train a plurality of ML models using a plurality of dataset group combinations, each dataset group combination including more than one dataset groups of the plurality of dataset groups (Baracaldo-Angel, Para. 0051, recombining the training dataset and full evaluation dataset after removing the one or more poisonous data points therefrom so as to create a final filtered training set; and training a final prediction model using the final filtered training set), wherein the first data signature is identified as being associated with the corrupted dataset based on a first subset of ML models trained using a second subset of dataset group combination including a first dataset group associated with the first data signature having lower performances than the plurality of ML models other than the first subset of the ML models (Baracaldo-Angel, Para. 0089, data is segmented by provenance signature and for each data segment two models are trained: a Modelunfiltered and a Modelfiltered. In line 7, the difference in performance between the Modelunfiltered and the Modelfiltered models is computed. The performance of each model is evaluated using trusted dataset DT. Different performance metrics can be used for this purpose, including but not limited to F1-measure and accuracy. In line 8, E serves as a tunable parameter to determine how large the performance decrease should be to conclude a segment of data points is poisonous).
Regarding claim 21, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 14, wherein the corrupted dataset associated with the first data signature is filtered out from the plurality of datasets for training the ML model based on a use case of the ML model (Baracaldo-Angel, Para. 0007, the method still further includes removing the one or more poisonous data points from the training dataset and the full evaluation dataset; recombining the training dataset and full evaluation dataset after removing the one or more poisonous data points therefrom so as to create a final filtered training set; and training a final prediction model using the final filtered training set).
Regarding claim 23, the combination of Baracaldo-Angel in view of Fong teaches an apparatus for a wireless communication at a user equipment (UE), comprising: memory (Baracaldo-Angel, Fig. 2); and at least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor is configured to (Baracaldo-Angel, Para. 0072, some combination of hardware and software logic that is accessible by the processor and configured to cause the processor to perform some functionality upon execution by the processor. Software logic may be stored on local and/or remote memory of any memory type, as known in the art): receive a first wireless transmission that includes a configuration from a network node assigning at least one data signature to be reported by the UE with a dataset for a machine learning (ML) model (Fong, Para. 0081, the secure server 430 aggregates the information, trains a neural network, and uses features extracted from the traits data to determine an identifiable score of users); and transmit a second wireless transmission that includes one or more datasets for the ML model from the UE to the network node with an indication the at least one data signature configured for the UE to report with the dataset (Fong, Para. 0061, to facilitate the traits data processing, the secure server 430 includes a neural network for machine learning of traits data obtained from the plurality of UESs 420. A variety of machine learning techniques can be used in place of the neural network, including, for example, recurrent neural networks (RNN), random forest classifiers, Bayes classifiers (e.g., naive Bayes), principal component analysis (PCA), support vector machines, linear discriminant analysis, and the like). Baracaldo-Angel and Fong are both considered to be analogous to the claim invention because they are in the same field of training the machine learning models to detect the vulnerabilities from the inaccurate data. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filling date of the claimed invention to have modified Baracaldo-Angel to incorporate the teachings of Fong to include (Fong, Para. 0081); and transmit a second wireless transmission that includes one or more datasets for the ML model from the UE to the network node with an indication the at least one data signature configured for the UE to report with the dataset (Fong, Para. 0061). Doing so would aid to provide location validity by validating the location data with its own data and generates the trust score. As such, the safe case provides data integrity and confidentiality (e.g., using secure storage and/or a secure communication channel to cloud)(Fong, Para. 0012).
Regarding claim 24, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 23, wherein the at least one data signature comprises at least one of: time or location of data collection, a first identifier (ID) of the UE, a second ID of the network node associated with the UE, a power class of the UE, a vendor of the UE, a component of the UE, age of the UE, or a model or a type of a sensor that collects data (Baracaldo-Angel, Para. 0030, provenance data” refers to the lineage or data associated with a data point and shows the operations that led to its creation, origin and manipulation. This may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system. Hence, provenance data are considered “trusted.” In preferred approaches, provenance data are associated with corresponding data as metadata) and (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.”).
Regarding claim 25, Baracaldo-Angel discloses a method of wireless communication at a network node, comprising: obtaining a plurality of datasets for training a machine learning (ML) model from at least one user equipment (UE) in wireless communication with the network node (Baracaldo-Angel, Fig. 3, and Fig. 10, and Paras 0006-0007, receiving a dataset and associated provenance data, wherein the dataset comprises a plurality of untrusted data points and excludes trusted data points, wherein each untrusted data point is associated with one or more provenance signatures of the provenance data; randomly assigning a first portion of the untrusted data points to a training dataset), each dataset including a set of metrics collected by a corresponding UE from the at least one UE (Baracaldo-Angel, Fig. 3, and Para. 0030, this may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system); assigning at least one data signature associated with a source to each dataset of the plurality of datasets (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.” The set of collected data points sharing a provenance signature is called the “data segment” of this signature);
Baracaldo-Angel does not explicitly disclose associating a legitimacy score with one or more of a plurality of data signatures,
wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value
; and sending the legitimacy score associated with the one or more of the plurality of data signatures to a network entity.
However, Fong teaches associating a legitimacy score with one or more of a plurality of data signatures (Fong, Para. 0045, in order to determine the trust score 230, the controller of the safe case compares the first set of data 210 and the second set of data 220, and determines deviations and/or correlations of values in the second set of data 220 from values in the first set of data 210. The controller of the safe case then assigns the trust score 230 as a function of the deviations and correlations), wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value (Fong, Para. 0046, the first set of data 210 may include biometric readings of the user of the user equipment, e.g., gaits, heart rhythm, plus rate, etc. Signatures, patterns, and/or traits of the biometric reading may show the user at a particular location) and (Fong, Fig. 5A and Para. 0078, various embodiments of the alert triggering are described below with reference to FIG. 5B. On the other hand, if the safe case determines that the trust score is not below the threshold (“No” branch from the decision block 560), as represented by block 580, the method 500 includes deriving trusted location data from one or more of the first set of data and the second set of data, and allowing access to the trusted location data by one or more trusted applications); and sending the legitimacy score associated with the one or more of the plurality of data signatures to a network entity (Fong, Fig. 5B and Para. 0080, the safe case transmits, via the remote communication device, the trust score to a secure server).
Baracaldo-Angel and Fong are both considered to be analogous to the claim invention because they are in the same field of training the machine learning models to detect the vulnerabilities from the inaccurate data. Therefore, it would have been obvious to someone ordinary skill in the art before the effective filling date of the claimed invention to have modified Baracaldo-Angel to incorporate the teachings of Fong to include associating a legitimacy score with one or more of a plurality of data signatures (Fong, Para. 0045), wherein a first data signature is identified as being associated with a corrupted dataset based on the legitimacy score being lower than a threshold value (Fong, Para. 0046) and (Fong, Fig. 5A and Para. 0078); and sending the legitimacy score associated with the one or more of the plurality of data signatures to a network entity (Fong, Fig. 5B and Para. 0080). Doing so would aid to provide location validity by validating the location data with its own data and generates the trust score. As such, the safe case provides data integrity and confidentiality (e.g., using secure storage and/or a secure communication channel to cloud)(Fong, Para. 0012).
Regarding claim 26, the combination of Baracaldo-Angel in view of Fong teaches the method of claim 25, wherein the at least one data signature indicates one or more of: time or location of data collection, a first identifier (ID) of the corresponding UE associated with a corresponding dataset, a second ID of the network node, a power class of the corresponding UE, a vendor of the corresponding UE, a component of the corresponding UE, age of the corresponding UE, or a model or a type of a sensor that collected a metric (Baracaldo-Angel, Para. 0030, provenance data” refers to the lineage or data associated with a data point and shows the operations that led to its creation, origin and manipulation. This may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system. Hence, provenance data are considered “trusted.” In preferred approaches, provenance data are associated with corresponding data as metadata) and (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.”).
Regarding claim 27, the combination of Baracaldo-Angel in view of Fong teaches the method of claim 25, further comprising: identifying the first data signature associated with the corrupted dataset; and filtering out at least one dataset associated with the first data signature from the plurality of datasets for training the ML model based on the first data signature being associated with the corrupted dataset (Baracaldo-Angel, Para. 0089, E serves as a tunable parameter to determine how large the performance decrease should be to conclude a segment of data points is poisonous. Methodologies for computing E are discussed above regarding the calibration procedure and thresholds generated thereby. Algorithm 1 returns a set of tuples containing data points that are suspected of being poisonous, associated provenance signatures and corresponding expected loss in performance if the suspect data points are not filtered (i.e. removed from the untrusted dataset)) and (Baracaldo-Angel, Para. 0142, accuracy and F1-score of a provenance-based defense against poison attacks in a fully untrusted data environment (FIG. 9)).
Regarding claim 28, the combination of Baracaldo-Angel in view of Fong teaches the method of claim 27, further comprising: training a first ML model using a first dataset of the plurality of datasets (Baracaldo-Angel, Para. 0085, one segment of data is randomly removed from the untrusted dataset 302 and one segment of legitimate data is selected at random from the trusted dataset 310); and applying a trusted testing dataset to the first ML model and a trusted ML model, the trusted ML model being trained using a trusted training dataset (Baracaldo-Angel, Para. 0083, the classifier trained with the particular segment (unfiltered model 308) on the trusted test dataset 310, the particular segment is considered poisoned and removed from the untrusted dataset 302, ultimately yielding a filtered dataset 312 excluding poison data points from the untrusted dataset), wherein the first dataset is identified as the corrupted dataset based on a performance difference between the first ML model and the trusted ML model being greater than the threshold value (Baracaldo-Angel, Para. 0087, a model is then trained on the randomly selected segments plus the segment being evaluated, while another model is trained only on the randomly selected segments. Performance of the two models is compared, and in at least one approach the procedure may be repeated several times in order to account for natural variance in the results. If the average change in performance is greater than the threshold value, the segment is deemed poisonous and filtered from the dataset).
Regarding claim 29, the combination of Baracaldo-Angel in view of Fong teaches the method of claim 27, further comprising: generating a plurality of dataset groups from the plurality of datasets based on the plurality of data signatures, each dataset groups being associated with one data signature of the plurality of data signatures (Baracaldo-Angel, Para. 0051, generating a new evaluation dataset by removing, from the full evaluation dataset, any data points sharing a provenance signature with one or more data points of the second training dataset; applying each of the complete prediction model and the filtered prediction model to the new evaluation dataset); and training a plurality of ML models using a plurality of dataset group combinations, each dataset group combination including more than one dataset groups of the plurality of dataset groups (Baracaldo-Angel, Para. 0051, recombining the training dataset and full evaluation dataset after removing the one or more poisonous data points therefrom so as to create a final filtered training set; and training a final prediction model using the final filtered training set), wherein the first data signature is identified as being associated with the corrupted dataset based on a first subset of ML models trained using a second subset of dataset group combination including a first dataset group associated with the first data signature having lower performances than the plurality of ML models other than the first subset of the ML models (Baracaldo-Angel, Para. 0089, data is segmented by provenance signature and for each data segment two models are trained: a Modelunfiltered and a Modelfiltered. In line 7, the difference in performance between the Modelunfiltered and the Modelfiltered models is computed. The performance of each model is evaluated using trusted dataset DT. Different performance metrics can be used for this purpose, including but not limited to F1-measure and accuracy. In line 8, E serves as a tunable parameter to determine how large the performance decrease should be to conclude a segment of data points is poisonous).
Regarding claim 31, the combination of Baracaldo-Angel in view of Fong teaches the apparatus of claim 1, wherein each of the at least one data signature indicates a first identifier (ID) of the corresponding UE associated with a corresponding dataset and at least one of a time or a location of data collection, and
wherein each of the at least one data signature further indicates one or more of:
a second ID of the network node, a power class of the corresponding UE, a vendor of the corresponding UE, a component of the corresponding UE, age of the corresponding UE, or a model or a type of a sensor that collected a metric (Baracaldo-Angel, Para. 0030, provenance data” refers to the lineage or data associated with a data point and shows the operations that led to its creation, origin and manipulation. This may include information about the device from which the data was gathered, including but not limited to: a firmware version, a user id, a timestamp (e.g. corresponding to transmission and/or receipt of data from the source), a unique network ID (such as a MAC address or other unique network identifier), physical location information (such as GPS coordinates, server rack and node number, etc.), data curator, annotator, transforming operations performed on the by any software agent, an identifier of the source of the data, historical statistics (such as reputation information collected from social media), and any equivalents thereof that would be appreciated by a person having ordinary skill in the art upon reading this disclosure. Importantly, provenance data are not easily modifiable by an adversary seeking to poison a given system. Hence, provenance data are considered “trusted.” In preferred approaches, provenance data are associated with corresponding data as metadata) and (Baracaldo-Angel, Para. 0032, a “provenance framework” is preferably deployed to record the lineage of data points received for training. The provenance framework provides a “provenance record” for each data point collected, and the provenance record contains one or more “provenance features” reflecting the lineage of the data point. A value of a given provenance feature, e.g., a specific environmental sensor or firmware version, is called a “provenance signature.”).
Claims 15 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Baracaldo-Angel et al. (US 2020/0019821 A1), hereinafter Baracaldo-Angel in view of Fong et al. (US 2022/0400383 A1), hereinafter Fong and further in view of BRENDEL (WO 2005/111805 A1), hereinafter BRENDEL.
Regarding claim 15, the combination of Baracaldo-Angel in view of Fong does not explicitly disclose the apparatus of claim 14, wherein the at least one processor is further configured to: instruct the at least one network node to assign the corresponding data signature to datasets obtained by the at least one network node. However, BRENDEL teaches wherein the at least one processor is further configured to: instruct the at least one network node to assign the corresponding data signature to datasets obtained by the at least one network node (BRENDEL, Page. 7, the apparatus 100, after identifying a signature of anomalous traffic may directly instruct a router or other device capable of filtering data to filter out traffic having that signature, and/or may output, through communication channel 100E, the results of its analysis to an operator via a text message, email, SNMP trap or otherwise. The latter option may be preferred to provide some administrative auditing of the signature analysis performed by the apparatus 100. In one embodiment, the identified signature and the proposed filter instructions for a filtering device may be displayed to an operator, with the filtering instructions not applied until the operator has approved them, for example using a suitable GUI. The apparatus 100 may include instructions to convert the detected bit and/or byte patterns into a different format if required, for example into a human readable form and output it through communication channel 100E using a suitable communication interface 105 for that channel (which may be the same as user interface 102), or into filtering instructions for a specific device). Baracaldo-Angel, Fong and BRENDEL are all considered to be analogues to the claimed invention because they are in the same field of obtaining datasets for training an ML model from at least one network node, each dataset being assigned with a corresponding data signature as corrupted or legitimate. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Baracaldo-Angel and Fong to incorporate the teachings of BRENDEL to include wherein the at least one processor is further configured to: instruct the at least one network node to assign the corresponding data signature to datasets obtained by the at least one network node (BRENDEL, Page. 7). Doing so would aid to prevent obscure false-positives (collisions with completely different packets, which accidentally have the same values at the specified offsets, but which might be sent to other systems, or other ports) it is possible to include those offset/value pairs in the signature. For example, the method may include adding to the signature the two or three packet parts that occur with the highest frequency and/or have a frequency over a certain threshold, the packets of which entirely contain the packets defined by the identified signature (BRENDEL, Page. 12).
Regarding claim 22, the combination of Baracaldo-Angel in view of Fong does not explicitly disclose the apparatus of claim 14, wherein the at least one processor is further configured to: instruct the at least one network node to filter out the corrupted dataset associated with the first data signature.
However, BRENDEL teaches wherein the at least one processor is further configured to: instruct the at least one network node to filter out the corrupted dataset associated with the first data signature (BRENDEL, Page. 7, the apparatus 100, after identifying a signature of anomalous traffic may directly instruct a router or other device capable of filtering data to filter out traffic having that signature, and/or may output, through communication channel 100E, the results of its analysis to an operator via a text message, email, SNMP trap or otherwise. The latter option may be preferred to provide some administrative auditing of the signature analysis performed by the apparatus 100. In one embodiment, the identified signature and the proposed filter instructions for a filtering device may be displayed to an operator, with the filtering instructions not applied until the operator has approved them, for example using a suitable GUI. The apparatus 100 may include instructions to convert the detected bit and/or byte patterns into a different format if required, for example into a human readable form and output it through communication channel 100E using a suitable communication interface 105 for that channel (which may be the same as user interface 102), or into filtering instructions for a specific device). Baracaldo-Angel, Fong and BRENDEL are all considered to be analogues to the claimed invention because they are in the same field of obtaining datasets for training an ML model from at least one network node, each dataset being assigned with a corresponding data signature as corrupted or legitimate. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Baracaldo-Angel and Fong to incorporate the teachings of BRENDEL to include wherein the at least one processor is further configured to: instruct the at least one network node to filter out the corrupted dataset associated with the first data signature (BRENDEL, Page. 7). Doing so would aid to prevent obscure false-positives (collisions with completely different packets, which accidentally have the same values at the specified offsets, but which might be sent to other systems, or other ports) it is possible to include those offset/value pairs in the signature. For example, the method may include adding to the signature the two or three packet parts that occur with the highest frequency and/or have a frequency over a certain threshold, the packets of which entirely contain the packets defined by the identified signature (BRENDEL, Page. 12).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO-892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GITA FARAMARZI whose telephone number is (571)272-0248. The examiner can normally be reached Monday- Friday 9:00 am- 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jorge L. Ortiz-Criado can be reached at (571)272-7624. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GITA FARAMARZI/Examiner, Art Unit 2496
/JORGE L ORTIZ CRIADO/Supervisory Patent Examiner, Art Unit 2496
Prosecution Insights