DETAILED ACTION
713.09 Interviews Between Final Rejection and Notice of Appeal [R-08.2017]
Normally, one interview after final rejection is permitted in order to place the application in condition for allowance or to resolve issues prior to appeal. However, prior to the interview, the intended purpose and content of the interview should be presented briefly, preferably in writing. Such an interview may be granted if the examiner is convinced that disposal or clarification for appeal may be accomplished with only nominal further consideration. Interviews merely to restate arguments of record or to discuss new limitations which would require more than nominal reconsideration or new search should be denied. See MPEP § 714.13.
Interviews may be held after the expiration of the shortened statutory period and prior to the maximum permitted statutory period of 6 months without an extension of time. See MPEP § 706.07(f).
A second or further interview after a final rejection may be held if the examiner is convinced that it will expedite the issues for appeal or disposal of the application.
For interviews after notice of appeal, see MPEP § 1204.03.
Interview time will be revised to a limit of 1 hour per new application or RCE (utility)/CPA (design), when during prosecution, the examiner conducts an interview. When more than one interview is needed in an application supervisors will have the flexibility to approve additional time and ensure that the interviews are being used to advance prosecution.
Authorization for Internet Communications
The examiner encourages Applicant to submit an authorization to communicate with the examiner via the Internet by making the following statement (from MPEP 502.03):
“Recognizing that Internet communications are not secure, I hereby authorize the USPTO to communicate with the undersigned and practitioners in accordance with 37 CFR 1.33 and 37 CFR 1.34 concerning any subject matter of this application by video conferencing, instant messaging, or electronic mail. I understand that a copy of these communications will be made of record in the application file.”
Please note that the above statement can only be submitted via Central Fax (not Examiner's Fax), Regular postal mail, or EFS Web using PTO/SB/439.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In response to the claims amendment, in view of the Remark, the claim objections have been withdrawn.
In response to the claims amendment, in view of the Remarks, the 112 and 101 rejections have been withdrawn.
Specification
The disclosure is objected to because the paragraph numbering in the specification contains duplicate paragraph numbers and therefore does not unambiguously identify each paragraphs, See 37 CFR 1.52(b)(6).
Applicant amendment added new paragraphs [0023], [0024] and [0025]. However, paragraphs [0023], [0024] and [0025] are present in the specification. Thus, two different paragraphs are presently identified by the same paragraph numbers.
Appropriate correction is required.
Claim Objections
Claims 1 - 20 are objected to because of the following informalities:
Regarding claims 1, 11 and 20; there appears to be a typographical error “;” of -- , -- (line 7, claim 1), (line 11, claim 11) and (line 10, claim 20).
Claims 2 – 10 and 12 – 19 are dependent claims and thus also objected.
Regarding claim 19, the limitation “the percent rate of error”, and “the actual resulting output” lack proper antecedent basis.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 – 9, 11 – 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over the prior art of record, MO et al., (US 2019/0034845 A1) (hereinafter “MO”) in view of ABU-MOSTAFA et al., (US 2015/0206065 A1) (hereinafter “Abu”)
Regarding claim 1, MO discloses; a computer-implemented method for analyzing cybersecurity data, comprising:
training, by one or more processors [i.e., processor 103 (see figure 1), (page 3, para 0029)], a first machine learning model using a first training dataset [i.e., a supervised machine learning model is trained (page 2, para 0022) i.e., learn during a training period using training data (page 4, para 0034) i.e., company attribute information can be input to attribute module 105 during training period (page 4, para 0035)] related to at least one area of interest of cybersecurity, the first training dataset comprising parameterized cybersecurity best-practice design data [i.e., slow patching cadence, weak cipher, inadequate use of firewalls and inadequate user of intrusion detection system (page 6, para 0067) Note; these are all measurable cybersecurity practices describing how an organization handles its security practices i.e., best-practice design], parameterized cybersecurity outcome information [i.e., occurrences of cybersecurity events across a random sampling of companies (page 6, para 0061) i.e., the number of K identified or discovered cybersecurity events (page 7, para 0076) i.e., data feeds of real-time reports on data breaches are analyzed, cybersecurity features most relevant to recent breach scenarios are identified, and a probability of a catastrophic breach occurring is predicted based on the prevalence of the identified cybersecurity feature (page 2, para 0023) i.e., the result of learning (page 4, para 0035)] and one or more of: (i) academic training data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)]; wherein training the first machine learning model comprises iteratively [i.e., continuously monitor and analyze the incoming attribute data (page 4, para 0039)] applying evidence-based weighing factors to the first training dataset [i.e., the identified attributes that are shared by a portfolio company and a company that experienced a cybersecurity risk event can be weighted (emphasis added) according to their correlation with the occurrence of the cybersecurity risk event (page 6, para 0063) Note; the weights are applied based on attributes that have been experienced by the company i.e., based on evidence].
storing, by the one or more processors [i.e., processor 103 (see figure 1), (page 3, para 0029)], the first machine learning model in one or more memories [i.e., in operation, system 100 can “learn” during a training period using training data to build a supervised training model and the result of the learning i.e., the supervised training model is then used to monitor whether new data exhibits the same pattern, categories, statistical relationship (page 4, para 0035) i.e., memory 104 stores modules (para 0029), (see figure 1) Note; the supervised model must be stored in memory];
retrieving, by the one or more processors, a first collection of data [i.e., new data (page 4, para 0035) i.e., company attribute information can be input to attribute module 105 during analysis period (page 4, para 0035), (see figure 1) i.e., data feeds of real-time reports on data breaches (page 2, para 0023)], the first collection of data including one or more of academic data [i.e., technical and non-technical data (page 3, para 0032)], open internet data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or corporate data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)], and the first collection of data is related to the at least one area of interest of cybersecurity [i.e., data breaches (page 2, para 0023);
analyzing, by the one or more processors using the first machine learning model stored in the one or more memories, the first collection of data [i.e., a machine learning model is utilized to identify the most significant cybersecurity event…(page 2, para 0022) i.e., data feeds of real-time reports on data breaches are analyzed…a probability of a catastrophic breach occurring …is predicted…(page 2, para 0023) i.e., the result of the leaning is then used to monitor whether new data exhibits the same patterns, categories, statistical relationships (page 4, para 0035)]; and
generating, by the one or more processors based upon the analysis, a resulting output [i.e., output of Cybersecurity Risk Level Module 108 (page 5, para 0051), (see figure 1) i.e., Cybersecurity Risk Level Module 108 quantifies a portfolio’s cybersecurity risk level based on the multiplier generated by multiplier module 107 (page 5, para 0049), (see figure 1) i.e., the multiplier is generated from data gathered using machine learning techniques discussed herein (page 7, para 0076)], the resulting output including one or more of: a strength of a cybersecurity strategy of an organization [i.e., utilize a machine learning model to quantify a portfolio’s cybersecurity risk (page 4, para 0034) i.e., correlate a portfolio’s risk of experiencing an adverse cybersecurity event (page 2, para 0023) i.e., company’s cybersecurity posture (page 2, para 0024)], a recommendation of a change to a cybersecurity strategy of an organization [i.e., output of Cybersecurity Risk Level Module 108 can be utilized by action module 109 to generate steps that, if executed, will change the portfolio’s cybersecurity risk level (page 5, para 0051), (see figure 1)], or a predicted outcome given a cybersecurity strategy of an organization [i.e., output of Cybersecurity Risk Level Module 108 (page 5, para 0051), (see figure 1) i.e., correlate a portfolio’s risk of experiencing an adverse cybersecurity event (page 2, para 0023)].
MO does not disclose;
cross-analyzing the evidence-based weighing factors against non-weighted input sets to increase accuracy of the evidence-based weighing factors in future iterations.
However, Abu-Mostafa discloses;
training a first machine learning model comprises applying weighing factors to a first training dataset [i.e., application of a set of weights 170 to training data 160 (see figure 1), (page 2, para 0018)] and cross-analyzing [i.e., evaluation module (see ref. 150 of figure 1), (page 2, para 0019)] the weighing factors against non-weighted input sets [i.e., the evaluation module 150 determines weight benefit 152 by comparing a function generated with weight and without weight (page 2, para 0017 - 0018), (see figure 1)] to increase accuracy of the evidence-based weighing factors in future iterations [i.e., may provide a method to determine whether application of weights to training data may benefit or hurt the performance of the machine learning system (page 5, para 0047)].
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the teachings of MO by adapting the teachings of Abu-Mostafa to provide a method to determine whether application of weights to training data may benefit or hurt the performance of the machine learning system (page 5, para 0047).
Regarding claim 2, MO discloses; the method of claim 1, wherein the first collection of data includes one or more of automatically retrieved data [i.e., data feeds of real-time reports on data breaches (page 2, para 0023), (page 3, para 0032) i.e., data paths i.e., a real-time processing path and a batch processing path (page 4, para 0039)].
Regarding claim 3, MO discloses; the method of claim 2, wherein the automatically retrieved data is retrieved using one or more artificial intelligence algorithms [i.e., data feeds of real-time reports on data breaches (page 2, para 0023), (page 3, para 0032) i.e., data paths i.e., a real-time processing path and a batch processing path (page 4, para 0039) i.e., machine leaning data handling (page 4, para 0035) i.e., an automated predictive model first processes raw supervised training data (page 4, para 0038)].
Regarding claim 4, MO discloses; the method of claim 1, wherein: (i) the academic data includes peer-reviewed academic research [i.e., technical and non-technical data (page 3, para 0032)]; (ii) the open internet data includes one or more of one or more news sources, one or more blogs, one or more forum posts, or one or more social media sources [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)]; and(iii) the corporate data includes one or more of anonymized corporate data or attributed corporate data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)].
Regarding claim 5, MO discloses; the method of claim 1, wherein the first machine learning model includes one or more of a descriptive analysis algorithm [i.e., a machine learning model is utilized to identify the most significant cybersecurity events and the most significant intercedences between companies to predict an occurrence of a cybersecurity risk event (para 0022) i.e., a probability of a catastrophic breach…is predicted (0023) i.e., statistical model can be trained…to fit a Bayesian model of likelihood of multiple cybersecurity event…an estimate of the risk that multiple companies will experience (para 0076 – 0077)] or a predictive analysis algorithm [i.e., analyzing degrees of dependency between a company that experienced a cybersecurity event and companies in a portfolio (0021) i.e., system 100 can learn during a training period by identifying pattern, category, statistical relationship exhibited by training data (para 0035, 0038 and 0049 - 0051].
Regarding claim 6, MO discloses; the method of claim 1, further comprising:
analyzing, by the one or more processors using one or more statistical modeling algorithms stored in the one or more memories, the first collection of data [i.e., a statistical model can be trained (page 7, para 0076)].
Regarding claim 7, MO discloses; the method of claim 6, wherein the one or more statistical modeling algorithms include a regression model [i.e., a Bayesian model of likelihood (page 7, para 0076)].
Regarding claim 8, MO discloses; the method of claim 1, wherein the at least one area of interest of cybersecurity includes one or more of: ransomware attacks [i.e., catastrophic breaches, or significant cybersecurity events (page 2, para 0023), denial of service attacks [i.e., data breaches or cybersecurity event (page 2, para 0023)], social engineering attacks [i.e., i.e., shard attributes (para 0045) i.e., data breaches or cybersecurity event (page 2, para 0023)], password attacks [i.e., authentication features (para 0073)], cloud attacks[i.e., data breaches or cybersecurity event (page 2, para 0023)], near misses [i.e., data breaches or cybersecurity event (page 2, para 0023)], or threat trends [i.e., data breaches or cybersecurity event (page 2, para 0023)].
Regarding claim 9, MO discloses; the method of claim 1, further comprising:
training, by the one or more processors, a second machine learning model using a second training dataset related to at least one area of interest of cybersecurity [i.e., a supervised machine learning model is trained (page 2, para 0022) i.e., learn during a training period using training data (page 4, para 0034) i.e., company attribute information can be input to attribute module 105 during training period (page 4, para 0035)], the second training dataset comprising outcome information and one or more of: (i) the academic training data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)];
storing, by the one or more processors, the second machine learning model in the one or more memories [i.e., in operation, system 100 can “learn” during a training period using training data to build a supervised training model and the result of the learning i.e., the supervised training model is then used to monitor whether new data exhibits the same pattern, categories, statistical relationship (page 4, para 0035) i.e., memory 104 stores modules (para 0029), (see figure 1) Note; the supervised model must be stored in memory]; and
identifying, by the one or more processors using the second machine learning model stored in the one or more memories, a second collection of data [i.e., new data (page 4, para 0035) i.e., company attribute information can be input to attribute module 105 during analysis period (page 4, para 0035), (see figure 1) i.e., data feeds of real-time reports on data breaches (page 2, para 0023)], the second collection of data including one or more of academic data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)].
Regarding claim 11, MO discloses; a computer system for analyzing cybersecurity data [i.e., a system 100 which quantifies a cybersecurity risk level of a portfolio of companies (page 3, para 0026), (see figure 1)], comprising:
one or more processors [i.e., the system comprises a processor 103 (page 3, para 0029), (see figure 1)];
one or more non-transitory program memories coupled to the one or more processors [i.e., the system comprises a memory 104 coupling to the processor (page 3, para 0029), (see figure 1)] and storing executable instructions that, when executed by the one or more processors, cause the computer system to [i.e., memory 104 stores executable instructions to perform by the processor to perform the following steps (para 0029), (see figure 1)]:
train a first machine learning model using a first training dataset [i.e., a supervised machine learning model is trained (page 2, para 0022) i.e., learn during a training period using training data (page 4, para 0034) i.e., company attribute information can be input to attribute module 105 during training period (page 4, para 0035)] related to at least one area of interest of cybersecurity, the first training dataset comprising parameterized cybersecurity best-practice design data [i.e., slow patching cadence, weak cipher, inadequate use of firewalls and inadequate user of intrusion detection system (page 6, para 0067) Note; these are all measurable cybersecurity practices describing how an organization handles its security practices i.e., best-practice design], parameterized cybersecurity outcome information [i.e., occurrences of cybersecurity events across a random sampling of companies (page 6, para 0061) i.e., the number of K identified or discovered cybersecurity events (page 7, para 0076) i.e., data feeds of real-time reports on data breaches are analyzed, cybersecurity features most relevant to recent breach scenarios are identified, and a probability of a catastrophic breach occurring is predicted based on the prevalence of the identified cybersecurity feature (page 2, para 0023) i.e., the result of learning (page 4, para 0035)] and one or more of: (i) academic training data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)]; wherein training the first machine learning model comprises iteratively [i.e., continuously monitor and analyze the incoming attribute data (page 4, para 0039)] applying evidence-based weighing factors to the first training dataset [i.e., the identified attributes that are shared by a portfolio company and a company that experienced a cybersecurity risk event can be weighted (emphasis added) according to their correlation with the occurrence of the cybersecurity risk event (page 6, para 0063) Note; the weights are applied based on attributes that have been experienced by the company i.e., based on evidence];
store the first machine learning model in one or more memories [i.e., in operation, system 100 can “learn” during a training period using training data to build a supervised training model and the result of the learning i.e., the supervised training model is then used to monitor whether new data exhibits the same pattern, categories, statistical relationship (page 4, para 0035) i.e., memory 104 stores modules (para 0029), (see figure 1) Note; the supervised model must be stored in memory];
retrieve a first collection of data [i.e., new data (page 4, para 0035) i.e., company attribute information can be input to attribute module 105 during analysis period (page 4, para 0035), (see figure 1) i.e., data feeds of real-time reports on data breaches (page 2, para 0023)], the first collection of data including one or more of academic data [i.e., technical and non-technical data (page 3, para 0032)], open internet data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or corporate data i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)], and the first collection of data is related to the at least one area of interest of cybersecurity [i.e., data breaches (page 2, para 0023);
analyze using the first machine learning model stored in the one or more memories, the first collection of data [i.e., a machine learning model is utilized to identify the most significant cybersecurity event…(page 2, para 0022) i.e., data feeds of real-time reports on data breaches are analyzed…a probability of a catastrophic breach occurring …is predicted…(page 2, para 0023) i.e., the result of the leaning is then used to monitor whether new data exhibits the same patterns, categories, statistical relationships (page 4, para 0035)]; and
generate based upon the analysis, a resulting output [i.e., output of Cybersecurity Risk Level Module 108 (page 5, para 0051), (see figure 1) i.e., Cybersecurity Risk Level Module 108 quantifies a portfolio’s cybersecurity risk level based on the multiplier generated by multiplier module 107 (page 5, para 0049), (see figure 1) i.e., the multiplier is generated from data gathered using machine learning techniques discussed herein (page 7, para 0076)], the resulting output including one or more of: a strength of a cybersecurity strategy of an organization [i.e., utilize a machine learning model to quantify a portfolio’s cybersecurity risk (page 4, para 0034) i.e., correlate a portfolio’s risk of experiencing an adverse cybersecurity event (page 2, para 0023)], a recommendation of a change to a cybersecurity strategy of an organization [i.e., output of Cybersecurity Risk Level Module 108 can be utilized by action module 109 to generate steps that, if executed, will change the portfolio’s cybersecurity risk level (page 5, para 0051), (see figure 1) i.e., company’s cybersecurity posture (page 2, para 0024)], or a predicted outcome given a cybersecurity strategy of an organization [i.e., output of Cybersecurity Risk Level Module 108 (page 5, para 0051), (see figure 1) i.e., correlate a portfolio’s risk of experiencing an adverse cybersecurity event (page 2, para 0023)].
MO does not disclose;
cross-analyzing the evidence-based weighing factors against non-weighted input sets to increase accuracy of the evidence-based weighing factors in future iterations.
However, Abu-Mostafa discloses;
training a first machine learning model comprises applying weighing factors to a first training dataset [i.e., application of a set of weights 170 to training data 160 (see figure 1), (page 2, para 0018)] and cross-analyzing [i.e., evaluation module (see ref. 150 of figure 1), (page 2, para 0019)] the weighing factors against non-weighted input sets [i.e., the evaluation module 150 determines weight benefit 152 by comparing a function generated with weight and without weight (page 2, para 0017 - 0018), (see figure 1)] to increase accuracy of the evidence-based weighing factors in future iterations [i.e., may provide a method to determine whether application of weights to training data may benefit or hurt the performance of the machine learning system (page 5, para 0047)].
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the teachings of MO by adapting the teachings of Abu-Mostafa to provide a method to determine whether application of weights to training data may benefit or hurt the performance of the machine learning system (page 5, para 0047).
Regarding claim 12, MO discloses; the system of claim 11, wherein the first collection of data includes one or more of automatically retrieved data [i.e., data feeds of real-time reports on data breaches (page 2, para 0023), (page 3, para 0032) i.e., data paths i.e., a real-time processing path and a batch processing path (page 4, para 0039)].
Regarding claim 13, MO discloses; the system of claim 12, wherein the automatically retrieved data is retrieved using one or more artificial intelligence algorithms [i.e., data feeds of real-time reports on data breaches (page 2, para 0023), (page 3, para 0032) i.e., data paths i.e., a real-time processing path and a batch processing path (page 4, para 0039) i.e., machine leaning data handling (page 4, para 0035) i.e., an automated predictive model first processes raw supervised training data (page 4, para 0038)].
Regarding claim 14, MO discloses; the system of claim 11, wherein: (i) the academic data includes peer-reviewed academic research [i.e., technical and non-technical data (page 3, para 0032)]; (ii) the open internet data includes one or more of one or more news sources, one or more blogs, one or more forum posts, or one or more social media sources [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)]; and(iii) the corporate data includes one or more of anonymized corporate data or attributed corporate data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)].
Regarding claim 15, MO discloses; the system of claim 11, wherein the first machine learning model includes one or more of a descriptive analysis algorithm [i.e., a machine learning model is utilized to identify the most significant cybersecurity events and the most significant intercedences between companies to predict an occurrence of a cybersecurity risk event (para 0022) i.e., a probability of a catastrophic breach…is predicted (0023) i.e., statistical model can be trained…to fit a Bayesian model of likelihood of multiple cybersecurity event…an estimate of the risk that multiple companies will experience (para 0076 – 0077)] or a predictive analysis algorithm [i.e., analyzing degrees of dependency between a company that experienced a cybersecurity event and companies in a portfolio (0021) i.e., system 100 can learn during a training period by identifying pattern, category, statistical relationship exhibited by training data (para 0035, 0038 and 0049 - 0051].
Regarding claim 16, MO discloses; the system of claim 11, wherein the executable instructions, when executed by the one or more processors, further cause the computer system to:
analyze, using one or more statistical modeling algorithms stored in the one or more non-transitory program memories, the first collection of data [i.e., a statistical model can be trained (page 7, para 0076)], the one or more statistical modeling algorithms include a regression model [i.e., a Bayesian model of likelihood (page 7, para 0076)].
Regarding claim 17, MO discloses; the system of claim 11, wherein the at least one area of interest of cybersecurity includes one or more of: ransomware attacks [i.e., catastrophic breaches, or significant cybersecurity events (page 2, para 0023), denial of service attacks [i.e., data breaches or cybersecurity event (page 2, para 0023)], social engineering attacks [i.e., i.e., shard attributes (para 0045) i.e., data breaches or cybersecurity event (page 2, para 0023)], password attacks [i.e., authentication features (para 0073)], cloud attacks[i.e., data breaches or cybersecurity event (page 2, para 0023)], near misses [i.e., data breaches or cybersecurity event (page 2, para 0023)], or threat trends [i.e., data breaches or cybersecurity event (page 2, para 0023)].
Regarding claim 18, MO discloses; the system of claim 11, wherein the executable instructions, when executed by the one or more processors, further cause the computer system to:
train a second machine learning model using a second training dataset related to at least one area of interest of cybersecurity [i.e., a supervised machine learning model is trained (page 2, para 0022) i.e., learn during a training period using training data (page 4, para 0034) i.e., company attribute information can be input to attribute module 105 during training period (page 4, para 0035)], the second train dataset comprising outcome information and one or more of: (i) the academic training data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)];
store the second machine learning model in the one or more memories [i.e., in operation, system 100 can “learn” during a training period using training data to build a supervised training model and the result of the learning i.e., the supervised training model is then used to monitor whether new data exhibits the same pattern, categories, statistical relationship (page 4, para 0035) i.e., memory 104 stores modules (para 0029), (see figure 1) Note; the supervised model must be stored in memory]; and
identify using the second machine learning model stored in the one or more memories, a second collection of data [i.e., new data (page 4, para 0035) i.e., company attribute information can be input to attribute module 105 during analysis period (page 4, para 0035), (see figure 1) i.e., data feeds of real-time reports on data breaches (page 2, para 0023)], the second collection of data including one or more of academic data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)].
Regarding claim 20, MO discloses; a tangible, non-transitory computer-readable medium storing executable instructions [i.e., memory 104 stores executable instructions to perform by the processor to perform the following steps (para 0029), (see figure 1)] for predicting the time to replace one or more vehicle seats, the instructions, when executed by one or more processors of a computer system, cause the computer system to:
train a first machine learning model using a first training dataset [i.e., a supervised machine learning model is trained (page 2, para 0022) i.e., learn during a training period using training data (page 4, para 0034) i.e., company attribute information can be input to attribute module 105 during training period (page 4, para 0035)] related to at least one area of interest of cybersecurity, the first training dataset comprising parameterized cybersecurity best-practice design data [i.e., slow patching cadence, weak cipher, inadequate use of firewalls and inadequate user of intrusion detection system (page 6, para 0067) Note; these are all measurable cybersecurity practices describing how an organization handles its security practices i.e., best-practice design], parameterized cybersecurity outcome information [i.e., occurrences of cybersecurity events across a random sampling of companies (page 6, para 0061) i.e., the number of K identified or discovered cybersecurity events (page 7, para 0076) i.e., data feeds of real-time reports on data breaches are analyzed, cybersecurity features most relevant to recent breach scenarios are identified, and a probability of a catastrophic breach occurring is predicted based on the prevalence of the identified cybersecurity feature (page 2, para 0023) i.e., the result of learning (page 4, para 0035)] and one or more of: (i) academic training data [i.e., technical and non-technical data (page 3, para 0032)], (ii) open internet training data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or (iii) corporate training data [i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)]; wherein training the first machine learning model comprises iteratively [i.e., continuously monitor and analyze the incoming attribute data (page 4, para 0039)] applying evidence-based weighing factors to the first training dataset [i.e., the identified attributes that are shared by a portfolio company and a company that experienced a cybersecurity risk event can be weighted (emphasis added) according to their correlation with the occurrence of the cybersecurity risk event (page 6, para 0063) Note; the weights are applied based on attributes that have been experienced by the company i.e., based on evidence];
store the first machine learning model in one or more memories [i.e., in operation, system 100 can “learn” during a training period using training data to build a supervised training model and the result of the learning i.e., the supervised training model is then used to monitor whether new data exhibits the same pattern, categories, statistical relationship (page 4, para 0035) i.e., memory 104 stores modules (para 0029), (see figure 1) Note; the supervised model must be stored in memory];
retrieve a first collection of data [i.e., new data (page 4, para 0035) i.e., company attribute information can be input to attribute module 105 during analysis period (page 4, para 0035), (see figure 1) i.e., data feeds of real-time reports on data breaches (page 2, para 0023)], the first collection of data including one or more of academic data [i.e., technical and non-technical data (page 3, para 0032)], open internet data [i.e., scraping online information from websites and news source (page 3, para 0032) i.e., data feeds of real-time reports on data breach (page 2, para 0023)], or corporate data i.e., company attribute information (page 4, para 0035) i.e., attribute of a company can be proprietary, technical, and non-technical data relating to a company…scraping data from corporate filings (page 3, para 0032)], and the first collection of data is related to the at least one area of interest of cybersecurity [i.e., data breaches (page 2, para 0023);
analyze using the first machine learning model stored in the one or more memories, the first collection of data [i.e., a machine learning model is utilized to identify the most significant cybersecurity event…(page 2, para 0022) i.e., data feeds of real-time reports on data breaches are analyzed…a probability of a catastrophic breach occurring …is predicted…(page 2, para 0023) i.e., the result of the leaning is then used to monitor whether new data exhibits the same patterns, categories, statistical relationships (page 4, para 0035)]; and
generate based upon the analysis, a resulting output [i.e., output of Cybersecurity Risk Level Module 108 (page 5, para 0051), (see figure 1) i.e., Cybersecurity Risk Level Module 108 quantifies a portfolio’s cybersecurity risk level based on the multiplier generated by multiplier module 107 (page 5, para 0049), (see figure 1) i.e., the multiplier is generated from data gathered using machine learning techniques discussed herein (page 7, para 0076)], the resulting output including one or more of: a strength of a cybersecurity strategy of an organization [i.e., utilize a machine learning model to quantify a portfolio’s cybersecurity risk (page 4, para 0034) i.e., correlate a portfolio’s risk of experiencing an adverse cybersecurity event (page 2, para 0023)], a recommendation of a change to a cybersecurity strategy of an organization [i.e., output of Cybersecurity Risk Level Module 108 can be utilized by action module 109 to generate steps that, if executed, will change the portfolio’s cybersecurity risk level (page 5, para 0051), (see figure 1) i.e., company’s cybersecurity posture (page 2, para 0024)], or a predicted outcome given a cybersecurity strategy of an organization [i.e., output of Cybersecurity Risk Level Module 108 (page 5, para 0051), (see figure 1) i.e., correlate a portfolio’s risk of experiencing an adverse cybersecurity event (page 2, para 0023)].
MO does not disclose;
cross-analyzing the evidence-based weighing factors against non-weighted input sets to increase accuracy of the evidence-based weighing factors in future iterations.
However, Abu-Mostafa discloses;
training a first machine learning model comprises applying weighing factors to a first training dataset [i.e., application of a set of weights 170 to training data 160 (see figure 1), (page 2, para 0018)] and cross-analyzing [i.e., evaluation module (see ref. 150 of figure 1), (page 2, para 0019)] the weighing factors against non-weighted input sets [i.e., the evaluation module 150 determines weight benefit 152 by comparing a function generated with weight and without weight (page 2, para 0017 - 0018), (see figure 1)] to increase accuracy of the evidence-based weighing factors in future iterations [i.e., may provide a method to determine whether application of weights to training data may benefit or hurt the performance of the machine learning system (page 5, para 0047)].
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify the teachings of MO by adapting the teachings of Abu-Mostafa to provide a method to determine whether application of weights to training data may benefit or hurt the performance of the machine learning system (page 5, para 0047).
Claim(s) 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over MO in view of Abu as applied to claims 1 and 11 above, and further in view of the prior art of record, Sabes et al., (US 11,817,214 B1) (hereinafter “Sabes”).
Regarding claim 10, MO discloses; the method of claim 1 [i.e., (see claim 1 above)].
MO and Abu-Mostafa do not disclose;
wherein: training the first machine learning model comprises: reducing, by the one or more processors, a percent rate of error of generating the resulting output by calculating one or more of: (i) an ordinary least squares of the difference between the generated resulting output and an actual resulting output of the first training data set, or (ii) an ordinary mean square of an aggregation of results between the generated resulting output and the actual resulting output of the first training data set; and generating, by the one or more processors, a confidence interval based upon one or more of: (i) the generated resulting output, (ii) the actual resulting output of the first training data set, and/or (iii) one or more standard deviations from the aggregated result.
However, Sabes discloses;
wherein: training the first machine learning model comprises:
reducing, by the one or more processors, a percent rate of error of generating the resulting output [i.e., determining loss/error based on the difference between an estimate and a label and updating model parameters to reduce the error (col. 21, line 16 – 27)] by calculating one or more of: (i) a ordinary least squares of the difference between the generated resulting output and the actual resulting output of the first training data set [i.e., ordinary least squares regression (OLSR) (col. 20, lines 60 – 65)], or (ii) an ordinary mean square of an aggregation of results between the generated resulting output and the actual resulting output of the first training data set [i.e., mean squared error (L2 loss) as a loss function used during model training (col. 21, lines 27 – 34)]; and
generating, by the one or more processors, a confidence interval based upon one or more of: (i) the generated resulting output, (ii) the actual resulting output of the first training data set, and/or (iii) one or more standard deviations from the aggregated result [i.e., ML outputs including a confidence and/or confidence interval, including embodiments with separate output nodes for prediction and confidence (col. 9, lines 8 – 35)].
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify teachings of MO and Abu-Mostafa by adapting the teachings of Sabes to improve the accuracy, reliability, and interpretability of the machine learning model (See Sabes; col. 1, lines 51 – 61).
Regarding claim 19, MO discloses; the system of claim 11 [i.e., (see claim 11 above)].
MO and Abu-Mostafa do not disclose;
wherein: training the first machine learning model comprises: reducing, by the one or more processors, the percent rate of error of generating the resulting output by calculating one or more of: (i) the ordinary least squares of the difference between the generated resulting output and the actual resulting output of the first training data set, or (ii) the ordinary mean square of an aggregation of results between the generated resulting output and the actual resulting output of the first training data set; and generating, by the one or more processors, a confidence interval based upon one or more of: (i) the generated resulting output, (ii) the actual resulting output of the first training data set, and/or (iii) one or more standard deviations from the aggregated result.
However, Sabes discloses;
wherein: training the first machine learning model comprises:
reducing, by the one or more processors, the percent rate of error of generating the resulting output [i.e., determining loss/error based on the difference between an estimate and a label and updating model parameters to reduce the error (col. 21, line 16 – 27)] by calculating one or more of: (i) the ordinary least squares of the difference between the generated resulting output and the actual resulting output of the first training data set [i.e., ordinary least squares regression (OLSR) (col. 20, lines 60 – 65)], or (ii) the ordinary mean square of an aggregation of results between the generated resulting output and the actual resulting output of the first training data set [i.e., mean squared error (L2 loss) as a loss function used during model training (col. 21, lines 27 – 34)]; and
generating, by the one or more processors, a confidence interval based upon one or more of: (i) the generated resulting output, (ii) the actual resulting output of the first training data set, and/or (iii) one or more standard deviations from the aggregated result [i.e., ML outputs including a confidence and/or confidence interval, including embodiments with separate output nodes for prediction and confidence (col. 9, lines 8 – 35)].
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify teachings of MO and Abu-Mostafa by adapting the teachings of Sabes to improve the accuracy, reliability, and interpretability of the machine learning model (See Sabes; col. 1, lines 51 – 61).
Response to Arguments
Applicant's arguments filed 07/02/2026 have been fully considered but they are not persuasive because of the followings;
Regarding claims 1 – 9, 11 – 18 and 20; applicant argued that “Mo contains no disclosure or suggestion of training on parameterized cybersecurity best-practice design data paired with parameterized cybersecurity outcome information. Mo’s training data concerns what companies have, i.e., shared attributes, not what cybersecurity strategy decision companies have made and what outcomes resulted” (see Remarks; page 16).
The Examiner respectfully disagrees with this argument because MO’s training data is not limited merely to what companies “have” as applicant incorrectly identified above. MO expressly identified cybersecurity best-practice attributes including slow patching cadence, weak ciphers, and inadequate use of firewalls and intrusion detection systems (see the Office’s rejection above), which reasonably correspond to parameterized cybersecurity best-practice design data. MO further teaches training by analyzing occurrences of cybersecurity event and evaluating and weighing the attributes of companies that experienced such events according to their correlation with occurrence of the cybersecurity risk event (see the Office’s rejection above). Thus, MO teaches training using cybersecurity practice parameters in association with corresponding cybersecurity outcome information, rather than merely identifying shared company attributes.
Applicant’s arguments with respect to Mo fail to disclose “wherein training the first machine learning model comprises iteratively applying evidence-based weighing factors to the first training dataset and cross-analyzing the evidence-based weighing factors against non-weighted input sets to increase accuracy of the evidence-based weighing factors in future iterations” (See Remarks; pages 16 – 17) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYED A RONI whose telephone number is (571)270-7806. The examiner can normally be reached M-F 9:00-5:00 pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeffrey L Nickerson can be reached at (469) 295-9235. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYED A RONI/Primary Examiner, Art Unit 2432
Prosecution Insights