DETAILED ACTION
This action is responsive to claims filed 05/16/2023.
Claims 1–20 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/16/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Drawings
The drawings are objected to because “[e]ach group of waveforms must be presented as a single figure, using a common vertical axis with time extending along the horizontal axis.” See MPEP 608.02.V(d). Specifically, Fig. 6D includes waveforms not presented in a single figure using a common vertical axis. Fig. 6D depicts two graphs within the same figure.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title
Claims 1–20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Step 1 — Is the claim to a process, machine, manufacture, or composition of matter?
Yes, claim 1 is directed to a method i.e., a process.
Step 2A — Prong 1 Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
“quantifying, [by the device,] how influential a particular telemetry metric is on the predictions”
These limitations, under their broadest reasonable interpretation, cover mental processes, concepts performed in the human mind (including an observation, evaluation, judgment, opinion). See MPEP 2106.04(a)(2). In particular, with the aid of pen and paper, a human can quantify how much of an impact particular data has on machine learning predictions.
Step 2A — Prong 2 — Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
“by the device” — This limitation is reciting generic computer components at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. See MPEP 2106.05(f).
“obtaining, by a device, a plurality of telemetry metrics regarding an online application accessed via a network” — This limitation is insignificant extra-solution activity and is merely data gathering. See MPEP 2106.05(g).
“training, by the device and based on the plurality of telemetry metrics, a gradient boosted tree-based prediction model to make predictions regarding a quality of experience for the online application” — This limitation amounts to no more than mere instructions to apply the exception and is the equivalent to mere instruction to implement the abstract idea on a computer. See MPEP 2106.05(f). Training prediction models to make predictions merely invokes computers or other machinery as a tool to perform an existing process.
“providing, by the device, a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Step 2B — Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception.
“obtaining, by a device, a plurality of telemetry metrics regarding an online application accessed via a network” — This limitation is directed to the activity of data gathering which is not an inventive concept because it is insignificant extra-solution activity of mere data gathering. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves transmitting information over a network. MPEP 2106.05(d)(II).
“providing, by the device, a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Regarding claim 2:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites a further limitation on the obtaining limitation which is directed to mere data gathering. Examiner notes the limitation merely limits the type of data being retrieved. The additional limitation:
“wherein the particular telemetry metric is a Layer-3 metric captured by the network” — This limitation is insignificant extra-solution activity and is merely data gathering. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“wherein the particular telemetry metric is a Layer-3 metric captured by the network” — This limitation is directed to the activity of data gathering which is not an inventive concept because it is insignificant extra-solution activity of mere data gathering. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves transmitting information over a network. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding claim 3:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites a further limitation on the obtaining limitation which is directed to mere data gathering. Examiner notes the limitation merely limits the type of data being retrieved. The additional limitation:
“wherein the particular telemetry metric is a Layer-7 metric computed by the online application” — This limitation is insignificant extra-solution activity and is merely data gathering. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“wherein the particular telemetry metric is a Layer-7 metric computed by the online application” — This limitation is directed to the activity of data gathering which is not an inventive concept because it is insignificant extra-solution activity of mere data gathering. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves transmitting information over a network. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding claim 4:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites an additional limitation on the overall method which is directed to an abstract idea including mental concepts performed in a human mind. The additional limitation:
“providing an indication of a threshold for the particular telemetry metric for display by the visualization tool at which a change occurs in the predictions” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“providing an indication of a threshold for the particular telemetry metric for display by the visualization tool at which a change occurs in the predictions” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding claim 5:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites a further limitation on the providing limitation which is directed to an insignificant extra-solution activity because it involves presenting data based on results. The additional limitation:
“providing a representation of a decision split in the gradient boosted tree-based prediction model that is contingent on the particular telemetry metric for display by the visualization tool” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“providing a representation of a decision split in the gradient boosted tree-based prediction model that is contingent on the particular telemetry metric for display by the visualization tool” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding claim 6:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites a further limitation on the providing limitation which is directed to an insignificant extra-solution activity because it involves presenting data based on results. The additional limitation:
“providing distributions of positive and negative samples of the particular telemetry metric for display by the visualization tool” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“providing distributions of positive and negative samples of the particular telemetry metric for display by the visualization tool” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding claim 7:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites an additional limitation on the overall method which is directed to an abstract idea including mental concepts performed in a human mind. The additional limitation:
“providing a dimensionality reduction between the particular telemetry metric and another metric for display by the visualization tool” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“providing a dimensionality reduction between the particular telemetry metric and another metric for display by the visualization tool” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding claim 8:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites a further limitation on the training limitation which is directed to mere instructions to apply an abstract idea that can be performed in the human mind. The additional limitation:
“wherein the device trains the gradient boosted tree-based prediction model using quality of experience feedback provided by users of the online application” — This limitation is reciting only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished such that it amounts no more than mere instructions to apply. See MPEP 2106.05(f); See also Electric Power Group, LLC v. Alstom, S.A., 830 F.3d 1350, 1356, 119 USPQ2d 1739 (Fed. Cir. 2016).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 9:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites a further limitation on the overall method which is directed to an abstract idea that can be performed in the human mind. The additional limitation:
“controlling the visualization tool to highlight anomalous data points in the plurality of telemetry metrics, based on a parameter set by a user of the visualization tool” — This limitation amounts to no more than mere instructions to apply the exception and is the equivalent to mere instruction to implement the abstract idea on a computer. See MPEP 2106.05(f). highlighting datapoints based on user input merely invokes computers or other machinery as a tool to perform an existing process.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding claim 10:
The claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1 above). This claim merely recites an additional limitation on the overall method which is directed to an abstract idea including mental concepts performed in a human mind. The additional limitation:
“providing an indication of a loss function associated with the gradient boosted tree-based prediction model for display by the visualization tool” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d)I.), failing Step 2A Prong 2.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
“providing an indication of a loss function associated with the gradient boosted tree-based prediction model for display by the visualization tool” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception (see MPEP 2106.05(I.), failing step 2B.
Regarding Claim 11:
Step 1 — Is the claim to a process, machine, manufacture, or composition of matter?
Yes, claim 1 is directed to an apparatus i.e., a machine.
Step 2A — Prong 1 Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
“quantify how influential a particular telemetry metric is on the predictions”
These limitations, under their broadest reasonable interpretation, cover mental processes, concepts performed in the human mind (including an observation, evaluation, judgment, opinion). See MPEP 2106.04(a)(2). In particular, with the aid of pen and paper, a human can quantify how much of an impact particular data has on machine learning predictions.
Step 2A — Prong 2 — Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
“a processor coupled to the one or more network interfaces and configured to execute one or more processes; and a memory configured to store a process that is executable by the processor, the process when executed configured to” — This limitation is reciting generic computer components at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. See MPEP 2106.05(f).
“obtain a plurality of telemetry metrics regarding an online application accessed via a network” — This limitation is insignificant extra-solution activity and is merely data gathering. See MPEP 2106.05(g).
“train, based on the plurality of telemetry metrics, a gradient boosted tree- based prediction model to make predictions regarding a quality of experience for the online application” — This limitation amounts to no more than mere instructions to apply the exception and is the equivalent to mere instruction to implement the abstract idea on a computer. See MPEP 2106.05(f). Training prediction models to make predictions merely invokes computers or other machinery as a tool to perform an existing process.
“provide a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Step 2B — Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception.
“obtain a plurality of telemetry metrics regarding an online application accessed via a network” — This limitation is directed to the activity of data gathering which is not an inventive concept because it is insignificant extra-solution activity of mere data gathering. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves transmitting information over a network. MPEP 2106.05(d)(II).
“provide a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Regarding Claim 20:
Step 1 — Is the claim to a process, machine, manufacture, or composition of matter?
Yes, claim 1 is directed to a computer-readable medium i.e., a machine.
Step 2A — Prong 1 Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites an abstract idea.
“quantifying, [by the device,] how influential a particular telemetry metric is on the predictions”
These limitations, under their broadest reasonable interpretation, cover mental processes, concepts performed in the human mind (including an observation, evaluation, judgment, opinion). See MPEP 2106.04(a)(2). In particular, with the aid of pen and paper, a human can quantify how much of an impact particular data has on machine learning predictions.
Step 2A — Prong 2 — Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
“a tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising” — This limitation is reciting generic computer components at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. See MPEP 2106.05(f).
“by a device” — This limitation is reciting generic computer components at a high-level of generality (i.e., as a generic computer component performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. See MPEP 2106.05(f).
“obtaining, by a device, a plurality of telemetry metrics regarding an online application accessed via a network” — This limitation is insignificant extra-solution activity and is merely data gathering. See MPEP 2106.05(g).
“training, by the device and based on the plurality of telemetry metrics, a gradient boosted tree-based prediction model to make predictions regarding a quality of experience for the online application” — This limitation amounts to no more than mere instructions to apply the exception and is the equivalent to mere instruction to implement the abstract idea on a computer. See MPEP 2106.05(f). Training prediction models to make predictions merely invokes computers or other machinery as a tool to perform an existing process.
“providing, by the device, a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions” — This limitation is insignificant extra-solution activity and is merely data outputting. See MPEP 2106.05(g).
Step 2B — Does the claim recite additional elements that amount to significantly more than the judicial exception?
No, there are no additional elements that amount to significantly more than the judicial exception.
“obtaining, by a device, a plurality of telemetry metrics regarding an online application accessed via a network” — This limitation is directed to the activity of data gathering which is not an inventive concept because it is insignificant extra-solution activity of mere data gathering. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves transmitting information over a network. MPEP 2106.05(d)(II).
“providing, by the device, a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions” — This limitation is directed to the activity of data outputting which is not an inventive concept because it is insignificant extra-solution activity of mere data outputting. See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3). This limitation is well-understood, routine, and conventional because it involves presenting data based on results. MPEP 2106.05(d)(II).
Regarding claims 12–19, although varying in scope, the limitations of claims 12–19 are substantially the same as the limitations of claims 2–9, respectively. Thus, claims 12–19 are rejected using the same reasoning and analysis as claims 2–9 above, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 4–11, and 14–20 are rejected under 35 U.S.C. 103 as being unpatentable over Cella et al., (US 20220197306 A1), hereinafter “Cella”, in view of Guim et al., (WO 2020226979 A2), hereinafter “Guim”.
Regarding claim 1, Cella teaches:
obtaining, by a device, a plurality of telemetry metrics regarding an online application accessed via a network (Cella ¶1032, ¶1076: “In embodiments, the EMP receives telemetry data from a client application associated with a particular user and learns the workflows performed by the particular user based on the telemetry data and the surrounding circumstances” and “In some embodiments, the digital twin generation system 8108 (in combination with the digital twin I/O system 8104) may obtain data streams from traditional data sources, such as relational databases, API interfaces, direct sensor input, human generated input, Hadoop file stores, graph databases that underlie operational and reporting tooling in the environment, telemetry data sources”—[(emphasis added)]);
training, by the device and based on the plurality of telemetry metrics, a gradient boosted tree-based prediction model to make predictions regarding a quality of experience for the online application (Cella ¶0114, ¶0263, ¶0464, ¶1032, ¶1671: “In embodiments, the artificial intelligence system makes use of an algorithm comprising an artificial neural network, a decision tree” and “In example embodiments, the presentation layer may include, but is not limited to, a user interface, and modules for investigation and discovery and tracking users' experience and engagements” and “The machine learning model 3000 builds one or more mathematical models based on training data to make predictions and/or decisions without being explicitly programmed to perform the specific tasks. The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics, simulation, decision making, and prediction making relating to the data processing, data analysis, simulation creation, and simulation analysis of the one or more of the value chain entities 652. The machine learning model 3000 may also use input data from a user or users of the information technology system. The machine learning model 3000 may include an artificial neural network, a decision tree” and “The executive agent may be trained to identify the COO's tendencies based on the COO's previous interaction with the COO digital twin” and “For example, one of the circuits may configure (e.g., using configuration parameters specified by an analytics library) a first AI-assisted technique to detect a particular condition (e.g., a gradient-boosted trees model), and then the same or another circuit may use a different AI-assisted technique (e.g., a neural network trained using deep learning techniques) to predict the response to a treatment plan for the particular condition”—[wherein the system is trained in part based on a previous interaction (i.e., quality of the previous interaction; e.g., users’ experience) including telemetry metrics]);
quantifying, by the device, how influential a particular telemetry metric is on the predictions (Cella ¶0318, ¶1562: “In embodiments, the set of predictions 3070 includes a least one prediction of an impact on a supply chain application based on a current state of a coordinated demand management application, such as a prediction that a demand for a good will decrease earlier than previously anticipated” and “In embodiments, the artificial intelligence modules 8804 may include and/or provide access to an analytics module 8818. In embodiments, an analytics module 8818 is configured to perform various analytical processes on data output from value chain entities or other data sources. In example embodiments, analytics produced by the analytics module 8818 may facilitate quantification of system performance as compared to a set of goals and/or metrics … In some example implementations, analytics processes may include tracking goals and/or specific metrics that involve coordination of value chain activities and demand intelligence, such as involving forecasting demand for a set of relevant items by location and time (among many others)”—[(emphasis added)]); and
providing, by the device, a visualization tool for display that indicates how influential the particular telemetry metric is on the predictions (Cella ¶0263, ¶0318, ¶0334, ¶0469, ¶0479: “In example embodiments, the presentation layer may include, but is not limited to, a user interface, and modules for investigation and discovery and tracking users' experience and engagements” and “In embodiments, the set of predictions 3070 includes a least one prediction of an impact on a supply chain application based on a current state of a coordinated demand management application, such as a prediction that a demand for a good will decrease earlier than previously anticipated” and “The adaptive intelligence systems 808 may provide further processing for automated control signal generation, such as by applying artificial intelligence to determine aspects of a value chain that impact automated control of the coordinated set of demand management applications and supply chain applications for a category of goods” and “The machine learning model 3000 may learn to analyze training data to output suggestions to the user of the information technology system regarding which types of sensor data are most relevant to achieving the modeling goal” and “Thus, input variation and testing of the impact of input variation on model effectiveness may be used to prune or enhance model performance for any of the machine learning systems described throughout this disclosure”—[wherein the user interface includes presentation layers (i.e., a visualization tool for display) that shows users output from a prediction model including variational input data that indicates which data is most relevant to the model (i.e., how influential the particular telemetry metric is on the predictions)]).
Cella does not appear to explicitly teach:
training, by the device and based on the plurality of telemetry metrics, a gradient boosted tree-based prediction model [to make predictions regarding a quality of experience for the online application.]
However, Guim teaches:
[training, by the device and based on the plurality of telemetry metrics, a gradient boosted tree-based prediction model] to make predictions regarding a quality of experience for the online application (Guim ¶0169, ¶0195, ¶¶0922–0923: “FIG.13 illustrates a comparison of operational considerations for edge computing among operational network layers. Specifically, FIG.13 illustrates the degrees and types of tradeoffs experienced among different compute processing locations” and “In addition to requirements and constraints provided from the mapping of workload types, other measurements or indicators may be used to select or configure an edge execution platform. For instance, mapping of services on a particular execution platform may consider: KPI performance benefits or user experience benefits (e.g., what latency is required to provide a good user experience for 360-degree video)” and “In an example, the gateway includes an automatic load balancer for tenant requests. The load balancer is sensitive to the tenant configuration for power budgeting to execute FaaS or AFaaS. Thus, if the power budget configuration specifies a low-performance requirement with a low power budget, the load balancer will prefer a low-power compute element over a high- power compute element. In an example, the gateway also includes interfaces to orchestrator components (e.g., cluster head nodes) to manage Service/AFaaS/FaaS, billing, or other configurable elements. In addition to the gateway, rack and platform power management may be employed. Here, the rack or platform expose interfaces to configure how much power is allocated per tenant. To avoid performance glass jaws (e.g., unexpected failure or performance degradation), the platform provides automatic service degradation prediction or monitoring per service, FaaS, or AFaaS. In an example, degradation, or the prediction thereof, is communicated to the orchestration layers”—[wherein predictions regarding the performance of the service (i.e., quality of experience) are made based on the online application]).
The methods of Cella, the teachings of Guim, and the instant application are analogous art because they pertain to using machine learning and telemetry data to make predictions.
It would be obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the methods of Cella with the teachings of Guim to provide predictions about the quality of the experience of the online application based on users. One would be motivated to do so to avoid unexpected failure or performance degradation (Guim ¶0924:).
Regarding claim 4, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
providing an indication of a threshold for the particular telemetry metric for display by the visualization tool at which a change occurs in the predictions (Cella ¶¶1794–1795: “Defects may be identified, e.g., by removing noise from the inspection data and subtracting a reference data set (e.g., a reference image of a defect-free part in the case that machine vision tools are being utilized for inspection), and classified using an unsupervised machine learning algorithm such as cluster analysis or an artificial neural network, to classify individual objects as either meeting or failing to meet a specified set of decision criteria (e.g., a decision boundary) in the feature space in which defects are being monitored. For example, a partially printed part may be compared with a render of the partial part and in case the partial part differs beyond a selected threshold from the render, the part may be classified as defective … In embodiments, in-process the defect classification data may be used by the machine learning algorithm to determine a set or sequence of process control parameter adjustments that will implement a corrective action, e.g., to adjust a layer dimension or thickness, so as to correct a defect when first detected. In some embodiments, in-process automated defect classification may be used by the machine learning algorithm to send a warning or error signal to an operator, or optionally, to automatically abort the deposition process”—[(emphasis added)]).
Regarding claim 5, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
wherein providing the visualization tool for display that indicates how influential the particular telemetry metric is on the predictions comprises: providing a representation of a decision split in the gradient boosted tree-based prediction model that is contingent on the particular telemetry metric for display by the visualization tool (Cella ¶1577–1578: “Non-limiting examples of analysis modules 8808 may include risk analysis module(s), security analysis module(s), decision tree analysis module(s), ethics analysis module(s), failure mode and effects (FMEA) analysis module(s), hazard analysis module(s), quality analysis module(s), safety analysis module(s), regulatory analysis module(s), legal analysis module(s), and/or other suitable analysis modules. In some embodiments, the analysis management module 8806 is configured to determine which types of analyses to perform based on the type of decision that was requested by an intelligence service client 8836. In some of these embodiments, the analysis management module 8806 may include an index or other suitable mechanism that identifies a set of analysis modules 8808 based on a requested decision type”—[(emphasis added)]).
Regarding claim 6, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
wherein providing the visualization tool for display that indicates how influential the particular telemetry metric is on the predictions comprises: providing distributions of positive and negative samples of the particular telemetry metric for display by the visualization tool (Cella ¶0486–0487: “In some embodiments, the machine learning model 3000 may be configured to transform the input training data prior to performing one or more classifications and/or predictions of the input training data. Thus, the machine learning model 3000 may be configured to reconstruct input training data from one or more unknown data-generating distributions without necessarily conforming to implausible configurations of the input training data according to the distributions. In some embodiments, the feature learning algorithm may be performed by the machine learning model 3000 in a supervised, unsupervised, or semi-supervised manner … In some embodiments, the machine learning model 3000 may be defined via anomaly detection, i.e., by identifying rare and/or outlier instances of one or more items, events and/or observations. The rare and/or outlier instances may be identified by the instances differing significantly from patterns and/or properties of a majority of the training data. Unsupervised anomaly detection may include detecting of anomalies, by the machine learning model 3000, in an unlabeled training data set under an assumption that a majority of the training data is ‘normal.’ Supervised anomaly detection may include training on a data set wherein at least a portion of the training data has been labeled as ‘normal’ and/or ‘abnormal.’”—[wherein the system provides distributions of normal and abnormal (i.e., positive and negative) training data (i.e., telemetry metrics)]).
Regarding claim 7, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
providing a dimensionality reduction between the particular telemetry metric and another metric for display by the visualization tool (Cella ¶0752: “In embodiments, the adaptive intelligent systems layer 614 may, for each set of features, execute a simulation based on the set of features and may collect the simulation outcome data resulting from the simulation. For example, in executing a simulation involving the interactions of an intelligent product digital twin 1780 representing an intelligent product 650 and a customer digital twin 1730, the adaptive intelligent systems layer 614 can vary the dimensions of the intelligent product digital twin 1780 and can execute simulations that generate outcomes in a simulation management system 5704. In this example, an outcome can be an amount of time taken by a customer digital twin 5502 to complete a task using the intelligent product digital twin 1780. During the simulations, the adaptive intelligent systems layer 614 may vary the intelligent product digital twin 1780 display screen size, available capabilities (processing, speech recognition, voice recognition, touch interfaces, remote control, self-organization, self-healing, process automation, computation, artificial intelligence, data storage, and the like), materials, and/or any other properties of the intelligent product digital twin 1780. Simulation data 5710 may be created for each simulation and may include feature data used to perform the simulations, as well as outcome data. In the example described above, the simulation data 5710 may be the properties of the customer digital twin 5502 and the intelligent product digital twin 1780 that were used to perform the simulation and the outcomes resulting therefrom. In embodiments, a machine learning system 5720 may receive training data 5730, outcome data 5740, simulation data 5710, and/or data from other types of external data sources 5702 (weather data, stock market data, sports event data, news event data, and the like). In embodiments, this data may be provided to the machine-learning system 5720 via an API of the adaptive intelligent systems layer 614. The machine learning system 5720 may train, retrain, or reinforce machine leaning models 5750 using the received data (training data, outcome data, simulation data, and the like)”—[wherein varying the dimensions includes dimension reduction]).
Regarding claim 8, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
wherein the device trains the gradient boosted tree-based prediction model using quality of experience feedback provided by users of the online application (Cella ¶0114, ¶0263, ¶0464, ¶1032, ¶1671: “In embodiments, the artificial intelligence system makes use of an algorithm comprising an artificial neural network, a decision tree” and “In example embodiments, the presentation layer may include, but is not limited to, a user interface, and modules for investigation and discovery and tracking users' experience and engagements” and “The machine learning model 3000 builds one or more mathematical models based on training data to make predictions and/or decisions without being explicitly programmed to perform the specific tasks. The machine learning model 3000 may receive inputs of sensor data as training data, including event data 1034 and state data 1140 related to one or more of the value chain entities 652. The sensor data input to the machine learning model 3000 may be used to train the machine learning model 3000 to perform the analytics, simulation, decision making, and prediction making relating to the data processing, data analysis, simulation creation, and simulation analysis of the one or more of the value chain entities 652. The machine learning model 3000 may also use input data from a user or users of the information technology system. The machine learning model 3000 may include an artificial neural network, a decision tree” and “The executive agent may be trained to identify the COO's tendencies based on the COO's previous interaction with the COO digital twin” and “For example, one of the circuits may configure (e.g., using configuration parameters specified by an analytics library) a first AI-assisted technique to detect a particular condition (e.g., a gradient-boosted trees model), and then the same or another circuit may use a different AI-assisted technique (e.g., a neural network trained using deep learning techniques) to predict the response to a treatment plan for the particular condition”—[wherein the system is trained in part based on a previous interaction (i.e., quality of the previous interaction; e.g., users’ experience) including telemetry metrics]).
Regarding claim 9, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
controlling the visualization tool to highlight anomalous data points in the plurality of telemetry metrics, based on a parameter set by a user of the visualization tool (Cella ¶0334, ¶0726: “The adaptive intelligence systems 808 may provide further processing for automated control signal generation, such as by applying artificial intelligence to determine aspects of a value chain that impact automated control of the coordinated set of demand management applications and supply chain applications for a category of goods” and “Clustering processes 5342 may be implemented to identify the failure pattern hidden in the failure data to train a model for detecting uncharacteristic or anomalous behavior. The failure data across multiple machines and their historical records may be clustered to understand how different patterns correlate to certain wear-down behavior. Analytics processes 5344 perform data analytics on various data to identify insights and predict outcomes. Natural language processes 4348 coordinate with machine twin 1770 to communicate the outcomes and results to the user of machine twin 1770”).
Regarding claim 10, Cella in view of Guim teaches all the limitations of claim 1.
Cella teaches:
providing an indication of a loss function associated with the gradient boosted tree-based prediction model for display by the visualization tool (Cella ¶1540: “For example, a result of forward propagation (e.g., output activation value(s)) determined using training input data is compared against a corresponding known reference output data to calculate a loss function gradient. The gradient may be then utilized in an optimization method to determine new updated weights in an attempt to minimize a loss function”).
Regarding claim 11, Cella teaches:
an apparatus, comprising:
one or more network interfaces;
a processor coupled to the one or more network interfaces and configured to execute one or more processes; and
a memory configured to store a process that is executable by the processor, the process when executed configured to (Cella ¶¶2772-2791: “A special-purpose system includes hardware and/or software and may be described in terms of an apparatus, a method, or a computer-readable medium. In various embodiments, functionality may be apportioned differently between software and hardware. For example, some functionality may be implemented by hardware in one embodiment and by software in another embodiment. Further, software may be encoded by hardware structures, and hardware may be defined by software, such as in software-defined networking or software-defined radio … The networking hardware may include one or more interface circuits. In some examples, the interface circuit(s) may implement wired or wireless interfaces that connect, directly or indirectly, to one or more networks. Examples of networks include a cellular network, a local area network (LAN), a wireless personal area network (WPAN), a metropolitan area network (MAN), and/or a wide area network (WAN). The networks may include one or more of point-to-point and mesh technologies. Data transmitted or received by the networking components may traverse the same or different networks. Networks may be connected to each other over a WAN or point-to-point leased lines using technologies such as Multiprotocol Label Switching (MPLS) and virtual private networks (VPNs) … Examples of cellular networks include GSM, GPRS, 3G, 4G, 5G, LTE, and EVDO. The cellular network may be implemented using frequency division multiple access (FDMA) network or code division multiple access (CDMA) network. Examples of a LAN are Institute of Electrical and Electronics Engineers (IEEE) Standard 802.11-2020 (also known as the WIFI wireless networking standard) and IEEE Standard 802.3-2018 (also known as the ETHERNET wired networking standard). Examples of a WPAN include IEEE Standard 802.15.4, including the ZIGBEE standard from the ZigBee Alliance. Further examples of a WPAN include the BLUETOOTH wireless networking standard, including Core Specification versions 3.0, 4.0, 4.1, 4.2, 5.0, and 5.1 from the Bluetooth Special Interest Group (SIG). A WAN may also be referred to as a distributed communications system (DCS). One example of a WAN is the internet”).
Regarding the remaining limitation of independent claim 11, the remaining limitations of claim 11 are substantially the same as the limitations in claim 1. Thus, claim 11 is rejected using the same reasoning as provided above in the rejection for claim 1.
Regarding claims 14–19, although varying in scope, the limitations of claims 14–19 are substantially the same as the limitations of claims 4–9, respectively. Thus, claims 14–19 are rejected using the same reasoning and analysis as claims 4–9 above, respectively.
Regarding claim 20, Cella teaches:
a tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising (Cella ¶0028, ¶¶2772-2791: “The set of job requirements is stored in a non-transitory computer readable memory that is accessible by at least one processor of the set of processors”).
Regarding the remaining limitation of independent claim 20, the remaining limitations of claim 20 are substantially the same as the limitations in claim 1. Thus, claim 20 is rejected using the same reasoning as provided above in the rejection for claim 1.
Claims 2–3 and 12–13 are rejected under 35 U.S.C. 103 as being unpatentable over Cella in view of Guim and further in view of Carranza et al., (US 20230140252 A1), hereinafter “Carranza”.
Regarding claim 2, Cella in view of Guim teaches all the limitations of claim 1.
Cella in view of Guim does not appear to explicitly teach:
wherein the particular telemetry metric is a Layer-3 metric captured by the network.
However, Carranza teaches:
wherein the particular telemetry metric is a Layer-3 metric captured by the network (Carranza ¶0031: “From a system design perspective, this arrangement provides important functionality. The IPU 410 is seen as a discrete device from the local host (e.g., the OS running in the compute platform CPUs 424) that is available to provide certain functionalities (networking, acceleration etc.). Those functionalities are typically provided via Physical or Virtual PCIe functions. Additionally, the IPU 410 is seen as a host (with its own IP etc.) that can be accessed by the infrastructure to setup an OS, run services, and the like. The IPU 410 sees all the traffic going to the compute platform 420 and can perform actions—such as intercepting the data or performing some transformation—as long as the correct security credentials are hosted to decrypt the traffic. Traffic going through the IPU goes to all the layers of the Open Systems Interconnection model (OSI model) stack (e.g., from physical to application layer)”—[wherein a person having skill in the art before the filling of the instant application would have known that an OSI model includes layer-3 metric data (i.e., physical layer; e.g., network layer)]).
The methods of Cella in view of Guim, the teachings of Carranza, and the instant application are analogous art because they pertain to using machine learning and telemetry data to make predictions.
It would be obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the methods of Cella in view of Guim, the teachings of Carranza to provide telemetry data that includes specific layer data. One would be motivated to do so to arrange the model to increase important functionality regarding the prediction data (Carranza ¶0031).
Regarding claim 3, Cella in view of Guim teaches all the limitations of claim 1.
Cella in view of Guim does not appear to explicitly teach:
wherein the particular telemetry metric is a Layer-7 metric computed by the online application.
However, Carranza teaches:
wherein the particular telemetry metric is a Layer-7 metric computed by the online application (Carranza ¶0031: “From a system design perspective, this arrangement provides important functionality. The IPU 410 is seen as a discrete device from the local host (e.g., the OS running in the compute platform CPUs 424) that is available to provide certain functionalities (networking, acceleration etc.). Those functionalities are typically provided via Physical or Virtual PCIe functions. Additionally, the IPU 410 is seen as a host (with its own IP etc.) that can be accessed by the infrastructure to setup an OS, run services, and the like. The IPU 410 sees all the traffic going to the compute platform 420 and can perform actions—such as intercepting the data or performing some transformation—as long as the correct security credentials are hosted to decrypt the traffic. Traffic going through the IPU goes to all the layers of the Open Systems Interconnection model (OSI model) stack (e.g., from physical to application layer)”—[wherein a person having skill in the art before the filling of the instant application would have known that an OSI model includes layer-7 metric data (i.e., application layer)]).
The methods of Cella in view of Guim, the teachings of Carranza, and the instant application are analogous art because they pertain to using machine learning and telemetry data to make predictions.
It would be obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the methods of Cella in view of Guim, the teachings of Carranza to provide telemetry data that includes specific layer data. One would be motivated to do so to arrange the model to increase important functionality regarding the prediction data (Carranza ¶0031).
Prior Art of Record
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
John T. Pugaczewski., (“Machine learning for quality of experience optimization”) discloses predictions regarding quality of experience for users including machine learning and telemetry data “Novel tools and techniques for machine learning based quality of experience optimization are provided. A system includes network elements, an orchestrator, and a server. The server includes a processor and non-transitory computer readable media comprising instructions to obtain telemetry information from a first protocol layer, obtain telemetry information from a second protocol layer, modify one or more attributes of the second protocol layer, observe a state of first protocol layer performance, assign a cost associated with changes to each of the one or more attributes of the second protocol layer, and optimize the first protocol layer performance based, at least in part, on the state of first protocol layer performance and the cost associated with the changes to one or more attributes of the second protocol layer. The orchestrator modifies the one or more attributes of the second protocol layer.” Pugaczewski, Abstract.
Bouchard et al., (“Adaptive broadcast media and data services”) discloses machine learning for OSI implementation including application layer 7 telemetry data “The link layer may be the layer supporting traffic between the physical layer and a network layer (e.g., OSI layer 7) such that input packet types are abstracted into a single format for processing by the RF physical layer, ensuring flexibility, efficiency (e.g., via compression of redundancies in input packet headers), and future extensibility of input types.” Bouchard ¶0236.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS SHINE whose telephone number is (571)272-2512. The examiner can normally be reached M-F, 11a-7p ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached on (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.B.S./Examiner, Art Unit 2126
/VAN C MANG/Primary Examiner, Art Unit 2126