Notice of Pre-AIA or AIA Status
Claims 1-19 are currently presented for Examination.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendments
The amendment filed on 02/11/2026 has been entered and considered by the examiner. By the
amendment, claim 1-2, 4-15, 17 and 19 are amended. The previous claim interpretation and its corresponding 112 rejection is withdrawn in view of amendment made. The previous 101 and 103 is still maintained in view of amendment made and an explanation is given below. See office action for detail.
Response Applicant 101 arguments (page 11-13)
Applicant’s arguments have been fully considered but are not persuasive. The rejection of claims 1–19 under 35 U.S.C. §101 is therefore maintained for the reasons set forth below. As amended, the claims still recite subject matter falling within abstract idea groupings identified in the MPEP 2106, including mathematical concepts and mental processes. The claims recite operations such as:
generate a plurality of models respectively corresponding to different one of a plurality of periods, each model indicating a relationship between the plurality of time-series data within the corresponding one of the plurality of periods;
detect an abnormality of the monitoring target based on the plurality of models and the plurality of time-series data by
calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models
determining if the output value exceeds a predetermined value;
detecting an abnormality based on the comparison;
determining whether a model is used or not used; and
excluding selected models from a plurality of models.
These limitations amount to analyzing information, performing mathematical comparisons, and making evaluative decisions based on the results. Such operations can be practically performed in the human mind or with pencil-and-paper or are mathematical relationships/computations. A human observer could, in principle: Receive time-series data (e.g., observing a graph or a list of numbers). Develop or recall a plurality of "models" (e.g., a set of rules, expectations, or mathematical formulas) Compare the received data against these models to identify deviations or "abnormalities" (e.g., "that data point is outside the expected range"). Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Also, claim recites “calculating a difference” which also falls under the Mathematical concepts of abstract idea. Thus, the claim falls under the combination of mental process and Mathematical concepts of abstract idea. Applicant’s amendments adding “calculating a difference” and “determining if the output value exceeds a predetermined value” do not remove the claim from the abstract idea category, but instead further specify the mathematical analysis being performed.
Applicant argues that the amended claims are tied to a concrete monitoring context involving sensors and model exclusion. This argument has been considered but is not persuasive. The limitation reciting acquisition of time-series data indicating changes in output values of a plurality of sensors merely collecting data and falls under the insignificant extra solution activity as discussed in MPEP 2106.05(g) and is well-understood, routine or conventional. ((See MPEP 2106.05(d) i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) The claims do not recite any improvement to sensor structure, sensor operation, data acquisition hardware, or signal generation. The sensors are used as a source of input data. Reciting that the data comes from a “monitoring target including the plurality of sensors,” or from an exposure apparatus or manufacturing apparatus, merely limits the use of the abstract idea to a particular technological environment. Limiting an otherwise abstract concept to use with industrial equipment or manufacturing systems does not integrate the exception into a practical application. The additional element of an information processing apparatus comprising: a processor; and a memory, including instructions stored thereon, which when executed by the processor cause the information processing apparatus in claim 1 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) The recited exclusion of models “not used to detect an abnormality” constitutes additional evaluation and management of mathematical models. It does not impose a meaningful technological limit, nor does it recite a technological improvement to computer functionality or sensor systems. The present claim an improvement in abstract idea (i.e., detect abnormality) and it is important to keep in mind that an improvement in the abstract idea itself and is not an improvement in technology. (See MPEP 2106.05(a)(II)
Applicant argues that claim 19 integrates the recited detection/exclusion into an article manufacturing workflow including exposure, development, processing, and maintenance steps. This argument is not persuasive. The additional manufacturing steps are broadly recited and are extra-solution activity surrounding the recited abstract analysis. Claim 19 does not require that the mathematical/modeling operations improve the exposure apparatus itself, alter exposure parameters, or technologically control substrate processing in a specific manner. The maintenance step is result-oriented and does not recite a particular technical mechanism that transforms the nature of the claim. Accordingly, claim 19 remains directed to the same judicial exception as claim 1. When considered individually and as an ordered combination, the additional claim elements amount to no more than generic implementation of the abstract idea using conventional components and functions, The ordered combination performs expected and conventional functions of receiving data, analyzing data, and acting on the results. No element or combination amounts to significantly more than the judicial exception. Thus, the 101 rejection is still maintained for claim 1-19.
Applicant 103 arguments
Independent claim recites calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models, and determining if the output value exceeds a predetermined value;
Oliner, Yun, Nakaya, Shizuka, Kuroda, Maeda, and Shiode fail to teach or suggest the above-noted features of amended independent claim 1.
Examiner response
Applicant’s arguments improperly isolate paragraph of Oliner [0271-0272] and [0312-0313] while ignoring Oliner as a whole. Although that paragraph discusses model evaluation/selection. Oliner expressly teaches detecting anomalies based on deviations between predicted and actual values. For further support the applicant arguments and the recent claim amendment See Oliner para 235-236 and see para 289, 292-293.
Oliner still teaches by calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models, (see para 271-Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 227-A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include …sensor data from real-time monitors (supply chains, military operation networks, or security systems. See para 243-244-Data source 1306 may be at least one source of incoming source data 1310 being fed into the data processing system 1302. Data source 1306 can be or include one or more external data sources, such as mobile devices, sensors. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like)
and determining if the output value exceeds a predetermined value; (see para 271-272- Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. In some implementations, the error corresponds to a residual between a predicted value and an actual value of the time series. For example, as shown, the predictive model generates predicted time series 1432 based on values of time series 1420 and time series 1422 within predicting period 1442. Although not shown, predicted time series 1432 may be generated within training period 1440 and could be used for training. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 289-In one approach, the anomaly detection tool determines the median absolute deviation (MAD) from the predicted and actual values over a moving window of time. An anomaly may be detected based on the MAD exceeding a threshold value (e.g., falling outside of a range of values). See para 292-293-There are many potential approaches for determining whether a residual(s) is sufficiently large to consider it an anomaly. Further, a user could provide an input parameter or configuration file setting various anomaly detection parameters, such as detection thresholds. The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device).)
Therefore, Oliner still teaches the claimed abnormality detection limitation.
Claim Rejections - 35 USC §101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. These claims are directed to an abstract idea without significantly more.
(Step 1) Is the claims to a process, machine, manufacture, or composition of matter?
Claims: 1-16 is directed to system or machine that falls on one of statutory category.
Claims: 17, 19 are directed to method or process that falls on one of statutory category.
Claim 18 is directed to a non-transitory computer readable medium, which falls under the manufacture of statutory category.
Claim 1, 17 and 18 recites
Step 2A, Prong 1
generate a plurality of models respectively corresponding to different one of a plurality of periods, each model indicating a relationship between the plurality of time-series data within the corresponding one of the plurality of periods; (The limitation describes the core concept of identifying relationships within data and creating models based on those relationships across different time periods. This is an analytical or mathematical task, which are often classified as abstract ideas. A person could, in principle, analyze time-series data, identify patterns or relationships, and formulate a rule or model for each period. Thus, it falls under the combination of mental process and mathematical concepts of abstract idea)
detect an abnormality of the monitoring target based on the plurality of models and the plurality of time-series data by calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models and determining if the output value exceeds a predetermined value; (A human observer could, in principle: Receive time-series data (e.g., observing a graph or a list of numbers). Develop or recall a plurality of "models" (e.g., a set of rules, expectations, or mathematical formulas) Compare the received data against these models to identify deviations or "abnormalities" (e.g., "that data point is outside the expected range"). Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Also, claim recites “calculating a difference” which also falls under the Mathematical concepts of abstract idea. Thus, the claim falls under the combination of mental process and Mathematical concepts of abstract idea)
exclude, from the plurality of models, a model that is not used to detect an abnormality of the monitoring target. (A person presented with a plurality of models and, using their own judgment, evaluate which ones are suitable for a specific task (abnormality detection) and mentally exclude the others. This involves observation, evaluation, and opinion formation, which are mental processes).
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception?
In accordance with Step 2A, Prong 2, the judicial exception is not integrated into a practical application. In particular, claim 1 and 17 recites the additional elements of acquire a plurality of time-series data indicating changes in output values of a plurality of sensors from a monitoring target including the plurality of sensors viewed as merely collecting data and falls under the insignificant extra solution activity as discussed in MPEP 2106.05(g). The additional element of an information processing apparatus comprising: a processor; and a memory, including instructions stored thereon, which when executed by the processor cause the information processing apparatus in claim 1 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) The additional element of a substrate processing apparatus in claim 17 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) The additional element of a non-transitory computer readable medium storing a program causing a computer to execute a monitoring method defined in claim 17 as described by claim 18 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) Thus, claim 1, 17 and 18 are directed to abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, claim 1 and 17 recites the additional elements of acquire a plurality of time-series data indicating changes in output values of a plurality of sensors from a monitoring target including the plurality of sensors viewed as merely collecting data and falls under the insignificant extra solution activity as discussed in MPEP 2106.05(g) and is well-understood, routine or conventional. ((See MPEP 2106.05(d) i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) The additional element of an information processing apparatus comprising: a processor; and a memory, including instructions stored thereon, which when executed by the processor cause the information processing apparatus in claim 1 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) The additional element of a substrate processing apparatus in claim 17 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) The additional element of a non-transitory computer readable medium storing a program causing a computer to execute a monitoring method defined in claim 17 as described by claim 18 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) Therefore, claims 1, 17 and 18 are directed to abstract idea and is not patent eligible.
Regarding claim 19
Step 2A, Prong 1
An article manufacturing method comprising:
a model generation step of generating a plurality of models respectively corresponding to a plurality of periods, each model indicating a relationship between the plurality of time-series data within the corresponding one of the plurality of periods; (The limitation describes the core concept of identifying relationships within data and creating models based on those relationships across different time periods. This is an analytical or mathematical task, which are often classified as abstract ideas. A person could, in principle, analyze time-series data, identify patterns or relationships, and formulate a rule or model for each period. Thus, it falls under the combination of mental process and mathematical concepts of abstract idea)
a detection step of detecting a state of the exposure apparatus based on the plurality of models and the plurality of time-series data by calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models, and determining if the output value exceeds a predetermined value; (A human observer could, in principle: Receive time-series data (e.g., observing a graph or a list of numbers). Develop or recall a plurality of "models" (e.g., a set of rules, expectations, or mathematical formulas) Compare the received data against these models to identify deviations or "abnormalities" (e.g., "that data point is outside the expected range"). Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Also, claim recites “calculating a difference” which also falls under the Mathematical concepts of abstract idea. Thus, the claim falls under the combination of mental process and Mathematical concepts of abstract idea)
a maintenance step of maintaining the exposure apparatus based on the state of the exposure apparatus detected in the detection step; (It is an abstract idea, specifically a mental process that can be performed by a human operator (e.g., an operator looks at the apparatus, sees it is in a certain state, and performs maintenance. The core idea of "detecting a state" and then "maintaining based on that state" (i.e., adjusting or fixing something based on observed conditions) can be a process performed by a human mind, possibly with the aid of pen and paper, prior to any specific technological implementation. A person can observe a machine, decide it needs maintenance, and perform that maintenance.) and
an exclusion step of excluding, from the plurality of models, a model that is not used to detect an abnormality of the exposure apparatus. (A person presented with a plurality of models and, using their own judgment, evaluate which ones are suitable for a specific task (abnormality detection) and mentally exclude the others. This involves observation, evaluation, and opinion formation, which are mental processes).
Step 2A, Prong 2: Does the claim recite additional elements that integrate the judicial exception?
In accordance with Step 2A, Prong 2, the judicial exception is not integrated into a practical application. In particular, claim 19 recites the additional elements of an acquisition step of acquiring a plurality of time-series data indicating changes in output values of the plurality of sensors from the exposure apparatus viewed as merely collecting data and falls under the insignificant extra solution activity as discussed in MPEP 2106.05(g). The additional element of an exposure step of exposing a substrate by using an exposure apparatus including a plurality of sensors; a development step of developing the substrate exposed in the exposure step; a processing step of obtaining an article by processing the substrate developed in the development step in claim 19 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) Thus, claim 19 is directed to abstract idea.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, claim 19 recites the additional elements of an acquisition step of acquiring a plurality of time-series data indicating changes in output values of the plurality of sensors from the exposure apparatus viewed as merely collecting data and falls under the insignificant extra solution activity as discussed in MPEP 2106.05(g) and is well-understood, routine or conventional. ((See MPEP 2106.05(d) i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); but see DDR Holdings, LLC v. Hotels.com, L.P., 773 F.3d 1245, 1258, 113 USPQ2d 1097, 1106 (Fed. Cir. 2014) The additional element of an exposure step of exposing a substrate by using an exposure apparatus including a plurality of sensors; a development step of developing the substrate exposed in the exposure step; a processing step of obtaining an article by processing the substrate developed in the development step in claim 19 are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f) The terms "exposure apparatus," "plurality of sensors," "development step," and "processing step" are very broad and describe standard, well-known components and steps in a typical semiconductor or lithographic manufacturing process. (see fig 1-2 US20120101622A1, US8947630B2T (see fig 1)) they lack specific structural or functional details that would distinguish them from a vast range of existing technologies. Thus, claim 19 is directed to abstract idea.
Claim 2 recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: acquire error information indicating occurrence of an error from the monitoring target, and determine, based on the error information, the plurality of periods during a period in which no error is occurring. The act of "acquiring error information" could be interpreted as a human observing a system and noting when an error occurs, a task considered a mental evaluation or observation. A person could, in their mind or using a simple recording method (like a checklist or a spreadsheet), observe a monitoring target and mark down the times when no errors were occurring. This indicates it falls under the "mental process" grouping of abstract ideas. Determining is a mental evaluation. Merely stating that this process is done "the processor" using "generic computer components" typically does not transform an abstract idea into patent-eligible subject matter. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 3 recites wherein the plurality of periods include two periods partially overlapping each other. A human can perform the process of comparing time periods and identifying overlap in their mind. Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 4 recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: determines a model to be excluded based on an elapsed time from generation of a model. Determining an elapsed time (e.g., "more than 24 hours have passed") and making a simple decision based on that time (e.g., "exclude the model") is a fundamental human mental activity. Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 5 recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded based on a change in output value with time of each of the plurality of sensors. A person could theoretically perform: observe the sensor readings over time, note the changes, and mentally exclude models that do not fit a certain profile. This is a key indicator of a "mental process" abstract idea. Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 6 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: calculate a difference between an output value of each of the plurality of sensors and an output value generated by a model. The operation of comparing two values and calculating their difference is something that can be performed entirely in the human mind, possibly with the aid of a pen and paper or basic mental arithmetic. Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 7 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: detect occurrence of an abnormality in the monitoring target if a value obtained by processing the difference exceeds a predetermined value. The core logic of the claim limitation involves: obtaining a value (calculation/processing), comparing that value to a threshold ("predetermined value") and detecting an abnormality based on the comparison result ("exceeds a predetermined value"). These steps can be conceptualized and performed mentally, making them susceptible to being categorized as an abstract idea. Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an evaluation or judgment that could be performed in the human mind or with the aid of pencil and paper. If a claim, under its broadest reasonable interpretation, covers a mental process, then it falls within the “Mental Process” grouping of abstract ideas. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 8 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine, as a model to be excluded, a model for which a frequency of generation of an output value at which the occurrence of an abnormality is detected exceeds a predetermined frequency. The process of monitoring a frequency and comparing it to a threshold to make an exclusion decision is a process that can be performed by a human mentally, perhaps with the aid of a pen and paper or a basic calculator, given the necessary data. This "mental process" characteristic is a strong indicator of an abstract idea. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 9 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded from at least three models respectively corresponding to at least three periods arranged in order of time elapsed among the plurality of periods, based on a frequency at which the occurrence of an abnormality is detected by using the at least three models. The process of determining which model to exclude based on the frequency of abnormalities is a decision-making process that a human could perform, at least in principle, using pen and paper or in their mind. The "processor" as a generic computer component performing a task that does not inherently require a specific, non-generic technological solution. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 10 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine, based on a threshold value determined based on a value obtained by processing the difference calculated by using each of already existing models, whether to exclude a model to be generated thereafter. The determination step is a type of decision-making process that could plausibly be performed by a human mind. The process of comparing values to a threshold and making a judgment is a example of a "mental process" grouping of abstract ideas. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 11 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: calculate an evaluation value while performing weighting on the plurality of differences respectively calculated by using the plurality of models, and detects a state of the monitoring target based on the evaluation value. The steps of "calculating an evaluation value," "performing weighting," and "detecting a state" based on those calculations are operations that can be performed entirely in the human mind using pen and paper. These concepts involve mathematical methods and decision-making processes, which are considered foundational tools of human thought. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 12 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: perform the weighting based on an elapsed time from generation of a model. A human can mentally, or using simple tools (like a calendar and a calculator/paper), track the elapsed time since a model was generated and apply a predetermined weighting factor. For example, a person managing data could use a rule like "models older than one week get half weight" and apply this manually. This ability to perform the core logic mentally or with pen-and-paper is a key indicator of an abstract idea. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 13 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: decrease the weight as the elapsed time of a model prolongs. The core idea of "decreasing a weight as time prolongs" can be characterized as a basic human judgment, observation, or evaluation that a person could perform mentally, perhaps with a pencil and paper (e.g., "the longer this goes on, the less important this factor is"). The specific values for the weight and time are simply variables in this mental framework. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 14 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: decrease the weight as the elapsed time of a model shortens. The determination step is a type of decision-making process that could plausibly be performed by a human mind. A person can be instructed to "decrease the weight" (e.g., a mental or physical factor used in an evaluation) as "time shortens" (or as a deadline approaches). This is a general instruction or a form of human judgment/evaluation that can be performed mentally, for example, deciding to give less consideration to an older piece of data. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 15 further recites wherein the instructions when executed by the processor, further cause the information processing apparatus to: notify about the detection of an abnormality of the monitoring target. The fundamental concept of "detecting an abnormality" and "notifying" could, in isolation, be considered a mental process (e.g., a human observing something and deciding it's abnormal) or an abstract idea (the general concept of flagging unusual events). Merely stating that this process is done "by the processor" is using generic computer components that typically does not transform an abstract idea into patent-eligible subject matter. Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim 16 further recites substrate processing apparatus including a plurality of sensors; and an information processing apparatus defined in claim 1 and configured to detect a state of the monitoring target are amounts to no more than mere instructions to apply the exception using generic computer components. (MPEP 2106.05(f). Claim therefore, when taken as a whole, still does not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception. Claim recites unpatentable ineligible subject matter for the same reasoning and analysis as mentioned for claim 1.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
4. Claim(s) 1, 2, 5-7, 10 and 15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Oliner et al. (PUB NO: US20180219889A1)
Regarding claim 1
Oliner teaches an information processing apparatus comprising: a processor; and a memory, including instructions stored thereon, which when executed by the processor cause the information processing apparatus to (see fig 13 and para 0227-0313) comprising:
acquire a plurality of time-series data indicating changes in output values of a plurality of sensors from a monitoring target including the plurality of sensors; (see para 226-227 and fig 12-13-Data may be collected as a time series data set, that is, a sequence of data points of a time series, often including successive measurements made over a time interval. A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include information processing logs, market transactions, and sensor data from real-time monitors (supply chains, military operation networks, or security systems). see para 243-246- Data source 1306 can be or include one or more external data sources, such as web servers, application servers, databases, firewalls, routers, operating systems, and software applications that execute on computer systems, mobile devices, sensors, and/or the like. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like. Indexer 1312 of data processing system 1302, when present, may receive source data 1310, for example, from a forwarder (not shown in FIG. 13) or data source 1306, and apportion source data 1310 into events.)
generate a plurality of models respectively corresponding to different one of a plurality of periods, each model indicating a relationship between the plurality of time-series data within the corresponding one of the plurality of periods; (see para 235-237- In some aspects of the present disclosure, relationships between time series data sets are automatically determined by analyzing values associated with the time series data sets. Each relationship is captured by a predictive model, which is configured to generate predicted values associated with at least one time series using values from at least one other time series. In some implementations, in order to search for and select a predictive model to use for anomaly detection on values associated with a time series (e.g., at least one time series), a plurality of predictive models is generated for the time series. Each predictive model can be configured to independently predict the values associated with the time series using values associated with at least one other time series. For example, one predictive model may predict values of time series data set 1202A based on values from time series data set 1202B (e.g., they may be explanatory variables), another predictive model may be similar, but use a different predictor function, and another may predict values of time series data set 1202A based on values from both time series data sets 1202B and 1202C. A predictive model is selected from the generated models for the anomaly detection based on an evaluation of the predicted values produced by the model. See para 269-270- As indicated in FIG. 14B, in some implementations, predictive models are trained over a training period, such as training period 1440. As indicated in FIG. 14B, in some implementations, predictive models are evaluated over a predicting period, such as predicting period 1442. As shown, the time series values used for predicting periods may be different values than those used for training and may follow the values used for training in the sequences of time series values. see para 279-280- For example, a subsequent training period 1440 and predicting period 1442 may be used for each iteration. Continued training of predictive models allow the predictive models to adapt to changes in the behavior of the data.)
detect an abnormality of the monitoring target based on the plurality of models and the plurality of time-series data, (see para 288-291-The predictive model(s) for each time series can be used to detect anomalies in values associated with the time series. Many approaches are available for anomaly detection using predicted values from the predictive models and actual values associated with the multiple time series. Some approaches to anomaly detection are based on determining and comparing characteristics of values of the time series being predicted, the predicted values of the time series, and/or the time series upon which the predicted value are based. As one example, an anomaly may be detected, at least in part, based on determining turbulence of the values in the various time series. For example, anomaly detection tool 1316 could determine a turbulence of values in the predicted time series, which is compared to the actual turbulence of values in the time series. As indicated above, any number of predictive models may be selected for anomaly detection (e.g., selected model 1416).)
by calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models, (see para 271-Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 227-A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include …sensor data from real-time monitors (supply chains, military operation networks, or security systems. See para 243-244-Data source 1306 may be at least one source of incoming source data 1310 being fed into the data processing system 1302. Data source 1306 can be or include one or more external data sources, such as mobile devices, sensors. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like)
and determining if the output value exceeds a predetermined value; (see para 271-272- Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. In some implementations, the error corresponds to a residual between a predicted value and an actual value of the time series. For example, as shown, the predictive model generates predicted time series 1432 based on values of time series 1420 and time series 1422 within predicting period 1442. Although not shown, predicted time series 1432 may be generated within training period 1440 and could be used for training. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 289-In one approach, the anomaly detection tool determines the median absolute deviation (MAD) from the predicted and actual values over a moving window of time. An anomaly may be detected based on the MAD exceeding a threshold value (e.g., falling outside of a range of values). See para 292-293-There are many potential approaches for determining whether a residual(s) is sufficiently large to consider it an anomaly. Further, a user could provide an input parameter or configuration file setting various anomaly detection parameters, such as detection thresholds. The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device).)
exclude, from the plurality of models, a model that is not used to detect an abnormality of the monitoring target. (See para 312-313 and fig 17- At block 1706, the trained set of predictive models is evaluated. For example, anomaly detection tool 1316 may evaluate predicted values generated using predictive models 1412 with respect to actual values of time series 1430. This may include determining at least one residual for each predictive model, such as residual time series 1434. At block 1708, at least one predictive model is removed from the set. For example, anomaly detection tool 1316 may remove, or filter out at least one of predictive models 1410 based on the evaluation. Various examples of criteria for narrowing the set of predictive models have been described above. Optionally, block 1704 and/or block 1706 may be repeated with the reduced set of predictive models (e.g., predictive models 1412 followed by predictive models 1414) unless an ending condition is satisfied. Also see para 0278-0279)
Regarding claim 2
Oliner further teaches The information processing apparatus according to claim 1 wherein the instructions when executed by the processor, further cause the information processing apparatus to: acquire error information indicating occurrence of an error from the monitoring target, and determine, based on the error information, the plurality of periods during a period in which no error is occurring. (see para 267- In some implementations, anomaly detection tool 1316 analyzes the times series' and engineer's features for the predictive models based on the analysis. For example, anomaly detection tool 1316 may analyze the time series to determine whether one or more particular features should be included in the predictive models. Based one the determination, the predictive models may be generated using the features, examples of which have been described above. Doing so can reduce processing needed for generating and selecting predictive models by including features likely to be relevant to the time series while excluding other features which are unlikely to be relevant. See also para 260)
Regarding claim 5
Oliner further teaches the information processing apparatus according to claim 1 wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded based on a change in output value with time of each of the plurality of sensors. (see para 271-272- Residual time series 1434 is determined from actual values of time series 1430 and corresponding predicted values of predicted time series 1432 within predicting period 1442. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. As mentioned above, the predictive model may be selected for anomaly detection based on the evaluation. This selection can be based, in part on the determined error (e.g., residual or one or more values derived from the residual) of the model. see para 312-313 and fig 17- At block 1706, the trained set of predictive models is evaluated. For example, anomaly detection tool 1316 may evaluate predicted values generated using predictive models 1412 with respect to actual values of time series 1430. This may include determining at least one residual for each predictive model, such as residual time series 1434. At block 1708, at least one predictive model is removed from the set. For example, anomaly detection tool 1316 may remove, or filter out at least one of predictive models 1410 based on the evaluation. Various examples of criteria for narrowing the set of predictive models have been described above. Optionally, block 1704 and/or block 1706 may be repeated with the reduced set of predictive models (e.g., predictive models 1412 followed by predictive models 1414) unless an ending condition is satisfied. Also see para 0278-0279)
Regarding claim 6
Oliner further teaches the information processing apparatus according to claim 5 wherein the instructions when executed by the processor, further cause the information processing apparatus to: calculate a difference between an output value of each of the plurality of sensors and an output value generated by a model. (see para 271-Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 227-A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include …sensor data from real-time monitors (supply chains, military operation networks, or security systems. See para 243-244-Data source 1306 may be at least one source of incoming source data 1310 being fed into the data processing system 1302. Data source 1306 can be or include one or more external data sources, such as mobile devices, sensors. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like)
Regarding claim 7
Oliner further teaches the information processing apparatus according to claim 6 wherein the instructions when executed by the processor, further cause the information processing apparatus to: detect occurrence of an abnormality in the monitoring target if a value obtained by processing the difference exceeds a predetermined value. (see para 271-272- Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. In some implementations, the error corresponds to a residual between a predicted value and an actual value of the time series. For example, as shown, the predictive model generates predicted time series 1432 based on values of time series 1420 and time series 1422 within predicting period 1442. Although not shown, predicted time series 1432 may be generated within training period 1440 and could be used for training. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 289-In one approach, the anomaly detection tool determines the median absolute deviation (MAD) from the predicted and actual values over a moving window of time. An anomaly may be detected based on the MAD exceeding a threshold value (e.g., falling outside of a range of values). See para 292-293-There are many potential approaches for determining whether a residual(s) is sufficiently large to consider it an anomaly. Further, a user could provide an input parameter or configuration file setting various anomaly detection parameters, such as detection thresholds. The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device).)
Regarding claim 10
Oliner further teaches the information processing apparatus according to claim 6 wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine, based on a threshold value determined based on a value obtained by processing the difference calculated by using each of already existing models, whether to exclude a model to be generated thereafter. (see para 293-The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device). For example, the anomaly detection tool can cause the indication to be automatically transmitted in response to detecting one or more anomalies (e.g., as an alert). The indication may take a variety of forms and many indications may be caused to be transmitted for a particular anomaly or particular anomalies. For example, an indication may be transmitted as part of an email, push notification, phone message, or display screen. As another example, an indication could be stored, retained, and/or analyzed to trigger actions such as retaining and/or generating of predictive models. See also para [0271] [0278-0279] and [0312-0313)
Regarding claim 15
Oliner further teaches the information processing apparatus according to claim 1 wherein the instructions when executed by the processor, further cause the information processing apparatus to: notify about the detection of an abnormality of the monitoring target. (see para 293- The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device). For example, the anomaly detection tool can cause the indication to be automatically transmitted in response to detecting one or more anomalies (e.g., as an alert). The indication may take a variety of forms and many indications may be caused to be transmitted for a particular anomaly or particular anomalies. For example, an indication may be transmitted as part of an email, push notification, phone message, or display screen. As another example, an indication could be stored, retained, and/or analyzed to trigger actions such as retaining and/or generating of predictive models.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
5. Claims 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Oliner et al. (PUB NO: US20180219889A1) in view of Yun et al., (PUB NO: US20120101622A1).
Regarding claim 17
Oliner teaches a monitoring method (see fig 13 and para 0227-0313) comprising:
an acquisition step of acquiring a plurality of time-series data indicating changes in output values of a plurality of sensors from a ; (see para 226-227 and fig 12-13-Data may be collected as a time series data set, that is, a sequence of data points of a time series, often including successive measurements made over a time interval. A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include information processing logs, market transactions, and sensor data from real-time monitors (supply chains, military operation networks, or security systems). see para 243-246- Data source 1306 can be or include one or more external data sources, such as web servers, application servers, databases, firewalls, routers, operating systems, and software applications that execute on computer systems, mobile devices, sensors, and/or the like. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like. Indexer 1312 of data processing system 1302, when present, may receive source data 1310, for example, from a forwarder (not shown in FIG. 13) or data source 1306, and apportion source data 1310 into events.)
a model generation step of generating a plurality of models respectively corresponding to different one of a plurality of periods, each model indicating a relationship between the plurality of time-series data within the corresponding one of the plurality of periods in the plurality of time-series data, thereby generating a plurality of models respectfully corresponding to the plurality of periods; (see para 235-237- In some aspects of the present disclosure, relationships between time series data sets are automatically determined by analyzing values associated with the time series data sets. Each relationship is captured by a predictive model, which is configured to generate predicted values associated with at least one time series using values from at least one other time series. In some implementations, in order to search for and select a predictive model to use for anomaly detection on values associated with a time series (e.g., at least one time series), a plurality of predictive models is generated for the time series. Each predictive model can be configured to independently predict the values associated with the time series using values associated with at least one other time series. For example, one predictive model may predict values of time series data set 1202A based on values from time series data set 1202B (e.g., they may be explanatory variables), another predictive model may be similar, but use a different predictor function, and another may predict values of time series data set 1202A based on values from both time series data sets 1202B and 1202C. A predictive model is selected from the generated models for the anomaly detection based on an evaluation of the predicted values produced by the model. See para 269-270- As indicated in FIG. 14B, in some implementations, predictive models are trained over a training period, such as training period 1440. As indicated in FIG. 14B, in some implementations, predictive models are evaluated over a predicting period, such as predicting period 1442. As shown, the time series values used for predicting periods may be different values than those used for training and may follow the values used for training in the sequences of time series values. see para 279-280- For example, a subsequent training period 1440 and predicting period 1442 may be used for each iteration. Continued training of predictive models allow the predictive models to adapt to changes in the behavior of the data.)
a detecting step of detecting an abnormality of the (see para 288-291-The predictive model(s) for each time series can be used to detect anomalies in values associated with the time series. Many approaches are available for anomaly detection using predicted values from the predictive models and actual values associated with the multiple time series. Some approaches to anomaly detection are based on determining and comparing characteristics of values of the time series being predicted, the predicted values of the time series, and/or the time series upon which the predicted value are based. As one example, an anomaly may be detected, at least in part, based on determining turbulence of the values in the various time series. For example, anomaly detection tool 1316 could determine a turbulence of values in the predicted time series, which is compared to the actual turbulence of values in the time series. As indicated above, any number of predictive models may be selected for anomaly detection (e.g., selected model 1416).)
by calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models, (see para 271-Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 227-A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include …sensor data from real-time monitors (supply chains, military operation networks, or security systems. See para 243-244-Data source 1306 may be at least one source of incoming source data 1310 being fed into the data processing system 1302. Data source 1306 can be or include one or more external data sources, such as mobile devices, sensors. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like)
and determining if the output value exceeds a predetermined value; (see para 271-272- Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. In some implementations, the error corresponds to a residual between a predicted value and an actual value of the time series. For example, as shown, the predictive model generates predicted time series 1432 based on values of time series 1420 and time series 1422 within predicting period 1442. Although not shown, predicted time series 1432 may be generated within training period 1440 and could be used for training. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 289-In one approach, the anomaly detection tool determines the median absolute deviation (MAD) from the predicted and actual values over a moving window of time. An anomaly may be detected based on the MAD exceeding a threshold value (e.g., falling outside of a range of values). See para 292-293-There are many potential approaches for determining whether a residual(s) is sufficiently large to consider it an anomaly. Further, a user could provide an input parameter or configuration file setting various anomaly detection parameters, such as detection thresholds. The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device).)
an exclusion step of excluding, from the plurality of models, a model that is not used to detect an abnormality of the (See para 312-313 and fig 17- At block 1706, the trained set of predictive models is evaluated. For example, anomaly detection tool 1316 may evaluate predicted values generated using predictive models 1412 with respect to actual values of time series 1430. This may include determining at least one residual for each predictive model, such as residual time series 1434. At block 1708, at least one predictive model is removed from the set. For example, anomaly detection tool 1316 may remove, or filter out at least one of predictive models 1410 based on the evaluation. Various examples of criteria for narrowing the set of predictive models have been described above. Optionally, block 1704 and/or block 1706 may be repeated with the reduced set of predictive models (e.g., predictive models 1412 followed by predictive models 1414) unless an ending condition is satisfied. Also see para 0278-0279)
However, Oliner does not explicitly teach substrate processing apparatus.
In the related field of invention, Yun teaches substrate processing apparatus.(see fig 1-2 and para 21-22- In a production setting, the timely detection of the fault condition depicted in FIG. 1 as well as the timely and accurate classification of the fault as one associated with a worn top cover ring would be highly desirable to prevent damage to subsequently processed substrates and/or damage to other components of the plasma processing system and to recover the system quickly after repair/maintenance. FIG. 2 shows, in accordance with an embodiment of the invention, a logic block diagram of various subcomponents of a plasma processing chamber 200 that is capable of automatic and timely detection of fault conditions as well as automatic and timely classification of faults)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include substrate processing apparatus as taught by Yun in the system of Oliner for producing semiconductor devices, for example, it is highly desirable that the plasma processing system produces electronic devices with the highest yield and with the lowest cost of ownership possible. To achieve a high yield and to reduce tool down time, which contributes to a higher cost of ownership, it is critical to detect and classify faults rapidly in order to minimize damage to wafers and/or to the plasma processing system components. Another motivation for automatically detecting fault conditions and for classifying fault conditions in an automatic and timely manner. (See para [0003-0006], Yun)
Regarding claim 16
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach a manufacturing apparatus comprising: a substrate processing apparatus including a plurality of sensors; and an information processing apparatus defined in claim 1 and configured to detect a state of the monitoring target.
In the related field of invention, Yun teaches a manufacturing apparatus comprising: a substrate processing apparatus including a plurality of sensors; and an information processing apparatus defined in claim 1 and configured to detect a state of the monitoring target. (See [0022] - [0032], [0056], [0057], claims 1, 5-7 and figures 1-3, 9)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include a manufacturing apparatus comprising: a substrate processing apparatus including a plurality of sensors; and an information processing apparatus defined in claim 1 and configured to detect a state of the monitoring target as taught by Yun in the system of Oliner for producing semiconductor devices, for example, it is highly desirable that the plasma processing system produces electronic devices with the highest yield and with the lowest cost of ownership possible. To achieve a high yield and to reduce tool down time, which contributes to a higher cost of ownership, it is critical to detect and classify faults rapidly in order to minimize damage to wafers and/or to the plasma processing system components. Another motivation for automatically detecting fault conditions and for classifying fault conditions in an automatic and timely manner. (See para [0003-0006], Yun)
Regarding claim 18
Oliner further teaches a non-transitory computer readable medium storing a program causing a computer to execute a monitoring method defined in claim 17. (See fig 13 and para 248- Data store 1314 (e.g., data store 208) may include a medium for the storage of data thereon. For example, data store 1314 may include non-transitory computer-readable medium storing data thereon that is accessible by entities of data processing environment 1300, such as indexer 1312 and anomaly detection tool 1316.)
6. Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Oliner et al. (PUB NO: US20180219889A1) in view of SHIZUKA et al., (PUB NO: JP2019159786A).
Regarding claim 3
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the plurality of periods include two periods partially overlapping each other.
In the related field of invention, SHIZUKA teaches wherein the plurality of periods include two periods partially overlapping each other. (see para 15- The score calculation unit 12 (period setting means, score calculation means) sets a period W having a predetermined time width as shown in FIG. 2 in the time-series data described above (step S2 in FIG. 10). At this time, a plurality of periods W are set as the period W, and the plurality of periods W include a period W having a specific time width at a constant time interval as indicated by reference numerals W1 to W3 in FIG. It is formed by moving along the time series. For example, the time width of the period W is 5 seconds, and the time interval between the adjacent periods W is set to 1 second. In this way, by setting a plurality of periods W, each period W is set so that a part of the time period overlaps with another period W adjacent to each other along the time series. In particular, in the present embodiment, three or more periods W (for example, W1 to W3) continuous along the time series are set to overlap in some time zones within each period.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include wherein the plurality of periods include two periods partially overlapping each other as taught by SHIZUKA in the system of Oliner in order to detect anomalies in a monitored system such as a plant or equipment, it is necessary to model the monitored system in advance using multidimensional time-series data output by the monitored system. (See para [0002], SHIZUKA)
7. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Oliner et al. (PUB NO: US20180219889A1) in view of Nakaya et al., (PUB NO: WO2014141682A1).
Regarding claim 4
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded based on an elapsed time from generation of a model.
In the related field of invention, Nakaya teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded based on an elapsed time from generation of a model. (See para [0053-0057]- First, access time which is another characteristic value given to each model or feature vector is defined. Numbers are assigned to each feature vector in order from the oldest acquisition time. As the acquisition time is older, a larger number is attached, and this number is taken as the access time of each feature vector. Then, the access time of the model is defined by the latest access time of the merged feature vectors. One example of unnecessity is this access time. The fact that the access time of the model is large (old) means that the signal pattern matching the model has not appeared for a long time. Therefore, in this case, deletion of the maximum unnecessary model in step 188 means discarding the model corresponding to the signal pattern which has not appeared for the longest. Also, during a certain period from the present to the past, unnecessary degree may be defined by the number of feature vectors merged into the model. In this case, it is defined that unnecessary degree becomes larger as the number of merged feature vectors is smaller. For example, the reciprocal of the number of merged feature vectors can be used for that purpose. In yet another example, the product of the unnecessary degrees of the above two examples may be used as the degree of unnecessity of each model. In yet another example, the access time / weight may be the unnecessity of each model. In this way, in step 188, a model corresponding to a signal pattern that does not appear for a long time or has a low appearance frequency is set as a model with high degree of unnecessity, and a model with the highest degree of unnecessity is deleted. This returns the number of all models to M pieces.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to determine a model to be excluded based on an elapsed time from generation of a model as taught by Nakaya in the system of Oliner in order to reduce the amount of data acquired by sensing while accurately sensing a signal including pulses that occur quasi-periodically, such as the R wave of an electrocardiogram, thereby reducing the power consumption of the analog digital converter. (See para [0015], Nakaya)
8. Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Oliner et al. (PUB NO: US20180219889A1) in view of Kuroda et al., (PUB NO: US20190179297A1).
Regarding claim 8
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine, as a model to be excluded, a model for which a frequency of generation of an output value at which the occurrence of an abnormality is detected exceeds a predetermined frequency.
In the related field of invention, Kuroda teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine, as a model to be excluded, a model for which a frequency of generation of an output value at which the occurrence of an abnormality is detected exceeds a predetermined frequency. (see para 118-Confirmation (labeling) as to whether information is obtained during a normal operation or not may be performed at an arbitrary timing after the information is stored in the storage unit 107 or the like, or may be performed in real time while the processing machine 200 is operating. It may be possible to generate a model by assuming that the information is normal, without performing labeling. If the information assumed as normal is in fact abnormal information, the determination process is not correctly performed because of the generated model. Such a situation can be discriminated based on the frequency that the information is determined as abnormal, and thus it is possible to take a countermeasure, such as deletion of a model generated by mistake. Furthermore, it may be possible to use a model generated from abnormal information as a model to determine an abnormality.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to determine, as a model to be excluded, a model for which a frequency of generation of an output value at which the occurrence of an abnormality is detected exceeds a predetermined frequency as taught by Kuroda in the system of Oliner in order to obtain a correct result of abnormality determination and perform the abnormality determination with high accuracy. Furthermore, in provide a diagnosis device, a learning device, and a diagnosis system capable of improving the accuracy in diagnosis of an abnormality. (See para [0004-0005], Kuroda)
Regarding claim 9
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded from at least three models respectively corresponding to at least three periods arranged in order of time elapsed among the plurality of periods, based on a frequency at which the occurrence of an abnormality is detected by using the at least three models.
In the related field of invention, Kuroda teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: determine a model to be excluded from at least three models respectively corresponding to at least three periods arranged in order of time elapsed among the plurality of periods, based on a frequency at which the occurrence of an abnormality is detected by using the at least three models. (see para 118-Confirmation (labeling) as to whether information is obtained during a normal operation or not may be performed at an arbitrary timing after the information is stored in the storage unit 107 or the like, or may be performed in real time while the processing machine 200 is operating. It may be possible to generate a model by assuming that the information is normal, without performing labeling. If the information assumed as normal is in fact abnormal information, the determination process is not correctly performed because of the generated model. Such a situation can be discriminated based on the frequency that the information is determined as abnormal, and thus it is possible to take a countermeasure, such as deletion of a model generated by mistake. Furthermore, it may be possible to use a model generated from abnormal information as a model to determine an abnormality.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to determine a model to be excluded from at least three models respectively corresponding to at least three periods arranged in order of time elapsed among the plurality of periods, based on a frequency at which the occurrence of an abnormality is detected by using the at least three models as taught by Kuroda in the system of Oliner in order to obtain a correct result of abnormality determination and perform the abnormality determination with high accuracy. Furthermore, in provide a diagnosis device, a learning device, and a diagnosis system capable of improving the accuracy in diagnosis of an abnormality. (See para [0004-0005], Kuroda)
9. Claims 11-14 are rejected under 35 U.S.C. 103 as being unpatentable over Oliner et al. (PUB NO: US20180219889A1) in view of Maeda et al., (PUB NO: US20120041575A1).
Regarding claim 11
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: calculate an evaluation value while performing weighting on the plurality of differences respectively calculated by using the plurality of models, and detects a state of the monitoring target based on the evaluation value.
In the related field of invention, Maeda teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: calculate an evaluation value while performing weighting on the plurality of differences respectively calculated by using the plurality of models, and detects a state of the monitoring target based on the evaluation value. (see para 148-152-Through modeling (1) 118, modeling is performed on learning data by using each piece of data as a plurality of models, determining similarities with observation data and applying the models, and calculating deviations from the observation data. Modeling (2) 108 is similar to FIG. 16 and is used to calculate a deviation from a model obtained from observation data. Subsequently, using respective deviations from modeling (1) and (2), a state change is calculated and a total deviation is calculated. In this case, while modeling (1) and (2) can be treated equally, weighting may be applied. In other words, if learning data is considered to be a basis, a weight of a model (1) is increased, and if observation data is considered to be a basis, a weight of a model (2) is increased. In accordance with the representation illustrated in FIG. 31, by comparing subspace models constituted by the model (1) between clusters, if the clusters originally have a same state, then a state change can be ascertained. In addition, if a subspace model of the observation data has moved from the original state, a state change can be identified. If the state change represents an intention to replace parts or the like or, in other words, if a designer is aware of the state change and the state change should be allowed, then the weight of the model (1) is reduced and the weight of the model (2) is increased. If the state change is unintended, then the weight of model (1) is increased.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to calculate an evaluation value while performing weighting on the plurality of differences respectively calculated by using the plurality of models, and detects a state of the monitoring target based on the evaluation value as taught by Maeda in the system of Oliner for generating quality learning data and, accordingly, to provide an anomaly detection method and system capable of reducing user load and detecting anomalies early at high sensitivity. (See para [0015], Maeda)
Regarding claim 12
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: perform the weighting based on an elapsed time from generation of a model.
In the related field of invention, Maeda teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: perform the weighting based on an elapsed time from generation of a model. (see para 148-152-Forgetting modeling may also be adopted in which the older the model (1), the smaller the weight thereof. In this case, emphasis is to be placed on models based on recent data.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to perform the weighting based on an elapsed time from generation of a model as taught by Maeda in the system of Oliner for generating quality learning data and, accordingly, to provide an anomaly detection method and system capable of reducing user load and detecting anomalies early at high sensitivity. (See para [0015], Maeda)
Regarding claim 13
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: decrease the weight as the elapsed time of a model prolongs.
In the related field of invention, Maeda teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: decreases the weight as the elapsed time of a model prolongs. (see para 148-152-Forgetting modeling may also be adopted in which the older the model (1), the smaller the weight thereof. In this case, emphasis is to be placed on models based on recent data.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to decrease the weight as the elapsed time of a model prolongs as taught by Maeda in the system of Oliner for generating quality learning data and, accordingly, to provide an anomaly detection method and system capable of reducing user load and detecting anomalies early at high sensitivity. (See para [0015], Maeda)
Regarding claim 14
Oliner teaches the information processing apparatus according to claim 1. Oliner does not explicitly teach wherein the instructions when executed by the processor, further cause the information processing apparatus to: decrease the weight as the elapsed time of a model shortens.
In the related field of invention, Maeda teaches wherein the instructions when executed by the processor, further cause the information processing apparatus to: decrease the weight as the elapsed time of a model shortens. (see para 148-152-Forgetting modeling may also be adopted in which the older the model (1), the smaller the weight thereof. In this case, emphasis is to be placed on models based on recent data.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include to decrease the weight as the elapsed time of a model shortens as taught by Maeda in the system of Oliner for generating quality learning data and, accordingly, to provide an anomaly detection method and system capable of reducing user load and detecting anomalies early at high sensitivity. (See para [0015], Maeda)
10. Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Oliner et al. (PUB NO: US20180219889A1) in view of SHIODE et al. (PUB NO: US 20140118728 A1) and further in view of Yun et al., (PUB NO: US20120101622A1).
Regarding claim 19
an acquisition step of acquiring a plurality of time-series data indicating changes in output values of a plurality of sensors from a ; (see para 226-227 and fig 12-13-Data may be collected as a time series data set, that is, a sequence of data points of a time series, often including successive measurements made over a time interval. A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include information processing logs, market transactions, and sensor data from real-time monitors (supply chains, military operation networks, or security systems). see para 243-246- Data source 1306 can be or include one or more external data sources, such as web servers, application servers, databases, firewalls, routers, operating systems, and software applications that execute on computer systems, mobile devices, sensors, and/or the like. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like. Indexer 1312 of data processing system 1302, when present, may receive source data 1310, for example, from a forwarder (not shown in FIG. 13) or data source 1306, and apportion source data 1310 into events.)
a model generation step of generating a plurality of models respectively corresponding to plurality of periods, each model indicating a relationship between the plurality of time-series data within the corresponding one of the plurality of periods in the plurality of time-series data, thereby generating a plurality of models respectfully corresponding to the plurality of periods; (see para 235-237- In some aspects of the present disclosure, relationships between time series data sets are automatically determined by analyzing values associated with the time series data sets. Each relationship is captured by a predictive model, which is configured to generate predicted values associated with at least one time series using values from at least one other time series. In some implementations, in order to search for and select a predictive model to use for anomaly detection on values associated with a time series (e.g., at least one time series), a plurality of predictive models is generated for the time series. Each predictive model can be configured to independently predict the values associated with the time series using values associated with at least one other time series. For example, one predictive model may predict values of time series data set 1202A based on values from time series data set 1202B (e.g., they may be explanatory variables), another predictive model may be similar, but use a different predictor function, and another may predict values of time series data set 1202A based on values from both time series data sets 1202B and 1202C. A predictive model is selected from the generated models for the anomaly detection based on an evaluation of the predicted values produced by the model. See para 269-270- As indicated in FIG. 14B, in some implementations, predictive models are trained over a training period, such as training period 1440. As indicated in FIG. 14B, in some implementations, predictive models are evaluated over a predicting period, such as predicting period 1442. As shown, the time series values used for predicting periods may be different values than those used for training and may follow the values used for training in the sequences of time series values. see para 279-280- For example, a subsequent training period 1440 and predicting period 1442 may be used for each iteration. Continued training of predictive models allow the predictive models to adapt to changes in the behavior of the data.)
a detecting step of detecting an abnormality of the (see para 288-291-The predictive model(s) for each time series can be used to detect anomalies in values associated with the time series. Many approaches are available for anomaly detection using predicted values from the predictive models and actual values associated with the multiple time series. Some approaches to anomaly detection are based on determining and comparing characteristics of values of the time series being predicted, the predicted values of the time series, and/or the time series upon which the predicted value are based. As one example, an anomaly may be detected, at least in part, based on determining turbulence of the values in the various time series. For example, anomaly detection tool 1316 could determine a turbulence of values in the predicted time series, which is compared to the actual turbulence of values in the time series. As indicated above, any number of predictive models may be selected for anomaly detection (e.g., selected model 1416).)
by calculating a difference between the output value of each of the plurality of sensors and an output value given by a model of the plurality of models, (see para 271-Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 227-A time series data set may be determined or derived from machine data. As indicated above, sources of the machine data can include …sensor data from real-time monitors (supply chains, military operation networks, or security systems. See para 243-244-Data source 1306 may be at least one source of incoming source data 1310 being fed into the data processing system 1302. Data source 1306 can be or include one or more external data sources, such as mobile devices, sensors. Source data 1310 may include, for example raw data (e.g., raw time-series data), such as server log files, activity log files, configuration files, messages, network packet data, performance measurements, sensor measurements, and/or the like)
and determining if the output value exceeds a predetermined value; (see para 271-272- Evaluating a predictive model can be based on determining an error between predicted values generated by the predictive model and actual values (e.g., what the values of time series 1430 actually were at those times) corresponding to those predicted values. In some implementations, the error corresponds to a residual between a predicted value and an actual value of the time series. For example, as shown, the predictive model generates predicted time series 1432 based on values of time series 1420 and time series 1422 within predicting period 1442. Although not shown, predicted time series 1432 may be generated within training period 1440 and could be used for training. The residuals represent differences between time series 1430 and predicted time series 1432 and therefore indicate the accuracy of the predictive model. For example, in general, the smaller the residuals, the more accurate the predictive model. Therefore, the residuals can serve as an effective basis for evaluating the predictive model for accuracy. See para 289-In one approach, the anomaly detection tool determines the median absolute deviation (MAD) from the predicted and actual values over a moving window of time. An anomaly may be detected based on the MAD exceeding a threshold value (e.g., falling outside of a range of values). See para 292-293-There are many potential approaches for determining whether a residual(s) is sufficiently large to consider it an anomaly. Further, a user could provide an input parameter or configuration file setting various anomaly detection parameters, such as detection thresholds. The anomaly detection tool may cause transmission of an indication of an anomaly detected using the selected predictive model (e.g., to a user and/or user device).)
an exclusion step of excluding, from the plurality of models, a model that is not used to detect an abnormality of the (See para 312-313 and fig 17- At block 1706, the trained set of predictive models is evaluated. For example, anomaly detection tool 1316 may evaluate predicted values generated using predictive models 1412 with respect to actual values of time series 1430. This may include determining at least one residual for each predictive model, such as residual time series 1434. At block 1708, at least one predictive model is removed from the set. For example, anomaly detection tool 1316 may remove, or filter out at least one of predictive models 1410 based on the evaluation. Various examples of criteria for narrowing the set of predictive models have been described above. Optionally, block 1704 and/or block 1706 may be repeated with the reduced set of predictive models (e.g., predictive models 1412 followed by predictive models 1414) unless an ending condition is satisfied. Also see para 0278-0279)
However, Oliner does not explicitly teach an article manufacturing method comprising: an exposure step of exposing a substrate by using an exposure apparatus including a plurality of sensors; a development step of developing the substrate exposed in the exposure step; a processing step of obtaining an article by processing the substrate developed in the development step; and a maintenance step of maintaining the exposure apparatus based on the state of the exposure apparatus detected in the detection step;
In the related field of invention, SHOIDE teaches an article manufacturing method comprising: an exposure step of exposing a substrate by using an exposure apparatus including a plurality of sensors; a development step of developing the substrate exposed in the exposure step; a processing step of obtaining an article by processing the substrate developed in the development step; (see para 60-nformation of the moire image detected by the sensor 136 includes that related to a relative position deviation between the patterns 104 and 106 on the flat plate 102. see para [0080-0081] and fig 13 -As described above, according to the lithography apparatus 300 of this embodiment, a foreign particle on a substrate can be detected immediately before transfer processing for transferring a pattern onto the substrate and inside the lithography apparatus 300. In other words, the lithography apparatus 300 can detect a foreign particle on a substrate, and can execute the transfer processing based on that detection result. Therefore, the lithography apparatus 300 can prevent damages of a mold, pattern transfer error, and the like caused a foreign particle, and can efficiently manufacture an article such as a semiconductor device. A method of manufacturing a device (semiconductor device, liquid crystal display element, or the like) as an article includes a step of transferring (forming) a pattern on a substrate (wafer, glass plate, film-shaped substrate, or the like using the lithography apparatus 300. This manufacturing step further includes a step of etching the substrate on which the pattern is transferred. Note that this manufacturing method includes another processing step of processing the substrate on which the pattern is transferred in place of an etching step when another article such as a pattern dot medium (recording medium) or optical element is to be manufactured. The lithography apparatus 300 may also be used as a charged particle beam lithography apparatus, projection exposure apparatus, and the like in addition to the imprint apparatus. The charged particle beam lithography apparatus is a lithography apparatus, which transfers a pattern onto a substrate by drawing on a substrate using a charged particle beam. The projection exposure apparatus is a lithography apparatus, which transfers a pattern on a reticle onto a substrate by projecting the pattern on the reticle onto the substrate via a projection optical system.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include maintenance step of maintaining the exposure apparatus based on the state of the exposure apparatus detected in the detection step as taught by SHIODE in the system of Oliner in order to detect a foreign particle on a substrate inside the lithographic apparatus with high precision and within a short period of time, thus suitable for the manufacture of semiconductor devices.(see para [002-006], SHIODE)
The combination of Oliner and SHIODE does not teach maintenance step of maintaining the exposure apparatus based on the state of the exposure apparatus detected in the detection step.
In the related field of invention, Yun teaches maintenance step of maintaining the exposure apparatus based on the state of the exposure apparatus detected in the detection step. (See para 003-In a plasma processing system employed for producing semiconductor devices, for example, it is highly desirable that the plasma processing system produces electronic devices with the highest yield and with the lowest cost of ownership possible. A fault condition may arise due to, for example, chamber component malfunction, chamber component wear, incorrectly installed chamber components, and/or any other condition that requires cleaning, maintenance, and/or replacement of one or more subsystems of the plasma processing system.)
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of anomaly detection based on relationships between multiple time series using the selected predictive model as disclosed by Oliner to include maintenance step of maintaining the exposure apparatus based on the state of the exposure apparatus detected in the detection step as taught by Yun in the system of Oliner for producing semiconductor devices, for example, it is highly desirable that the plasma processing system produces electronic devices with the highest yield and with the lowest cost of ownership possible. To achieve a high yield and to reduce tool down time, which contributes to a higher cost of ownership, it is critical to detect and classify faults rapidly in order to minimize damage to wafers and/or to the plasma processing system components. Another motivation for automatically detecting fault conditions and for classifying fault conditions in an automatic and timely manner. (See para [0003-0006], Yun)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
11. All claims 1-19 are rejected.
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to PURSOTTAM GIRI whose telephone number is (469)295-9101. The examiner can normally be reached 7:30-5:30 PM, Monday to Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RENEE CHAVEZ can be reached at 5712701104. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PURSOTTAM GIRI/
Examiner, Art Unit 2186
/RENEE D CHAVEZ/Supervisory Patent Examiner, Art Unit 2186
Prosecution Insights