DETAILED ACTION
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Regarding Chinese Patent App. No. CN202211654652.8 (filed 12/22/2022), receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Regarding PCT Application No. PCT/CN2023/088253 (filed 4/14/2023), Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Information Disclosure Statement
The information disclosure statement submitted on 2/5/2024 has been submitted.
Claim Objections
Claim 1 is objected to because of the following informalities:
In claim 1, line 2, “comprising following steps” should read “comprising the following steps”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-8 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim 1 recites: “quantitatively characterizing a discrimination of a data category by means of cluster evaluation indexes”. It is unclear what the metes and bounds of the “discrimination of a data category” limitation. The plain and ordinary meaning of “discrimination” typically relates to prejudices or biases, but could also refer to the act of perceiving differences. It’s unclear if this limitation is meant to refer to (1) a bias value (with respect to discrimination), (2) a mathematical discriminant, and/or (3) simply meaning that there is a difference between data categories. For purposes of compact prosecution, this will be interpreted as “quantitatively characterizing a difference of a data category by means of cluster evaluation indexes”.
Claim 1 recites: “using weights of the cluster evaluation indexes of different dimensions as dynamic rewards”. This is indefinite because one of ordinary skill in the art would not be able to understand what is claimed in view of the specification. MPEP 2173.02. It is unclear how the “cluster evaluation indexes” have weights, and the specification does not provide any further explanation. It’s also unclear how weights can have “different dimensions,” as weights are typically thought of as integer or real numbers, meaning a dimension of 1. So how could the dimensions be different? For purposes of compact prosecution, this will be interpreted as “using different values of the cluster evaluation index as dynamic rewards.”
Claim 1 recites: “updating a neural network parameter of deep reinforcement learning through characterization of an interactive relation between a model and a discrete manufacturing decision-making analysis system”. This is indefinite because one of ordinary skill in the art would not be able to understand what is claimed in view of the specification. MPEP 2173.02. It is unclear how a neural network parameter (or hyper-parameter) is updated “through characterization of an interactive relation between a model and a discrete manufacturing decision-making analysis system”. The specification is unclear what “an interactive relation” refers to and how such “interaction relation” is characterized, and is further unclear how such characterization can be used to update a neural network parameter. For purposes of compact prosecution, this will be interpreted as “updating a neural network parameter trained using deep reinforcement learning based at least in part on a model utilized by a discrete manufacturing decision-making analysis system.”
Claims 2-8 depend from claim 1, do not remedy the deficiencies of claim 1, and are also rejected for the same reasons explained above with respect to claim 1.
Claim 4 is further rejected because in line 5, with respect to
PNG
media_image1.png
30
110
media_image1.png
Greyscale
, the variable “h” is not defined or explained in the claim or specification, so it’s unclear what “h” refers to.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Step 1 of the Alice/Mayo framework, Claims 1-8 are directed to a method (a process), which falls within one of the four statutory categories of inventions.
Regarding Claim 1
Step 2A, prong 1 (Is the claim directed to a law of nature, a natural phenomenon or an abstract idea).
Claim 1 recites the following mental processes, that in each case under the broadest reasonable interpretation, covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components (e.g., “deep reinforcement learning”, “neural network”).
A characterization method ... for discrete manufacturing industry data, comprising following steps: (under the broadest reasonable interpretation, a human can mentally review and characterize discrete manufacturing industry data, such as reviewing the output from 2 or more temperature sensors)
(2) dividing the discrete manufacturing industry data into a discrete feature and a
continuous feature, ... (under the broadest reasonable interpretation, a human can mentally divide the collected data into both discrete and continuous features, such as discrete temperature highs and lows, and temperature over time)
(3) quantitatively characterizing a discrimination of a data category by means of
cluster evaluation indexes; and (under the broadest reasonable interpretation, a human can mentally characterizing a difference of a data category by means of cluster evaluation indexes, for example, a human can mentally cluster the feature data, determine a representative data category for each cluster, and quantitatively determine (e.g., using a distance metric) the differences between the clusters, where each cluster is given an index (e.g., cluster 1, cluster 2))
(4) using weights of cluster evaluation indexes of different dimensions as dynamic
rewards, ... (under the broadest reasonable interpretation, a human can mentally determine a rewards function for deep reinforcement learning, where such rewards function utilizes the different cluster evaluation index values)
Step 2A, prong 2 (Does the claim recite additional elements that integrate the judicial exception into a practical application?).
The judicial exception is not integrated into a practical application.
Regarding the “... based on deep reinforcement learning ... ” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of deep reinforcement learning. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (deep reinforcement learning). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “(1) collecting discrete manufacturing industry data, and creating a spatio-temporal database” limitation, such additional element of a data gathering and storage step is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of gathering data for use in the claimed process (see MPEP 2106.05(g)).
Regarding the “creating a data coupling coding network, converting a coding vector in
the data coding network into a characterization vector, and creating a data characterization
model” limitation, such limitations are recited at a high-level of generality and amount to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional elements of different types of computing networks and models. These additional elements are recited at a high-level of generality and amount to no more than mere instructions to apply the exception using generic computer components (computing networks and models). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Regarding the “creating a deep reinforcement learning model, and updating a neural network parameter of deep reinforcement learning through characterization of an interactive relation between a model and a discrete manufacturing decision-making analysis system” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception. In particular, the claim only recites the additional element of creating a deep RL model and updating a neural network. This additional element is recited at a high-level of generality and amounts to no more than mere instructions to apply the exception using a generic computer component (creating a deep RL model and updating a neural network). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f)).
Accordingly, at Step 2A, prong two, after considering all claim elements individually and as an ordered combination, it is determined that the claims do not integrate the judicial exception into a practical application.
Step 2B (Does the claim recite additional elements that amount to significantly more than the judicial exception?)
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception.
Regarding the “... based on deep reinforcement learning ... ” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “(1) collecting discrete manufacturing industry data, and creating a spatio-temporal database” limitation, as discussed above, the additional element of a data gathering step is recited at a high level of generality and amounts to extra-solution activity of receiving data, i.e. pre-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Regarding the “creating a data coupling coding network, converting a coding vector in
the data coding network into a characterization vector, and creating a data characterization
model” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Regarding the “creating a deep reinforcement learning model, and updating a neural network parameter of deep reinforcement learning through characterization of an interactive relation between a model and a discrete manufacturing decision-making analysis system” limitation, such limitation is recited at a high-level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, because the limitation merely provides instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not add significantly more than the judicial exception. (See MPEP 2106.05(f)).
Accordingly, at Step 2B after considering all claim elements individually and as an ordered combination, it is determined that the claims do not integrate the judicial exception into a practical application.
Regarding Claim 2
Step 2A, Prong 2
Regarding the “wherein the discrete manufacturing industry data in step (1) comprises real-time workshop device data, advanced planning and scheduling (APS) production scheduling data, product data management (PDM) product data, enterprise resource planning (ERP)
purchase-sale-stock data, and manufacturing execution system (MES) production execution
data” limitation, such limitation merely describes the data being collected and processed, and therefore such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate a judicial exception into a practical application.
Step 2B
Regarding the “wherein the discrete manufacturing industry data in step (1) comprises real-time workshop device data, advanced planning and scheduling (APS) production scheduling data, product data management (PDM) product data, enterprise resource planning (ERP)
purchase-sale-stock data, and manufacturing execution system (MES) production execution
data” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use as explained above, which does not amount to significantly more than the judicial exception. MPEP 2106.05(h).
Claim 7 - The characterization method according to claim 6, wherein the cluster evaluation
indexes of the different dimensions comprise a Calinski-Harabasz (CH) index, a Davies-Bouldin index (DBI), and/or a silhouette coefficient.
Regarding Claim 3
Step 2A, Prong 1
PNG
media_image2.png
120
678
media_image2.png
Greyscale
PNG
media_image3.png
258
678
media_image3.png
Greyscale
PNG
media_image4.png
418
654
media_image4.png
Greyscale
PNG
media_image5.png
202
666
media_image5.png
Greyscale
PNG
media_image6.png
96
266
media_image6.png
Greyscale
PNG
media_image7.png
152
658
media_image7.png
Greyscale
(under the broadest reasonable interpretation, such limitations merely pertain to the mathematical concepts and formulations specified in claim 3, which is another type of abstract idea)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 4
Step 2A, Prong 1
PNG
media_image8.png
366
641
media_image8.png
Greyscale
(under the broadest reasonable interpretation, such limitations merely pertain to the mathematical concepts and formulations specified in claim 4, which is another type of abstract idea)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 5
Step 2A, Prong 1
PNG
media_image9.png
270
653
media_image9.png
Greyscale
(under the broadest reasonable interpretation, such limitations merely pertain to the mathematical concepts and formulations specified in claim 5, which is another type of abstract idea)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 6
Step 2A, Prong 1
PNG
media_image10.png
306
662
media_image10.png
Greyscale
(under the broadest reasonable interpretation, such limitations merely pertain to the mathematical concepts and formulations specified in claim 6, which is another type of abstract idea)
Regarding Step 2A, Prong 2, the claim does not include any additional elements that integrate the judicial exception into a practical application and regarding Step 2B, there are no additional elements recited that amount to significantly more than the judicial exception.
Regarding Claim 7
Step 2A, Prong 2
Regarding the “wherein the cluster evaluation indexes of the different dimensions comprise a Calinski-Harabasz (CH) index, a Davies-Bouldin index (DBI), and/or a silhouette coefficient” limitation, this limitation merely describes types of indexing with respect to clusters, and therefore such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate a judicial exception into a practical application.
Step 2B
Regarding the “wherein the cluster evaluation indexes of the different dimensions comprise a Calinski-Harabasz (CH) index, a Davies-Bouldin index (DBI), and/or a silhouette coefficient” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use as explained above, which does not amount to significantly more than the judicial exception. MPEP 2106.05(h).
Regarding Claim 8
Step 2A, Prong 2
Regarding the “wherein the deep reinforcement learning model in step (4) further comprises one of deep deterministic policy gradient (DDPG), Advanced-Actor-Critic (A2C)/ Asynchronous-Advanced-Actor-Critic (A3C), proximal policy optimization (PPO)/trust region policy optimization (TRPO), soft actor critic (SAC), and twin delayed deep deterministic policy gradient (TD3)” limitation, such limitation merely describes types of deep reinforcement learning techniques, and therefore such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not integrate a judicial exception into a practical application.
Step 2B
Regarding the “wherein the deep reinforcement learning model in step (4) further comprises one of deep deterministic policy gradient (DDPG), Advanced-Actor-Critic (A2C)/ Asynchronous-Advanced-Actor-Critic (A3C), proximal policy optimization (PPO)/trust region policy optimization (TRPO), soft actor critic (SAC), and twin delayed deep deterministic policy gradient (TD3)” limitation, such limitation amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use as explained above, which does not amount to significantly more than the judicial exception. MPEP 2106.05(h).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 8 rejected under 35 U.S.C. 103 as being unpatentable over US 20230281427 A1, hereinafter referenced as SAKHINANA, in view of Ma, Xiao, et al. "Clustered reinforcement learning." arXiv preprint arXiv:1906.02457 (2019), hereinafter referenced as MA, and further in view of US 20210149384 A1, hereinafter referenced as DITTMER.
Regarding Claim 1
SAKHINANA teaches:
A characterization method ... for discrete manufacturing industry data, comprising following steps: (SAKHINANA, para. 0005: “There is a need and necessity for a mathematical framework to enhance privacy-preserving, trade-off to preserve utility for data monetization of the large-scale industrial and manufacturing plants multivariate mixed-variable time series data.”;
SAKHINANA, para. 0028: “Referring FIG. 2, illustrates a block diagram (200) of the system (100) for privacy preserving generative mechanism for data-disclosure of the industrial data. Herein, the one or more I/O interfaces (104) are configured to receive a multivariate mixed-variable time series data of a plurality of sensory observations, the cluster-labels associated with the multivariate mixed-variable time series data and a cluster-independent random noise.”;
Examiner’s Note: SAKHINAKA performs privacy preservation techniques to industrial data (corresponding to recited “discrete manufacturing industry data”, where the broadest reasonable interpretation of “discrete” merely means that such manufacturing industry data is “individually separate and distinct”, meaning from more than one source (such as the more than one sensors), and is not required to be mathematically discretized)
(1) collecting discrete manufacturing industry data, ... (SAKHINANA, para. 0005: “There is a need and necessity for a mathematical framework to enhance privacy-preserving, trade-off to preserve utility for data monetization of the large-scale industrial and manufacturing plants multivariate mixed-variable time series data.”;
SAKHINANA, para. 0028: “Referring FIG. 2, illustrates a block diagram (200) of the system (100) for privacy preserving generative mechanism for data-disclosure of the industrial data. Herein, the one or more I/O interfaces (104) are configured to receive a multivariate mixed-variable time series data of a plurality of sensory observations, the cluster-labels associated with the multivariate mixed-variable time series data and a cluster-independent random noise.”;
Examiner’s Note: SAKHINAKA collects industrial data from more than one sensor (corresponding to recited “discrete manufacturing industry data”, where the broadest reasonable interpretation of “discrete” merely means that such manufacturing industry data is “individually separate and distinct”, meaning from more than one source (such as the more than one sensors), and is not required to be mathematically discretized)
(2) dividing the discrete manufacturing industry data into a discrete feature and a
continuous feature, (SAKHINANA, para. 0028: “The received multivariate mixed-variable time series data is pre-processed to normalize continuous feature variables by bounding heterogeneous measurements between a predefined range of a min-max scaling technique. Discreate feature variables are transformed by representing as a sparse binary vector through a one-hot encoding technique.”
SAKHINANA, para. 0071: “At the next step (304), pre-processing the received multivariate mixed-variable time series data. Wherein, the pre-processing comprising normalizing continuous feature variables by bounding heterogeneous measurements between a predefined range of a min-max scaling technique and transforming discreate feature variables by representing as a sparse binary vector through a one-hot encoding technique.”;
Examiner’s Note: pre-processing continuous feature variables and discrete feature variables separately corresponds to recited “dividing the discrete manufacturing industry data...” limitation)
creating a data coupling coding network, converting a coding vector in the data coding network into a characterization vector, and creating a data characterization model; (SAKHINANA, para. 0029: “The plurality of neural networks (116) are trained in two phases. In the first phase the embedding neural network (118) is trained using a predefined low-dimensional mixed feature training dataset to obtain a high-dimensional mixed feature embeddings. Wherein, the obtained high-dimensional mixed feature embeddings are used to train a supervisory neural network (128) for a single step ahead predictions of the high-dimensional mixed-feature embeddings.”;
SAKHINANA, para. 0030: “The embedding neural network (118) ... learns a high-dimensional representation,... by transforming the corresponding low-dimensional real sequences, ... The embedding neural network (118) assists in effective learning by incorporating the semantics of the mixed-feature variables in its feature embeddings, ...”
Examiner’s Note: As shown in Fig. 2, time series data is input into the data coupling coding network (corresponding to recited “embedding neural network 118” because such embedding neural network both couples and encodes the time series data into an embedded representation), where the embedding neural network 118 takes the features input into the network (corresponding to recited “coding vector”) and converts such features to an output embedding representation (corresponding to recited “characterization vector”), and then, based on this information, the supervisory neural network (128) is trained (corresponding to recited “data characterization model”) because such supervisory neural network takes the output embedding representation and outputs a prediction, or a characterization)
However, SAKHINANA fails to explicitly teach:
... based on deep reinforcement learning ...
... and creating a spatio-temporal database
(3) quantitatively characterizing a discrimination of a data category by means of
cluster evaluation indexes; and
(4) using weights of cluster evaluation indexes of different dimensions as dynamic
rewards, creating a deep reinforcement learning model, and updating a neural network
parameter of deep reinforcement learning through characterization of an interactive relation
between a model and a discrete manufacturing decision-making analysis system.
However, in a related field of endeavor (agent and model training, see p. 1, section 1), MA teaches and makes obvious:
... based on deep reinforcement learning ... (MA, p. 1, section 1: “reinforcement learning, especially deep RL (DRL), has recently attracted much attention and achieved significant performance in a variety of applications”
MA, p. 1, section 2: “To tackle this problem, we propose a novel RL framework, called clustered reinforcement learning (CRL), for efficient exploration in RL.”;
Examiner’s Note: the SAKHINANA-MA combination now modifies the training of the neural networks of SAKHINANA to utilize the clustered reinforcement learning teachings of MA)
(3) quantitatively characterizing a discrimination of a data category by means of
cluster evaluation indexes; and (MA, p. 4, section 3.2.1:
PNG
media_image11.png
94
650
media_image11.png
Greyscale
Examiner’s Note: the i with respect to Ci corresponds to the recited “cluster evaluation indexes” and such Ci are used to cluster different states (corresponding to recited “data category”) using a clustering algorithm (corresponding to recited “quantitatively characterizing”)
(4) using weights of cluster evaluation indexes of different dimensions as dynamic
rewards, creating a deep reinforcement learning model,
. (MA, p. 3, section 3.1: “we model the RL problem as a finite horizon discounted Markov decision process (MDP), which can be defined by ...”
MA, p. 5, section 3.2.2:
PNG
media_image12.png
354
668
media_image12.png
Greyscale
Examiner’s Note: MA teaches a specific reward function (see equation 1, corresponding to recited “dynamic rewards” because the reward is calculable as opposed to static), where such reward function is based on the cluster assignment function φ(si) (corresponding to recited “using weights of cluster evaluation indexes ... as dynamic rewards” because values (or weights) of Ck are used with respect to dynamic rewards); the SAKHINANA-MA combination now uses the deep RL modeling system of MA with the industrial neural network models of SAKHINANA, such that the neural network models of SAKHINANA are updated and trained using deep reinforcement learning as in MA)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of SAKHINANA and MA as explained above. As disclosed by MA, one of ordinary skill would have been motivated to do so because MA teaches that its clustered reinforcement learning framework “can outperform other state-of-the-art methods to achieve the best performance in most cases.” (p. 2, section 1).
However, SAKHINANA and MA fail to explicitly teach:
... and creating a spatio-temporal database
updating a neural network parameter of deep reinforcement learning through characterization of an interactive relation between a model and a discrete manufacturing decision-making analysis system
However, in a related field of endeavor (manufacturing systems using machine learning, see paras. 0003, 0058), DITTMER teaches and makes obvious:
... and creating a spatio-temporal database (DITTMER, para. 0055: “The time-series database 86 stores data associated with a particular manufacturing application system 54 and data from corresponding industrial components 62. In contrast, the application-specific time-series database 90 stores data for multiple manufacturing applications systems 54. In fact, the application-specific time-series database serves as a record or history of the manufacturing application systems 54 collected by the time-series database 86. Each time the time-series database collects data from components and related to application-specific metrics, that data is sent to the application-specific time-series database 90 as well. After the aggregate monitoring system 40 collects data from the time-series database 86, organizes the data, and pushes the data to the application-specific time-series database, the data stored in the application-specific time-series database may be processed and analyzed to determine whether any events are present or likely to occur.”;
Examiner’s Note: the SAKHINANA-MA-DITTMER combination now uses the time-series database of DITTMER (corresponding to spatio-temporal database) to store data collected from sensors as in SAKHINANA)
updating a neural network parameter of deep reinforcement learning through characterization of an interactive relation between a model and a discrete manufacturing decision-making analysis system (DITTMER, para. 0040: “The system 80 also includes Internet of Things (IoT) data analytics component 84, which allows the local monitoring system 42 to collect and analyze data from the industrial components 62 (e.g., sensors 66) disposed on manufacturing equipment, pipelines, and other types of machinery. After the local monitoring system 42 receives data from the industrial components 62 and data related to application-specific metrics of the manufacturing application system 54, the local monitoring system 42 may send both sets of data to time-series database 84, which may store both sets of data with respect to a time in which the data was collected. In some embodiments, the time-series database 84 may export the data as log files 88 to another database, to other components, or the like.”
DITTMER, para. 0058: “For example, machine learning component 142 may be used to perform trend analyses and identify boundary conditions based on correlations between the data from the industrial components 62 and application-specific metrics corresponding to the manufacturing application system 54. Machine learning circuitry (e.g., circuitry used to implement machine learning logic) may access the data stored in the universal fields of information platform 140 to identify patterns of the data. Because data is provided from a multitude of diverse online services, new data patterns not previously attainable may emerge. As used herein, the machine learning component 142 may include circuitry or software that implements certain algorithms and/or statistical models that computer systems may use to perform specific tasks without using explicit instructions, relying instead on patterns and inference instead. In one example, the machine learning component 142 may generate a mathematical model based on sample data, known as “training data”, in order to make predictions or decisions without being explicitly programmed to perform the task.”;
Examiner’s Note: the SAKHINANA-MA-DITTMER combination now modifies the neural network models of SAKHINANA such that they are updated using deep reinforcement learning as in MA, and further using the interactions between the monitoring system 42 of DITTMER, utilizing machine learning models, to inform how such neural network models are updated)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of SAKHINANA, MA, and DITTMER as explained above. As disclosed by DITTMER, one of ordinary skill would have been motivated to do so because DITTMER teaches improvements “related to monitoring data representative of the health or status of various components employed in the manufacturing application systems, such that the components are suitable for executing various types of operations that may be related to manufacturing or production in an industrial automation system.” (para. 0002). As further disclosed by DITTMER, “[s]toring the organized data in an application-specific time-series database 90 provides improved querying operations for the data and improved trend analyses.” (para. 0043).
Regarding Claim 2
SAKHINANA, MA, and DITTMER teach the method of claim 1 as explained above. However, SAKHINANA and MA fail to explicitly teach:
wherein the discrete manufacturing industry data in step (1) comprises real-time
workshop device data, advanced planning and scheduling (APS) production scheduling data, product data management (PDM) product data, enterprise resource planning (ERP)
purchase-sale-stock data, and manufacturing execution system (MES) production execution
data
However, in a related field of endeavor (manufacturing systems using machine learning, see paras. 0003, 0058), DITTMER teaches and makes obvious:
wherein the discrete manufacturing industry data in step (1) comprises real-time
workshop device data, (DITTMER, para. 0034: “Further, the SCADA 60 may analyze real or near real-time data from industrial components 62 and subsequently control the industrial components 62.”)
advanced planning and scheduling (APS) production scheduling data, (DITTMER, para. 0035: “In some embodiments, the manufacturing application system 54 may include MES or MOM that manage production workflow to produce the desired products, batch management, laboratory, maintenance and plant performance management systems, data historians, related middleware, and the like.”
Examiner’s Note: the MOM (manufacturing operations management system, see para. 0003) manages production workflow, correspond to recited “APS production scheduling data”)
product data management (PDM) product data, (DITTMER, para. 0054: “the application-specific agents 136 may probe into specific industrial components 62 and retrieve data that is relevant to application-specific metrics or operations being performed by the manufacturing application system 54.”)
enterprise resource planning (ERP) purchase-sale-stock data (DITTMER, para. 0037: “Positioned above the manufacturing operations system level, a business logistics system level may manage business-related activities of the manufacturing operation. For instance, enterprise resource planning (ERP) system 52 may establish production schedule, material use, shipping, and inventory levels to support the operations monitored by the components (e.g., databases, servers) in the manufacturing application system level 56.”)
manufacturing execution system (MES) production execution data (DITTMER, para. 0020: “The MES may include software or a computer system used to monitor, control, and log processes employed to transform raw materials to products in the industrial automation system.”;
Examiner’s Note: MES is defined as manufacturing execution system in para. 0003; the SAKHINANA-MA-DITTMER combination now collects the various types of manufacturing data as set forth by DITTMER)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of SAKHINANA, MA, and DITTMER as explained above. As disclosed by DITTMER, one of ordinary skill would have been motivated to do so because DITTMER teaches improvements “related to monitoring data representative of the health or status of various components employed in the manufacturing application systems, such that the components are suitable for executing various types of operations that may be related to manufacturing or production in an industrial automation system.” (para. 0002). As further disclosed by DITTMER, “[s]toring the organized data in an application-specific time-series database 90 provides improved querying operations for the data and improved trend analyses.” (para. 0043).
Regarding Claim 8
SAKHINANA, MA, and DITTMER teach the method of claim 1 as explained above. However, SAKHINANA fails to explicitly teach:
wherein the deep reinforcement learning model in step (4) further comprises one of deep deterministic policy gradient (DDPG), Advanced-Actor-Critic (A2C)/ Asynchronous-Advanced-Actor-Critic (A3C), proximal policy optimization (PPO)/trust region policy optimization (TRPO), soft actor critic (SAC), and twin delayed deep deterministic policy gradient (TD3).
However, in a related field of endeavor (agent and model training, see p. 1, section 1), MA teaches and makes obvious:
wherein the deep reinforcement learning model in step (4) further comprises one of deep deterministic policy gradient (DDPG), Advanced-Actor-Critic (A2C)/ Asynchronous-Advanced-Actor-Critic (A3C), proximal policy optimization (PPO)/trust region policy optimization (TRPO), soft actor critic (SAC), and twin delayed deep deterministic policy gradient (TD3). (MA, p. 6, section 3: “Many policy updating algorithms, such as trust region policy optimization (TRPO) [24], can be adopted”
MA, p. 9, section 4.3: “We also compare our results to double-DQN [31], dueling network [32], A3C+ [5], double DQN with pseudo-count [5], ...”;
Examiner’s Note: the SAKHINANA-MA-DITTMER combination now uses the deep RL modeling system of MA, which can use either TRPO or A3C as specified by M).
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of SAKHINANA, MA, and DITTMER as explained above. As disclosed by MA, one of ordinary skill would have been motivated to do so because MA teaches that its clustered reinforcement learning framework “can outperform other state-of-the-art methods to achieve the best performance in most cases.” (p. 2, section 1).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over SAKHINANA, MA, and DITTMER, and further in view of CN114139629A (submitted in Applicant’s 2/5/2024 IDS), hereinafter referenced as SONGLEI.
Regarding Claim 3
SAKHINANA, MA, and DITTMER teach the method of claim 1 as explained above. However, SAKHINANA, MA, and DITTMER fail to explicitly teach:
PNG
media_image2.png
120
678
media_image2.png
Greyscale
PNG
media_image3.png
258
678
media_image3.png
Greyscale
PNG
media_image4.png
418
654
media_image4.png
Greyscale
PNG
media_image5.png
202
666
media_image5.png
Greyscale
PNG
media_image6.png
96
266
media_image6.png
Greyscale
PNG
media_image7.png
152
658
media_image7.png
Greyscale
However, in a related field of endeavor (mixed data representations), SONGLEI teaches and makes obvious:
PNG
media_image2.png
120
678
media_image2.png
Greyscale
PNG
media_image3.png
258
678
media_image3.png
Greyscale
(SONGLEI, claim 3:
PNG
media_image13.png
200
400
media_image13.png
Greyscale
In the above formula
PNG
media_image14.png
200
400
media_image14.png
Greyscale
,
PNG
media_image15.png
200
400
media_image15.png
Greyscale
represents discrete features,
PNG
media_image16.png
200
400
media_image16.png
Greyscale
correlation between continuous features vj,
PNG
media_image17.png
200
400
media_image17.png
Greyscale
represents continuous features, and the joint density between discrete features vj, t is the threshold parameter, λ is the scale coefficient, where the joint density
PNG
media_image18.png
200
400
media_image18.png
Greyscale
) (see translation provided via Google Patents)
PNG
media_image4.png
418
654
media_image4.png
Greyscale
(SONGLEI, claim 3: The calculation function expression is
PNG
media_image19.png
200
400
media_image19.png
Greyscale
In the above formula, N is the number of data objects,
PNG
media_image20.png
200
400
media_image20.png
Greyscale
are discrete eigenvalues
PNG
media_image21.png
200
400
media_image21.png
Greyscale
and the kernel function between vj,
PNG
media_image22.png
200
400
media_image22.png
Greyscale
is the kernel function of continuous features,
PNG
media_image23.png
200
400
media_image23.png
Greyscale
represents the continuous eigenvalue f1 of Variable Ai on the kth data object,
PNG
media_image24.png
200
400
media_image24.png
Greyscale
represents the continuous eigenvalue fi of the variable Ai on the xth data object,m and hi represents the bandwidth parameter of the continuous feature where the kernel function
PNG
media_image25.png
200
400
media_image25.png
Greyscale
)
PNG
media_image5.png
202
666
media_image5.png
Greyscale
(SONGLEI, claim 3:
PNG
media_image26.png
200
400
media_image26.png
Greyscale
In the above formula,
PNG
media_image27.png
200
400
media_image27.png
Greyscale
represents the eigenvalue corresponding to the discrete feature vj on the kth data object and λ is the scale coefficient)
PNG
media_image6.png
96
266
media_image6.png
Greyscale
PNG
media_image7.png
152
658
media_image7.png
Greyscale
(SONGLEI, para. 0070:
PNG
media_image28.png
70
334
media_image28.png
Greyscale
Examiner’s Note: the SAKHINANA-MA-DITTMER-SONGLEI combination now utilizes the mathematical formulations explicitly defined in SONGLEI in combination with the mixed continuous and discrete data of SAKHINANA)
Before the effective filing date of the present application, it would have been obvious to one of ordinary skill in the art to combine the teachings of SAKHINANA, MA, DITTMER, and SONGLEI as explained above. As disclosed by SONGLEI, one of ordinary skill would have been motivated to do so in order to learn the “difference between the data objects.” (abstract). One of ordinary skill would further be motivated to do so in order to attempt to determine correlations between continuous and discrete values as taught by SONGLEI.
Allowable Subject Matter
Claims 4-7 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 and 112(b) are overcome.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 4 would be considered allowable because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in the independent claims, including at least:
PNG
media_image8.png
366
641
media_image8.png
Greyscale
The closest prior art of record discloses:
US 20230281427 A1, hereinafter referenced as SAKHINANA discloses collecting industrial multivariate, mixed-variable, time-series data for more than one sensor. (paras. 0005, 0028). The data is pre-processed for both continuous and discrete feature variables. (para. 0028).
Ma, Xiao, et al. "Clustered reinforcement learning." arXiv preprint arXiv:1906.02457 (2019), hereinafter referenced as MA discloses performing deep reinforcement learning using a clustered reinforcement learning algorithm. (MA, p. 1, section 1 and p. 4, section 3.2.1).
US 20210149384 A1, hereinafter referenced as DITTMER storing time-series data in a database and utilizing a local monitoring system having access to various types of manufacturing data. (paras. 0040, 0055, 0058).
CN114139629A (submitted in Applicant’s 2/5/2024 IDS), hereinafter referenced as SONGLEI, in claim 3 (of SONGLEI), teaches some of the same formulas claimed with respect to claim 3 of the present application).
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in claim 4. Therefore, because claim 4 is not anticipated nor made obvious by the prior art of record, claim 4 would be allowed if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 and 112(b) are overcome.
Claim 5 would be considered allowable because none of the references of record either alone or in combination fairly disclose or suggest the combination of limitations specified in the independent claims, including at least:
PNG
media_image9.png
270
653
media_image9.png
Greyscale
The closest prior art of record discloses:
US 20230281427 A1, hereinafter referenced as SAKHINANA discloses collecting industrial multivariate, mixed-variable, time-series data for more than one sensor. (paras. 0005, 0028). The data is pre-processed for both continuous and discrete feature variables. (para. 0028).
Ma, Xiao, et al. "Clustered reinforcement learning." arXiv preprint arXiv:1906.02457 (2019), hereinafter referenced as MA discloses performing deep reinforcement learning using a clustered reinforcement learning algorithm. (MA, p. 1, section 1 and p. 4, section 3.2.1).
US 20210149384 A1, hereinafter referenced as DITTMER storing time-series data in a database and utilizing a local monitoring system having access to various types of manufacturing data. (paras. 0040, 0055, 0058).
However, the examiner has found that the distinct feature of the Applicant's claimed invention over the prior art is the explicit claiming of the aforementioned limitations in combination with all the other limitations as specified in claim 5. Therefore, because claim 5 is not anticipated nor made obvious by the prior art of record, claim 5 would be allowed if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 and 112(b) are overcome.
Claims 6-7 depend from claim 5 and would be allowed over the prior art of record for the same reasons explained with respect to claim 5, if rewritten in independent form including all of the limitations of the base claim and any intervening claims, provided that the rejections under 35 U.S.C. 101 and 112(b) are overcome.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230351202 A1 (Sakhinana). “Health monitoring of complex industrial assets remains the most critical task for avoiding downtimes, improving system reliability, safety and maximizing utilization. Deep learning-driven generative models encapsulate the operational behavior from adversarial losses through adversarial training of the complex large-scale industrial-plant or asset multivariate time series data. Recent advances in time-series synthetic data generation have several inherent limitations for realistic applications. The existing solutions do not provide a unified approach and do not generate the realistic data which can be used in the industrial processes. Further, the existing solution is not able to incorporate condition and constraint prior knowledge while sampling the synthetic data.” (para. 0021).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL C LEE whose telephone number is (571)272-4933. The examiner can normally be reached M-F 12:00 pm - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL C. LEE/Examiner, Art Unit 2128
Prosecution Insights