DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on 6/3/2024. Claims 1-20 are pending in the case. Claims 1, 8, and 15 are independent claims.
Specification
The disclosure is objected to because of the following informalities:
Paragraph 0014 recites “cannel” where “channel” was apparently intended.
Paragraph 0015 is cut off.
Appropriate correction is required.
Claim Objections
Claims 5, 12, and 18-19 are objected to because of the following informalities:
Claim 5 recites “expanding, by the processor, active” where “expanding, by the processor, the active” was apparently intended.
Claims 12 and 19 recite “expand active” where “expand the active” was apparently intended.
Claim 18 recites “instruction” where “instructions” was apparently intended.
Appropriate correction is required.
Claim Rejections - 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. § 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention. Each independent claim recites “phases” which is not described in the specification and is not a term of art for the usage in the claim. It is unclear what a phase is, and therefore how to expand or compress them. For the purposes of prior art and subject matter eligibility analyses Examiner assumes phases are any division of the data into units suitable for mixing, expanding is the process of separating the data into units, and compression is the process of remerging them. Examiner’s assumption is not evidence of support in the original disclosure. Dependent claims inherit the same issue from parent claims and do not resolve it.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 8-10, and 15-17 are rejected under 35 U.S.C. § 103 as being unpatentable over He et al. (US 2024/0096072 A1, hereinafter He) in view of Zhang et al. (“Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection,” 21 August 2023, https://arxiv.org/abs/2209.12244v2, hereinafter Zhang).
As to independent claim 1, He teaches a computer-implemented method for training a phase mixer masked autoencoder model, the computer-implemented method comprising:
expanding, by a processor, a plurality of input features into from an initial size to n number of phases (“Referring back to FIG. 1, in particular embodiments, the input image 102 may be divided into a plurality of patches 104,” paragraph 0030 lines 1-3), wherein the input features are from a multi-variate time series data set (“Particular embodiments described herein relates to using a collection of images (or videos),” paragraph 0022 lines 1-2);
encoding, by the processor, the expanded input features (“the encoder 110 may output or generate a latent representation 112a corresponding to a visible patch 106a, a latent representation 112b corresponding to a visible patch 106b, a latent representation 112n corresponding to a visible patch 106n, so on and so forth,” paragraph 0031 lines 14-18);
masking, by the processor, the [] encoding (“Returning to FIG. 1, once the latent representations 112 corresponding to the visible patches 106 are obtained through the encoder 110, mask tokens 114a, 114b, . . . , 114n (also individually and/or collectively herein referred to as 114) corresponding to the masked patches 108 may be added to the latent representations 112,” paragraph 0036 lines 1-6);
decoding, by the processor, the expanded masked encodings, based on a decoder head (“reconstructed patches (e.g., patches 118a, 118b, . . . , 118n) generated by the decoder,” paragraph 0054 lines 9-10);
compressing, by the processor, the decoded masked encodings (“the reconstructed patches and the first subset of patches (e.g., visible patches 106) may be used to generate a reconstructed image, such as image 120 as shown in FIG. 1,” paragraph 0054 lines 12-15);
determining, by the processor, the error of the decoded masked encodings compared to the plurality of input features (“The loss function may compute a mean squared error (MSE) between the reconstructed and original images in the pixel space,” paragraph 0055 lines 6-8); and
updating, by the processor, one or more weights of the decoder head based on the error (“Based on the loss function and/or the comparison, the computing system (e.g., computing system 1000) may update one or more parameters of the encoder and/or the decoder of the first machine-learning model,” paragraph 0055 lines 15-19).
He does not appear to expressly teach a method comprising:
compressing, by the processor, the encoded input features to the initial size; and
expanding, by the processor, the masked encodings from an initial size to n number of phases.
Zhang teaches a method comprising:
compressing, by the processor, the [] input features to the initial size (page 4 figure 2 left side – the RGB/Depth/Thermal channels are merged into a single “Channel Mixing” space);
masking, by the processor, the compressed [features] (page 4 figure 2 left side – the “Channel Mixing” space is combined with a spatial mask); and
expanding, by the processor, the masked encodings from an initial size to n number of phases (page 4 figure 2 right side – the single output from the encoder is split into RGB/Depth/Thermal channels and fed to three separate decoders).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the masking of He to comprise the compression and expansion of Zhang. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely masking the compressed form (Zhang page 4 figure 2). Therefore, the rationale to support a conclusion that the claim would have been obvious is that combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 2, the rejection of claim 1 is incorporated. He/Zhang further teaches a method wherein decoding the expanded masked encodings further comprises correlation encoding, by the processor, the expanded masked encodings, based at least in part on mixing the masked encodings through at least one of the following, intra patch mixing, interpatch mixing (“each patch is randomly dropped with two channels and then shuffled in the channel,” Zhang page 3 section “3.1. Channel-Mixing” lines 8-9), and channel mixing (“The Channel-Mixing module in our approach involves concatenating the channels of visual RGB, Depth, and Thermal modalities (RGBDT),” Zhang page 1 figure 1 caption lines 1-3).
As to dependent claim 3, the rejection of claim 1 is incorporated. He/Zhang further teaches a method wherein encoding the expanded input features further comprises time-series independent (“Particular embodiments described herein relates to using a collection of images (or videos),” He paragraph 0022 lines 1-2) encoding, the expanded masked encodings, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing (“each patch is randomly dropped with two channels and then shuffled in the channel,” Zhang page 3 section “3.1. Channel-Mixing” lines 8-9).
As to independent claim 8, He teaches a computer system for training a phase mixer masked autoencoder model, the computer system comprising:
a processor (“processor 1002,” paragraph 0061 line 4);
a memory in communication with the processor (“processor 1002 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1004, or storage 1006,” paragraph 0061 lines 4-6);
one or more computer program instructions stored on the memory, when executed by the processor, cause the processor to perform one or more operations (“processor 1002 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1004, or storage 1006; decode and execute them,” paragraph 0061 lines 4-7), the operations comprising:
expand a plurality of input features into from an initial size to n number of phases (“Referring back to FIG. 1, in particular embodiments, the input image 102 may be divided into a plurality of patches 104,” paragraph 0030 lines 1-3), wherein the input features are from a multi-variate time series data set (“Particular embodiments described herein relates to using a collection of images (or videos),” paragraph 0022 lines 1-2);
encode the expanded input features (“the encoder 110 may output or generate a latent representation 112a corresponding to a visible patch 106a, a latent representation 112b corresponding to a visible patch 106b, a latent representation 112n corresponding to a visible patch 106n, so on and so forth,” paragraph 0031 lines 14-18);
mask the [] encoding (“Returning to FIG. 1, once the latent representations 112 corresponding to the visible patches 106 are obtained through the encoder 110, mask tokens 114a, 114b, . . . , 114n (also individually and/or collectively herein referred to as 114) corresponding to the masked patches 108 may be added to the latent representations 112,” paragraph 0036 lines 1-6);
decode the expanded masked encodings, based on a decoder head (“reconstructed patches (e.g., patches 118a, 118b, . . . , 118n) generated by the decoder,” paragraph 0054 lines 9-10);
compressing, by the processor, the decoded masked encodings (“the reconstructed patches and the first subset of patches (e.g., visible patches 106) may be used to generate a reconstructed image, such as image 120 as shown in FIG. 1,” paragraph 0054 lines 12-15);
determine the error of the decoded masked encodings compared to the plurality of input features (“The loss function may compute a mean squared error (MSE) between the reconstructed and original images in the pixel space,” paragraph 0055 lines 6-8); and
update one or more weights of the decoder head based on the error (“Based on the loss function and/or the comparison, the computing system (e.g., computing system 1000) may update one or more parameters of the encoder and/or the decoder of the first machine-learning model,” paragraph 0055 lines 15-19).
He does not appear to expressly teach instructions to:
compress the encoded input features to the initial size; and
expand the masked encodings from an initial size to n number of phases.
Zhang teaches instructions to:
compress the [] input features to the initial size (page 4 figure 2 left side – the RGB/Depth/Thermal channels are merged into a single “Channel Mixing” space);
mask the compressed [features] (page 4 figure 2 left side – the “Channel Mixing” space is combined with a spatial mask); and
expand the masked encodings from an initial size to n number of phases (page 4 figure 2 right side – the single output from the encoder is split into RGB/Depth/Thermal channels and fed to three separate decoders).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the masking of He to comprise the compression and expansion of Zhang. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely masking the compressed form (Zhang page 4 figure 2). Therefore, the rationale to support a conclusion that the claim would have been obvious is that combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 9, the rejection of claim 8 is incorporated. He/Zhang further teaches a system wherein decoding the expanded masked encodings further comprises operations to correlation encode the expanded masked encodings, based at least in part on mixing the masked encodings through at least one of the following, intra patch mixing, interpatch mixing (“each patch is randomly dropped with two channels and then shuffled in the channel,” Zhang page 3 section “3.1. Channel-Mixing” lines 8-9), and channel mixing (“The Channel-Mixing module in our approach involves concatenating the channels of visual RGB, Depth, and Thermal modalities (RGBDT),” Zhang page 1 figure 1 caption lines 1-3).
As to dependent claim 10, the rejection of claim 8 is incorporated. He/Zhang further teaches a system wherein encoding the expanded input features further comprises operations to time-series independent (“Particular embodiments described herein relates to using a collection of images (or videos),” He paragraph 0022 lines 1-2) encode, the expanded masked encodings, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing (“each patch is randomly dropped with two channels and then shuffled in the channel,” Zhang page 3 section “3.1. Channel-Mixing” lines 8-9).
As to independent claim 15, He teaches a computer program product for training a phase mixer masked autoencoder model, the computer program product comprising:
program instructions stored on a memory device, executable by a processor to perform one or more operations (“processor 1002 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1004, or storage 1006; decode and execute them,” paragraph 0061 lines 4-7), where in the program instructions comprise instructions to:
expand a plurality of input features into from an initial size to n number of phases (“Referring back to FIG. 1, in particular embodiments, the input image 102 may be divided into a plurality of patches 104,” paragraph 0030 lines 1-3), wherein the input features are from a multi-variate time series data set (“Particular embodiments described herein relates to using a collection of images (or videos),” paragraph 0022 lines 1-2);
encode the expanded input features (“the encoder 110 may output or generate a latent representation 112a corresponding to a visible patch 106a, a latent representation 112b corresponding to a visible patch 106b, a latent representation 112n corresponding to a visible patch 106n, so on and so forth,” paragraph 0031 lines 14-18);
mask the [] encoding (“Returning to FIG. 1, once the latent representations 112 corresponding to the visible patches 106 are obtained through the encoder 110, mask tokens 114a, 114b, . . . , 114n (also individually and/or collectively herein referred to as 114) corresponding to the masked patches 108 may be added to the latent representations 112,” paragraph 0036 lines 1-6);
decode the expanded masked encodings, based on a decoder head (“reconstructed patches (e.g., patches 118a, 118b, . . . , 118n) generated by the decoder,” paragraph 0054 lines 9-10);
compressing, by the processor, the decoded masked encodings (“the reconstructed patches and the first subset of patches (e.g., visible patches 106) may be used to generate a reconstructed image, such as image 120 as shown in FIG. 1,” paragraph 0054 lines 12-15);
determine the error of the decoded masked encodings compared to the plurality of input features (“The loss function may compute a mean squared error (MSE) between the reconstructed and original images in the pixel space,” paragraph 0055 lines 6-8); and
update one or more weights of the decoder head based on the error (“Based on the loss function and/or the comparison, the computing system (e.g., computing system 1000) may update one or more parameters of the encoder and/or the decoder of the first machine-learning model,” paragraph 0055 lines 15-19).
He does not appear to expressly teach instructions to:
compress the encoded input features to the initial size; and
expand the masked encodings from an initial size to n number of phases.
Zhang teaches instructions to:
compress the [] input features to the initial size (page 4 figure 2 left side – the RGB/Depth/Thermal channels are merged into a single “Channel Mixing” space);
mask the compressed [features] (page 4 figure 2 left side – the “Channel Mixing” space is combined with a spatial mask); and
expand the masked encodings from an initial size to n number of phases (page 4 figure 2 right side – the single output from the encoder is split into RGB/Depth/Thermal channels and fed to three separate decoders).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the masking of He to comprise the compression and expansion of Zhang. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely masking the compressed form (Zhang page 4 figure 2). Therefore, the rationale to support a conclusion that the claim would have been obvious is that combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 16, the rejection of claim 15 is incorporated. He/Zhang further teaches a computer program product wherein decoding the expanded masked encodings further comprises program instructions to correlation encode the expanded masked encodings, based at least in part on mixing the masked encodings through at least one of the following, intra patch mixing, interpatch mixing (“each patch is randomly dropped with two channels and then shuffled in the channel,” Zhang page 3 section “3.1. Channel-Mixing” lines 8-9), and channel mixing (“The Channel-Mixing module in our approach involves concatenating the channels of visual RGB, Depth, and Thermal modalities (RGBDT),” Zhang page 1 figure 1 caption lines 1-3).
As to dependent claim 17, the rejection of claim 15 is incorporated. He/Zhang further teaches a computer program product wherein encoding the expanded input features further comprises program instructions to time-series independent (“Particular embodiments described herein relates to using a collection of images (or videos),” He paragraph 0022 lines 1-2) encode, the expanded masked encodings, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing (“each patch is randomly dropped with two channels and then shuffled in the channel,” Zhang page 3 section “3.1. Channel-Mixing” lines 8-9).
Claims 4-7, 11-14, and 18-20 are rejected under 35 U.S.C. § 103 as being unpatentable over He in view of Zhang and Cella et al. (US 2020/0225655 A1, hereinafter Cella).
As to dependent claim 4, the rejection of claim 1 is incorporated.
He/Zhang does not appear to expressly teach a method comprising:
inputting, by the processor, an active multi-variate time series into the updated decoder head;
predicting, by the processor, a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.
Cella teaches a method comprising:
inputting, by the processor, an active multi-variate time series into the updated decoder head (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” paragraph 0962 lines 17-19);
predicting, by the processor, a future state variable value for the active multi-variate time series (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” paragraph 1168 lines 21-36), based at least in part on the updated decoder head (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” paragraph 0962 lines 17-19).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the autoencoder of He/Zhang to comprise the active multi-variate time series of Cella. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely predicting an active multi-variate time series (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36). Therefore, the rationale to support a conclusion that the claim would have been obvious is that combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 5, the rejection of claim 4 is incorporated. He/Zhang/Cella further teaches a method wherein predicting the future state variable value comprises:
encoding (“the encoder 110 may output or generate a latent representation 112a corresponding to a visible patch 106a, a latent representation 112b corresponding to a visible patch 106b, a latent representation 112n corresponding to a visible patch 106n, so on and so forth,” He paragraph 0031 lines 14-18), by the processor, the active multi-variate time series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19);
expanding, by the processor, active multi-variate time series encodings from an initial size to n number of phases (“Referring back to FIG. 1, in particular embodiments, the input image 102 may be divided into a plurality of patches 104,” He paragraph 0030 lines 1-3);
decoding, by the processor, the expanded active multi-variate time series encodings, based on a decoder head (“reconstructed patches (e.g., patches 118a, 118b, . . . , 118n) generated by the decoder,” He paragraph 0054 lines 9-10);
compressing, by the processor, the decoded active multi-variate time series encodings (“the reconstructed patches and the first subset of patches (e.g., visible patches 106) may be used to generate a reconstructed image, such as image 120 as shown in FIG. 1,” He paragraph 0054 lines 12-15);
generating, by the processor, the state variable value from the decoded active multi-variate time series encodings (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36) based on a linear prediction head (“Currently TSFM include a patch time-series transformer model and a time-series mixer model. The PatchTST and the TSMixer use a transformer backbone and a simple linear head,” Applicant admitted prior art, specification paragraph 0013 lines 1-2).
As to dependent claim 6, the rejection of claim 4 is incorporated. He/Zhang/Cella further teaches a method wherein the active multivariate time-series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19) is associated with forecasting (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36) temperature and pressure (“the platform 100 may include the local data collection system 102 deployed in the environment 104 to monitor signals from heavy industrial equipment and processes including monitoring one or more sensors. By way of this example, sensors may be devices that may be used to detect or respond to some type of input from a physical environment, such as an electrical, heat, or optical signal. In embodiments, the local data collection system 102 may include multiple sensors such as, without limitation, a temperature sensor, a pressure sensor, a torque sensor, a flow sensor, a heat sensor, a smoke sensor, an arc sensor, a radiation sensor, a position sensor, an acceleration sensor, a strain sensor, a pressure cycle sensor, a pressure sensor, an air temperature sensor, and the like,” Cella paragraph 0352 lines 14) of a natural gas production system (“the sensor kit 29700 may be deployed in an oil or natural gas fields, off-shore oil rigs, mines, oil or gas pipelines, solar fields, wind farms, hydroelectric power stations, and the like,” Cella paragraph 2380 lines 9-15).
As to dependent claim 7, the rejection of claim 4 is incorporated. He/Zhang/Cella further teaches a method wherein the active multivariate time-series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19) is associated with chemical impurity output levels (“An example system characterization value 11030 includes a predicted outcome for a process associated with the industrial system—for example a product quality description, a product quantity description, a product variability description (e.g., the expected variability of a product parameter predicted according to the operating conditions of the system), a product yield description, a net present value (NPV) for a process, a process completion time, a process chance of completion success, and/or a product purity result,” Cella paragraph 1168 lines 3-11).
As to dependent claim 11, the rejection of claim 8 is incorporated.
He/Zhang does not appear to expressly teach a system comprising operations to:
input an active multi-variate time series into the updated decoder head;
predict a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.
Cella teaches a system comprising operations to:
input an active multi-variate time series into the updated decoder head (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” paragraph 0962 lines 17-19);
predict a future state variable value for the active multi-variate time series (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” paragraph 1168 lines 21-36), based at least in part on the updated decoder head (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” paragraph 0962 lines 17-19).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the autoencoder of He/Zhang to comprise the active multi-variate time series of Cella. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely predicting an active multi-variate time series (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36). Therefore, the rationale to support a conclusion that the claim would have been obvious is that combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 12, the rejection of claim 11 is incorporated. He/Zhang/Cella further teaches a system wherein predicting the future state variable value comprises operations to:
encode (“the encoder 110 may output or generate a latent representation 112a corresponding to a visible patch 106a, a latent representation 112b corresponding to a visible patch 106b, a latent representation 112n corresponding to a visible patch 106n, so on and so forth,” He paragraph 0031 lines 14-18) the active multi-variate time series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19);
expand active multi-variate time series encodings from an initial size to n number of phases (“Referring back to FIG. 1, in particular embodiments, the input image 102 may be divided into a plurality of patches 104,” He paragraph 0030 lines 1-3);
decode the expanded active multi-variate time series encodings, based on a decoder head (“reconstructed patches (e.g., patches 118a, 118b, . . . , 118n) generated by the decoder,” He paragraph 0054 lines 9-10);
compress the decoded active multi-variate time series encodings (“the reconstructed patches and the first subset of patches (e.g., visible patches 106) may be used to generate a reconstructed image, such as image 120 as shown in FIG. 1,” He paragraph 0054 lines 12-15);
generate the state variable value from the decoded active multi-variate time series encodings (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36) based on a linear prediction head (“Currently TSFM include a patch time-series transformer model and a time-series mixer model. The PatchTST and the TSMixer use a transformer backbone and a simple linear head,” Applicant admitted prior art, specification paragraph 0013 lines 1-2).
As to dependent claim 13, the rejection of claim 12 is incorporated. He/Zhang/Cella further teaches a system wherein the active multivariate time-series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19) is associated with forecasting (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36) temperature and pressure (“the platform 100 may include the local data collection system 102 deployed in the environment 104 to monitor signals from heavy industrial equipment and processes including monitoring one or more sensors. By way of this example, sensors may be devices that may be used to detect or respond to some type of input from a physical environment, such as an electrical, heat, or optical signal. In embodiments, the local data collection system 102 may include multiple sensors such as, without limitation, a temperature sensor, a pressure sensor, a torque sensor, a flow sensor, a heat sensor, a smoke sensor, an arc sensor, a radiation sensor, a position sensor, an acceleration sensor, a strain sensor, a pressure cycle sensor, a pressure sensor, an air temperature sensor, and the like,” Cella paragraph 0352 lines 14) of a natural gas production system (“the sensor kit 29700 may be deployed in an oil or natural gas fields, off-shore oil rigs, mines, oil or gas pipelines, solar fields, wind farms, hydroelectric power stations, and the like,” Cella paragraph 2380 lines 9-15).
As to dependent claim 14, the rejection of claim 12 is incorporated. He/Zhang/Cella further teaches a system wherein the active multivariate time-series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19) is associated with chemical impurity output levels (“An example system characterization value 11030 includes a predicted outcome for a process associated with the industrial system—for example a product quality description, a product quantity description, a product variability description (e.g., the expected variability of a product parameter predicted according to the operating conditions of the system), a product yield description, a net present value (NPV) for a process, a process completion time, a process chance of completion success, and/or a product purity result,” Cella paragraph 1168 lines 3-11).
As to dependent claim 18, the rejection of claim 15 is incorporated.
He/Zhang does not appear to expressly teach a computer program product comprising program instruction to:
input an active multi-variate time series into the updated decoder head;
predict a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.
Cella teaches a computer program product comprising program instruction to:
input an active multi-variate time series into the updated decoder head (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” paragraph 0962 lines 17-19);
predict a future state variable value for the active multi-variate time series (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” paragraph 1168 lines 21-36), based at least in part on the updated decoder head (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” paragraph 0962 lines 17-19).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the autoencoder of He/Zhang to comprise the active multi-variate time series of Cella. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely predicting an active multi-variate time series (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36). Therefore, the rationale to support a conclusion that the claim would have been obvious is that combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 19, the rejection of claim 18 is incorporated. He/Zhang/Cella further teaches a computer program product wherein predicting the future state variable value comprises program instructions to:
encode (“the encoder 110 may output or generate a latent representation 112a corresponding to a visible patch 106a, a latent representation 112b corresponding to a visible patch 106b, a latent representation 112n corresponding to a visible patch 106n, so on and so forth,” He paragraph 0031 lines 14-18) the active multi-variate time series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19);
expand active multi-variate time series encodings from an initial size to n number of phases (“Referring back to FIG. 1, in particular embodiments, the input image 102 may be divided into a plurality of patches 104,” He paragraph 0030 lines 1-3);
decode the expanded active multi-variate time series encodings, based on a decoder head (“reconstructed patches (e.g., patches 118a, 118b, . . . , 118n) generated by the decoder,” He paragraph 0054 lines 9-10);
compress the decoded active multi-variate time series encodings (“the reconstructed patches and the first subset of patches (e.g., visible patches 106) may be used to generate a reconstructed image, such as image 120 as shown in FIG. 1,” He paragraph 0054 lines 12-15);
generate the state variable value from the decoded active multi-variate time series encodings (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36) based on a linear prediction head (“Currently TSFM include a patch time-series transformer model and a time-series mixer model. The PatchTST and the TSMixer use a transformer backbone and a simple linear head,” Applicant admitted prior art, specification paragraph 0013 lines 1-2).
As to dependent claim 20, the rejection of claim 19 is incorporated. He/Zhang/Cella further teaches a computer program product wherein the active multivariate time-series (“The streaming data collection system 10150 may include an auto encoder neural network 10152,” Cella paragraph 0962 lines 17-19) is associated with forecasting (“An example system characterization value 11030 includes a predicted future state for a process associated with the industrial system—for example an operating temperature at a given future time, an energy consumption value, a volume in a tank, an emitted noise value at a school adjacent to the industrial system, and/or a rotational speed of a pump. The predicted future state may be time based (e.g., at 4 PM on Thursday), based on a state of the process (e.g., during the third stage, during system shutdown, etc.), and/or based on a future state of particular interest (e.g., peak energy consumption, highest temperature value, maximum noise value, time or process stage when a maximum number of personnel will be within 50 feet of a sensitive area, time or process stage when an aspect of the system redundancy is at a lowest point—e.g., for determining high risk points in a process, etc.),” Cella paragraph 1168 lines 21-36) temperature and pressure (“the platform 100 may include the local data collection system 102 deployed in the environment 104 to monitor signals from heavy industrial equipment and processes including monitoring one or more sensors. By way of this example, sensors may be devices that may be used to detect or respond to some type of input from a physical environment, such as an electrical, heat, or optical signal. In embodiments, the local data collection system 102 may include multiple sensors such as, without limitation, a temperature sensor, a pressure sensor, a torque sensor, a flow sensor, a heat sensor, a smoke sensor, an arc sensor, a radiation sensor, a position sensor, an acceleration sensor, a strain sensor, a pressure cycle sensor, a pressure sensor, an air temperature sensor, and the like,” Cella paragraph 0352 lines 14) of a natural gas production system (“the sensor kit 29700 may be deployed in an oil or natural gas fields, off-shore oil rigs, mines, oil or gas pipelines, solar fields, wind farms, hydroelectric power stations, and the like,” Cella paragraph 2380 lines 9-15).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure:
US 2023/0376729 A1 disclosing “masking the latent representation and supplying the masked latent representation to the decoder to generate the decoder output” (paragraph 0009 lines 2-4)
Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
In the interests of compact prosecution, Applicant is invited to contact the examiner via electronic media pursuant to USPTO policy outlined MPEP § 502.03. All electronic communication must be authorized in writing. Applicant may wish to file an Internet Communications Authorization Form PTO/SB/439. Applicant may wish to request an interview using the Interview Practice website: http://www.uspto.gov/patent/laws-and-regulations/interview-practice.
Applicant is reminded Internet e-mail may not be used for communication for matters under 35 U.S.C. § 132 or which otherwise require a signature. A reply to an Office action may NOT be communicated by Applicant to the USPTO via Internet e-mail. If such a reply is submitted by Applicant via Internet e-mail, a paper copy will be placed in the appropriate patent application file with an indication that the reply is NOT ENTERED. See MPEP § 502.03(II).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan Barrett whose telephone number is 571 270 3311. The examiner can normally be reached 9:00am to 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Michelle Bechtold can be reached at 571 431 0762. The fax phone number for the organization where this application or proceeding is assigned is 571 273 8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ryan Barrett/
Primary Examiner, Art Unit 2148
Prosecution Insights