DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013 is being examined under the first inventor to file provisions of the AIA .
Status of the Application
This action is a first action on the merits in response to the application filed on 06/03/2024.
Status of Claims
Claims 1-20 filed on 06/03/2024 are currently pending and have been examined in this application.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 06/03/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. Specifically, claims 1-20 are directed to an abstract idea without additional elements to integrate the claims into a practical application or to amount to significantly more than the abstract idea.
Claims 1-20 even if the claims were directed to a process, machine, or manufacture (Step 1), however the claims are directed to the abstract mathematical concept Mental Process because it manipulates data using general-purpose computer processors.
With respect to Step 2A Prong One of the frameworks, claim 1 recites an abstract idea. Claim 1 includes limitations for “A method for training a multi-level mixer masked autoencoder with channel mixing across group dimensions model, the method comprising: receiving a plurality of input features; expanding an input encoding from a time-series independent encoder; correlation encoding the expanded encodings, based at least in part on mixing the encodings through a least one of the following, channel mixing, spatial mixing, and channel mixing compressing the correlation encodings; decoding the correlation encodings, based on a decoder head; determining the error of the decoded correlation encodings compared to the plurality of input features; and updating one or more weights of the decoder head based on the error”
The limitations above recite an abstract idea under Step 2A Prong One. More particularly, the limitations above recite mathematical concept and Mental Process as mentioned above. As a result, claim 1 recites an abstract idea under Step 2A Prong One.
Claims 8 and 15 recite substantially similar limitations to those presented with respect to claim 1. As a result, claims 8 and 15 recite an abstract idea under Step 2A Prong One for the same reasons as stated above with respect to claim 1. Similarly, claims 2-7, 9-14, and 16-20 recite a Mental Process and mathematical concept as explained above. As a result, claims 2-7, 9-14, and 16-20 recite an abstract idea under Step 2A Prong One.
With respect to Step 2A Prong Two of the framework, claim 1 does not include additional elements that integrate the abstract idea into a practical application. Claim 1 includes additional elements that do not recite an abstract idea. The additional elements of claim 1 include “computer-implemented” and “by a processor”. When considered in view of the claim as a whole, the step of “receiving” does not integrate the abstract idea into a practical application because “receiving” is an insignificant extra solution activity to the judicial exception. When considered in view of the claim as a whole, the recited computer elements do not integrate the abstract idea into a practical application because the computer elements are generic computer elements that are merely used as a tool to perform the recited abstract idea. As set forth in the 2019 Eligibility Guidance, 84 Fed. Reg. at 55 “merely include[ing] instructions to implement an abstract idea on a computer” is an example of when an abstract idea has not been integrated into a practical application. Therefore, the claim is directed to an abstract idea.
As a result, claim 1 does not include additional elements that integrate the abstract idea into a practical application under Step 2A Prong Two.
As noted above, claims 8 and 15 recite substantially similar limitations to those recited with respect to claim 1. Although claim 8 further recites “A computer system” and claim 15 further recites “A computer program”, when considered in view of the claim as a whole, the recited computer elements do not integrate the abstract idea into a practical application because the computer elements are generic computer elements that are merely used as a tool to perform the recited abstract idea. As a result, claims 8 and 15 do not include additional elements that integrate the abstract idea into a practical application under Step 2A Prong Two.
Claims 2-7, 9-14, and 16-20 do not include any additional elements beyond those recited by independent claims 1, 8, and 15. As a result, claims 2-7, 9-14, and 16-20 do not include additional elements that integrate the abstract idea into a practical application under Step 2A Prong Two.
With respect to Step 2B of the framework, claim 1 does not include additional elements amounting to significantly more than the abstract idea. As noted above, claim 1 includes additional elements that do not recite an abstract idea. The additional elements of claim 1 include “computer-implemented” and “by a processor”. The step of “receiving” does not amount to significantly more than the abstract idea because “receiving” is well-understood, routine, and conventional computer function in view of MPEP 2106.05(d)(ll). The recited computer elements do not amount to significantly more than the abstract idea because the computer elements are generic computer elements that are merely used as a tool to perform the recited abstract idea. As a result, claim 1 does not include additional elements that amount to significantly more than the abstract idea under Step 2B.
As noted above, claims 8 and 15 recite substantially similar limitations to those recited with respect to claim 1. Although claim 8 further recites “A computer system” and claim 15 further recites “A computer program”, the recited computer elements do not amount to significantly more than the abstract idea because the computer elements are generic computer elements that are merely used as a tool to perform the recited abstract idea. Further, looking at the additional elements as an ordered combination adds nothing that is not already present when considering the additional elements individually. As a result, claims 8 and 15 do not include additional elements that amount to significantly more than the abstract idea under Step 2B.
Claims 2-7, 9-14, and 16-20 do not include any additional elements beyond those recited by independent claims 1, 8, and 15. As a result, claims 2-7, 9-14, and 16-20 do not include additional elements that amount to significantly more than the abstract idea under Step 2B.
Therefore, the claims are directed to an abstract idea without additional elements amounting to significantly more than the abstract idea. Accordingly, claims 1-20 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the claims are directed to software per se. Applicant has claimed “A computer program product” with the broadest reasonable interpretation, since the computer program product does not comprise physical elements, it can be interpreted as software elements, i.e. printed matter. Printed matter is not statutory subject matter under 35 USC 101. As a result, this claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007). In order to overcome this rejection under 35 U.S.C. 101.
Claims 16-20 are rejected under 35 U.S.C. 101 because they depend from claim 15 and therefore inherent the same rejection for the same reasons mentioned (above).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claims 1-20 are rejected under 35 U.S.C. 103 as being un-patentable over Lee et al. “LEARNING TO EMBED TIME SERIES PATCHES INDEPENDENTLY,” arXiv repository on December 26, 2023, hereinafter Lee in view of Si-An Chen et al. “TSMixer: An All-MLP Architecture for Time Series Forecasting” Published in Transactions on Machine Learning Research (09/2023) and in further view of He et al. “Masked Autoencoders Are Scalable Vision Learners” arXiv December 19, 2021
Regarding claim 1. Lee teaches A computer-implemented method for training a multi-level mixer masked autoencoder with channel mixing across group dimensions model, the computer-implemented method comprising: receiving, by the processor, a plurality of input features; expanding, by a processor, an input encoding from a time-series independent encoder; [Lee, Introduction, Lee teaches “Masked time series modeling (MTM) task partially masks out TS and predicts the masked parts from the unmasked parts using encoders capturing dependencies among the patches, such as Transformers (Zerveas et al., 2021; Nie et al., 2023). However, we argue that learning such dependencies among patches, e.g., predicting the unmasked parts based on the masked parts and utilizing architectures capturing dependencies among the patches, might not be necessary for representation learning.” wherein Lee teaches time-series representation learning inherently teaches segmenting multivariate time series into patches and passing them through an independent encoder]
Lee does not specifically teach, however, Chen teaches correlation encoding, by the processor, the expanded encodings, based at least in part on mixing the encodings through a least one of the following, channel mixing, spatial mixing, and channel mixing [Chen, page 3, Chen teaches “The resulting TSMixer alternatively applies MLPs across time and feature dimensions, conceptually corresponding to time-mixing and feature-mixing operations, efficiently capturing both temporal patterns and cross-variate information, as illustrated in Fig. 1. The residual designs ensure that TSMixer retains the capacity of temporal linear models while still being able to exploit cross-variate information.” wherein Chen use of MLP-Mixer architectures to replace attention layers for time series by interleaving spatial (or temporal/patch) mixing and channel mixing]
Lee and Chen are in the same field of endeavor as the claimed invention of managing machine learning system designed to analyze complex time-series data and predict future events. It would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to modify/combine converting the data into an abstract format (encoding) of Lee with mixing information across different channels (variables) and spaces (time steps or patches) to find hidden correlations and patterns of Chen since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, with the predictable results of training a specialized deep learning architecture, Masked Autoencoder (MAE) with an integrated MLP-Mixer.
Lee in view of Chen does not specifically teach, however, He teaches compressing, by the processor, the correlation encodings; decoding, by the processor, the correlation encodings, based on a decoder head; determining, by the processor, the error of the decoded correlation encodings compared to the plurality of input features; and updating, by the processor, one or more weights of the decoder head based on the error [He teaches the basic mechanics of masking input features, running them through an encoder-decoder architecture, calculating a reconstruction loss/error, and performing backpropagation to update weights. See He figure 2 and page 4 “Our loss function computes the mean squared error (MSE) between the reconstructed and original images in the pixel space. We compute the loss only on masked patches”. See also weight decay in Table 10]
Lee and He are in the same field of endeavor as the claimed invention of managing machine learning system designed to analyze complex time-series data and predict future events. It would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to modify/combine converting the data into an abstract format (encoding) of Lee and mixing information across different channels (variables) and spaces (time steps or patches) to find hidden correlations and patterns of Chen with masking input features, running them through an encoder-decoder architecture, calculating a reconstruction error of He since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, with the predictable results of training a specialized deep learning architecture, Masked Autoencoder (MAE) with an integrated MLP-Mixer.
Regarding claim 2. Lee in view of Chen and He teaches all of the limitations of claim 1 (as above). Further, Lee teaches wherein the decoder head is comprised of a thin decoder with mixer architecture and a linear prediction head [Lee, page 10, Lee teaches “we foresee that the concept of utilizing lightweight architectures will serve as a source of inspiration for future endeavors across domains where substantial computational resources are not readily accessible” wherein a "thin" (lightweight) network]
Regarding claim 3. Lee in view of Chen and He teaches all of the limitations of claim 1 (as above). Further, Lee teaches wherein encoding the expanded input features further comprises: expanding the time-scries independent encoding, the, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing [Lee, page 3, Lee teaches “while our proposed CL operates with patches within a single TS” wherein interpatch mixing. Further, Lee’ Abstract teaches “Transformers to capture the dependencies between patches by predicting masked patches from unmasked patches” wherein dependencies between patches/ intra patch].
Regarding claims 4-5. Lee in view of Chen and He teaches all of the limitations of claim 1 (as above). further comprising: inputting, by the processor, an active multi-variate time series into the updated decoder head; and predicting, by the processor, a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head. Further wherein predicting the future state variable value comprises: encoding, by the processor, the active multi-variate time series; expanding, by the processor, active multi-variate time series encodings from an initial size to n number of phases; decoding, by the processor, the expanded active multi-variate time series encodings, based on a decoder head; compressing, by the processor, the decoded active multi-variate time series encodings; and generating, by the processor, the state variable value from the decoded active multi-variate time series encodings based on a linear prediction head [predicting a future state variable value for the active multi-variate time series is a standard practices for foundation models and masked autoencoders involve taking the pre-trained encoder/decoder, feeding active unseen data, and replacing or leveraging the head for forecasting. Expanding... encodings from an initial size to n number of phase, changing dimension size or reshaping temporal/channel variables to match forecasting windows (phases) is a standard element of sequence mapping in deep learning.
Lee in view of He does not specifically teach, however, Chen teaches Generating the state variable value based on a linear prediction head [Chen, page 20, Chen teaches “OK, is then linearly projected to the prediction space, which can be real values or the parameters of a probability distribution” wherein Chen use bypass complex recurrent/attention mechanics by using direct linear mapping heads to map historical encodings directly to future time steps]
It would have been obvious for one having ordinary skill in the art before the effective filing date of the claimed invention to modify/combine converting the data into an abstract format (encoding) of Lee with generating the state variable value based on a linear prediction head of Chen since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, with the predictable results of training a specialized deep learning architecture, Masked Autoencoder (MAE) with an integrated MLP-Mixer.
Regarding claims 6-7. Lee in view of Chen and He teaches all of the limitations of claim 1 (as above). Further, Wherein the active multivariate time-series is associated with natural gas production system pressures. wherein the active multivariate time-series is associated with chemical impurity output levels [Applying a generic multivariate time-series forecasting model to specific telemetry fields such as system pressures or chemical levels represents a purely non-functional limitation. See MPEP 2111.05].
Regarding claim 8, the claim recites analogous limitations to claim 1 above, and is therefore rejected on the same premise. Claim 1 is a method claim while claim 8 is directed to a system which is anticipated by Lee introduction.
Regarding claims 9-14, claims 9-14 recite substantially similar limitations as claim 2-7, respectively; therefore, claims 9-14 are rejected with the same rationale, reasoning, and motivation provided above for claims 2-7, respectively. Claims 2-7 are method claims while claims 9-14 are directed to a system which is anticipated by Lee introduction.
Regarding claim 15, the claim recites analogous limitations to claim 1 above, and is therefore rejected on the same premise. Claim 1 is a method claim while claim 15 is directed to a computer program product which is anticipated by Lee introduction.
Regarding claims 16-20, claims 16-20 recite substantially similar limitations as claim 2-6, respectively; therefore, claims 16-20 are rejected with the same rationale, reasoning, and motivation provided above for claims 2-6, respectively. Claims 2-6 are method claims while claims 16-20 are directed to a a computer program product which is anticipated by Lee introduction.
Conclusion
The following prior art made of record and not relied upon are considered pertinent to applicant's disclosure. Ando et al. (US 20240289615 A1) teaches a neural network update device comprising a processor comprising hardware, the processor being configured to: with respect to a plurality of output data obtained as a result of inputting a plurality of training data into a neural network, compare the plurality of output data with a plurality of pieces of correct answer information associated with the plurality of training data, to calculate a loss value for each of the plurality of output data; select, among the plurality of output data, relevant output data, the loss value for which meets a predetermined reference, and irrelevant output data, the loss value for which does not meet the predetermined reference; and create processed correct answer information by processing the correct answer information compared with the relevant output data, compare the relevant output data with the processed correct answer information to output a processed loss value, and update the neural network by using the processed loss value, or create processed training data by processing the training data associated with the relevant output data, input the processed training data into the neural network, to cause the neural network to output processed output data obtained as a result of classifying the processed training data, compare the processed output data with the correct answer information associated with the relevant output data, to output a processed loss value, and update the neural network by using the processed loss value.
Any inquiry concerning this communication from the examiner should be directed to Abdallah El-Hagehassan whose contact information is (571) 272-0819 and Abdallah.el-hagehassan@uspto.gov The examiner can normally be reached on Monday- Friday 8 am to 5 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on (571) 272-6045. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-3734.
Information regarding the status of an application may be obtained from the patent application information retrieval (PAIR) system. Status information of published applications may be obtained from either private PAIR or public PAIR. Status information of unpublished applications is available through private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have any questions on access to the private PAIR system, contact the electronic business center (EBC) at (866) 271-9197 (toll-free). If you would like assistance from a USPTO customer service representative or access to the automated information system, call (800) 786-9199 (in US or Canada) or (571) 272-1000.
/ABDALLAH A EL-HAGE HASSAN/
Primary Examiner, Art Unit 3623
Prosecution Insights