DETAILED ACTION
Claims 1-20 are pending in the Instant Application.
Claims 1-20 are rejected (Non-Final Rejection).
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The Instant Application, filed 12/28/2023, claims foreign priority to 23383320.1, EP filed 12/19/2023. Thus, the earliest effective filing date is 12/19/2023 for what was recited therein.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/19/2024 was considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101.
Claims 10-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because claim 10 recites a system, but does not show any explicit hardware. Without hardware, a system can be implemented as pure software. Since pure software is considered non-statutory, claim 10 is rejected. Claims 11-20 are dependent on claim 10, do not cure the issue, and thus, are also rejected.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Step 1: Claims 1 recites a process, and thus is a statutory category of invention.
Step 2A Prong One: Claim 1 recites receiving input to be removed from a machine learning model, and retraining and compressing that model. There limitations recite mathematical operations performed on data and model parameters. It is the mere manipulation of numerical information. Thus, the claimed are directed to an abstract idea.
Step 2A Prong Two: The judicial exception is not integrated into a practical application. The claim recites the additional elements of an “information processing system,” “computational model,” “retraining module,” and “tensorization module.” These elements merely provide a generic computer environment to perform the abstract idea. There is no improvement to computer functionality, memory usage, processor operation, or even machine learning. The removal of certain elements from the model, and the compression of the model are improvements on the model, but not improvements to computer technology. Therefore, these additional elements do not integrate the abstract idea into practical application.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Dependent claims 2-6 are rejected for the same reasons as claim 1, because they merely add limitations specifying the type of computational models, and do not add a technological improvement or are significantly more than the abstract idea.
Dependent claim 7 limits the mathematical structures to tensor networks. Tensor networks are mathematical representations and would just further the abstract idea from claim 1. Claims 8 and 9 further limit the tensor networks and would also just further the abstract idea.
Therefore, claims 1-9 are rejected as being directed to an abstract idea without significantly more.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 7-9 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor regards as the invention.
Claims 7 and 9 recite the limitation "the mathematical structures”" in line 1. There is insufficient antecedent basis for this limitation in the claim. Claim 8 is rejected for being dependent on claims 7 and does not cure the issue.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8, 10-17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Mele, United States Patent Application Publication No. 2023/0252336, in view of Novikov et al. (“Novikov”), Tensorizing neural networks, 2015.
As per claim 1, Mele discloses a computer-implemented process, the process including the following steps comprising: receiving input for a computational model in an information processing system, wherein the input is data that is to be removed ([0406] wherein a bias is input to be removed); retraining the computational model using a retraining module, wherein the retraining results in an uncorrelated output ([0406] wherein the model is retrained using a low bias training set, or adjust parameters); but does not disclose compressing the computational model using a tensorization module, wherein the compression uses tensor networks. However, Novikov teaches compressing the computational model using a tensorization module ([Page 7] wherein Tensor compression results are described), wherein the compression uses tensor networks ([Page 4, 4 TT-Layer] wherein a tensor network is described as a neural network with one or more TT layers).
Both Mele and Novikov both describe altering a neural network. One could apply the compression from Novikov with the relearning in Mele to teach the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the method of retraining to unlearn in Mele with the compression in Novikov in order to save space when storing a machine learning model.
As per claim 2, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 1. Mele further discloses wherein the computational model is a layered computational model ([0175] wherein a convolutional neural network is described, which is a layered computational model).
As er claim 3, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 2. Mele further discloses wherein the layered computational model is a layered computational model with convolutional operations ([0175] wherein a convolutional neural network is described, which has convolutional operations).
As per claim 4. note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 1. Mele further discloses wherein the computational model is a computational model for language processing ([0185] wherein language processing is described).
As per claim 5, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 1. Mele further discloses wherein the computational model is a binary classification model ([0403] wherein a binary classification is described).
As per claim 6, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 1. Novikov further teaches wherein the computational model is a prediction model ([Page 1, 1. Introduction} wherein an image classification model is described which is a prediction model).
As per claim 7, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 1. Novikov further teaches wherein the mathematical structures are tensor networks ([Page 4, 4 TT-Layer] wherein a tensor network is described as a neural network with one or more TT layers).
As per claim 8, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 7. Novikov further teaches wherein the tensor networks are used to compress layers of the computational model ([Page 7] wherein Tensor compression results are described).
As per claim 10, Mele discloses an information processing system, comprising: a computational model that processes input data ([0406] wherein a bias is input to be removed);
a retraining module that retrains the computational model to produce an uncorrelated output ([0406] wherein the model is retrained using a low bias training set, or adjust parameters; but does not disclose a tensorization module that compresses the computational model using mathematical structures. However, Novikov teaches a tensorization module that compresses the computational model using mathematical structures ([Page 7] wherein Tensor compression results are described).
Both Mele and Novikov both describe altering a neural network. One could apply the compression from Novikov with the relearning in Mele to teach the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the method of retraining to unlearn in Mele with the compression in Novikov in order to save space when storing a machine learning model.
As per claim 11, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Mele further discloses wherein the computational model is a layered computational model ([0175] wherein a convolutional neural network is described, which is a layered computational model).
As per claim 12, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Mele further discloses wherein the layered computational model is a layered computational model with convolutional operations ([0175] wherein a convolutional neural network is described, which has convolutional operations).
As per claim 13, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Mele further discloses wherein the computational model is a computational model for language processing ([0185] wherein language processing is described).
As per claim 14, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Mele further discloses wherein the computational model is a binary classification model ([0403] wherein a binary classification is described).
As per claim 15, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Novikov further teaches wherein the computational model is a prediction model ([Page 1, 1. Introduction} wherein an image classification model is described which is a prediction model).
As per claim 16, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Novikov further teaches wherein the mathematical structures are tensor networks ([Page 4, 4 TT-Layer] wherein a tensor network is described as a neural network with one or more TT layers).
As per claim 17, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 16. Novikov further teaches wherein the tensor networks are used to compress layers of the computational model ([Page 7] wherein Tensor compression results are described).
As per claim 19, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10. Mele further discloses comprising a user interface for inputting the data to be removed ([0406] wherein a bias is input to be removed).
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mele in view of Novikov in further view of Wang et al. (“Wang”), United States Patent Application Publication No. 2022/0148293.
As per claim 9, note the rejection of claim 1 where Mele and Novikov are combined. The combination teaches the process of claim 1, but does not teach wherein the mathematical structures are tensor networks used to compress layers with convolutional operations and layers with attention mechanism. However, Wang teaches wherein the mathematical structures are tensor networks used to compress layers with convolutional operations and layers with attention mechanism ([0062] wherein attention mechanisms are used).
Both Novikov and Wang describe tensorizing. One could use the attention mechanism in Wang with the tensorizing compression in Novikov to teach the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the method of retraining a model to unlearn an input and compressing the model as in the combination of Mele and Novikov with the attention mechanism in Wang in order to be able to provide importance to certain features and improve the discriminatory ability of the model.
As per claim 18, claim 18 is the system that performs the process of claim 9 and is rejected for the same rationale and reasoning.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Mele in view of Novikov in further view of Parangi et al. (“Parangi”), United States Patent Application Publication No. 2021/0232920, in view of von Elgg et al. (“Elgg”), United States Patent Application Publication No. 2013/0054837.
As per claim 20, note the rejection of claim 10 where Mele and Novikov are combined. The combination teaches the system of claim 10, but does not teach wherein the user interface provides feedback on the progress of the retraining and compression.
However, Parangi teaches wherein the user interface provides feedback on the progress of the training ([0054]), but does not teach wherein the user interface provides feedback on the progress of the compression. However, Elgg teaches wherein the user interface provides feedback on the progress of the compression ([0044] wherein the client can monitor the progress of the compression).
Mele describes retraining a model to erase information from model, but does not expressly describe providing feedback of the progress of the learning/relearning. However, Parangi teaches providing feedback of the progress of the training to the user. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the method of retraining a machine learning model in Mele with the display of the progress of the training/retraining as in Parangi in order to provide the user with an indication of when they can use the model. Novikov and Elgg both describe methods of compression. One could use the method of showing the progress of the compression as in Elgg with the compression in Novikov to teach the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the method of compressing the machine learning model as in Novikov with the display of the progress of compression in Elgg in order to alert the user of the condition of the file.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KANNAN SHANMUGASUNDARAM whose telephone number is (571)270-7763. The examiner can normally be reached M-F 9:00 AM -6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Charles Rones can be reached at (571) 272-4085. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KANNAN SHANMUGASUNDARAM/Primary Examiner, Art Unit 2168
Prosecution Insights