Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This is Non-Final Office Action, in responses to Patent Application filed 07/14/2023. Claim(s) 1-20 are pending. Claim(s) 1, 8 and 15 is/are independent.
In addition, in the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Objections
Claim(s) 18 and 20 are objected to because of the following informalities: Claim 18 as claimed “The system of claim 14” (i.e., claim 14 is a “medium” claim). Also, Claim 20 as claimed “The system of claim 6” (i.e., claim 6 is a “method” claim). Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 fail to recite statutory subject matter, as defined in 35 U.S.C. 101, because: The claimed invention is/are directed to a judicial exception (i.e., abstract idea) without significantly more.
Step 1: YES (Claim(s) is/are process, machine, manufacture or composition of the matter). ... A method, comprising: receiving an application-dependent dataset representative of an application to which an original machine learning model can be applied;
providing, to the original model, each of data samples in the application-dependent dataset so that each node/layer in the original model produces an output vector in response to the data sample;
obtaining, with respect to each of a plurality of nodes of each of multiple layers in the original machine learning model, an aggregated output vector based on the output vectors produced by the each node/layer in response to the data samples, respectively;
computing similarity metrics that measure the similarity of aggregated output vectors of a pair of nodes or a pair of layers in the original machine learning model;
selecting removal candidate nodes and removal candidate layers based on the similarity metrics;
performing loss-based evaluation with respect to each of the removal candidate nodes and each of the removal candidate layers to identify redundant nodes and layers in the original machine learning model;
removing redundant nodes and layers from the original model to generate a compressed model; and deploying the compressed model for the application... and therefore, fall into one of the four categories of patent eligible subject matter (process, machine, manufacture or composition of the matter).
Step 2A : Prong One: ( whether a claim recites a judicial exception ?) the claim(s) recite ... “receiving an application-dependent dataset” representative of an application to which an original “machine learning model can be applied”;
providing, to the original model, each of data samples in the application-“dependent dataset” so that each node/layer in the original model “produces an output vector in response to the data sample”;
“obtaining”, with respect to each of a plurality of nodes of each of multiple layers in the original machine learning model, an “aggregated output vector” based on the output vectors produced by the each node/layer in response to the data samples, respectively;
“computing similarity metrics” that measure the similarity of aggregated output vectors of a pair of nodes or a pair of layers in the original machine learning model;
“selecting removal candidate nodes and removal candidate layers based on the similarity metrics;”
“performing loss-based evaluation” with respect to each of the removal candidate nodes and each of the removal candidate layers to identify redundant nodes and layers in the original machine learning model;
“ removing redundant nodes and layers from the original model to generate a compressed model; and deploying the compressed model for the application”...
These limitation(s) recite mental processes and mathematical calculation...since the similarity metrics that measure the similarity of aggregated output vectors of a pair of nodes or a pair of layers in the original machine learning model... and performing loss-based evaluation... [is a high level mathematical calculation(s)] ...then [APPLY IT] to generate a compressed model; and deploying the compressed model for the application....Thus these limitation(s) recite mental processes and mathematical calculation(s).
--------------Step 2A : Prong Two: (Do the claim(s) recite “additional element(s) that integrate the “Judicial Exception” into “A Practical Application” ? The claim(s) recite additional limitation(s) such as applied the “original machine learning model” and “selecting removal candidate nodes and removal candidate layers based on the similarity metrics”... to generate a compressed model; and deploying the compressed model for the application...it is noted, the improvement in the abstract idea itself ... but do not integrate the judicial exception into a practical application, i.e., applied the “original machine learning model” and “selecting removal candidate nodes and removal candidate layers based on the similarity metrics”... to generate a compressed model; and deploying the compressed model for the application... using the system/computer program product.
These limitation(s) only recite a generic computer component(s) that only amounts to mere instructions to implement the abstract idea on a computer, and therefore, do not integrate the judicial exception into a practical application. (MPEP 2106.04(d), 2106.05(f)).
Step 2B: (Whether a Claim Amounts to Significantly More) ? The claim(s) recite additional limitation(s) such as ... applied the “original machine learning model” and “selecting removal candidate nodes and removal candidate layers based on the similarity metrics”... to generate a compressed model; and deploying the compressed model for the application using the system/computer program product...These limitation(s) only recite a generic computer component(s) that only amounts to mere instructions to implement the abstract idea on a computer, and therefore, do not amount to significantly more than the abstract idea itself (MPEP 2106.05, 2106.04(d) and 2106.05(f)).
As to the dependent claim(s) 2-7, 9-14 and 16-20, further recite, addition limitation(s) such as, (artificial neural network (ANN), input/output layers/nodes, two aggregated output vectors of a pair of nodes/layers, removal candidate layers/nodes are selected based on the layer level similarity metrics, another layer satisfies a second pre-determined condition, removal candidate from the original model; during the simulation, a first overall loss ... without the removal candidate based on a pre-determined loss function; assessing whether the first overall loss and a second overall loss of the original with the removal candidate therein satisfy a pre-determined condition; designating, if the pre-determined condition is satisfied, the removal candidate as redundant so that it is to be removed from the original model, pre-determined condition not satisfied, etc.,) These limitation(s) only amounts to mere instructions to implement the abstract idea ...and do not include elements that amount to significantly more than the abstract idea and are also rejected under the same rational.
Accordingly, claims 1-20 fail to recite statutory subject matter, as defined in 35 U.S.C. 101.
In addition, Claim(s) 8-14 recite “A machine readable and non-transitory medium”. The Specification in USPGPUB 2005/0021818 A1, paragraph 44, stated, “... a machine-readable medium may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like..”. Thus, the claim is construed broadly as including carrier wave medium or physical transmission medium, etc. Accordingly, Claim 8-14 fails to recite statutory subject matter, as defined in 35 U.S.C. 101.
Also, Claim(s) 15-20, recite “system”. The Specification in USPGPUB 2005/0021818 A1, paragraph(s) 20 and 29 and Fig 3A and 5A, stated, “[0020] FIG. 3Adepicts an exemplary high level system diagram of the model compression pipeline 110,...”, and “[0029] FIG. 5A depicts an exemplary high level system diagram of the loss-based removal candidate determiner 330. ...”. Thus, the claim is construed broadly as a computer “Program”, etc. Accordingly, Claim 15-20 fails to recite statutory subject matter, as defined in 35 U.S.C. 101.
Allowable Subject Matter
Claim(s) 1-20 would be allowable if rewritten and/or amending to remedy the 101 rejection(s) and the claim(s) objections.
Reason for Allowance
Under the broadest reasonable interpretation of the claimed limitation which is consistence with the Applicant's Specification, the prior arts of recorded when taken individually or in combination do not expressly teach or render obvious the limitations recited in claim(s) 1, 8 and 15 when taken in the context of the claims as a whole, especially the concept of, … “receiving an application-dependent dataset representative of an application to which an original machine learning model can be applied; providing, to the original model, each of data samples in the application-dependent dataset so that each node/layer in the original model produces an output vector in response to the data sample; and obtaining, with respect to each of a plurality of nodes of each of multiple layers in the original machine learning model, an aggregated output vector based on the output vectors produced by the each node/layer in response to the data samples, respectively; wherein computing similarity metrics that measure the similarity of aggregated output vectors of a pair of nodes or a pair of layers in the original machine learning model; and selecting removal candidate nodes and removal candidate layers based on the similarity metrics; that is performing loss-based evaluation with respect to each of the removal candidate nodes and each of the removal candidate layers to identify redundant nodes and layers in the original machine learning model; and removing redundant nodes and layers from the original model to generate a compressed model; and deploying the compressed model for the application..”.As claimed and further supports in the specifications PGPUB 20250021818 A1- The Abstract and Para(s) 20-22, 29 and Fig(s). 3A and 5A.
In addition, neither a reference uncovered that would have provided a basis of evidence for asserting a motivation, nor one of ordinary skilled in the art before the effective filing date of the claimed invention, would have combined them to arrive at the present invention as recited in the context of independent claim(s) 1, 8 and 15 as a whole.
Thus, claim(s) 1, 8 and 15 is/are allowed over the prior arts of record. Dependent claims 2-9, 10-14 and 16-20 are also allowable due to its dependency of independent claim(s) 1, 8 and 15.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Holtham (“ US 20180247193 A1” filed 02/23/2018, relates to methods for improving the operation of computer-implemented neural networks. Some aspects relate to training a neural network using a compressed representation of the inputs either through efficient discretization of the inputs, or choice of compression. This approach allows a multiscale approach where the input discretization is adaptively changed during the learning process, or the loss of the compression is changed during the training. Once a network has been trained, the approach allows for efficient predictions and classifications using compressed inputs. One approach can generate a larger more diverse training dataset based on both simulations from physical models, as well as incorporating domain expertise and other available information. One approach can automatically match the documents to the list, while still allowing a user to input information to update and correct the matching process... [the Abstract].
Cheng et al., NPL (“Model Compression and Acceleration for Deep Neural Networks” Published 2018 by IEEE, 11 pages, describing, as larger neural networks with more layers and nodes are considered, reducing their storage and computational cost becomes critical, especially for some real-time applications such as online learning and incremental learning. In addition, recent years witnessed significant progress in virtual reality, augmented reality, and smart wearable devices, creating unprecedented opportunities for researchers to tackle fundamental challenges in deploying deep-learning systems to portable devices with limited resources [e.g., memory, central processing units (CPUs), energy, bandwidth]. Efficient Deep learning methods can have a significant impact on distributed systems, embedded devices, and field-programmable gate array (FPGA) for artificial intelligence (AI)...wherein regarding training protocols, models based on parameter pruning/sharing low rank factorization can be extracted from pretrained ones or trained from scratch, while the transferred/ compact filter and KD models can only support training from scratch. These methods are independently designed and complement each other. ...[The Introduction section]
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUOC A TRAN whose telephone number is (571)272-8664. The examiner can normally be reached Monday-Friday 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at 571-272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QUOC A TRAN/Primary Examiner, Art Unit 2145