CTNF 18/530,683 CTNF 90426 DETAILED ACTION This action is in response to claims filed 06 December 2023 for application 18530683 filed 06 December 2023. Currently claims 1-20 are pending. Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority 02-26 AIA Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. Claim Rejections - 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15-aia AIA Claim(s) 1, 5, 6, 10-13, 16, and 18-20 is/are rejected under 35 U.S.C. 102 (A)(1) as being anticipated by Volodarskiy et al. (US 20200175354 A1) . Regarding claims 1, 12, 13, and 19 , Volodarskiy teaches: A processor-implemented method, the method comprising: while training a neural network (NN) using a current training mode selected from a plurality of training modes for training of the NN, measuring time data of a plurality of sub-operations for the training of the NN (“An underappreciated aspect of AutoML is that trying all possible algorithms and their hyperparameter values is an infinite time problem. Because of this infinite time problem, all AutoML tools and well-known academic algorithms constrain the time for algorithm selection and training. However, these AutoML tools do not formulate the problem as selecting the best algorithm based on the time constraint. Thus, in an embodiment, a method is disclosed that comprises using at least one hardware processor to select a plurality of machine-learning algorithms, generate a batch of trials from the plurality of machine-learning algorithms, begin executing at least a portion of the batch of trials, and, during execution of the batch of trials, provide intermediate evaluation results. It may involve estimating a training time and accuracy for two or more models represented in the batch of trials, and then selecting the best algorithm and hyperparameter settings to train from an available set of algorithm/hyperparameter setting combinations based on a time constraint set by the user. The method may use both an estimate of the training time for each algorithm and an estimated accuracy of the algorithm and settings to make the selection. For example, the method may use a combination of the estimated training time and the a priori (pre-training) estimated accuracy of the algorithm to choose the next best algorithm to train.” [0049], claim 6 model may be neural network) ; based on the time data, determining a computation time to perform computation operations among the plurality of sub-operations and a communication time to perform communication operations among the plurality of sub-operations [0049] ; based on a comparison result of the computation time and the communication time, selecting a next training mode from the plurality of training modes (“For example, the method may use a combination of the estimated training time and the a priori (pre-training) estimated accuracy of the algorithm to choose the next best algorithm to train.” [0049]) ; and training the NN based on the next training mode (“For example, the method may use a combination of the estimated training time and the a priori (pre-training) estimated accuracy of the algorithm to choose the next best algorithm to train.” [0049]) . Claims 13 and 19 also recite: processing modules configured to execute workloads corresponding to the plurality of sub-operations. Volodarskiy discloses: processing modules configured to execute workloads corresponding to the plurality of sub-operations (“The infrastructure may comprise a platform 110 (e.g., one or more servers) which hosts and/or executes one or more of the various functions, processes, methods, and/or software modules described herein. Platform 110 may comprise dedicated servers, or may instead comprise cloud instances, which utilize shared resources of one or more servers. These servers or cloud instances may be collocated and/or geographically distributed. Platform 110 may also comprise or be communicatively connected to a server application 112 and/or one or more databases 114. In addition, platform 110 may be communicatively connected to one or more user systems 130 via one or more networks 120. Platform 110 may also be communicatively connected to one or more external systems 140 (e.g., other platforms, websites, etc.) via one or more networks 120.” [0019]) . Regarding claims 5 and 16 , Volodarskiy teaches: The method of claim 1, wherein the plurality of sub-operations comprises any one or any combination of any two or more of a backward computation operation related to backward propagation, a gradient communication operation related to sharing of a layer gradient, an update computation operation related to model update, and a parameter communication operation related to sharing of a model parameter (“It may involve estimating a training time and accuracy for two or more models represented in the batch of trials, and then selecting the best algorithm and hyperparameter settings to train from an available set of algorithm/hyperparameter setting combinations based on a time constraint set by the user.” [0049]) . Regarding claim 6 , Volodarskiy teaches: The method of claim 5, wherein the determining of the computation time and the communication time comprises, based on the time data, recording first temporary data of any one or any combination of any two or more of the backward computation operation, the gradient communication operation, the update computation operation, and the parameter communication operation in a timetable for each layer of the NN (“It may involve estimating a training time and accuracy for two or more models represented in the batch of trials, and then selecting the best algorithm and hyperparameter settings to train from an available set of algorithm/hyperparameter setting combinations based on a time constraint set by the user.” [0049]) . Regarding claims 10, 18 and 20 , Volodarskiy teaches: The method of claim 1, wherein the selecting of the next training mode comprises: in response to a value of the computation time being larger among the computation time and the communication time, selecting the next training mode such that the value of the computation time decreases; and in response to the value of the computation time being larger among the computation time and the communication time, selecting the next training mode such that the value of the computation time increases (“It may involve estimating a training time and accuracy for two or more models represented in the batch of trials, and then selecting the best algorithm and hyperparameter settings to train from an available set of algorithm/hyperparameter setting combinations based on a time constraint set by the user.” [0049]) . Regarding claim 11 , Volodarskiy teaches: The method of claim 1, further comprising: predicting a change in total training time according to the next training mode based on dependency between computation operations and communication operations of a plurality of layers of the NN; and in response to the total training time increasing according to the next training mode, selecting an alternative training mode of the next training mode from the plurality of training modes (“It may involve estimating a training time and accuracy for two or more models represented in the batch of trials, and then selecting the best algorithm and hyperparameter settings to train from an available set of algorithm/hyperparameter setting combinations based on a time constraint set by the user.” [0049]) . Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim (s) 2-3 and 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Volodarskiy in view of Mopur et al. (US 20200151619 A1) . Regarding claims 2 and 14 , Volodarskiy does not explicitly disclose, however, Mopur teaches: The method of claim 1, wherein the plurality of training modes are distinguished from each other according to an update range of the NN by each processing module used for the training of the NN [0045-46] either a full range historical and new retraining or only new retraining can be used). Volodarskiy and Mopur are in the same field of endeavor of training ML models and are analogous. Volodarskiy discloses a method for determining training time and changing training methods depending on user constraints. Mopur teaches training using historical and new or only new data depending on metrics. It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the training modification due to time constraints as disclosed by Volodarskyi with the know training using various amounts and types of data as disclosed by Mopur to yield predictable results of faster and better training. Regarding claims 3 and 15 , Volodarskiy does not explicitly disclose, however, Mopur teaches: The method of claim 1, wherein the plurality of training modes comprises any one or any combination of any two or more of: a first training mode in which full update of a corresponding model of the NN is performed by each of processing modules used for the training of the NN (“Another form of training the impact of data drift may indicate the need to add additional prediction metrics to the original training regime. For example, in an image classification scenario, a classification done by the production model version (i.e., v1.A) may be with low confidence and be erroneous. The result could be manually inspected by a user (i.e., data expert) and manually reclassifies the image. This new classification data is therein used to train the model. This type of user controlled training would apply additional prediction metrics to the original production ML model 208 in an attempt to identify the unknown relationships impacting performance In various embodiments, more than one user controlled training can be launched.” [0046]) ; a second training mode in which partial update of 1/N of the corresponding model is performed by each of the processing modules (“Other impacts may indicate the need to perform new data training. For example, in some embodiments a sequence of negative or low correlation values may indicate that the drift has resulted in a drastic change in performance. In such cases, utilizing only new streaming data may be preferable, as the historical data has consistently resulted in poor predicted performance. Accordingly, training may be performed on the first production ML model 208 as deployed may be trained only with new streaming data to try and achieve a faster improvement in performance. This would result in a new version of the production ML model (production ML model 208.2).” [0045]) ; and a third training mode in which partial update of 1/M of the corresponding model is performed by each of the processing modules, and wherein the N represents a total number of the processing modules, and the M represents an integer greater than 1 and smaller than the N ([0045-46] . Allowable Subject Matter Claims 4, 7-9 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. None of the prior art of record teaches or discloses modifying the training mode and/or training data on a per layer basis in a neural network. Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yuan et al. (Distributed Learning of Fully Connected Neural Networks using Independent Subnet Training) discloses splitting distributed training among subnetworks . Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC NILSSON whose telephone number is (571)272-5246. The examiner can normally be reached M-F: 7-3. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571)-272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ERIC NILSSON/ Primary Examiner, Art Unit 2151 Application/Control Number: 18/530,683 Page 2 Art Unit: 2151 Application/Control Number: 18/530,683 Page 3 Art Unit: 2151 Application/Control Number: 18/530,683 Page 4 Art Unit: 2151 Application/Control Number: 18/530,683 Page 5 Art Unit: 2151 Application/Control Number: 18/530,683 Page 6 Art Unit: 2151 Application/Control Number: 18/530,683 Page 7 Art Unit: 2151 Application/Control Number: 18/530,683 Page 8 Art Unit: 2151 Application/Control Number: 18/530,683 Page 9 Art Unit: 2151 Application/Control Number: 18/530,683 Page 10 Art Unit: 2151 Application/Control Number: 18/530,683 Page 11 Art Unit: 2151
Prosecution Insights