DETAILED ACTION
This action is in response to the claims filed 06/17/2024 for Application number 18/653,096. Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 09/06/2024 and 02/26/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the claim can be interpreted as directed to signal per se. The specification fails to provide a clear disavowal that excludes transitory signals. Examiner proposes the applicant to amend computer readable medium to be non-transitory computer readable medium.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,
Step 1 Analysis: Claim 1 is directed to a process, which falls within one of the four statutory categories.
Step 2A Prong 1 Analysis: Claim 1 recites, in part, The limitations of:
searching a weight library based on the first structure information in order to obtain a reference weight, [wherein the weight library stores a plurality of groups of historical weights], and wherein the reference weight is a second weight that is in the plurality of groups of historical weights and that is of a second neural network model having a second structure similar to a first structure of the first neural network model; can be considered to be an evaluation in the human mind
These limitations as drafted, are processes that, under broadest reasonable interpretation, covers performance of the limitation in the mind or with the aid of pen and paper which falls within the “Mental Processes” grouping of abstract ideas.
The limitation of: converting the reference weight to obtain a first weight of the first neural network model can be considered to be a mathematical calculation.
This limitation as drafted, is a process that, under broadest reasonable interpretation, covers mathematical calculations which falls within the “Mathematical concepts” grouping of abstract ideas.
Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 Analysis: This judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements – “first neural network model” and “second neural network model. These elements that are recited are only generally linked to the judicial exception. Please see MPEP 2106.05(h). Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim further recites: obtaining first structure information of a first neural network model. This limitation is a mere data gathering step and thus is an insignificant extra-solution activity.
wherein the weight library stores a plurality of groups of historical weights. This limitation is an insignificant extra-solution activity.
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea.
Step 2B Analysis: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of utilizing a first and second neural network model to perform the steps of the claimed process amount to generally linking the additional elements to the judicial exception. Furthermore, the limitation of obtaining first structure information of a first neural network model is well-understood, routine, and conventional, as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network”.
The limitation of: wherein the weight library stores a plurality of groups of historical weights is well-understood, routine, and conventional, as evidenced by MPEP §2106.05(d)(II)(iv), “Storing and retrieving information in memory”.
These limitations therefore remain insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, these additional elements amount to generally linking the additional elements to the judicial exception and insignificant extra-solution activity, which cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 2, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein the reference weight is represented through a reference matrix corresponding to the second structure, and wherein converting the reference weight to obtain the first weight comprises at least one of cropping, concatenating, or copying the reference matrix in order to obtain a target matrix corresponding to the first structure, wherein the target matrix represents the first weight. This claim recites additional mathematical steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exceptions into a practical application, nor to significantly more than the judicial exceptions. The claim is not patent eligible.
Regarding claim 3, the rejection of claim 2 is further incorporated, and further, the claim recites: wherein the reference matrix comprises a matrix corresponding to a second convolution kernel of the second neural network model, and wherein the at least one of cropping, concatenating, or copying the reference matrix, matrix in order to obtain the target matrix corresponding to the first structure comprises:
cropping the reference matrix to obtain the target matrix when a first window size of a first convolution kernel of the first neural network model is less than a second window size of the second convolution kernel or a first quantity of input/output channels of the first convolution kernel is less than a second quantity of input/output channels of the second convolution kernel; or
copying or concatenating the reference matrix to obtain the target matrix when a first quantity of convolutional layers of the first neural network model is greater than a second quantity of convolutional layers of the second neural network model. This claim recites additional mathematical steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exceptions into a practical application, nor to significantly more than the judicial exceptions. The claim is not patent eligible.
Regarding claim 4, the rejection of claim 1 is further incorporated, and further, the claim recites: initializing the first neural network model based on the first weight to obtain an initialized first neural network model; and
training the initialized first neural network model using a dataset. These limitations amount to mere instructions to apply the judicial exception using a generic computer component. Please see MPEP 2106.05(f).
The claim does not include any additional elements that amount to an integration of the judicial exceptions into a practical application, nor to significantly more than the judicial exceptions. The claim is not patent eligible.
Regarding claim 5, the rejection of claim 3 is further incorporated, and further, the claim recites: further comprising updating, when training of the first neural network model is completed, a third weight of a trained first neural network model to the weight library. This claim recites additional mental steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exceptions into a practical application, nor to significantly more than the judicial exceptions. The claim is not patent eligible.
Regarding claim 6, the rejection of claim 1 is further incorporated, and further, the claim recites: wherein searching the weight library based on the first structure information in order to obtain the reference weight comprises: determining, based on the first structure information and second structure information of a third neural network model corresponding to at least one of the plurality of groups of historical weights in the weight library, a similarity between the first structure and a third structure of the third neural network model; This claim recites additional mental steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim further recites: obtaining the second weight as the reference weight based on the similarity between the first structure and the third structure, wherein a similarity between the first structure and the second structure meets a preset condition. This limitation is an insignificant extra-solution activity.
The claim does not include any additional elements that amount to significantly more than the judicial exception. This limitation is just a nominal or tangential addition to the claim, and is also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network”. This limitation therefore remains insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, this additional element represents an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 7, the rejection of claim 6 is further incorporated, and further, the claim recites: determining, using a graph neural network based on the first graph structure and the at least one second graph structure, the similarity between the first structure and the third structure. This claim recites additional mental steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim further recites: wherein determining the similarity between the first structure and the third structure comprises:
obtaining a first graph structure based on the first structure information;
obtaining at least one second graph structure based on the second structure information. These limitations are insignificant extra-solution activities.
The claim does not include any additional elements that amount to significantly more than the judicial exception. These limitations are just a nominal or tangential addition to the claim, and are also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network”. This limitation therefore remains insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, this additional element represents an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 8, the rejection of claim 6 is further incorporated, and further, the claim recites wherein the label identifies at least one of a type of a neural network model corresponding to the historical weights or a type of a task to which the historical weights are applicable and
wherein searching the weight library based on the first structure information in order to obtain the reference weight comprises: determining a search set from the weight library based on the label corresponding to each group of historical weights, wherein the search set comprises at least one of the plurality of groups of historical weights whose label matches at least one of a type of the first neural network model or a type of a task to which the first neural network model is applicable; and determining the reference weight from the search set based on the first structure information.
This claim recites additional mental steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim further recites: wherein the weight library stores a label corresponding to each group of the plurality of groups of historical weights This limitation is an insignificant extra-solution activity.
The claim does not include any additional elements that amount to significantly more than the judicial exception. These limitations are just a nominal or tangential addition to the claim, and are also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(iv), “Storing and retrieving information in memory”. This limitation therefore remains insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, this additional element represents an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 9, the rejection of claim 1 is further incorporated, and further, the claim recites wherein searching the weight library based on the first structure information in order to obtain the reference weight comprises: querying a historical search record of the weight library based on the first structure information, wherein the historical search record comprises third structure information of a historically searched neural network model and a found historical weight;.
This claim recites additional mental steps in addition to the judicial exception identified in the rejection of claim 1, thus recites a judicial exception.
The claim further recites: obtaining, when the first structure information is found in the historical search record, the found historical weight as the reference weight. This limitation is an insignificant extra-solution activity.
The claim does not include any additional elements that amount to significantly more than the judicial exception. This limitation is just a nominal or tangential addition to the claim, and is also well-understood, routine and conventional as evidenced by MPEP §2106.05(d)(II)(I), “receiving or transmitting data over a network”. This limitation therefore remains insignificant extra-solution activity even upon reconsideration, and does not amount to significantly more. Even when considered in combination, this additional element represents an insignificant extra-solution activity which cannot provide an inventive concept. The claim is not patent eligible.
Claim 10 recites features similar to claim 1 and is rejected for at least the same reasons therein. Claim 10 additionally requires analysis for “A computing device cluster comprising at least one computing device, wherein the at least one computing device comprises: at least one memory configured to store instructions; and one or more processors coupled to the at least one memory, wherein the one or more processors are configured to execute the instructions to cause the computing device cluster to” however the claim recites the additional elements of “A computer device cluster comprising … at least one computing device, one memory configured to …, and one or more processors configured to…” which amounts to mere instructions to apply the exception using a generic computer component. Please see MPEP 2106.05(f).
Regarding Claims 11-18, they recite features similar to claims 2-9 and are rejected for at least the same reasons therein.
Regarding Claim 19, it recites features similar to claim 11 and is rejected for at least the same reasons therein.
Claim 20 recites features similar to claims 1 and 10 and is rejected for at least the same reasons therein. Claim 20 additionally recites “A computer program product comprising instructions that are stored on a computer-readable medium and that, when executed by a processor, cause an apparatus to…” however as noted above, claim 20 recites “computer readable medium” which fails the requirement for step 1 analysis as it does not fall within any one of the four statutory categories.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, 10-15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. ("Accelerating DNN Architecture Search at Scale Using Selective Weight Transfer", hereinafter "Liu") in view of Fielding et al. ("Evolving Deep DenseBlock Architecture Ensembles for Image Classification", hereinafter "Fielding").
Regarding claim 1, Liu teaches A method comprising:
obtaining first structure information of a first neural network model (“When the search strategy has trained enough new candidates from scratch, it generates child models by mutating one variable node of the provider models acting as parents. The scheduler passes the architecture sequences of both parent and the child model (“first neural network model”) to the evaluators(4).” [pg. 87, top left col]);
and wherein the reference weight is a second weight that is in the plurality of groups of historical weights (“Starting from this insight, we propose the idea that new candidates need not be trained starting from random weights, but rather from the weights of similar layers of previously evaluated candidates.” [pg. 82, bottom para]) and that is of a second neural network model having a second structure similar to a first structure of the first neural network model (“To address this problem, we propose an alternative approach that is based on a key insight: candidates from the same search space are functionally and structurally similar with each other.” [pg. 82, right col, ¶3]); and
converting the reference weight to obtain a first weight of the first neural network model. (“4) if they have shareable tensors, initialize the weights of the current model with the weights of the parent’s model.” [pg. 88, C. Methodology])
However fails to explicitly teach searching a weight library based on the first structure information in order to obtain a reference weight, wherein the weight library stores a plurality of groups of historical weights
Fielding teaches searching a weight library based on the first structure information in order to obtain a reference weight, wherein the weight library stores a plurality of groups of historical weights (See Algorithms 2 and 3, discloses storing weights and searching from lookup tables (corresponds to “weight library”), “we design the central weight inheritance lookup table method to take into account the size of the parameter matrix for a layer (rather than a block), ensuring that layers will only inherit weights from the table that match the position and the parameter matrix size.” [pg. 9, 3.3.1.])
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Liu’s teachings with the weight lookup table storage and retrieval method as taught by Fielding. One would have been motivated to make this modification in order introduce a weight inheritance method that can perform joint optimization and training of neural network architectures to reduce the computational costs. [Abstract, Fielding]
Regarding claim 2, Liu/Fielding teaches The method of claim 1, where Liu teaches wherein the reference weight is represented through a reference matrix corresponding to the second structure, and wherein converting the reference weight to obtain the first weight comprises at least one of cropping, concatenating, or copying the reference matrix in order to obtain a target matrix corresponding to the first structure, wherein the target matrix represents the first weight. (See pg. 84, Tensor Matching, Liu discloses Tensor matching which corresponds to “copying the reference matrix in order to obtain a target matrix corresponding to the first structure, wherein the target matrix represents the first weight”)
Regarding claim 3, Liu/Fielding teaches The method of claim 2, wherein the reference matrix comprises a matrix corresponding to a second convolution kernel of the second neural network model (“Figure 3 shows an illustrative example of a convolutional NN and its shape sequence. The neural network begins with f ×w×h filters, followed by several layers such as pooling layer, batch normalization layer, or another convolutional layer (omitted in the figure).” [pg. 84, A. Tensor Matching; filter is a convolutional kernel]), and
wherein the at least one of cropping, concatenating, or copying the reference matrix, matrix in order to obtain the target matrix corresponding to the first structure comprises:
cropping the reference matrix to obtain the target matrix when a first window size of a first convolution kernel of the first neural network model is less than a second window size of the second convolution kernel or a first quantity of input/output channels of the first convolution kernel is less than a second quantity of input/output channels of the second convolution kernel; or
copying or concatenating the reference matrix to obtain the target matrix when a first quantity of convolutional layers of the first neural network model is greater than a second quantity of convolutional layers of the second neural network model. (“However, since the receiver model has an additional convolutional layer (2), even if the shape of the last dense layer is the same between the provider and the receiver (3), LP cannot transfer the weights of the last dense layer. “ [pg. 85, Longest Prefix (LP), ¶1; note: The claim recites the alternative language of “or” thus under BRI, the examiner is only required to map to one of the corresponding alternatives.])
Regarding claim 4, Liu/Fielding teaches The method of claim 1, further comprising:
Liu teaches initializing the first neural network model based on the first weight to obtain an initialized first neural network model; and training the initialized first neural network model using a dataset. (“The training lasts for one epoch. If after one epoch, the objective metrics (validation accuracy) are better when the receiver is trained from random weight initialization” [pg. 85, top right col])
Regarding claim 5, Liu/Fielding teaches The method of claim 4, Fielding teaches further comprising updating, when training of the first neural network model is completed, a third weight of a trained first neural network model to the weight library. (See Algorithm 1, pg. 7, discloses weight updating steps 3-9)
Same motivation to combine the teachings of Liu/Fielding as claim 1.
Regarding claim 6, Liu/Fielding teaches The method of claim 1, wherein searching the weight library based on the first structure information in order to obtain the reference weight comprises:
Liu teaches determining, based on the first structure information and second structure information of a third neural network model corresponding to at least one of the plurality of groups of historical weights in the weight library, a similarity between the first structure and a third structure of the third neural network model (“Our key insight (discussed in Section III) is that weight transfer from a similar model is likely to accelerate the convergence of the receiver model. Hence, we use the number of different variable node choices as the distance d to measure the similarity of two models in the same search space. The larger d is, the less similar the two models are. The choices made by variable nodes are recorded in the architecture sequences (discussed in Section II).” [pg. 85, V, ¶1]); and
obtaining the second weight as the reference weight based on the similarity between the first structure and the third structure, wherein a similarity between the first structure and the second structure meets a preset condition. (See pg. 86, algorithm 1, steps 6-9 “Initialize child with the weights of parent d between the parent and the child is always one!”)
Claim 10 recites features similar to claim 1 and is rejected for at least the same reasons therein. Claim 10 additionally requires A computing device cluster comprising at least one computing device, wherein the at least one computing device comprises: at least one memory configured to store instructions; and one or more processors coupled to the at least one memory, wherein the one or more processors are configured to execute the instructions to cause the computing device cluster to (Liu, Table II, pg. 88 discloses hardware configuration involving CPU/GPU/memory)
Regarding Claim 11, it recites features similar to claim 2 and is rejected for at least the same reasons therein. Claim 11 further recites “or input the reference matrix into a weight conversion model for conversion in order to obtain the target matrix.” However this is recited in an alternative language, thus the examiner under BRI is only required to map to one of the alternatives.
Regarding Claims 12-15, they recite features similar to claims 3-6 and are rejected for at least the same reasons therein.
Claim 20 recites features similar to claims 1 and 10 and is rejected for at least the same reasons therein. Claim 20 additionally requires A computer program product comprising instructions that are stored on a computer-readable medium and that, when executed by a processor, cause an apparatus to… (Liu, Table II, pg. 88 discloses hardware configuration involving CPU/GPU/memory)
Claims 7 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Fielding and further in view of Bai et al. ("SimGNN: A Neural Network Approach to Fast Graph Similarity Computation, hereinafter "Bai").
Regarding claim 7, Liu/Fielding teaches The method of claim 6, wherein
determining the similarity between the first structure and the third structure comprises:
Liu teaches obtaining a first graph structure based on the first structure information (See pg. 84, A. Tensor Matching, ¶1-2; computation graph);
obtaining at least one second graph structure based on the second structure information (“A provider and a receiver model can have many tensors of identical shapes (which we refer to as transferable). Therefore, finding a mapping of the transferable tensors from the provider to the receiver is equivalent to mapping sub-topologies of the two corresponding computation graphs” [pg. 84, A. Tensor Matching, ¶2]); and
However Liu/Fielding fails to explicitly teach determining, using a graph neural network based on the first graph structure and the at least one second graph structure, the similarity between the first structure and the third structure.
Bai teaches determining, using a graph neural network based on the first graph structure and the at least one second graph structure, the similarity between the first structure and the third structure. (“First, we propose an efficient and effective attention mechanism to select the most relevant parts of a graph to generate a graph-level embedding, which preserves the similarity between graphs. Second, we propose a pairwise node comparison method to supplement the graph-level embeddings for more effective modeling of the similarity between two graphs.” [pg. 385, 2nd bullet right col])
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Liu’s/Fielding’s teachings in order to use a GNN to determine similarities between graph structures as taught by Bai. One would have been motivated to make this modification in order to speed up the graph similarity computation [pg. 384, Introduction, bottom para, Bai]
Regarding Claim 16, it recites features similar to claim 7and is rejected for at least the same reasons therein.
Claims 8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Fielding and further in view of Gur et al. ("US 20210019665 A1", hereinafter "Gur").
Regarding claim 8, Liu/Fielding teaches The method of claim 6, where Liu further teaches determining the reference weight from the search set based on the first structure information. (“Our key insight (discussed in Section III) is that weight transfer from a similar model is likely to accelerate the convergence of the receiver model. Hence, we use the number of different variable node choices as the distance d to measure the similarity of two models in the same search space” [pg. 85, §V.A, ¶1])
however fails to explicitly teach wherein the weight library stores a label corresponding to each group of the plurality of groups of historical weights, wherein the label identifies at least one of a type of a neural network model corresponding to the historical weights or a type of a task to which the historical weights are applicable, and wherein searching the weight library based on the first structure information in order to obtain the reference weight comprises: determining a search set from the weight library based on the label corresponding to each group of historical weights, wherein the search set comprises at least one of the plurality of groups of historical weights whose label matches at least one of a type of the first neural network model or a type of a task to which the first neural network model is applicable;
Gur teaches wherein the weight library stores a label corresponding to each group of the plurality of groups of historical weights (“Collections of data may have associated metadata specifying various characteristics of the collections, including labels associated with the content of the collection of data, where the labels describe the content of the collection of data in some way” [¶0018]),
wherein the label identifies at least one of a type of a neural network model corresponding to the historical weights or a type of a task to which the historical weights are applicable (“These task characteristics are converted to corresponding search criteria for searching the model index 142 to determine if there are existing trained ML models registered in the trained model repository 140 that satisfy the search criteria of the task” [¶0038]), and
wherein searching the weight library based on the first structure information in order to obtain the reference weight comprises:
determining a search set from the weight library based on the label corresponding to each group of historical weights, wherein the search set comprises at least one of the plurality of groups of historical weights whose label matches at least one of a type of the first neural network model or a type of a task to which the first neural network model is applicable (“These task characteristics are converted to corresponding search criteria for searching the model index 142 to determine if there are existing trained ML models registered in the trained model repository 140 that satisfy the search criteria of the task” [¶0038]); and
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Liu’s/Fielding’s teachings in order to implement the model repository management and search engine of Gur. One would have been motivated to make this modification in order to provide a search engine capability for finding ML algorithms that provide functionality needed for performing desired training of a ML model to perform a desired task. [¶0019, Gur]
Regarding Claim 17, it recites features similar to claim 8 and is rejected for at least the same reasons therein.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Rozantsev et al. ("Residual Parameter Transfer for Deep Domain Adaptation", hereinafter "Rozantsev").
Regarding claim 19, Liu/Fielding teaches The method of claim 1, however fails to explicitly teach wherein the reference weight is represented through a reference matrix corresponding to the second structure, and
wherein converting the reference weight to obtain the first weight comprises inputting the reference matrix into a weight conversion model for conversion to obtain a target matrix.
Rozantsev teaches wherein the reference weight is represented through a reference matrix corresponding to the second structure (“For each layer i ∈ Ω, let us first consider a vector representation of the source and target stream parameters as θs i and θt i, respectively.” [pg. 3, §3.1, ¶1]), and
wherein converting the reference weight to obtain the first weight comprises inputting the reference matrix into a weight conversion model for conversion to obtain a target matrix. (“Note that ki, the second dimension of the Ai and Bi matrices, controls the complexity of the transformation by limiting the rank of the matrices. ki = 0 corresponds to the degenerate case where the parameters of the source and target streams are identical, that is, shared between the two streams.” [pg. 3, §3.1, ¶2, See Eq (1)])
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Liu’s/Fielding’s teachings to use the model adaptation technique of Rozantsev. One would have been motivated to make this modification in order to flexibly preserve the similarities between model domains. [Abstract, Rozantsev]
Allowable Subject Matter
Claims 9 and 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 9 and 18:
querying a historical search record of the weight library based on the first structure information, wherein the historical search record comprises third structure information of a historically searched neural network model and a found historical weight; and
obtaining, when the first structure information is found in the historical search record, the found historical weight as the reference weight.
No prior art was uncovered which fairly discloses the limitations of claims 9 and 18 as recited.
The closest prior art uncovered was Liu et al. (“Accelerating DNN Architecture Search at Scale Using Selective Weight Transfer”) which discloses a weight transfer mechanism between a receiver and provider model however does not go into details of querying a historical search of a weight library based on the first structure information.
Fielding et al. (“Evolving Deep DenseBlock Architecture Ensembles for Image Classification”) discloses lookup tables with stored trained weights however does not mention any querying of a historical search of the weight library based on the first structure information
Gur et al. (“US 20210019665 A1”) does disclose querying the framework to see if there are any ML models that exist to solve a certain task or problem, however the reference does not disclose any details of querying a historical search of a weight library based on the first structure information
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Creedon et al. (“US 20210256418 A1”) discloses caching of ML model training parameters but does not explicitly disclose the querying of a historical search of a weight library based on the first structure information
Wang et al. (“US 20230267307 A1”) discloses generating a ML multi-task model to perform tasks and further discloses a centralized repository for data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL H HOANG whose telephone number is (571)272-8491. The examiner can normally be reached Mon-Fri 8:30AM-4:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL H HOANG/ PRIMARY EXAMINER, Art Unit 2122