DETAILED ACTION
Claims 1 – 20 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
The Specification in [0006], [0008] and [0009] recites ‘a output value of a corresponding network modules’ which the Examiner believes should be --an output value of a corresponding network module-- or corrected to maintain the inventor’s intent while still being grammatically correct.
Appropriate correction is required.
Claims Objections
Claims 1, 12 and 20 are objected to because of the following informalities:
Claims 1, 12 and 20 recite in their first limitation ‘a output value of a corresponding network modules’ which the Examiner believes should be --an output value of a corresponding network module-- or corrected to maintain the inventor’s intent while still being grammatically correct.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 7, 8, 9, 10, 12, 18 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception without significantly more.
Independent claims 1, 12 and 20 recite the limitations of determining a first task, a second task, and a neural network for training, the neural network comprising network modules and importance coefficients corresponding to the network modules, each importance coefficient being used to scale an output value of the corresponding network modules, performing a target operation using each of the first task and the second task to obtain respective embedding features through training the neural network using text samples that correspond to a target task, obtaining trained importance coefficients and using the trained importance coefficients to determine an embedding feature of the target task, and then determining the similarity between the first and second tasks based on the embedding feature of the first task and the embedding feature of the second task.
Nothing in the claims preclude the claimed technique from being performed in the human mind. The entire process involves data gathering through the determination of the first task, the second task, and the neural network along with the constituents of the neural network, data manipulation through performing a target operation to obtain an embedding feature of the first task and an embedding feature of the second task, and data inference through the determining of a task similarity between the first and second tasks based on their respective embedding features. A human may be presented with two separate tasks, and also information for learning how to obtain a similarity between two tasks, the human may then perform target operation using the first task, and perform the target operation using the second task, these involving having the human learn, using text samples that correspond to a target task, for each of the first and second tasks, obtain initial results (as the trained importance coefficients), and then also for each of the first and second tasks, obtain second results (as the embedding features) making use of the initial results, and finally make a comparison of both separate second results in order to make a decision on how similar the first and second tasks are. The claims hereby recite a mental process.
This judicial exception is not integrated into a practical application as the claims simply teach of gathering data, manipulating data, and inferring data from available results. Claim 12 provides one or more processors and a memory, claim 20 provides a non-transient computer-readable storage medium, but these are mentioned in generic terms.
The invention is noted to any particular defining structures and simply provides instructions to apply the judicial exception. The techniques can be performed by a generic computer which would be presented as a tool to implement the abstract idea (classifiable as automation of the mental process steps). The Specification in [0032] provides general-purpose computer devices (known to comprise a processor and storage memory) to read upon the limitations of the claims. The neural network presented here could be one of a Transformer architecture [0055] and also that this is a large language model [0071]. The claims here simply present the training of a neural network without specifying the details of the application other than that text samples are used to obtain trained importance coefficients and embedding features, which are tasks that can be mentally performed making use of mental algorithms. By this presentation, mentioning the neural network in the claims can simply refer to a random or general-purpose neural network being applied to performing a mental process. Training a neural network in its base form could be interpreted as a human learning to perform a function, classifiable as a mental process, whereby the human receives text samples corresponding to a target task and determines values which are the trained importance coefficients, and then uses the determined values of the trained importance coefficients to then compute further values/vectors as the embedding features of the task. As indicated, these can be performed mentally. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the invention is not tied to a practical application.
The claims provide techniques that amount to no more than mere instructions that apply the judicial exception which can be performed by a generic device. Merely mentioning the processors and the storage memory amount to no more than general-purpose hardware used as tools to implement the abstract idea and do not provide any particular application other than applying them for the purpose of implementing a judicial exception. While the claims mention of a neural network, the neural network do not recite specifics on how it is being trained and applied to determining task similarity, and therefore, the claims still do not amount to significantly more than the mentioned judicial exception. Mere instructions to apply an exception using a generic device cannot provide an inventive concept. Claims 1, 12 and 20 are not eligible.
Claim 7 provides that the neural network is of a Transformer architecture having self-attention modules and feed forward neural network modules, thereby providing the type of neural network being used. This does not integrate any practical application nor does it provide any additional element sufficient to amount to more than the mentioned judicial exception.
Claim 8 provides that the neural network is a large language model. It provides the general application of the neural network. This does not integrate any practical application nor does it provide any additional element sufficient to amount to more than the mentioned judicial exception.
Claim 9 provides that the initial values of the plurality of importance coefficients are obtained by random initialisation. A human may mentally perform this initialisation procedure by randomly allotting importance coefficients. This does not integrate any practical application nor does it provide any additional element sufficient to amount to more than the mentioned judicial exception.
Claims 10 and 18 provide performing task migration between the first and second tasks upon determining that the similarity between both tasks is higher than a threshold. A human may, upon determining a reasonable similarity between tasks, move certain aspects/features of each of the first and second tasks between themselves so as to improve task performance. This does not integrate any practical application nor does it provide any additional element sufficient to amount to more than the mentioned judicial exception.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 20 recites a ‘non-transient computer-readable storage medium’ which is unclear by its presentation given that it can be interpreted to cover both transitory and non-transitory media. The Specification in [0009] mentions a ‘non-transient computer-readable storage medium’ but does not provide proper clarification on what this entails, thereby not giving the scope of what it is meant to encompass. Transitory media including carrier waves or communication media are viewed as physical characteristics of a form or energy, such as frequency, voltage, or the strength of a magnetic field, defined energy or magnetism, per se, and as such are non-statutory natural phenomena. O’Reilly, 56 U.S. (15 How.) at 112-14. Moreover, it does not appear that the claim reciting the signal are encoded with functional descriptive material that fall within any of the categories of the patentable subject matter set forth in § 101. Hence, by the claim’s presentation, one of ordinary skill in the art can interpret the claim to include transitory signals and non-transitory signals.
In order to overcome the present rejection, the Applicant is advised to amend the claim by using the following terminology: ‘non-transitory computer-readable storage medium’ as this defines structural and functional interrelationships between the computer program and the rest of the computer, which permits the computer program’s functionality to be realised, and is thus statutory. Such example terminology has been also found in the Official Gazette 1351 OG 212. See MPEP 2106.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 9, 12 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu et al. (US 2023/0185568 A1: hereafter — Wu) in view of Annangi et al. (US 2024/0379226 A1: hereafter — Annangi).
For claim 1, Wu discloses a method of determining similarity between text processing tasks (Wu: [0068] — searching for similar functions), comprising:
determining a first task, a second task, and a neural network to be trained, the neural network to be trained comprises a plurality of network modules and a plurality of [[importance coefficients corresponding to the plurality of network modules, [[and each importance coefficient of the plurality of importance coefficients are used to scale a output value of a corresponding network modules]] (Wu: [0053] — performing function similarity based on observation of target functions being mapped to the same indirect function call site (indicating the presence of a first and second function/task) and generating feature embeddings from a neural network that is trained to be able to determine the similarity between functions);
respectively performing a target operation using the first task and the second task as a target task to obtain an embedding feature of the first task and an embedding feature of the second task (Wu: [0073] — identifying similar target functions based on feature embeddings generated by a graph neural network), wherein the target operation comprises: …
determining a task similarity between the first task and the second task based on the embedding feature of the first task and the embedding feature of the second task (Wu: [0068] — determining similar functions based on obtained feature embeddings from a neural network).
The reference of Wu provides teaching for the training of a neural network to be able to determine similarity between a first task and a second other task through finding other functions that the first function is similar to. This however differs from the claimed invention in that the claimed invention further provides teaching for a neural network which provides obtaining importance samples to be used in the training of the neural network to obtain embedding features of the tasks.
This is however not new to the art as the reference of Annangi provides teaching for the presence of this as:
determining a first task, a second task, and a neural network to be trained, the neural network to be trained comprises a plurality of network modules and a plurality of importance coefficients corresponding to the plurality of network modules, and each importance coefficient of the plurality of importance coefficients are used to scale a output value of a corresponding network modules (Annangi: [0068] — for the purpose of training a neural network, obtaining trainable internal parameters (as the importance coefficients); [0080], [0101] — the trainable parameters being weights that can be used as scale factors at the appropriate neural network layer);
training the neural network to be trained using text samples corresponding to the target task and obtaining a plurality of trained importance coefficients (Annangi: [0068] — for the purpose of training a neural network, obtaining trainable internal parameters (as the importance coefficients); [0164] — random initialisation of trainable internal parameters of the neural network; [0081] — data candidate for training being electronic textual file, one or more strings of text); and
determining an embedding feature of the target task based on the plurality of trained importance coefficients (Annangi: [0068] — generating embeddings (embedding features) at the neural network based on the internal parameters (trained importance coefficients); [0029] — the neural network being applied to performing an inferencing task).
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to modify the teaching of the training of the neural network for determining similarities between a first function and another, by incorporating the known teaching of Annangi which selects trainable parameters (as importance coefficients) for the neural network so they can be applied to obtaining outputs of the neural network being used for inferring a task, to thereby come up with the claimed invention. The combination of prior art elements would have provided the predictable result of being able to determine how much weights being applied to the input features of the neural network would produce desirable output results, and effectively learning the intended task. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
For claim 9, claim 1 is incorporated and the combination of Wu in view of Annangi discloses the method, wherein initial values of the plurality of importance coefficients are obtained by random initialization (Annangi: [0164] — random initialisation of trainable internal parameters (the importance coefficients) of the neural network).
As for claim 12, electronic device claim 12 and method claim 1 are related as electronic device and the method of using same, with each claimed element’s function corresponding to the claimed method step. Wu in [0015] provides a processor as well as storage devices suitable to read upon the limitations of this claim. Accordingly, claim 12 is similarly rejected under the same rationale as applied above with respect to method claim 1.
As for claim 20, computer program product claim 20 and method claim 1 are related as computer program product storing executable instructions required for performing the claimed method steps on a computer. Wu in [0015] provides computer-readable storage medium suitable to read upon the limitations of this claim. Accordingly, claim 20 is similarly rejected under the same rationale as applied above with respect to method claim 1.
Claims 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 2023/0185568 A1) in view of Annangi (US 2024/0379226 A1) as applied to claim 1, further in view of JONES et al. (US 2025/0148258 A1: hereafter — Jones).
For claim 7, claim 1 is incorporated but the combination of Wu in view of Annangi fail to teach the limitations of this claim, for which the reference of Jones is now introduced to teach as the method, wherein the neural network to be trained is of a Transformer architecture, and the plurality of network modules includes a plurality of self-attention modules and a plurality of feed forward neural network modules (Jones: [0087] — the model to be used is a transformer model, which has a feed-forward neural network as well as one or more self-attention layers).
The combination of Wu in view of Annangi provides teaching for training a neural network for performing a task. This however differs from the claimed invention in that the claimed invention further provides that the trained neural network is of a transformer architecture having a plurality of self-attention modules and a plurality of feed-forward neural network modules. This is however not new to the art as the reference of Jones is seen to teach this above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found the combination of the teaching of the neural network as provided by the combination of Wu in view of Annangi, with the known teaching of Jones which presents a transformer architecture neural network having a plurality of self-attention modules and feed-forward neural network modules, as an obvious method to try, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of providing faster training over large datasets and multi-modal tasks, as provided by transformers. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
For claim 8, claim 7 is incorporated and the combination of Wu in view of Annangi further in view of Jones discloses the method, wherein the neural network to be trained is a large language model (Jones: [0094] — a neural network such as a large language model).
Claims 10 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 2023/0185568 A1) in view of Annangi (US 2024/0379226 A1) as applied to claims 1 and 12, further in view of Watson et al. (US 2020/0320379 A1: hereafter — Watson)
For claim 10, claim 1 is incorporated but the combination of Wu in view of Annangi fails to disclose the limitations of this claim, for which the reference of Watson is now introduced to teach as the method, further comprising:
performing task migration between the first task and the second task in response to determining that the task similarity between the first task and the second task is higher than a predetermined similarity (Watson: [0030] — determining that a neural network model is similar to another model up to a similarity threshold; [0031] — performing transfer learning based on both models (tasks) being similar (higher than a predetermined similarity)).
The combination of Wu in view of Annangi provides teaching for determining if a task is similar to another task. This however differs from the claimed invention in that the claimed invention further provides performing task migration when both tasks are determined to meet a predetermined similarity. This is however not new to the art as the reference of Watson is seen to teach above.
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to improve upon the technique of the combination of Wu in view of Annangi which determines that one task is similar to another task, by incorporating the known technique of Watson which performs transfer learning as a task migration technique from one model to another similar model, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result of being able to adapt a current model or task to perform another function/task without needing to be trained right from the very beginning. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
As for claim 18, electronic device claim 18 and method claim 10 are related as electronic device and the method of using same, with each claimed element’s function corresponding to the claimed method step. Accordingly, claim 18 is similarly rejected under the same rationale as applied above with respect to method claim 10.
Claims 11 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 2023/0185568 A1) in view of Annangi (US 2024/0379226 A1) as applied to claim 1, further in view of Watson (US 2020/0320379 A1) as applied to claims 10 and 18, and further in view of Rowe et al. (US 2022/0110018 A1: hereafter — Rowe).
For claim 11, claim 10 is incorporated and the combination of Wu in view of Annangi further in view of Watson discloses the method, wherein, the task migration includes at least one of the following:
augmenting a training set of the second task using a training set of the first task (Watson: [0031] — providing enhancement to the target machine learning tasks as target data set, such that the prior-trained model (first task) can be used for transfer learning to the other similar model (second task));
migrating at least a part of the model parameters in a trained neural network for the first task to a neural network for the second task (Watson: [0031] — providing enhancement to the target machine learning tasks as target data set (migration of model parameters) from a the prior-trained neural network model (first task) to enhance the performance of the target machine learning task (the neural network of the second task)); and
[[simultaneously]] training a neural network for the first task and a neural network for the second task, wherein the neural network for the first task and the neural network for the second task share a portion of the structures or parameters (Watson: [0025] — measuring similarity based on one or more source data sets of a pre-trained neural network model (neural network of the first task) being compared to one a target machine learning task (neural networks of the second task) in order to assess similarity metrics (the similarity being an indication of data sets being common between them); [0029] — checking for a similarity metric between two neural network models (first and second tasks) by computing the distance between the representations of both models (indicating a shared portion of the structures or parameters between both neural networks), with both neural network models having been pre-trained).
The combination of Wu in view of Annangi further in view of Watson however fail to explicitly teach of a simultaneous training of both neural network tasks.
This is however not new to the art as the reference of Rowe is introduced to teach this simultaneous training as:
simultaneously training a neural network for the first task and a neural network for the second task, wherein the neural network for the first task and the neural network for the second task share a portion of the structures or parameters (Rowe: [0088] — simultaneous/multi-task training of a neural network for related tasks).
Hence, before the effective filing date of the claimed invention, one of ordinary skill in the art would have found it obvious to improve upon the technique of the combination of Wu in view of Annangi further in view of Watson which finds similarities between two neural network models/tasks, by incorporating the known technique of Rowe which performs simultaneous training of neural networks attributed to both similar tasks, to thereby come up with the claimed invention. The combination of both prior art elements would have provided the predictable result that such a simultaneous training would provide a way for the similar tasks to obtain information from, and to learn from each other, leading to improved combine performance. See KSR Int’l Co. v. Teleflex Inc., 550 U.S. 398, 415-421, 82 USPQ2d 1385, 1395-97 (2007).
As for claim 19, electronic device claim 19 and method claim 11 are related as electronic device and the method of using same, with each claimed element’s function corresponding to the claimed method step. Accordingly, claim 19 is similarly rejected under the same rationale as applied above with respect to method claim 11.
Allowable Subject Matter
Claims 2 – 6 and 13 – 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
With regard to dependent claim 2, the prior art of record taken alone or in
combination fail to teach, inter alia, a method for training the neural network involving determining a first loss value for the plurality of importance coefficients that is positively correlated with absolute values of the importance coefficients, and then obtaining a text processing output by the neural network based on the text sample, to then determine a second loss value used to evaluate the text processing result, such that the importance coefficients and learnable parameters are adjusted based on the first loss and the second loss values.
Claim 2 is hereby objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form.
Dependent claims 3, 4, 5 and 6 depend on claim 2 and are also objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form.
With regard to dependent claim 13, the prior art of record taken alone or in
combination fail to teach, inter alia, an electronic device for training the neural network involving determining a first loss value for the plurality of importance coefficients that is positively correlated with absolute values of the importance coefficients, and then obtaining a text processing output by the neural network based on the text sample, to then determine a second loss value used to evaluate the text processing result, such that the importance coefficients and learnable parameters are adjusted based on the first loss and the second loss values.
Claim 13 is hereby objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form.
Dependent claims 14, 15, 16 and 17 depend on claim 13 and are also objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure.
Burr et al. (US 2024/0086677 A1) provides teaching for the presence of trained weights in a neural network layer being present to provide one or more scalar values to be applied to the output of the neural network [0015].
Puigcerver i Perez et al. (US 2022/0108171 A1) provides teaching for a system that takes advantage of transfer learning techniques when a first task and a second task are similar [0012], and that a predicted similarity can be determined between each candidate neural network [0153].
CHO et al. (US 2025/00534661 A1) provides teaching for migrating a first task based on the first task not meeting a predetermined target performance [0023].
SONG et al. (US 2022/0103203 A1) provides teaching for performing random initialisation on a coefficient of each layer of the neural network model [0063].
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to OLUWADAMILOLA M. OGUNBIYI whose telephone number is (571)272-4708. The Examiner can normally be reached Monday – Thursday (8:00 AM – 5:30 PM Eastern Standard Time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s Supervisor, PARAS D. SHAH can be reached at (571) 270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUWADAMILOLA M OGUNBIYI/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
05/02/2026
Prosecution Insights