USING TENSORS IN PORTIONS OF A NEURAL NETWORK

§101 §102 §103

DETAILED ACTION This communication is in response to the application filed 5/10/24 in which claims 1-20 were presented for examination. Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Information Disclosure Statement The information disclosure statement (IDS) submitted on 8/30/24 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. The information disclosure statement (IDS) submitted on 9/29/25 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 101 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 14-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because they are directed to a machine readable medium. The broadest reasonable interpretation of a claim drawn to a machine readable medium typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of machine readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 U.S.C. 101 as covering non-statutory subject matter. See In re Nuitjen, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter. A claim drawn to such a machine readable medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. 101 by adding the limitation “non-transitory” to the claim. Claim Rejections - 35 USC § 102 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. Claims 1, 2, 4, 8, 11, and 14 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Syvatkovskiy (US 2021/0034335 A1; published Feb. 4, 2021). Regarding claim 1, Syvatkovskiy discloses [a] processor, comprising: (Syvatkovskiy Figure 9 ¶ 111 (“The computing devices 902, 904 may include one or more processors 908, 940, one or more communication interfaces 910, 942, one or more storage devices 912, 944, one or more input/output devices 914, 949, and one or more memory devices 919, 948.”)) one or more circuits to cause one or more neural networks to use one or more tensors in two or more portions of the one or more neural networks (Syvatkovskiy Figure 2B (illustrates the neural transformer model 224 [e.g., one or more neural networks] receiving context tensor 216 [e.g., one or more tensors] being used by transformer blocks 226…226B [e.g., two or more portions of the one or more neural networks]) PNG media_image1.png 490 784 media_image1.png Greyscale ). Regarding claim 2, Syvatkovskiy discloses the invention of claim 1 as discussed above. Syvatkovskiy further discloses wherein the two or more portions comprise two or more transformer blocks (Syvatkovskiy Figure 2B (transformer blocks 226A…226B)). Regarding claim 4, Syvatkovskiy discloses the invention of claim 1 as discussed above. Syvatkovskiy further discloses wherein the one or more neural networks comprise one or more transformers (Syvatkovskiy Figure 2B (transformer blocks 226A…226B)). Regarding claim 8, Syvatkovskiy discloses [a] system comprising: one or more processors to cause one or more neural networks to use one or more tensors in two or more portions of the one or more neural networks (Syvatkovskiy Figure 2B (illustrates the neural transformer model 224 [e.g., one or more neural networks] receiving context tensor 216 [e.g., one or more tensors] being used by transformer blocks 226…226B [e.g., two or more portions of the one or more neural networks]) PNG media_image1.png 490 784 media_image1.png Greyscale ). Regarding claim 11, Syvatkovskiy discloses the invention of claim 8 as discussed above. Syvatkovskiy further discloses wherein the one or more neural networks comprise one or more transformers (Syvatkovskiy Figure 2B (transformer blocks 226A…226B)). Regarding claim 14, Syvatkovskiy discloses [a] machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least: (Syvatkovskiy ¶ 112 (“A memory device 916, 948 may be any non-transitory computer-readable storage media that may store executable procedures, applications, and data.”)) cause one or more neural networks to use one or more tensors in two or more portions of the one or more neural networks (Syvatkovskiy Figure 2B (illustrates the neural transformer model 224 [e.g., one or more neural networks] receiving context tensor 216 [e.g., one or more tensors] being used by transformer blocks 226…226B [e.g., two or more portions of the one or more neural networks]) PNG media_image1.png 490 784 media_image1.png Greyscale ). Claim Rejections - 35 USC § 103 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or non-obviousness. Claims 3 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Syvatkovskiy as applied to claim 1 above, and further in view of Lee (US 2010/0241596 A1; published Sep. 23, 2010). Regarding claim 3, Syvatkovskiy discloses the invention of claim 1 as discussed above. Syvatkovskiy does not expressly disclose wherein at least one of the two or more portions comprises a linear combination of two or more tensors that are each in at least one other portion of the one or more neural networks (but see Lee ¶ 66 (“Initially, the default weight assignments for each component classifier are that each of the weights is equal. When the method first begins, the entire ensemble confusion matrix is selected and each weight of the component classifies in the entire ensemble confusion matrix are equal. In addition, whenever a partition is created by the user, each partition by default starts with an initial weight assignment where the weights are all equal. Thus, the default setting at the start of the method and for each new partition created is that the weights of the component classifiers are equal.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lee to create an ensemble classifier as a weighted combination of previously-trained component classifiers, at least because doing so would enable adjusting the overall accuracy of the ensemble classifier. Regarding claim 5, Syvatkovskiy discloses the invention of claim 1 as discussed above. Syvatkovskiy does not expressly disclose wherein the two or more portions approximate one or more portions of another neural network (but see Lee ¶ 66 (“Initially, the default weight assignments for each component classifier are that each of the weights is equal. When the method first begins, the entire ensemble confusion matrix is selected and each weight of the component classifies in the entire ensemble confusion matrix are equal. In addition, whenever a partition is created by the user, each partition by default starts with an initial weight assignment where the weights are all equal. Thus, the default setting at the start of the method and for each new partition created is that the weights of the component classifiers are equal.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lee to create an ensemble classifier as a weighted combination of previously-trained component classifiers, at least because doing so would enable adjusting the overall accuracy of the ensemble classifier. Claims 6, 7, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Syvatkovskiy as applied to claims 1, 8, and 14 above, and further in view of Jebara (US 2014/0029840 A1; published Jan. 30, 2014). Regarding claim 6, Syvatkovskiy discloses the invention of claim 1 as discussed above. Syvatkovskiy does not expressly disclose wherein the two or more portions each comprise a coefficient of the one or more tensors that is learned during training of the one or more neural networks (but see Jebara ¶ 119 (“Embodiments of the disclosed subject matter include a computer-implemented method for optimizing classifiers, the computer-implemented method comprising receiving by a processor, training data representing classifiable events or objects including label data identifying respective classes of the events or objects, wherein each training example comprises one or more elements; associating a respective loss parameter with each of the training examples; (a) calculating a weight from each of the loss parameters that is a non-linear function of the loss parameter; (b) optimizing a weak learner or selecting a classifier from a pool of classifiers, to minimize an exponential loss of the weak learner or classifier on the training examples; (c) calculating a coefficient for the optimized weak learner or the selected classifier which is proportional to a logarithm of the ratio of the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier and the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier; (d) updating the loss parameters to the product of each with the exponential loss of the weak learner or classifier on its respective training example; repeating the operations defined in clauses a through d until a stop criterion is met; forming a linear combination of the optimized weak learners or the selected classifiers obtained from multiple iterations of operations a through d, each weighted by a respective one of the coefficients calculating in operation c; and outputting data representing said linear combination.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Jebara to use a coefficient calculated during training for the optimized weak learner, at least because doing so would enable to create a strong composite classifier. Regarding claim 7, Syvatkovskiy discloses the invention of claim 1 as discussed above. Syvatkovskiy does not expressly disclose wherein the one or more circuits are to select the two or more portions of the one or more neural networks during training of the one or more neural networks (but see Jebara ¶ 119 (“Embodiments of the disclosed subject matter include a computer-implemented method for optimizing classifiers, the computer-implemented method comprising receiving by a processor, training data representing classifiable events or objects including label data identifying respective classes of the events or objects, wherein each training example comprises one or more elements; associating a respective loss parameter with each of the training examples; (a) calculating a weight from each of the loss parameters that is a non-linear function of the loss parameter; (b) optimizing a weak learner or selecting a classifier from a pool of classifiers, to minimize an exponential loss of the weak learner or classifier on the training examples; (c) calculating a coefficient for the optimized weak learner or the selected classifier which is proportional to a logarithm of the ratio of the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier and the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier; (d) updating the loss parameters to the product of each with the exponential loss of the weak learner or classifier on its respective training example; repeating the operations defined in clauses a through d until a stop criterion is met; forming a linear combination of the optimized weak learners or the selected classifiers obtained from multiple iterations of operations a through d, each weighted by a respective one of the coefficients calculating in operation c; and outputting data representing said linear combination.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Jebara to use a coefficient calculated during training for the optimized weak learner, at least because doing so would enable to create a strong composite classifier. Regarding claim 13, Syvatkovskiy discloses the invention of claim 8 as discussed above. Syvatkovskiy does not expressly disclose wherein one or more processors of the system are to select the two or more portions of the one or more neural networks during training of the one or more neural networks (but see Jebara ¶ 119 (“Embodiments of the disclosed subject matter include a computer-implemented method for optimizing classifiers, the computer-implemented method comprising receiving by a processor, training data representing classifiable events or objects including label data identifying respective classes of the events or objects, wherein each training example comprises one or more elements; associating a respective loss parameter with each of the training examples; (a) calculating a weight from each of the loss parameters that is a non-linear function of the loss parameter; (b) optimizing a weak learner or selecting a classifier from a pool of classifiers, to minimize an exponential loss of the weak learner or classifier on the training examples; (c) calculating a coefficient for the optimized weak learner or the selected classifier which is proportional to a logarithm of the ratio of the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier and the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier; (d) updating the loss parameters to the product of each with the exponential loss of the weak learner or classifier on its respective training example; repeating the operations defined in clauses a through d until a stop criterion is met; forming a linear combination of the optimized weak learners or the selected classifiers obtained from multiple iterations of operations a through d, each weighted by a respective one of the coefficients calculating in operation c; and outputting data representing said linear combination.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Jebara to use a coefficient calculated during training for the optimized weak learner, at least because doing so would enable to create a strong composite classifier. Regarding claim 15, Syvatkovskiy discloses the invention of claim 14 as discussed above. Syvatkovskiy does not expressly disclose further comprising a set of instructions, which if performed by the one or more processors, cause the one or more processors to select the two or more portions of the one or more neural networks during training of the one or more neural networks (but see Jebara ¶ 119 (“Embodiments of the disclosed subject matter include a computer-implemented method for optimizing classifiers, the computer-implemented method comprising receiving by a processor, training data representing classifiable events or objects including label data identifying respective classes of the events or objects, wherein each training example comprises one or more elements; associating a respective loss parameter with each of the training examples; (a) calculating a weight from each of the loss parameters that is a non-linear function of the loss parameter; (b) optimizing a weak learner or selecting a classifier from a pool of classifiers, to minimize an exponential loss of the weak learner or classifier on the training examples; (c) calculating a coefficient for the optimized weak learner or the selected classifier which is proportional to a logarithm of the ratio of the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier and the sum of the assigned weights corresponding to the examples classified correctly by the optimized weak learner or the selected classifier; (d) updating the loss parameters to the product of each with the exponential loss of the weak learner or classifier on its respective training example; repeating the operations defined in clauses a through d until a stop criterion is met; forming a linear combination of the optimized weak learners or the selected classifiers obtained from multiple iterations of operations a through d, each weighted by a respective one of the coefficients calculating in operation c; and outputting data representing said linear combination.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Jebara to use a coefficient calculated during training for the optimized weak learner, at least because doing so would enable to create a strong composite classifier. Claims 9, 12, 16, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Syvatkovskiy as applied to claims 1, 8, and 14 above, and further in view of Lan, Zhenzhong, et al. "Albert: A lite bert for self-supervised learning of language representations." arXiv preprint arXiv:1909.11942 (2019) (“Lan”). Regarding claim 9, Syvatkovskiy discloses the invention of claim 8 as discussed above. Syvatkovskiy does not expressly disclose wherein a tensor of the one or more tensors comprises a set of weights, the one or more neural networks comprise a particular neural network comprising the two or more portions, and the one or more processors are to calculate outputs of the two or more portions based, at least in part, on a set of input values and the set of weights (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., one or more tensors], e.g., only sharing feed-forward network (FFN) parameters across layers [e.g., two or more portions of the one or more neural networks], or only sharing attention parameters. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing [e.g., set of weights] has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Regarding claim 12, Syvatkovskiy discloses the invention of claim 8 as discussed above. Syvatkovskiy further discloses wherein the one or more neural networks comprises a first number of blocks, (Syvatkovskiy Figure 2B (transformer blocks 226A…226B)). Syvatkovskiy does not expressly disclose and the one or more processors are to select the one or more tensors from a group of tensors comprising a second number of tensors that is less than the first number (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., group of tensors], e.g., only sharing feed-forward network (FFN) parameters [e.g., one or more tensors] across layers, or only sharing attention parameters [e.g., one or more tensors]. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Regarding claim 16, Syvatkovskiy discloses the invention of claim 14 as discussed above. Syvatkovskiy does not expressly disclose wherein a tensor of the one or more tensors comprises a set of weights, the one or more neural networks comprise a neural network comprising the two or more portions (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., one or more tensors], e.g., only sharing feed-forward network (FFN) parameters across layers [e.g., two or more portions of the one or more neural networks], or only sharing attention parameters. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing [e.g., set of weights] has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Regarding claim 19, Syvatkovskiy discloses the invention of claim 16 as discussed above. Syvatkovskiy further discloses wherein the one or more neural networks comprise one or more transformers (Syvatkovskiy Figure 2B (transformer blocks 226A…226B)). Regarding claim 20, Syvatkovskiy discloses the invention of claim 16 as discussed above. Syvatkovskiy further discloses wherein the one or more neural networks comprises a first number of blocks, (Syvatkovskiy Figure 2B (transformer blocks 226A…226B)). Syvatkovskiy does not expressly disclose and the one or more processors are to select the one or more tensors from a group of tensors comprising a second number of tensors that is less than the first number (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., group of tensors], e.g., only sharing feed-forward network (FFN) parameters [e.g., one or more tensors] across layers, or only sharing attention parameters [e.g., one or more tensors]. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Claims 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Syvatkovskiy as applied to claims 8 and 14 above, and further in view of Lee and Lan. Regarding claim 10, Syvatkovskiy discloses the invention of claim 8 as discussed above. Syvatkovskiy does not expressly disclose the one or more processors are to select a combination of two or more tensors from a group of tensors to use in a particular portion of the two or more portions, (but see Lee ¶ 66 (“Initially, the default weight assignments for each component classifier are that each of the weights is equal. When the method first begins, the entire ensemble confusion matrix is selected and each weight of the component classifies in the entire ensemble confusion matrix are equal. In addition, whenever a partition is created by the user, each partition by default starts with an initial weight assignment where the weights are all equal. Thus, the default setting at the start of the method and for each new partition created is that the weights of the component classifiers are equal.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lee to create an ensemble classifier as a weighted combination of previously-trained component classifiers, at least because doing so would enable adjusting the overall accuracy of the ensemble classifier. Syvatkovskiy and Lee do not expressly disclose and at least one of the two or more tensors is used another of the two or more portions (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., at least one of the two or more tensors], e.g., only sharing feed-forward network (FFN) parameters across layers [e.g., another of the two or more portions], or only sharing attention parameters. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Regarding claim 17, Syvatkovskiy discloses the invention of claim 14 as discussed above. Syvatkovskiy does not expressly disclose wherein the one or more processors are to select a combination of two or more tensors from a group of tensors to use in a portion of the two or more portions, (but see Lee (US 2010/0241596 A1; published Sep. 23, 2010) ¶ 66 (“Initially, the default weight assignments for each component classifier are that each of the weights is equal. When the method first begins, the entire ensemble confusion matrix is selected and each weight of the component classifies in the entire ensemble confusion matrix are equal. In addition, whenever a partition is created by the user, each partition by default starts with an initial weight assignment where the weights are all equal. Thus, the default setting at the start of the method and for each new partition created is that the weights of the component classifiers are equal.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lee to create an ensemble classifier as a weighted combination of previously-trained component classifiers, at least because doing so would enable adjusting the overall accuracy of the ensemble classifier. Syvatkovskiy and Lee do not expressly disclose and at least one of the two or more tensors is used another of the two or more portions (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., at least one of the two or more tensors], e.g., only sharing feed-forward network (FFN) parameters across layers [e.g., another of the two or more portions], or only sharing attention parameters. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Syvatkovskiy as applied to claim 15 above, and further in view of Lan. Regarding claim 18, Syvatkovskiy discloses the invention of claim 15 as discussed above. Syvatkovskiy does not expressly disclose further comprising a set of instructions, which if performed by the one or more processors, cause the one or more processors to calculate outputs of the two or more portions based at least in part on the one or more tensors, a set of input values, and two or more coefficients be learned during training of the one or more neural networks (but see Lan page 4 Section 3.1 Cross-layer parameter sharing (“For ALBERT, we propose cross-layer parameter sharing as another way to improve parameter efficiency. There are multiple ways to share parameters [e.g., one or more tensors], e.g., only sharing feed-forward network (FFN) parameters across layers [e.g., two or more portions of the one or more neural networks], or only sharing attention parameters. The default decision for ALBERT is to share all parameters across layers…. Figure 1 shows the L2 distances and cosine similarity of the input and output embeddings for each layer, using BERT-large and ALBERT-large configurations (see Table 1). We observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing [e.g., set of weights] has an effect on stabilizing network parameters.”)). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Syvatkovskiy to incorporate the teachings of Lan to share weights across layers, at least because doing so would have the effect of stabilizing network parameters. Conclusion The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Dehghani, Mostafa, et al. "Universal transformers." arXiv preprint arXiv:1807.03819 (2018). Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHID KHAN whose telephone number is (571)270-0419. The examiner can normally be reached M-F, 9-5 est. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571)272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /SHAHID K KHAN/ Primary Examiner, Art Unit 2146