CTNF 18/450,571 CTNF 100578 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. This action is responsive to the application filed on August 16 th , 2023. Claims 1-7 are pending in the case. Claim 1 is the independent claim. Information Disclosure Statement The information disclosure statement filed July 16 th , 2024 fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include a concise explanation of the relevance, as it is presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the content of the information, of each reference listed that is not in the English language. “Chinese language office action dated 2024-04-26, issued in application no. TW 112142228” ( cited as Non-Patent Literature No. 1 ) has not been considered because no English copy of the document has been provided. It has been placed in the application file, but the information referred to therein has not been considered. Specification 07-29 AIA The disclosure is objected to because of the following informalities: “ lable ” in ¶39 should be “label”. “ psudo-labels ” in ¶39 should be “pseudo-labels” . Appropriate correction is required. Claim Rejections - 35 USC § 112 07-30-02 AIA The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. The following is a quotation of 35 U.S.C. 112 (pre-AIA), second paragraph: The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention. 07-34-01 Claim 4 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 4 recites the limitation “ the first sub-block doubles channel number of the original input ” in lines 1-2 of the claim. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, this limitation has been interpreted as “the first sub-block doubles a channel number of the original input”. Claim 4 recites the limitation “ maintain the same channel number ” in line 4 of the claim. It is unclear what channel number is being maintained and at what value. For examination purposes, and in light of figure 2 of the application, this limitation has been interpreted as “maintain the doubled channel number of the original input”. Claim 4 recites the limitation “ restores the channel number to that of the original input ” in lines 5-6 of the claim. It is unclear what channel number is being restored and at what value. For examination purposes, and in light of figure 2 of the application, this limitation has been interpreted as “restores the doubled channel number to that of the original input”. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Regarding claim 1: Step 1 : Claim 1 is directed to [a] method , therefore it falls under the statuary category of a process. Step 2A Prong 1 : The claim recites, in part: “ (ii) removing the shortcut connection from the branch structure ” this encompasses the mental removing of an observed shortcut connection from an observed branch structure. “ (iii) building a reparameterization model by linearly expanding each of the original convolutional layers into a reparameterization block in the reparameterization model ” this limitation is a mathematical concept. “ (iv) optimizing parameters of the reparameterization blocks by training the reparameterization model ” this limitation is a mathematical concept. “ (v) transforming each of the optimized reparameterization blocks into a reparameterized convolutional layer to form a branchless structure that replaces the branch structure in the trained model ” this limitation is a mathematical concept. Step 2A Prong 2 : The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “ (i) obtaining a trained model… ” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). “a trained model… that has a branch structure involving one or more original convolutional layers and a shortcut connection ” the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h). Step 2B : The additional elements, “a trained model… that has a branch structure involving one or more original convolutional layers and a shortcut connection ”, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “ (i) obtaining a trained model… ” the limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc. , 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). See MPEP § 2106.05(d)/(II). Therefore, the claim is ineligible. Regarding claim 2, the rejection of claim 1 is incorporated and further: Step 2A Prong 1 : a continuation of the abstract idea identified in the parent claim. Step 2A Prong 2 : The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “ the reparameterization block expanded from the original convolutional layer using an original convolutional kernel with a size of N×N comprises a first sub-block, a second sub-block, a third sub-block, a fourth sub-block, a fifth sub-block, and a sixth sub-block, using convolutional kernels with sizes of 1×1, 1×1, N×N, N×1, 1×N, and 1×1, respectively ” the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h). “ the first sub-block takes original input of the original convolutional layer as input ”, “ the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block take output from the first sub- block as input ”, “ the sixth sub-block takes outputs from the first sub-block, the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block as input ” these limitations are an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Step 2B : The additional elements, “ the reparameterization block expanded from the original convolutional layer using an original convolutional kernel with a size of N×N comprises a first sub-block, a second sub-block, a third sub-block, a fourth sub-block, a fifth sub-block, and a sixth sub-block, using convolutional kernels with sizes of 1×1, 1×1, N×N, N×1, 1×N, and 1×1, respectively ”, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Further, “ the first sub-block takes original input of the original convolutional layer as input ”, “ the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block take output from the first sub-block as input ”, “ the sixth sub-block takes outputs from the first sub-block, the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block as input ” Furthermore the additional element is directed to storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc. , 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015). See MPEP § 2106.05(d)/(II). Therefore, the claim is ineligible. Regarding claim 3, the rejection of claim 2 is incorporated and further: Step 2A Prong 1 : The claim recites, in part: “ merging the convolutional kernels used by the first sub-block, the second sub-block, the third sub-block, the fourth sub-block, the fifth sub-block, and the sixth sub-block into a reparameterized convolutional kernel with a size of N×N used by the reparameterized convolutional layer ” a continuation of the abstract idea identified in the parent claim. Step 2A Prong 2 : The claim does not recite any additional limitations, thus does not further recite any additional elements that integrates the judicial exception into a practical application or amount to significantly more. Regarding claim 4, the rejection of claim 2 is incorporated and further: Step 2A Prong 1 : The claim recites, in part: “ the first sub-block doubles channel number of the original input ”, “ wherein the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block maintain the same channel number ”, ‘ wherein the sixth sub-block restores the channel number to that of the original input ” these limitations are all mathematical concepts. Step 2A Prong 2 : The claim does not recite any additional limitations, thus does not further recite any additional elements that integrates the judicial exception into a practical application or amount to significantly more. Regarding claim 5, the rejection of claim 1 is incorporated and further: Step 2A Prong 1 : The claim recites, in part: “ using a loss function to calculate a loss value of the second set of feature maps relative to the first set of feature maps ” this limitation is a mathematical concept. “ using an optimization algorithm to adjust the parameters of the reparameterization blocks based on the loss value ” this limitation is a mathematical concept. Step 2A Prong 2 : The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “ inputting training data into the branch structure involving the original convolutional layers and the shortcut connection, and obtaining a first set of feature maps output by the branch structure ”, “ inputting the training data into the reparameterization model, and obtaining a second set of feature maps output by the reparameterization model ” the limitations are an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2). Step 2B : The additional elements, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Therefore, the claim is ineligible. Regarding claim 6, the rejection of claim 1 is incorporated and further: Step 2A Prong 1 : The claim recites, in part: “ inputting labeled data…to perform a specific task, and obtaining prediction result ” this encompasses the mental obtaining of a prediction from observing labeled data. “ using a loss function to calculate a loss value of the prediction result relative to a label of the labeled data ” this limitation is a mathematical concept. “ using an optimization algorithm to adjust the parameters of the reparameterization blocks in the reparameterization model based on the loss value ” this limitation is a mathematical concept. Step 2A Prong 2 : The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “ into the reparameterization model ” the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h). “ output by the reparameterization model ” the limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2). Step 2B : The additional elements, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Therefore, the claim is ineligible. Regarding claim 7, the rejection of claim 1 is incorporated and further: Step 2A Prong 1 : The claim recites, in part: “ marking all the original convolutional layers ” this encompasses the mental marking of observed layers. “ searching for the next branch structure ” this encompasses the mental search of observed branch structures for a next branch structure. “ checking if all the original convolutional layers involved in the searched branch structure have a mark ” this encompasses the mental checking of observed layers for an observed mark. “ in response to all the original convolutional layers involved in the searched branch structure having the mark, unmarking all the original convolutional layers involved in the searched branch structure, performing steps (ii)-(v) on the searched branch structure, and searching for the next branch structure ” this encompasses the mental unmarking of observed layers involved in a search and performing previous mental steps. Step 2A Prong 2 : The judicial exception is not integrated into a practical application; the remaining limitations of the claim are as follows: “ of the trained model ”, “ in the trained model ” lines 4 and 11 of the claim the limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h). Step 2B : The additional elements, taken individually and in combination, do not provide an inventive concept of significantly more than the abstract idea itself for the reasons set forth in step 2A prong 2 above. Therefore, the claim is ineligible. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1 and 6 are rejected under 35 U.S.C. § 103 as being unpatentable over Meng et al. (“RMNET: Equivalently Removing Residual Connection From Networks”, Meng et al., 1 Nov 2021) ( hereinafter “Meng”) in view of Hu et al. (“Online Convolutional Re-parameterization”, Hu et al., 2 Apr 2022) ( hereinafter “Hu”) . Regarding claim 1: Meng teaches [a] method for removing branches from trained deep learning models, comprising the following steps: (i) obtaining a trained model (Meng, page 2, ¶2 “With this method, we can equivalently convert a pre-trained ResNet or MobileNetV2 to an RMNet model to increase the degree of parallelism.”) that has a branch structure involving one or more original convolutional layers and a shortcut connection (Meng, page 3, section 3, fig 1(a) PNG media_image1.png 713 1605 media_image1.png Greyscale figure 1(a) shows a branch structure with multiple convolutional layers, and a residual connection which can be considered the shortcut connection ) ; (ii) removing the shortcut connection from the branch structure (Meng, page 2, ¶2 “In this paper, we introduce a novel approach named RM operation, which can remove the residual connection with non-linear layers inside it and keep the result of the model unchanged.”) ; …form a branchless structure that replaces the branch structure in the trained model (Meng, page 2, ¶5 “With RM operation, we can convert the ResBlocks to a stack of convolutions and ReLUs, which help get a deeper network without residual connection and make it very friendly for pruning.” Here, the layer stack without residual connections can be considered the branchless structure ) . Meng does not teach “ (iii) building a reparameterization model by linearly expanding each of the original convolutional layers into a reparameterization block in the reparameterization model; (iv) optimizing parameters of the reparameterization blocks by training the reparameterization model; and (v) transforming each of the optimized reparameterization blocks into a reparameterized convolutional layer ” However, Hu teaches (iii) building a reparameterization model by linearly expanding each of the original convolutional layers into a reparameterization block in the reparameterization model (Hu, page 2, col 1, ¶3 “In the first stage, block linearization, we remove all the non-linear norm layers and introduce the linear scaling layers.” Furthermore , Hu, page 4, col 1, section 3.2, ¶2 “Based on the linear scaling layers, we modify the reparameterization blocks as illustrated in Fig. 3.”) ; (iv) optimizing parameters of the reparameterization blocks by training the reparameterization model (Hu, page 6, col 2, section 4.1, ¶1 “By default, we apply an SGD optimizer to train the models, with initial learning rate 0.1 and cosine annealed in 120 epochs.”) ; and (v) transforming each of the optimized reparameterization blocks into a reparameterized convolutional layer (Hu, page 2, col 1, ¶3 “The second stage, named block squeezing, simplifies the complicated linear block into a single convolutional layer”) Meng and Hu are analogous art because both references concern methods for reparameterization. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Meng’s reparameterization method to incorporate the expansions and optimizations taught by Hu. The motivation for doing so would have been to outperform other methods using linear scaling and squeezing as stated in Hu, page 1, Abstract, ¶1 “we present online convolutional re parameterization (OREPA), a two-stage pipeline, aiming to reduce the huge training overhead by squeezing the complex training-time block into a single convolution. To achieve this goal, we introduce a linear scaling layer for better optimizing the online blocks. Assisted with the reduced training cost, we also explore some more effective re-param components. Compared with the state-of-the-art re-param models, OREPA is able to save the training-time memory cost by about 70% and accelerate the training speed by around 2×. Meanwhile, equipped with OREPA, the models outperform previous methods on ImageNet by up to +0.6%.” Regarding claim 6: Meng in view of Hu teaches [t]he method as claimed in claim 1, wherein step (iv) further comprises: inputting labeled data into the reparameterization model to perform a specific task, and obtaining prediction result output by the reparameterization model (Hu, page 6, col 2, section 4.1, ¶1 “We conduct experiments on the ImageNet-1k [13] dataset.” Here, the ImageNet-1k dataset is labeled data for the specific task of image classification ) ; using a loss function to calculate a loss value of the prediction result relative to a label of the labeled data (Hu, page 5, col 2, ¶2 “…where L is the loss function of the entire model and η is the learning rate.”) ; using an optimization algorithm to adjust the parameters of the reparameterization blocks in the reparameterization model based on the loss value (Hu, page 6, col 2, section 4.1, ¶1 “By default, we apply an SGD optimizer to train the models, with initial learning rate 0.1 and cosine annealed in 120 epochs.”) . It would have been obvious to combine the teachings of Meng and Hu for the reasons set forth in connection with claim 1 above . 07-21-aia AIA Claim 5 is rejected under 35 U.S.C. § 103 as being unpatentable over Meng in view of Hu in further view of Li et al. (“Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts”, Li et al., 2020) ( hereinafter “Li”) . Regarding claim 5: Meng in view of Hu teaches [t]he method as claimed in claim 1, Meng in view of Hu does not teach “ wherein step (iv) further comprises: inputting training data into the branch structure involving the original convolutional layers and the shortcut connection, and obtaining a first set of feature maps output by the branch structure; inputting the training data into the reparameterization model, and obtaining a second set of feature maps output by the reparameterization model; using a loss function to calculate a loss value of the second set of feature maps relative to the first set of feature maps; and using an optimization algorithm to adjust the parameters of the reparameterization blocks based on the loss value ” However, Li teaches wherein step (iv) further comprises: inputting training data into the branch structure involving the original convolutional layers and the shortcut connection, and obtaining a first set of feature maps output by the branch structure (Li, page 5, section 3.4, algorithm 1 “Feed the data into ResNet t using to get the feature maps f t (V ; x,y) ”) ; inputting the training data into the reparameterization model, and obtaining a second set of feature maps output by the reparameterization model (Li, page 5, section 3.4, algorithm 1 “Feed the same batch of data into Path 1, 2, 3, 4 in Section 3.2, to obtain feature maps f s t i W , V ; x , y and losses L s t i W , L s W , i = 1 , … , N ”) ; using a loss function to calculate a loss value of the second set of feature maps relative to the first set of feature maps (Li, page 1, Abstract, ¶1 “During forward step, the feature maps of the early stages of plain CNN are passed through later stages of both itself and the ResNet counterpart to calculate the loss.”) ; and using an optimization algorithm to adjust the parameters of the reparameterization blocks based on the loss value (Li, page 1, Abstract, ¶1 “During backpropagation, gradients calculated from a mixture of these two parts are used to update the plainCNN network to solve the gradient vanishing problem.”) . Meng in view of Hu and Li are analogous art because both references concern methods for removing shortcut connections from convolutional neural networks. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Meng/Hu’s reparameterization method to incorporate the feature map student/teacher methods taught by Li. The motivation for doing so would have been to outperform other knowledge distillation methods as stated in Li, page 6, ¶1 “We compare our methods with previous knowledge distillation methods [42, 28] and plain-CNN models initialized with the Dirac delta matrix [29] and the related results demonstrate that our method can outperform these methods in most cases.” Examiner Notes Claims 2, 3, 4 and 7 have been rejected under 35 U.S.C. § 101 but have not been rejected under prior art. A complete prior art search was performed for claims 2, 3, 4 and 7, however no prior art was uncovered that teaches or fairly suggests the following: “ using an original convolutional kernel with a size of N×N comprises a first sub-block, a second sub-block, a third sub-block, a fourth sub-block, a fifth sub-block, and a sixth sub-block, using convolutional kernels with sizes of 1×1, 1×1, N×N, N×1, 1×N, and 1×1, respectively; wherein the first sub-block takes original input of the original convolutional layer as input; wherein the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block take output from the first sub-block as input; wherein the sixth sub-block takes outputs from the first sub-block, the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block as input. ” as recited in claim 2. “ merging the convolutional kernels used by the first sub-block, the second sub-block, the third sub-block, the fourth sub-block, the fifth sub-block, and the sixth sub-block into a reparameterized convolutional kernel with a size of N×N used by the reparameterized convolutional layer. ” as recited in claim 3, which depends on claim 2. “ the first sub-block doubles channel number of the original input; wherein the second sub-block, the third sub-block, the fourth sub-block, and the fifth sub-block maintain the same channel number; and wherein the sixth sub-block restores the channel number to that of the original input. ” as recited in claim 4, which depends on claim 2. “ marking all the original convolutional layers of the trained model… checking if all the original convolutional layers involved in the searched branch structure have a mark; in response to all the original convolutional layers involved in the searched branch structure having the mark, unmarking all the original convolutional layers involved in the searched branch structure, performing steps (ii)-(v) on the searched branch structure, and searching for the next branch structure in the trained model. ” as recited in claim 7 . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yang et al. (“Expansion-Squeeze-Block: Linear Over-parameterization with Shortcut Connections to Train Compact Convolutional Networks”, Yang et al., 2022) discloses “Expansion-Squeeze-Block: Linear Over-parameterization with Shortcut Connections to Train Compact Convolutional Networks. The structure expands the width of convolutional layers and adds shortcut connections for better performance without adding any nonlinearity. The expanded networks can be contracted back to the original format algebraically at inference time without loss of information.” Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB Z SUSSMAN MOSS whose telephone number is (571) 272-1579. The examiner can normally be reached Monday - Friday, 9 a.m. - 5 p.m. ET. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /J.S.M./Examiner, Art Unit 2122 /KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122 Application/Control Number: 18/450,571 Page 2 Art Unit: 2122 Application/Control Number: 18/450,571 Page 3 Art Unit: 2122 Application/Control Number: 18/450,571 Page 4 Art Unit: 2122 Application/Control Number: 18/450,571 Page 5 Art Unit: 2122 Application/Control Number: 18/450,571 Page 6 Art Unit: 2122 Application/Control Number: 18/450,571 Page 7 Art Unit: 2122 Application/Control Number: 18/450,571 Page 8 Art Unit: 2122 Application/Control Number: 18/450,571 Page 9 Art Unit: 2122 Application/Control Number: 18/450,571 Page 10 Art Unit: 2122 Application/Control Number: 18/450,571 Page 11 Art Unit: 2122 Application/Control Number: 18/450,571 Page 12 Art Unit: 2122 Application/Control Number: 18/450,571 Page 13 Art Unit: 2122 Application/Control Number: 18/450,571 Page 14 Art Unit: 2122 Application/Control Number: 18/450,571 Page 15 Art Unit: 2122 Application/Control Number: 18/450,571 Page 16 Art Unit: 2122
Prosecution Insights