DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The specification has been checked, but not to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Drawings
The applicant’s submitted drawings (including the replacement sheet filed 22 August 2023) appear to be acceptable for examination purposes. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the drawings.
Information Disclosure Statement
As required by M.P.E.P. 609(c), the applicant's submission of the Information Disclosure Statement, dated 24 October 2024, is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending. As required by M.P.E.P 609 C(2), a copy of the PTOL-1449 initialed and dated by the examiner is attached to the instant office action.
Claim Objections
Claim 12 is objected to because of the following informalities: “for tensor processing operation” appears as though it should be “for a tensor processing operation” or “for tensor processing operations” or similar. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-18 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) mathematical concepts. This judicial exception is not integrated into a practical application and does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception, as described below.
Step 1 for all claims:
Under the first part of the analysis, claims 13-18 recite a method and claims 1-12 recite a device. Accordingly, these claims fall within the four statutory categories of invention and the analysis proceeds to Step 2A, prongs 1 and 2, and Step 2B, as described below.
As per claim 1:
Under step 2A, prong 1, the claim recites an abstract idea including the following mathematical concept elements:
perform a 1x1 convolution on the input data to obtain a plurality of data groups – a 1x1 convolution is a mathematical calculation.
perform a group convolution on the plurality of data groups – the group convolution is a mathematical calculation.
perform a 1x1 convolution on the intermediate data – a 1x1 convolution is a mathematical calculation.
If a claim, under the broadest reasonable interpretation covers a mathematical relationship between variables or numbers, a numerical formula or equation, or a mathematical calculation, it will be considered as falling within the “mathematical concepts” grouping of abstract ideas. See MPEP § 2106.04(a)(2).
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
A device for implementing a tensor-train decomposition operation for a respective convolutional layer of a convolutional neural network (CNN) – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the device being configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
receive input data comprising a first number of channels – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the plurality of data groups comprising a second number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
to obtain intermediate data comprising a third number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
to obtain output data comprising a fourth number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
A device for implementing a tensor-train decomposition operation for a respective convolutional layer of a convolutional neural network (CNN) – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
the device being configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
receive input data comprising a first number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
the plurality of data groups comprising a second number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
to obtain intermediate data comprising a third number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
to obtain output data comprising a fourth number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 2:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein: the group convolution is performed based on a kernel shared between the plurality of data groups – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein: the group convolution is performed based on a kernel shared between the plurality of data groups – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 3:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein: the third number of channels is determined based on a number of data groups in the plurality of data groups – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein: the third number of channels is determined based on a number of data groups in the plurality of data groups – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 4:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein: the third number of channels is further determined based on one or more hardware characteristics of the device – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein: the third number of channels is further determined based on one or more hardware characteristics of the device – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 5:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein: each data group comprises a fifth number of channels, and wherein the second number of channels is determined based on the third number of channels and the fifth number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
wherein: each data group comprises a fifth number of channels, and wherein the second number of channels is determined based on the third number of channels and the fifth number of channels – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 6:
The claim recites the following additional mathematical concept elements:
wherein the first number of convolutional layers equals a sum of the second number of convolutional layers and the third number of decomposed convolutional layers – this is a mathematical formula/calculation (sum).
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the device according to claim 1, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
obtain the CNN comprising a first number of convolutional layers, wherein each convolutional layer is associated with a respective first ranking number – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
and provide a decomposed CNN comprising a second number of convolutional layers and a third number of decomposed convolutional layers based on a training of the CNN – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
and wherein each decomposed convolutional layer is associated with a respective second ranking number – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the device according to claim 1, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
obtain the CNN comprising a first number of convolutional layers, wherein each convolutional layer is associated with a respective first ranking number – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
and provide a decomposed CNN comprising a second number of convolutional layers and a third number of decomposed convolutional layers based on a training of the CNN – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
and wherein each decomposed convolutional layer is associated with a respective second ranking number – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 7:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the device according to claim 6, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
determine, for a respective convolutional layer of the CNN, a weighting pair based on: a weighted convolutional layer obtained by allocating a first weighting trainable parameter to the respective convolutional layer; and
a weighted decomposed convolutional layer obtained by allocating a second weighting trainable parameter to a decomposed convolutional layer determined for the respective convolutional layer – this is recited at a high level of generality and amounts to insignificant extra-solution activity as data gathering/storage that is limited to a particular type of data, generally linking the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(g) and (h), and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the device according to claim 6, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
determine, for a respective convolutional layer of the CNN, a weighting pair based on: a weighted convolutional layer obtained by allocating a first weighting trainable parameter to the respective convolutional layer; and
a weighted decomposed convolutional layer obtained by allocating a second weighting trainable parameter to a decomposed convolutional layer determined for the respective convolutional layer – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data). The courts have also found limitations directed to obtaining and storing information electronically, recited at a high level of generality, to be well-understood, routine, and conventional. See MPEP § 2106.05(d)(II) “receiving or transmitting data over a network,” "electronic record keeping,” and "storing and retrieving information in memory.”
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 8:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the device according to claim 7, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
perform an initial training iteration of the CNN based on at least the weighting pair – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the device according to claim 7, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
perform an initial training iteration of the CNN based on at least the weighting pair – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 9:
The claim recites the following additional mathematical concept elements:
determine, after performing the initial training iteration, at least one convolutional layer having a minimal first weighting trainable parameter – determining the minimal parameter from a group of parameters is a mathematical calculation.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the device according to claim 8, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the device according to claim 8, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 10:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the device according to claim 9, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
perform an additional training iteration of the CNN – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
based on substituting a weighting pair of the at least one convolutional layer having the minimal first weighting trainable parameter with a corresponding decomposed convolutional layer, and a remaining of the at least one weighting pair from a previous iteration – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the device according to claim 9, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
perform an additional training iteration of the CNN – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
based on substituting a weighting pair of the at least one convolutional layer having the minimal first weighting trainable parameter with a corresponding decomposed convolutional layer, and a remaining of the at least one weighting pair from a previous iteration – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 11:
The claim recites the following additional mathematical concept elements:
determining a respective convolutional layer having a minimal first weighting trainable parameter – determining the minimal parameter from a group of parameters is a mathematical calculation.
until a predetermined number of convolutional layers are substituted with corresponding decomposed convolutional layers – comparing the iteration/number of substituted layers value to a predetermined number is a mathematical calculation.
Accordingly, at step 2A, prong one, the claim is directed to an abstract idea.
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
the device according to claim 8, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
iteratively perform, … substituting the weighting pair of the respective convolutional layer having the minimal first weighting trainable parameter with a corresponding decomposed convolutional layer, and performing a next training iteration – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
the device according to claim 8, further configured to – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
iteratively perform, … substituting the weighting pair of the respective convolutional layer having the minimal first weighting trainable parameter with a corresponding decomposed convolutional layer, and performing a next training iteration – this amounts to generally linking the use of the judicial exception to a particular technological environment or field of use by limiting it to a particular data source or type. See MPEP § 2106.05(h) and Electric Power, 830 F.3d at 1354, 119 USPQ2d at 1742 (limiting application of abstract idea to power grid data).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 12:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
comprising an artificial intelligence accelerator adapted for tensor processing operation of the CNN – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
comprising an artificial intelligence accelerator adapted for tensor processing operation of the CNN – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f).
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 13, see the rejection of claim 1, above.
As per claim 14:
Under step 2A, prong two, the judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
A tangible, non-transitory computer-readable medium having instructions thereon, which, upon being executed by a computer, cause the steps of the method of claim 13 to be performed – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f). [Examiner’s Note: see above for the method steps themselves.]
Accordingly, at step 2A, prong two, these additional elements do not integrate the abstract idea into a practical application for the claim as a whole, because it does not impose any meaningful limits on practicing the abstract idea. See MPEP § 2106.04(d).
Under step 2B, the claims do not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the claim recites the additional elements of:
A tangible, non-transitory computer-readable medium having instructions thereon, which, upon being executed by a computer, cause the steps of the method of claim 13 to be performed – this amounts to mere instructions to apply the exception using a generic computer component, recited at a high level of generality. See MPEP § 2106.05(f). [Examiner’s Note: see above for the method steps themselves.]
Accordingly, at step 2B, these additional elements, both individually and in combination, do not amount to significantly more than the judicial exception. See MPEP § 2106.05.
Therefore, the claim is not eligible subject matter under 35 U.S.C. 101.
As per claim 15, see the rejection of claim 2, above.
As per claim 16, see the rejection of claim 3, above.
As per claim 17, see the rejection of claim 4, above.
As per claim 18, see the rejection of claim 5, above.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 13-16, and 18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Wang et al. (Wide Compression: Tensor Ring Nets, Feb 2018, pgs. 1-12).
As per claim 1, Wang teaches a device for implementing a tensor-train decomposition operation for a respective convolutional layer of a convolutional neural network (CNN) [tensor ring networks (TR-Nets) compress the fully-connected and convolutional layers of deep neural networks (pg. 1, abstract, etc.) using tensor ring decomposition (pg. 2, section 3; etc.)], the device being configured to:
receive input data comprising a first number of channels [each convolutional layer decomposes the number of channels entering the layer, which is 1 at the first input (pg. 5, fig. 5, etc.); where 1 is the first number of channels in this case];
perform a 1x1 convolution on the input data to obtain a plurality of data groups, the plurality of data groups comprising a second number of channels [in TRN, tensor decomposition is applied by combining kernel tensor factorization with convolution operations in three steps (11)-(13), where (11) is a convolutional layer from I feature maps to R2 feature maps with a 1x1 patch (pg. 5, section 3.2; etc.) and each convolutional layer decomposes the number of channels entering the layer (pg. 5, fig. 5, etc.); where the output channels of the 1x1 convolution are the second number of channels];
perform a group convolution on the plurality of data groups to obtain intermediate data comprising a third number of channels [in TRN, tensor decomposition is applied by combining kernel tensor factorization with convolution operations in three steps (11)-(13), where (12) contains R convolutional layers from R feature maps to R feature maps with a D x D patch (group convolution) (pg. 5, section 3.2; etc.) and each convolutional layer decomposes the number of channels entering the layer (pg. 5, fig. 5, etc.); where the output of the group convolution is the intermediate data with a third number of channels]; and
perform a 1x1 convolution on the intermediate data to obtain output data comprising a fourth number of channels [in TRN, tensor decomposition is applied by combining kernel tensor factorization with convolution operations in three steps (11)-(13), where (13) is a convolutional layer from R2 feature maps to O feature maps with a 1 x 1 patch. (pg. 5, section 3.2; etc.) and each convolutional layer decomposes the number of channels entering the layer (pg. 5, fig. 5, etc.); where the output of the third convolutional layer is the output data with a fourth number of channels].
As per claim 2, Wang teaches wherein: the group convolution is performed based on a kernel shared between the plurality of data groups [in TRN, tensor decomposition is applied by combining kernel tensor factorization with convolution operations in three steps (11)-(13), where (12) contains R convolutional layers from R feature maps to R feature maps with a D x D patch (group convolution) (pg. 5, section 3.2; etc.); which is a group convolution based on a shared kernel].
As per claim 3, Wang teaches wherein: the third number of channels is determined based on a number of data groups in the plurality of data groups [each convolutional layer decomposes the number of channels entering the layer (pg. 5, fig. 5, etc.); where the third number of channels is based upon the channels entering the third convolutional layer, which is based on the second, which is based on the first].
As per claim 5, Wang teaches wherein: each data group comprises a fifth number of channels, and wherein the second number of channels is determined based on the third number of channels and the fifth number of channels [each convolutional layer decomposes the number of channels entering the layer (pg. 5, fig. 5, etc.); where the number of channels of the grouped convolution (second number of channels, above) is based upon the number of channels in each member of the group (the fifth number of channels) and the desired output (third number of channels)].
As per claim 13, see the rejection of claim 1, above.
As per claim 14, Wang teaches a tangible, non-transitory computer-readable medium having instructions thereon, which, upon being executed by a computer, cause the steps of the method of claim 13 to be performed [the system may be implemented in resource-constrained devices, such as a mobile, wearable, or IOT device including application, storage, and memory constraints (pg. 1, abstract and section 1; etc.); which includes a computer executing instructions from memory].
As per claim 15, see the rejection of claim 2, above.
As per claim 16, see the rejection of claim 3, above.
As per claim 18, see the rejection of claim 5, above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 4, 6-12, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (Wide Compression: Tensor Ring Nets, Feb 2018, pgs. 1-12) in view of Choudhury (US 2020/0410336 – cited in an IDS).
As per claim 4, Wang teaches the device according to claim 3, as described above.
While Wang teaches that the decomposition is based on resource constraints (see, e.g., Wang: pg. 1, section 1), it has not been relied upon for teaching wherein: the third number of channels is further determined based on one or more hardware characteristics of the device.
Choudhury teaches wherein: the third number of channels is further determined based on one or more hardware characteristics of the device [a reconstructed tensor may be generated based on the core and factor matrices that essentially approximate the weight tensor of, e.g., a given layer of a neural network such that the reconstructed tensor has a smaller rank than the weight tensor, thereby reducing the required size and/or flops (para. 0021, etc.) and the determination of ranking for the layers and factorization is based upon the resource constraints of the system (paras. 0030-31, etc.); so, the number of channels output by the layer is based upon the resource constraints of the system, which is a hardware characteristic of the device].
Wang and Choudhury are analogous art, as they are within the same field of endeavor, namely decomposition of neural networks.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize the system’s resource constraints as part of the factorization/decomposition of the layer, which determines the number of channels, as taught by Choudhury, in the factorization/decomposition of the layers in the system/method taught by Wang.
Choudhury provides motivation as [One way to reduce the amount of required computing resources is to compress the model using one or more compression mechanisms, which include, for example, pruning, weight sharing, encoding, and performing low rank decomposition of connection weights (para. 0019, etc.) and at least one embodiment of the present invention may provide a beneficial effect such as, for example, reducing size and/or flops of neural networks compared to existing compression techniques. One or more embodiments of the present invention may also provide a beneficial effect such as, for example, allowing deep learning models with a greater number of layers and parameter to be used on device with limited resources (such as mobile phones and other edge devices in the context of IOT, for example) (para. 0079), which is also a desired outcome contemplated by Wang (see, e.g., Wang: pg. 1, section 1; etc.)].
As per claim 6, Wang teaches the device according to claim 1, further configured to:
obtain the CNN comprising a first number of convolutional layers, wherein each convolutional layer is associated with a respective first ranking number [TRN compresses a CNN including fully connected and convolutional layers (pg. 1, abstract, etc.) where each is associated with a tensor ring rank R (pg. 2, section 3; etc.)].
While Wang also teaches decomposing convolutional layers of a CNN (see above), it has not been relied upon for teaching the device further configured to: provide a decomposed CNN comprising a second number of convolutional layers and a third number of decomposed convolutional layers based on a training of the CNN; wherein the first number of convolutional layers equals a sum of the second number of convolutional layers and the third number of decomposed convolutional layers, and wherein each decomposed convolutional layer is associated with a respective second ranking number.
Choudhury teaches a device [the invention may be implemented in a mobile of IoT device, etc. (paras. 0002, 0079, etc.)] configured to: provide a decomposed CNN comprising a second number of convolutional layers and a third number of decomposed convolutional layers based on a training of the CNN [the system receives and trains a CNN model for a number of epochs, performs weighted decomposition for some of the convolutional layers based upon the determined rankings, and retrains a compressed/decomposed version of the model, (paras. 0030-31; figs. 3-4; etc.); where the low rank factorization produces a (third) number of decomposed convolutional layers, and the remaining convolutional layers are the second number of convolutional layers];
wherein the first number of convolutional layers equals a sum of the second number of convolutional layers and the third number of decomposed convolutional layers, and wherein each decomposed convolutional layer is associated with a respective second ranking number [the system receives and trains a CNN model for a number of epochs, performs weighted decomposition for some of the convolutional layers based upon the determined rankings, and retrains a compressed/decomposed version of the model, (paras. 0030-31; figs. 3-4; etc.); where the low rank factorization produces a (third) number of decomposed convolutional layers, and the remaining convolutional layers are the second number of convolutional layers].
Wang and Choudhury are analogous art, as they are within the same field of endeavor, namely decomposition of neural networks.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to include layer-wise ranking and decomposition, as taught by Choudhury, in the convolutional layer decomposition in the system/method taught by Wang.
Choudhury provides motivation as [One way to reduce the amount of required computing resources is to compress the model using one or more compression mechanisms, which include, for example, pruning, weight sharing, encoding, and performing low rank decomposition of connection weights (para. 0019, etc.) and at least one embodiment of the present invention may provide a beneficial effect such as, for example, reducing size and/or flops of neural networks compared to existing compression techniques. One or more embodiments of the present invention may also provide a beneficial effect such as, for example, allowing deep learning models with a greater number of layers and parameter to be used on device with limited resources (such as mobile phones and other edge devices in the context of IOT, for example) (para. 0079), which is also a desired outcome contemplated by Wang (see, e.g., Wang: pg. 1, section 1; etc.)].
As per claim 7, Wang/Choudhury teaches the device according to claim 6, further configured to determine, for a respective convolutional layer of the CNN, a weighting pair based on:
a weighted convolutional layer obtained by allocating a first weighting trainable parameter to the respective convolutional layer [the filter significance determination module performs function based on the set of statistics to determine filter significance (W) for each layer of the trained model, the significance values are then provided as input to the weighted decomposition module, which multiplies the filter values with the filter significance. At 336, the module 308 determines a rank based at least in part on one or more resource constraints 314, and then performs a low rank factorization process as represented by 334. A compressed version 312 of the trained model 302 is output by the weighted decomposition module 308 (Choudhury: paras. 0030-31; figs. 3-4; etc.); where the determined rank weighting for the convolutional layer before decomposition is the first weighting trainable parameter]; and
a weighted decomposed convolutional layer obtained by allocating a second weighting trainable parameter to a decomposed convolutional layer determined for the respective convolutional layer [the filter significance determination module performs function based on the set of statistics to determine filter significance (W) for each layer of the trained model, the significance values are then provided as input to the weighted decomposition module, which multiplies the filter values with the filter significance. At 336, the module 308 determines a rank based at least in part on one or more resource constraints 314, and then performs a low rank factorization process as represented by 334. A compressed version 312 of the trained model 302 is output by the weighted decomposition module 308 (Choudhury: paras. 0030-31; figs. 3-4; etc.); where the determined weight for the convolutional layer after decomposition is the second weighting trainable parameter].
As per claim 8, Wang/Choudhury teaches the device according to claim 7, further configured to:
perform an initial training iteration of the CNN based on at least the weighting pair [The filter significance determination module 306 performs a few epochs of training (forward and backward pass) as shown at 322, and then computes a set of statistics at 324. The trained model 302 is retrained at 332 using the dataset 304 and the resulting tensor. A compressed version 312 of the trained model 302 is then output by the weighted decomposition module 308 (Choudhury: paras. 0030-31; figs. 3-4; etc.); where the retraining iterations are based upon the ranking and decomposed weightings].
As per claim 9, Wang/Choudhury teaches the device according to claim 8, further configured to:
determine, after performing the initial training iteration, at least one convolutional layer having a minimal first weighting trainable parameter [At 326, the filter significance determination module 306 performs one or more of a step, sigmoid, and logistic function based on the set of statistics to determine filter significance (W) for each layer of the trained model. The filter significance values are then provided as input to the weighted decomposition module 308. The weighted decomposition module 308 multiplies the filter values with the filter significance as shown at 338 to obtain an input tensor for a given layer of the neural network. At 336, the module 308 determines a rank based at least in part on one or more resource constraints 314, and then performs a low rank factorization process as represented by 334 (Choudhury: paras. 0030-31; figs. 3-4; etc.); where the low rank factorization is the determination of the minimal first weighting trainable parameter].
As per claim 10, Wang/Choudhury teaches the device according to claim 9, further configured to:
perform an additional training iteration of the CNN, based on substituting a weighting pair of the at least one convolutional layer having the minimal first weighting trainable parameter with a corresponding decomposed convolutional layer, and a remaining of the at least one weighting pair from a previous iteration [The trained model 302 is retrained at 332 using the dataset 304 and the resulting tensor. A compressed version 312 of the trained model 302 is then output by the weighted decomposition module 308 (Choudhury: paras. 0030-31; figs. 3-4; etc.); which includes the decomposed convolutional layer and its associated minimal first weighting and ranking (see above)].
As per claim 11, Wang/Choudhury teaches the device according to claim 8, further configured to:
iteratively perform, determining a respective convolutional layer having a minimal first weighting trainable parameter, substituting the weighting pair of the respective convolutional layer having the minimal first weighting trainable parameter with a corresponding decomposed convolutional layer, and performing a next training iteration, until a predetermined number of convolutional layers are substituted with corresponding decomposed convolutional layers [At 326, the filter significance determination module 306 performs one or more of a step, sigmoid, and logistic function based on the set of statistics to determine filter significance (W) for each layer of the trained model. The filter significance values are then provided as input to the weighted decomposition module 308. The weighted decomposition module 308 multiplies the filter values with the filter significance as shown at 338 to obtain an input tensor for a given layer of the neural network. At 336, the module 308 determines a rank based at least in part on one or more resource constraints 314, and then performs a low rank factorization process as represented by 334. The low factorization process generates a resulting tensor having a rank equal to the determined rank, wherein the resulting tensor approximates the input tensor. The trained model 302 is retrained at 332 using the dataset 304 and the resulting tensor. A compressed version 312 of the trained model 302 is then output by the weighted decomposition module 308 (Choudhury: paras. 0030-31; figs. 3-4; etc.); which ranking, low rank factorization/decomposition, and retraining is iteratively performing the steps].
As per claim 12, Wang/Choudhury teaches comprising an artificial intelligence accelerator adapted for tensor processing operation of the CNN [the processor(s) used to implement the invention can include special purpose processors/hardware for executing the operations (Choudhury: paras. 0049-51), which is an AI accelerator adapted for tensor processing operations of the CNN, as it is hardware specifically designed for the CNN tensor operations described by the embodiments of Choudhury (see above)].
As per claim 17, see the rejection of claim 4, above.
Conclusion
The following is a summary of the treatment and status of all claims in the application as recommended by M.P.E.P. 707.07(i): claims 1-18 are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Cheng (US 2021/0241094) – discloses using reinforcement learning for rank selection in tensor decomposition of neural network layers.
Sather (US 12,061,988) – discloses layer-wise decomposition of weight tensors, including decomposing a layer into multiple layers.
Kim et al. (Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications, Feb 2016, pgs. 1-16) – discloses decomposition/compression of CNNs.
Lebedev et al. (Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition, April 2015, pgs. 1-11) – discloses low-rank CP-decomposition of convolutional layers.
The examiner requests, in response to this Office action, that support be shown for language added to any original claims on amendment and any new claims. That is, indicate support for newly added claim language by specifically pointing to page(s) and line number(s) in the specification and/or drawing figure(s). This will assist the examiner in prosecuting the application.
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the references cited or the objections made. He or she must also show how the amendments avoid such references or objections. See 37 CFR 1.111(c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEORGE GIROUX whose telephone number is (571)272-9769. The examiner can normally be reached M-F 10am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at 571-272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEORGE GIROUX/Primary Examiner, Art Unit 2128
Prosecution Insights