DETAILED ACTION
This action is in response to the Applicant Response filed 25 June 2026 for application 16/177,282 filed 31 October 2018.
Claim(s) 1, 11 is/are currently amended.
Claim(s) 1-20 is/are pending.
Claim(s) 1-20 is/are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments regarding the objections to the claims have been fully considered and, in light of the amendments to the claims, are persuasive. However, in light of the amendments to the claims, new claim objections have arisen, as noted below.
Applicant's arguments regarding the 35 U.S.C. 112(a) rejection(s) of claim(s) 1-20 have been fully considered and, in light of the amendments to the claims, are persuasive. The 35 U.S.C. 112(a) rejection(s) of claim(s) 1-20 has/have been withdrawn.
Applicant’s arguments regarding the 35 U.S.C. 102 and/or 35 U.S.C. 103 rejections of the claims are based on the newly amended subject matter. All arguments are addressed in the 35 U.S.C. 102 and/or 35 U.S.C. 103 rejections of the claims below.
Claim Objections
Claim(s) 1-20 is/are objected to because of the following informalities:
Claim 1, line 8, “and” should be removed
Claim 1, line 10, by a processor should read “by [[a]]the processor”
Claim 10, lines 1-2, wherein constructing the new neural network using the plurality of layers selected from one or more neural networks in the ecosystem should read “wherein constructing the new neural network using the plurality of layers selected from at least two different pre-trained neural networks having different network architectures in the ecosystem”
Claim 11, line 11, construct the fully-layered executable neural network instance should read “construct [[the]]a fully-layered executable neural network instance”
Claim 12, line 2, the fully-layered ad hoc neural network should read “the fully-layered executable neural network instance”
Claim 13, lines 1-2, the layer activation processor based neural network model should read “the trained layer-selection processor-based neural network model”
Claim 15, line 2, the fully-layered ad hoc neural network should read “the fully-layered executable neural network instance”
Claim 16, line 3, the layer activation processor based neural network model should read “the trained layer-selection processor-based neural network model”
Claim 17, lines 1-2, the layer activation processor based neural network model should read “the trained layer-selection processor-based neural network model”
Claim 18, lines 1-2, the layer activation processor based neural network model should read “the trained layer-selection processor-based neural network model”
Claim 19, lines 1-2, the layer activation processor based neural network model should read “the trained layer-selection processor-based neural network model”
Claim 20, lines 1-2, forming the fully-layered ad hoc neural network should read “constructing the fully-layered executable neural network instance”
Claim 20, line 2, the plurality of layers activated should read “the plurality of layers selected”
Claims 2-10, 12-20 are objected to due to their dependence, either directly or indirectly, on claims 1, 10-13, 15-20
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101, because the claim(s) is/are directed to an abstract idea, and because the claim elements, whether considered individually or in combination, do not amount to significantly more than the abstract idea, see Alice Corporation Pty. Ltd. V. CLS Bank International et al., 573 US 208 (2014).
Regarding claim 1, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file.
The limitation of analyzing ... attributes of the input file to identify media characteristics, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of selecting ... a plurality of layers from the candidate layer combinations, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of constructing, in real-time ... a new neural network using the plurality of layers selected from at least two different pre-trained neural networks having different network architectures in the ecosystem, wherein the new neural network is fully-layered, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of classifying ... the input file ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of determining a performance score through computational analysis of classification results, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a score.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – processor. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – trained layer-selection neural network, ecosystem of pre-trained neural networks, new fully-layered neural network. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites executing, by the processor, a trained layer-selection neural network, distinct from a classification neural network, the trained layer-selection neural network receiving the media characteristics as inputs and outputting candidate layer combinations selected from an ecosystem of pre-trained neural networks; ... using the new fully-layered neural network, wherein the classification employs a dynamic configuration of the selected layers that are activated based on the input file's attributes which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
processor amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
trained layer-selection neural network, ecosystem of pre-trained neural networks, new fully-layered neural network amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 2, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 2 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the input file comprises an image having a first and second object, wherein the new neural network is constructed based on one or more attributes of the first object.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the input and the neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the input and the neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 3, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file.
The limitation of selecting ... a second plurality of layers from one or more neural networks in the ecosystem based on one or more attributes of the second object, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of constructing, in real-time, a second neural network using the second plurality of layers selected from the ecosystem, wherein the second neural network is fully-layered, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of classifying the second object ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – second fully-layered neural network. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
second fully-layered neural network amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 4, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 4 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the ecosystem of pre-trained neural networks comprises multiple neural networks of different network architectures.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the ecosystem of pre-trained neural networks comprises multiple neural networks of different network architectures which is simply additional information regarding the ecosystem, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – multiple neural networks of different network architectures. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
multiple neural networks of different network architectures amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the ecosystem do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 5, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 5 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the input file is a multimedia file, and wherein the new neural network comprises layers of different classes of data comprising audio, object, and text.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the input and the neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the input and the neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 6, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file.
The limitation of analyzing the input file to identify one or more attributes of the input file prior to selecting the plurality of layers from the ecosystem, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 7, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 7 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the trained layer selection neural network is trained to match one or more attributes of the input file to features of a layer of a neural network in the ecosystem.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the trained layer selection neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the trained layer selection neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 8, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 8 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the trained layer selection neural network is trained to match one or more attributes of the input file to a portion of a layer of a neural network in the ecosystem.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the trained layer selection neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the trained layer selection neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 9, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file. The Step 2A Prong One Analysis for claim 1 is applicable here since claim 9 carries out the method of claim 1 but for the recitation of additional element(s) of wherein the trained layer selection neural network is trained to match one or more attributes of the input file to one or more neurons of a layer of a neural network in the ecosystem.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the trained layer selection neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the trained layer selection neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 10, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) method of classifying an input file.
The limitation of wherein constructing the new neural network using the plurality of layers selected from one or more neural networks in the ecosystem comprises activating the selected plurality of layers while disabling non-selected layers from the ecosystem of layers of pre-trained neural networks, wherein only activated layers can receive or output encoded data in classifying the input file, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 11, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system.
The limitation of ... output candidate layer combinations selected from the ecosystem of pre-trained processor based neural networks, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... select a plurality of layers from the candidate layer combinations, wherein the selected plurality of layers are selected from at least two different pre-trained neural networks having different network architectures in the ecosystem, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... construct the fully-layered executable neural network instance using the selected plurality of layers, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... classify the input file ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... determine a performance score through computational analysis of classification results, as drafted, is a process that, under its broadest reasonable interpretation, covers a mathematical concept. The limitation encompasses calculating a score.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. If a claim limitation, under its broadest reasonable interpretation, covers performance of mathematical concepts, then it falls within the "Mathematical Concepts" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – processor, memory. The additional element(s) is/are recited at a high-level of generality (i.e., as generic computer components performing generic computer functions of executing instructions on the computers) such that it amounts to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b)).
The claim recites additional element(s) – an ecosystem of pre-trained processor based neural networks having a plurality of network architectures, trained layer-selection processor-based neural network model, fully-layered executable neural network instance. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
The claim recites ... using the fully-layered executable neural network instance, wherein the classification employs a dynamic configuration of the selected layers that are activated based on the input file's attributes which is simply applying a model recited at a high level of generality and amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer (MPEP 2106.05(f)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
processor, memory amount(s) to no more than mere instructions to apply the exception using generic computer components (MPEP 2106.05(b))
applying a model amount(s) to no more than mere instructions to apply the exception (MPEP 2106.05(f))
an ecosystem of pre-trained processor based neural networks having a plurality of network architectures, trained layer-selection processor-based neural network model, fully-layered executable neural network instance amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 12, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 12 carries out the neural network system of claim 11 but for the recitation of additional element(s) of wherein the input file comprises an image having a first and second object, wherein the fully-layered ad hoc neural network is formed based on one or more attributes of the first object.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the input and the neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the input and the neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 13, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system.
The limitation of ... form, in real-time, a second fully-layered ad hoc neural network by activating a set of layers selected from the ecosystem, wherein the second fully-layered ad hoc neural network is fully-layered, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
The limitation of ... classify the second object ..., as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites additional element(s) – second fully-layered ad hoc neural network. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
second fully-layered neural network amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 14, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 14 carries out the neural network system of claim 11 but for the recitation of additional element(s) of wherein the ecosystem of pre-trained processor based neural networks comprises multiple neural networks of different network architectures.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application.
The claim recites wherein the ecosystem of pre-trained processor based neural networks comprises multiple neural networks of different network architectures which is simply additional information regarding the ecosystem, and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)).
The claim recites additional element(s) – multiple neural networks of different network architectures. The additional element(s) is/are recited at a high-level of generality such that it amounts to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h)).
Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of:
multiple neural networks of different network architectures amount(s) to no more than indicating a field of use or technological environment in which to apply the judicial exception (MPEP 2106.05(h))
additional information regarding the ecosystem do(es) not apply the exception in a meaningful way (MPEP 2106.05(e))
The additional element(s) do(es) not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 15, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 15 carries out the neural network system of claim 11 but for the recitation of additional element(s) of wherein the input file is a multimedia file, and wherein the fully-layered ad hoc neural network comprises layers of different classes of data comprising audio, object, and text.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the input and the neural network and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the input and the neural network do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 16, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system.
The limitation of ... analyze the input file to identify one or more attributes of the input file prior to the layer activation processor based neural network model activating the plurality of layers from the ecosystem, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 17, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 17 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 17 carries out the neural network system of claim 11 but for the recitation of additional element(s) of wherein the layer activation processor based neural network model is trained to match one or more attributes of the input file to features of a layer of a pre-trained processor based neural network in the ecosystem.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the layer activation processor based neural network model and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the layer activation processor based neural network model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 18, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 18 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 18 carries out the neural network system of claim 11 but for the recitation of additional element(s) of wherein the layer activation processor based neural network model is trained to match one or more attributes of the input file to a portion of a layer of a pre-trained processor based neural network in the ecosystem.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the layer activation processor based neural network model and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the layer activation processor based neural network model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 19, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system. The Step 2A Prong One Analysis for claim 11 is applicable here since claim 19 carries out the neural network system of claim 11 but for the recitation of additional element(s) of wherein the layer activation processor based neural network model is trained to match one or more attributes of the input file to one or more neurons of a layer of a pre-trained processor based neural network in the ecosystem.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated into a practical application. In particular, the claim recites additional information regarding the layer activation processor based neural network model and the element(s) do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Accordingly, the additional element(s) do(es) not integrate the abstract idea into a practical application because the additional element(s) do(es) not impose any meaningful limits on practicing the abstract idea, and, therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element(s) of additional information regarding the layer activation processor based neural network model do(es) not apply the exception in a meaningful way (MPEP 2106.05(e)). Not applying the exception in a meaningful way does not provide an inventive concept, and, therefore, the claim is not patent eligible.
Regarding claim 20, the claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 20 is directed to a system with a processor, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The claim recites a(n) neural network system.
The limitation of wherein forming the fully-layered ad hoc neural network using the plurality of layers activated from one or more pre-trained processor based neural networks in the ecosystem comprises activating a selected plurality of layers while disabling non-selected layers from the ecosystem of pre-trained processor based neural networks having two or more layers, wherein only activated layers can receive or output encoded data in classifying the input file, as drafted, is a process that, under its broadest reasonable interpretation, covers a mental process. The limitation is directed to observation, evaluation, judgment and opinion and is a process capable of being performed by a human mentally or using pen and paper.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, then it falls within the "Mental Processes" grouping. Accordingly, the claim recites an abstract idea.
Step 2A Prong Two Analysis: With respect to the abstract idea, the judicial exception is not integrated
into a practical application. The claim does not recite any additional elements which integrate the
abstract idea into a practical application and, therefore, does not impose any meaningful limits on
practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception. As discussed above with respect to the integration of the
abstract idea into a practical application, the claim does not recite any additional elements which
provide an inventive concept, and, therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 4, 6-11, 14, 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (SkipNet: Learning Dynamic Routing in Convolutional Networks, hereinafter referred to as “Wang”) in view of Chou et al. (Unifying and Merging Well-trained Deep Neural Networks for Inference Stage, hereinafter referred to as “Chou”).
Regarding claim 1 (Currently amended), Wang teaches a method of classifying an input file (Wang, section 4 – teaches using method for classification), the method comprising:
analyzing, by a processor (Wang, section 4 – teaches performing experiments using pre-existing known computer models and datasets and evaluating computational costs and accuracy [It would be obvious to a person skilled in the art that a processor is needed to perform these experiments and evaluations]), attributes of the input file to identify media characteristics (Wang, section 4 – teaches data augmentation (mirroring/shifting) and normalization with the channel means and standard deviations);
executing, by the processor, a trained layer-selection neural network, distinct from a classification neural network (Wang, section 3 – teaches layer selection is accomplished using gating networks interposed between layers; Wang, section 3.1 – teaches neural network gating networks), the trained layer-selection neural network receiving the media characteristics as inputs and outputting candidate layer combinations selected from an ecosystem of pre-trained neural networks (ang, section 3 – teaches the gating networks [computational analysis] mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer); and
selecting, by the processor, a plurality of layers from the candidate layer combinations (Wang, section 3 – teaches SkipNets are CNNs [pre-trained] in which individual layers are selectively included or excluded for a given input using gating networks);
constructing, in real-time, by a processor (Wang, section 4 – teaches performing experiments using pre-existing known computer models and datasets and evaluating computational costs and accuracy [It would be obvious to a person skilled in the art that a processor is needed to perform these experiments and evaluations]), a new neural network using the plurality of layers selected from … pre-trained neural networks … in the ecosystem (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded for a given input using gating networks mapping the output of the previous layer or group of layers to a binary decision to execute or bypass the next layer [selecting for each input based on previous layer means real-time construction]), wherein the new neural network is fully-layered (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded using gating networks; Wang, section 4 – teaches using SkipNets for classification [Fully-layered networks are interpreted as models which take input and produce classification output using layer selection]);
classifying, by a processor (Wang, section 4 – teaches performing experiments using pre-existing known computer models and datasets and evaluating computational costs and accuracy [It would be obvious to a person skilled in the art that a processor is needed to perform these experiments and evaluations]), the input file using the new fully-layered neural network (Wang, section 4 – teaches using method for classification using the SkipNets), wherein the classification employs a dynamic configuration of the selected layers that are activated based on the input file's attributes (Wang, section 4 – teaches using method for classification using the SkipNets); and
determining a performance score through computational analysis of classification results (Wang, sections 3.2-3.3 – teaches supervised learning which uses objective optimization with reinforcement learning which determines a performance score based on the classification used to update the skip gates).
While Wang teaches selecting pre-existing layers from an ecosystem, Wang does not explicitly teach that the layers are selected from at least two different pre-trained neural networks having different network architectures in the ecosystem.
Chou teaches constructing a new neural network using the plurality of layers selected from at least two different pre-trained neural networks having different network architectures in the ecosystem (Chou, section 1 – teaches merging multiple well-trained feed forward networks with differing architectures) …
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Wang with the teachings of Chou in order to produce a more compact model to handle original tasks simultaneously while consuming less computational time and storage than the compound model of the original networks in the field of model generation using layers of pre-existing models (Chou, section 1 – “… (2) The proposed method produces a more compact model to handle the original tasks simultaneously. The compact model consumes less computational time and storage than the compound model of the original networks. It has a great potential to be fitted in low-end systems.”).
Regarding claim 4 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Chou further teaches wherein the ecosystem of pre-trained neural networks comprises multiple neural networks of different network architectures (Chou, section 1 – teaches merging multiple well-trained feed forward networks with differing architectures).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou in order to combine networks of different architectures to produce a more compact model to handle original tasks simultaneously while consuming less computational time and storage than the compound model of the original networks (Chou, section 1).
Regarding claim 6 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches analyzing the input file to identify one or more attributes of the input file prior to selecting the plurality of layers from the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 1 above.
Regarding claim 7 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches wherein the trained layer selection neural network is trained to match one or more attributes of the input file to features of a layer of a neural network in the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer; [Choosing to execute the next layer based on attributes from the previous layer is interpreted as matching features to the features of the next layer]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 1 above.
Regarding claim 8 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches wherein the trained layer selection neural network is trained to match one or more attributes of the input file to a portion of a layer of a neural network in the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer; [Choosing to execute the next layer based on attributes from the previous layer is interpreted as matching features to the features of the next layer, including portions thereof]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 1 above.
Regarding claim 9 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches wherein the trained layer selection neural network is trained to match one or more attributes of the input file to one or more neurons of a layer of a neural network in the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer; [Choosing to execute the next layer based on attributes from the previous layer is interpreted as matching features to the features of the next layer, including neurons thereof]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 1 above.
Regarding claim 10 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches wherein constructing the new neural network using the plurality of layers selected from one or more neural networks in the ecosystem comprises activating the selected plurality of layers while disabling non-selected layers from the ecosystem of layers of pre-trained neural networks (Wang, section 3 – teaches SkipNets are CNNs [pre-trained] in which individual layers are selectively included or excluded for a given input), wherein only activated layers can receive or output encoded data in classifying the input file (Wang, section 3 – teaches the gating networks map the output of the previous layer or group of layers to a binary decision to execute or bypass the subsequent layer or group of layers; see also Wang, Figs. 1, 2).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 1 above.
Regarding claim 11 (Currently amended), Wang teaches a neural network system (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded for a given input) comprising:
a trained layer-selection processor-based neural network model distinct from a classification neural network (Wang, section 3 – teaches layer selection is accomplished using gating networks interposed between layers; Wang, section 3.1 – teaches neural network gating networks), configured to output candidate layer combinations selected from the ecosystem of pre-trained processor based neural networks (Wang, section 3 – teaches SkipNets are CNNs [pre-trained] in which individual layers are selectively included or excluded for a given input using gating networks); and
a processor coupled to a memory (Wang, section 4 – teaches performing experiments using pre-existing known computer models and datasets and evaluating computational costs and accuracy [It would be obvious to a person skilled in the art that a processor is needed to perform these experiments and evaluations]), the processor configured to select a plurality of layers from the candidate layer combinations, wherein the selected plurality of layers are selected from … pre-trained neural networks … in the ecosystem (Wang, section 3 – teaches SkipNets are CNNs [pre-trained] in which individual layers are selectively included or excluded for a given input using gating networks), construct the fully-layered executable neural network instance using the selected plurality of layers (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded for a given input using gating networks mapping the output of the previous layer or group of layers to a binary decision to execute or bypass the next layer [selecting for each input based on previous layer means real-time construction]; Wang, section 4 – teaches using SkipNets for classification [Fully-layered networks are interpreted as models which take input and produce classification output using layer selection]), classify the input file using the fully-layered executable neural network instance (Wang, section 4 – teaches using method for classification using the SkipNets), wherein the classification employs a dynamic configuration of the selected layers that are activated based on the input file's attributes (ang, section 4 – teaches using method for classification using the SkipNets), and determine a performance score through computational analysis of classification results (ang, sections 3.2-3.3 – teaches supervised learning which uses objective optimization with reinforcement learning which determines a performance score based on the classification used to update the skip gates).
While Wang teaches selecting pre-existing layers from an ecosystem, Wang does not explicitly teach that the layers are selected from at least two different pre-trained neural networks having different network architectures in the ecosystem.
Chou teaches
an ecosystem of pre-trained processor based neural networks having a plurality of network architectures (Chou, section 1 – teaches merging multiple well-trained feed forward networks with differing architectures);
… select a plurality of layers from the candidate layer combinations, wherein the selected plurality of layers are selected from at least two different pre-trained neural networks having different network architectures in the ecosystem (Chou, section 1 – teaches merging multiple well-trained feed forward networks with differing architectures) …
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Wang with the teachings of Chou in order to produce a more compact model to handle original tasks simultaneously while consuming less computational time and storage than the compound model of the original networks in the field of model generation using layers of pre-existing models (Chou, section 1 – “… (2) The proposed method produces a more compact model to handle the original tasks simultaneously. The compact model consumes less computational time and storage than the compound model of the original networks. It has a great potential to be fitted in low-end systems.”).
Regarding claim 14 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Chou further teaches wherein the ecosystem of pre-trained processor based neural networks comprises multiple neural networks of different network architectures (Chou, section 1 – teaches merging multiple well-trained feed forward networks with differing architectures).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou in order to combine networks of different architectures to produce a more compact model to handle original tasks simultaneously while consuming less computational time and storage than the compound model of the original networks (Chou, section 1).
Regarding claim 16 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein the processor is further configured to analyze the input file to identify one or more attributes of the input file prior to the layer activation processor based neural network model activating the plurality of layers from the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 11 above.
Regarding claim 17 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein the layer activation processor based neural network model is trained to match one or more attributes of the input file to features of a pre-trained layer of a processor based neural network in the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer for a pre-trained network; [Choosing to execute the next layer based on attributes from the previous layer is interpreted as matching features to the features of the next layer]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 11 above.
Regarding claim 18 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein the layer activation processor based neural network model is trained to match one or more attributes of the input file to a portion of a layer of a pre-trained processor based neural network in the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer for a pre-trained network; [Choosing to execute the next layer based on attributes from the previous layer is interpreted as matching features to the features of the next layer, including portions thereof]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 11 above.
Regarding claim 19 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein the layer activation processor based neural network model is trained to match one or more attributes of the input file to one or more neurons of a layer of a pre-trained processor based neural network in the ecosystem (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer for a pre-trained network; [Choosing to execute the next layer based on attributes from the previous layer is interpreted as matching features to the features of the next layer, including neurons thereof]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 11 above.
Regarding claim 20 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein forming the fully-layered ad hoc neural network using the plurality of layers activated from one or more pre-trained processor based neural networks in the ecosystem comprises activating a selected plurality of layers while disabling non-selected layers from the ecosystem of pre-trained processor based neural networks having two or more layers (Wang, section 3 – teaches SkipNets are CNNs [pre-trained] in which individual layers are selectively included or excluded for a given input; see also Wang, Figs. 1-2 – multiple layers), wherein only activated layers can receive or output encoded data in classifying the input file (Wang, section 3 – teaches the gating networks map the output of the previous layer or group of layers to a binary decision to execute or bypass the subsequent layer or group of layers for a pre-trained network; see also Wang, Figs. 1, 2).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang and Chou for the same reasons as disclosed in claim 11 above.
Claims 2-3, 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Chou and further in view of Girshick et al. (Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, hereinafter referred to as “Girshick”).
Regarding claim 2 (Original), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches wherein the new neural network is constructed based on one or more attributes of the first object (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer for a given input [first object]).
While Wang in view of Chou teaches performing SkipNets for a given input, Wang in view of Chou does not explicitly teach wherein the input file comprises an image having a first and second object.
Girshick teaches wherein the input file comprises an image having a first and second object (Girshick, Figure 1 – teaches an image having multiple regions, e.g., a person and a horse), wherein the new neural network is constructed based on one or more attributes of the first object (Girshick, section 2 – teaches developing regions for each object in the region and propagating each region/object individually through the CNN to classify the object).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Wang in view of Chou with the teachings of Girshick in order to develop a simple and scalable image detection and classification model for multiple objects in an image which outperforms existing methods in the field of neural network generation and object classification (Girshick, Abstract – “Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012—achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features...”).
Regarding claim 3 (Original), Wang in view of Chou and further in view of Girshick teaches all of the limitations of the method of claim 2 as noted above. Wang further teaches
selecting, using the trained layer selection neural network (Wang, section 3 – teaches layer selection is accomplished using gating networks interposed between layers; Wang, section 3.1 – teaches neural network gating networks), a second plurality of layers from one or more neural networks in the ecosystem (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded for a given input; [Because the layers are selected in real-time based on the input, a second plurality of layers would be selected for a second object]) based on one or more attributes of the second object (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer; [Because the layers are selected in real-time based on the input, a second plurality of layers would be selected for a second object]);
constructing, in real-time, a second neural network using the second plurality of layers selected from the ecosystem (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded for a given input using gating networks mapping the output of the previous layer or group of layers to a binary decision to execute or bypass the next layer [selecting for each input based on previous layer means real-time construction]), wherein the second neural network is fully-layered (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded using gating networks; Wang, section 4 – teaches using SkipNets for classification [Fully-layered networks are interpreted as models which take input and produce classification output using layer selection]); and
classifying the second object using the second fully-layered neural network (Wang, section 4 – teaches using method for classification using the SkipNets; [Because the layers are selected in real-time based on the input, a second plurality of layers would be selected to classify a second object]).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang, Chou and Girshick for the same reasons as disclosed in claim 2 above.
Regarding claim 12 (Original), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein the fully-layered ad hoc neural network is formed based on one or more attributes of the first object (Wang, section 3 – teaches the gating networks mapping the output of the previous layer or group of layers [feature maps/attributes for a CNN] to a binary decision to execute or bypass the next layer for a given input [first object]).
While Wang in view of Chou teaches performing SkipNets for a given input, Wang in view of Chou does not explicitly teach wherein the input file comprises an image having a first and second object.
Girshick teaches wherein the input file comprises an image having a first and second object (Girshick, Figure 1 – teaches an image having multiple regions, e.g., a person and a horse), wherein the fully-layered ad hoc neural network is formed based on one or more attributes of the first object (Girshick, section 2 – teaches developing regions for each object in the region and propagating each region/object individually through the CNN to classify the object).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Wang in view of Chou with the teachings of Girshick in order to develop a simple and scalable image detection and classification model for multiple objects in an image which outperforms existing methods in the field of neural network generation and object classification (Girshick, Abstract – “Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012—achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features...”).
Regarding claim 13 (Previously Presented), Wang in view of Chou and further in view of Girshick teaches all of the limitations of the neural network system of claim 12 as noted above. Wang further teaches
wherein the layer activation processor based neural network model (Wang, section 3 – teaches layer selection is accomplished using gating networks interposed between layers; Wang, section 3.1 – teaches neural network gating networks) is further configured to form, in real-time, a second fully-layered ad hoc neural network (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded for a given input using gating networks mapping the output of the previous layer or group of layers to a binary decision to execute or bypass the next layer [selecting for each input based on previous layer means real-time construction; Because the layers are selected in real-time based on the input, a second plurality of layers would be selected for a second object]) by activating a set of layers selected from the ecosystem (Wang, section 3 – teaches SkipNets are CNNs [pre-trained] in which individual layers are selectively included or excluded for a given input using gating networks), wherein the second fully-layered ad hoc neural network is fully-layered (Wang, section 3 – teaches SkipNets are CNNs in which individual layers are selectively included or excluded using gating networks; Wang, section 4 – teaches using SkipNets for classification [Fully-layered networks are interpreted as models which take input and produce classification output using layer selection]); and
wherein the processor is configured to classify the second object using the second fully-layered ad hoc neural network (Wang, section 4 – teaches using method for classification using the SkipNets; [Because the layers are selected in real-time based on the input, a second plurality of layers would be selected to classify a second object])).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to combine the teachings of Wang, Chou and Girshick for the same reasons as disclosed in claim 12 above.
Claim 5, 15 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Chou and further in view of Zahavy et al. (Is a Picture Worth a Thousand Words? A Deep Multimodal Architecture for Product Classification in E-Commerce, hereinafter referred to as “Zahavy”).
Regarding claim 5 (Previously Presented), Wang in view of Chou teaches all of the limitations of the method of claim 1 as noted above. Wang further teaches wherein the input file is a multimedia file (Wang, section 4 – teaches datasets of images).
While Wang in view of Chou teaches that SkipNets can be applied to different models, Wang in view of Chou does not explicitly teach wherein new neural network comprises of layers of different classes of data comprising audio, object, and text.
Zahavy teaches wherein the input file is a multimedia file (Zahavy, Experiments section - teaches dataset of products comprising title image and shelf [text/image]), and wherein the new neural network comprises layers of different classes of data comprising audio, object, and text (Zahavy, Methods and Architectures section – teaches multimodal text CNN architecture [text] and VCC network for images [object]; see also Zahavy, Abstract, Multi-Modality section – teaches expanding algorithms to other modalities, including audio).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Wang in view of Chou with the teachings of Zahavy in order to improve classification accuracy in the field of neural network generation (Zahavy, Abstract – “Classifying products precisely and efficiently is a major challenge in modern e-commerce. The high traffic of new products uploaded daily and the dynamic nature of the categories raise the need for machine learning models that can reduce the cost and time of human editors. In this paper, we propose a decision level fusion approach for multi-modal product classification based on text and image neural network classifiers. We train input specific state-of-the-art deep neural networks for each input source, show the potential of forging them together into a multi-modal architecture and train a novel policy network that learns to choose between them. Finally, we demonstrate that our multi-modal network improves classification accuracy over both networks on a real-world largescale product classification dataset that we collected from Walmart.com. While we focus on image-text fusion that characterizes e-commerce businesses, our algorithms can be easily applied to other modalities such as audio, video, physical sensors, etc.”).
Regarding claim 15 (Previously Presented), Wang in view of Chou teaches all of the limitations of the neural network system of claim 11 as noted above. Wang further teaches wherein the input file is a multimedia file (Wang, section 4 – teaches datasets of images).
While Wang in view of Chou teaches that SkipNets can be applied to different models, Wang in view of Chou does not explicitly teach wherein new neural network comprises of layers of different classes of data comprising audio, object, and text.
Zahavy teaches wherein the input file is a multimedia file (Zahavy, Experiments section - teaches dataset of products comprising title image and shelf [text/image]), and wherein the fully-layered ad hoc neural network comprises layers of different classes of data comprising audio, object, and text (Zahavy, Methods and Architectures section – teaches multimodal text CNN architecture [text] and VCC network for images [object]; see also Zahavy, Abstract, Multi-Modality section – teaches expanding algorithms to other modalities, including audio).
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to modify Wang in view of Chou with the teachings of Zahavy in order to improve classification accuracy in the field of neural network generation (Zahavy, Abstract – “Classifying products precisely and efficiently is a major challenge in modern e-commerce. The high traffic of new products uploaded daily and the dynamic nature of the categories raise the need for machine learning models that can reduce the cost and time of human editors. In this paper, we propose a decision level fusion approach for multi-modal product classification based on text and image neural network classifiers. We train input specific state-of-the-art deep neural networks for each input source, show the potential of forging them together into a multi-modal architecture and train a novel policy network that learns to choose between them. Finally, we demonstrate that our multi-modal network improves classification accuracy over both networks on a real-world largescale product classification dataset that we collected from Walmart.com. While we focus on image-text fusion that characterizes e-commerce businesses, our algorithms can be easily applied to other modalities such as audio, video, physical sensors, etc.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to MARSHALL WERNER whose telephone number is (469) 295-9143. The examiner can normally be reached on Monday – Thursday 7:30 AM – 4:30 PM ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARSHALL L WERNER/ Primary Examiner, Art Unit 2125
Prosecution Insights