Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are presented for examination.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 21st, 2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1-11 and 18-19 are objected to because of the following informalities:
Claim 1 recites “A neural network device comprising: one or more processors; and a memory storing one or more program executed by the one or more processors, wherein the processors…”. Examiner suggests changing to the following: “A neural network device comprising: one or more processors; and a memory storing one or more program instructions that, when executed by the one or more processors, cause the one or more processors to…”
Claim 2 should be changed from “…and store variable layers whose weights is to be changed according to each class combination…” to “…and store variable layers whose weights are to be changed according to each class combination…”
Claim 3 should be changed from “the processors outputs” to “the processors output”
Claims 8 and 18 should be changed from “a plurality of feature extraction layers that receives the input data” to “a plurality of feature extraction layers that receive the input data”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1-11 recite the limitations "the processors" in the first line. There is insufficient antecedent basis for these limitations in the claims.
Claims 6, 8, 16, and 18 recite the limitations "…or the output of the previous layer…" in the first line. There is insufficient antecedent basis for this limitation in the claims.
Claims 6 and 16 recite the limitation “the input of the set variable layer…" in the second line. There is insufficient antecedent basis for this limitation in the claims.
Claims 12-18 and 20 recite the limitation “the steps of…" in the first line. There is insufficient antecedent basis for this limitation in the claims.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 19 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 19 recites “[t]he neural network device according to claim 12…”. Claim 19 does not contain all of the limitations of claim 12 and is rejected because the claim is trying to claim the neural network device of claim 12 without claiming the method involving the device.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an
abstract idea without significantly more.
Claim 1
Step 1: The claim recites a neural network device; therefore, it is directed to the statutory category of
a machine.
Step2A Prong 1: The claim recites:
select at least one class combination of different numbers and types from N (where N is a natural number) classes designated for input data: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of selecting different numbers and types of classes, which can be performed mentally.
set, according to each class combination selected in a … module including a plurality of layers, at least one layer among the layers of the … module as a changeable variable layer: This limitation is a mental process because it involves the evaluation/judgement/opinion of setting a layer from the plurality of layers to be a changeable variable layer, which can be performed mentally.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
one or more processors; and a memory storing one or more program executed by the one or more processors, wherein the processors…: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
and output a result of performing a neural network operation on the input data by changing the variable layer set according to the selected class combination, and a weight of the variable layer: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
neural network module: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
one or more processors; and a memory storing one or more program executed by the one or more processors, wherein the processors…: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
and output a result of performing a neural network operation on the input data by changing the variable layer set according to the selected class combination, and a weight of the variable layer: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
neural network module: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
The elements in combination as an ordered whole still do not amount to significantly more than the judicial exception (i.e., the abstract idea of mental processes for selecting a class combination and designating a layer of a neural network as a variable layer). The claim merely describes a process of making a mental selection of at least one class combination of different numbers and types from among N classes designated for input data, setting at least one layer of the neural network module as a changeable variable layer according to the selected class combination, and then performing standard data processing steps (receiving input data, changing the variable layer and weights, and outputting the result of the neural network operation). The recitation of a neural network module comprising a plurality of layers executed by one or more processors and a memory merely indicates a generic technological environment in which the abstract ideas are applied, without improving the functioning of a computer or the neural network itself.
Therefore, the claim as a whole remains focused on the abstract idea and fails Step 2B of the eligibility
analysis.
Claim 2
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
pre-designate and store variable layers whose weights is to be changed according to each class combination, and the weights, and change the weights of the designated variable layers to the stored weights according to the selected class combination: This limitation is a mental process because it involves the evaluation/judgement/opinion of designating variable layers whose weights are to be changed based on the class combination, and changing the weights of the layers which can be performed mentally or by pen and paper.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 3
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
outputs a likelihood for each class in the number of variable layers corresponding to the number of classes included in each class combination: This limitation is a mental process because it involves the evaluation/judgement/opinion of producing a likelihood for each class in the variable layers based on each class combination, which can be performed mentally or by pen and paper.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 4
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
…output a likelihood for each of the N classes as the variable layer: This limitation is a mental process because it involves the evaluation/judgement/opinion of producing a likelihood for each class in the variable layers.
and change the structures and weights of the variable layers to output a likelihood for a number of classes smaller than N according to the class combination: This limitation is a mental process because it involves the evaluation/judgement/opinion of changing the weights/ parameters of the variable layers and produce a likelihood for a subset of classes, which can be performed mentally or by pen and paper.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
set a final fully-connected (FC) layer of the neural network module configured to…: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
set a final fully-connected (FC) layer of the neural network module configured to…: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 5
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
output a likelihood for each of the N classes, and change the structures and weights of the variable layers to output a likelihood for a number of classes smaller than N according to the class combination: This limitation is a mental process because it involves the evaluation/judgement/opinion of changing the weights/ parameters of the variable layers and produce a likelihood for a subset of classes, which can be performed mentally or by pen and paper.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
as the variable layer, an adaptive decision layer additionally arranged to receive an output of a final fully-connected (FC) layer of the neural network module configured to…: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
as the variable layer, an adaptive decision layer additionally arranged to receive an output of a final fully-connected (FC) layer of the neural network module configured to…: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 6
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
select at least one feature extraction layer according to the class combination among a plurality of feature extraction layers… and set it as a selection feature extraction layer: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of selecting at least one feature extraction layer.
and concatenate the output of the selection feature extraction layer with the input of the set variable layer and apply them together: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of combining the output of the at least one feature extraction layers with the at least one set variable layer.
and perform a ... operation to estimate and output features: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of performing an operation to estimate and output features.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
that receive the input data or the output of the previous layer from the neural network module: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
that receive the input data or the output of the previous layer from the neural network module: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself which cannot provide inventive concept (MPEP 2106.05(h)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 7
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
concatenate some outputs designated among the outputs of the selection feature extraction layer with the input of the variable layer according to the class combination: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of combining the output of the at least one feature extraction layers with the at least one set variable layer.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 8
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
and performs a … operation to estimate and output features: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of performing an operation to estimate and output features.
and concatenate the input of the set variable layer with the output of the sub-feature extraction layer and apply them together: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of combining the output of the at least one feature extraction layers with the at least one set variable layer.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
to the neural network module: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
add… a sub-feature extraction layer, configured separately from a plurality of feature extraction layers that receives the input data: Mere data gathering recited at a high level of generality, and thus is an insignificant extra-solution activity (MPEP 2106.05(g)).
or the output of the previous layer according to the class combination: Insignificant extra-solution activity as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
and extracting features by receiving the output of one feature extraction layer of the plurality of feature extraction layers: Mere data gathering recited at a high level of generality, and thus is an insignificant extra-solution activity (MPEP 2106.05(g)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
to the neural network module: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
add… a sub-feature extraction layer, configured separately from a plurality of feature extraction layers that receives the input data: The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
or the output of the previous layer according to the class combination: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
and extracting features by receiving the output of one feature extraction layer of the plurality of feature extraction layers: The additional element of “receiving” does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of receiving steps amounts to no more than mere data gathering. This element amounts to receiving data over a network and are well-understood, routine, conventional activity. See MPEP 2106.05(d), subsection II (i). This cannot provide an inventive concept.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 9
Step 1: A machine, as above.
Step2A Prong 1: This claim does not recite an additional abstract idea, but the claim depends on
claim 8 which recites an abstract idea.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
the sub-feature extraction layer is configured so that the output is not transmitted to other layers of the neural network module: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself which cannot provide inventive concept (MPEP 2106.05(h)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
the sub-feature extraction layer is configured so that the output is not transmitted to other layers of the neural network module: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself which cannot provide inventive concept (MPEP 2106.05(h)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 10
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
select the class combination based on at least one of an external situation according to an environment in which the neural network device is used or an internal situation according to an output of the neural network module: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of selecting a class combination based on an environment from a neural network device or from an output of a neural network module.
Step 2A Prong Two and Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim is ineligible.
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 11
Step 1: A machine, as above.
Step2A Prong 1: The claim recites, inter alia:
thereby determining the variable layer set according to each class combination and the weight changed in the variable layer: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of determining the variable layer set based on each class combination and the altered weight.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
perform learning based on learning data for the N classes to determine a weight of each layer provided in the neural network module, and then perform additional learning based on learning data including classes according to at least one class combination among the learning data: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself (MPEP 2106.05(h)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
perform learning based on learning data for the N classes to determine a weight of each layer provided in the neural network module, and then perform additional learning based on learning data including classes according to at least one class combination among the learning data: The limitation amounts to merely indicating a field of use or technological environment in which to apply a judicial exception. This does not amount to significantly more than the exception itself which cannot provide inventive concept (MPEP 2106.05(h)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 12
Step 1: The claim recites an output handling method; therefore, it is directed to the statutory category of a process.
Step2A Prong 1: The claim recites, inter alia:
selecting at least one class combination of different numbers and types from N (where N is a natural number) classes designated for input data: This limitation encompasses a mental process because it involves the evaluation/judgement/opinion of selecting different numbers and types of classes, which can be performed mentally.
setting, according to each class combination selected in a neural network module including a plurality of layers, at least one layer among the layers of the … module as a changeable variable layer: This limitation is a mental process because it involves the evaluation/judgement/opinion of setting a layer from the plurality of layers to be a changeable variable layer, which can be performed mentally.
Step2A Prong 2: This judicial exception is not integrated into a practical application because the
additional elements are as follows:
a neural network device, performed by a computing device having one or more processors and a memory that stores one or more programs to be executed by the one or more processors, the method comprising the steps of: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
and outputting a result of performing a neural network operation on the input data by changing a weight of the variable layer set according to the selected class combination to a preset and stored weight: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)).
neural network module: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05(f)).
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly
more than the judicial exception because the additional elements are as follows:
a neural network device, performed by a computing device having one or more processors and a memory that stores one or more programs to be executed by the one or more processors, the method comprising the steps of: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
and outputting a result of performing a neural network operation on the input data by changing a weight of the variable layer set according to the selected class combination to a preset and stored weight: Insignificant extra-solution as the limitation amounts to necessary data outputting (MPEP 2106.05(g)(3)). This falls under Well-Understood, Routine, Conventional activity -see MPEP 2106.05(d)(II)(vi).
neural network module: Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and cannot provide inventive concept (MPEP 2106.05(f)).
Even when considered in combination, these additional elements represent mere instructions to apply
an exception and therefore do not provide an inventive concept. The claim is ineligible.
Claim 13 recites similar limitations to claim 3. Therefore, claim 13 is rejected using the same rationale as claim 3.
Claim 14 recites similar limitations to claim 4. Therefore, claim 14 is rejected using the same rationale as claim 4.
Claim 15 recites similar limitations to claim 5. Therefore, claim 15 is rejected using the same rationale as claim 5.
Claim 16 recites similar limitations to claim 6. Therefore, claim 16 is rejected using the same rationale as claim 6.
Claim 17 recites similar limitations to claim 7. Therefore, claim 17 is rejected using the same rationale as claim 7.
Claim 18 recites similar limitations to claims 8 and 9. Therefore, claim 18 is rejected using the same rationale as claims 8 and 9.
Claim 19 recites similar limitations to claim 10. Therefore, claim 19 is rejected using the same rationale as claim 10.
Claim 20 recites similar limitations to claim 11. Therefore, claim 20 is rejected using the same rationale as claim 11.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 6, 8, and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018).
Regarding claim 1,
Yan teaches A neural network device comprising: one or more processors; and a memory storing one or more program executed by the one or more processors (Paragraph 73 of Yan, "The machine 900 may include processors 910, memory 930, and I/O components 950", Paragraph 74, "a machine-readable medium 938 on which is stored the instructions 916").
wherein the processors select at least one class combination of different numbers and types from N (where N is a natural number) classes designated for input data (Paragraph 39, "the result of each branching CNN 540-550 is only valid for a subset of categories", Paragraph 17, "the challenging classes are routed to downstream fine CNNs that focus solely on confusing classes… the structure of an HD-CNN (e.g., the structure of each component CNN, the number of fine classes, and so on) may be determined by a designer...", Paragraph 54, “the 100 categories of the CIFAR100 dataset can be divided into coarse categories by training a deep CNN model based on the 50,000 training images and 10,000 testing images of the dataset. The number of coarse categories may be provided as an input (e.g., four coarse categories may be selected) and the process 600 used to divide the fine categories into the coarse categories.”
Yan’s HD-CNN routes each input to a downstream fine CNN whose prediction is valid only for a subset of the full category set, and Yan forms those subsets by clustering the N fine categories into coarse groups of differing membership. Each coarse group is a class combination of a particular number and type.).
Yan does not teach set, according to each class combination selected in a neural network module including a plurality of layers, at least one layer among the layers of the neural network module as a changeable variable layer, output a result of performing a neural network operation on the input data by changing the variable layer set according to the selected class combination, and a weight of the variable layer.
Mullapudi, in the same field of endeavor, teaches set, according to each class combination selected in a neural network module including a plurality of layers, at least one layer among the layers of the neural network module as a changeable variable layer (Page 2 Section 3 of Mullapudi, "Branches which are specialized for computing features on visually similar classes. We view computing features relevant to a subset of the network inputs as a subtask of the larger classification task.", Page 3 Section 3.2, "…the job of the gating function is to choose which features (which branches) to compute. Since the gating function only needs to narrow down the final classification problem to determining which k subtasks to compute", Page 5 Section 4.2, "We empirically observe that partitioning the first three ResSep blocks into the stem and replicating the fourth block in each of the branches gives better accuracy per unit computation cost... we retain the structure of Block 4 but scale the number of filters in each convolution layer by wb to reduce training time."
Mullapudi provides branches specialized to class clusters and a gate that selects which branches are active per input. Each branch is a replicated 4th block where the 4th block corresponds to a convolutional 3x3 nn (Fig. 2b). When the gating function is activated, the branch gets assigned certain layers pertaining to the subtask for computation. The branches that get activated for a given class cluster are the layer(s) treated as the changeable variable layer for that combination.).
output a result of performing a neural network operation on the input data by changing the variable layer set according to the selected class combination, and a weight of the variable layer (Page 1 Introduction, “We evaluate HydraNets in the context of image classification where we specialize network components for visually similar classes. The gating mechanism in the HydraNet template enables a simple training process which effectively specializes the components to similar classes and enables conditional execution for faster inference.", Section 3.3, "We evaluate all branches on all inputs and mask out features from branches not picked by the gating function. In effect, the branch outputs from the branches other than the top-k for each input are ignored and not seen by the combiner. Masking ensures that weights of the branches not chosen by the gating remain unchanged during back propagation."
Mullapudi evaluates only the top-k selected branches and masks out the rest, so the combiner produces its output from the selected branches’ weights alone. Selecting a different combination changes both which branch-layers are active and which trained weights contribute to the output.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan's coarse to-fine subset classification with Mullapudi's dynamic per-input branch selection to reduce inference cost while concentrating model capacity on the selected classes (Section 4.2 and Discussion of Mullapudi).
Regarding claim 6,
Yan does not teach select at least one feature extraction layer according to the class combination among a plurality of feature extraction layers ... and set it as a selection feature extraction layer.
Mullapudi, in the same field of endeavor, teaches select at least one feature extraction layer according to the class combination among a plurality of feature extraction layers that receive the input data or the output of the previous layer from the neural network module and perform a neural network operation to estimate and output features and set it as a selection feature extraction layer (Page 2 Section 3, " A stem that computes features used by all branches and in deciding which subtasks to perform for an input. ", Page 3 Section 3.2, " the job of the gating function is to choose which features (which branches) to compute… determining which k subtasks to compute…"
Mullapudi runs a shared stem and then gates, per input, which downstream branch feature-extractors execute. The gate-selected branches are the selection feature extraction layer.).
concatenate the output of the selection feature extraction layer with the input of the set variable layer and apply them together (Page 2 Section 3, "A combiner which aggregates features from multiple branches to make final predictions.", Page 3 Section 3.2, " We denote gating function score for an input I by g(I) ∈ [0, 1], the output of branch b by branch(b), and indices of the top-k branches by topk(g(I)). The combined output of the branches comb is given by:
PNG
media_image1.png
41
262
media_image1.png
Greyscale
We find that combining the features from the branches by concatenation works equally well in practice…"
The combiner aggregates the features from the selected branches and feeds them to the prediction stage. Feeding the aggregated/concatenated branch features into the prediction layer corresponds to concatenating the selection-layer output with the variable-layer input.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan's coarse to-fine subset classification with Mullapudi's dynamic selection of specialized components in order to reduce computation cost and improve accuracy while specializing capacity to the selected classes (Section 4.2 and Discussion of Mullapudi).
Regarding claim 8,
Yan does not teach add, to the neural network module, a sub-feature extraction layer, configured separately from a plurality of feature extraction layers that receives the input data or the output of the previous layer according to the class combination and performs a neural network operation to estimate and output features, and extracting features by receiving the output of one feature extraction layer of the plurality of feature extraction layers, and concatenate the input of the set variable layer with the output of the sub-feature extraction layer and apply them together.
Mullapudi, in the same field of endeavor, teaches add, to the neural network module, a sub-feature extraction layer, configured separately from a plurality of feature extraction layers that receives the input data or the output of the previous layer according to the class combination and performs a neural network operation to estimate and output features, extracting features by receiving the output of one feature extraction layer (Page 2 Section 3 of Mullapudi, "A stem that computes features used by all branches and in deciding which subtasks to perform for an input.", Page 3 Section 3.1, " We compute a feature representation for each class by averaging the features from the final fully connected layer of an image classification network for several training images of the same class. Clustering these average class features using k-means with nb cluster centers results in a partitioning of the feature space… Each of the nb class partitions is assigned to one of the branches, which we refer to as a subtask."
Each branch is a separately-configured feature extractor that receives the shared stem output. The separately-configured branch fed by the stem corresponds to the added sub-feature extraction layer receiving the output of one feature extraction layer.).
concatenate the input of the set variable layer with the output of the sub-feature extraction layer and apply them together (Page 3 Section 3.2, " We denote gating function score for an input I by g(I) ∈ [0, 1], the output of branch b by branch(b), and indices of the top-k branches by topk(g(I)). The combined output of the branches comb is given by:
PNG
media_image1.png
41
262
media_image1.png
Greyscale
We find that combining the features from the branches by concatenation works equally well in practice…"
The combiner concatenates the selected branch features and feeds them to the prediction stage. Feeding the concatenated branch outputs into the prediction layer corresponds to concatenating the selection layer output with the variable layer input.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan's coarse to-fine subset classification with Mullapudi's dynamic selection of specialized components in order to reduce computation cost and improve accuracy while specializing capacity to the selected classes (Section 4.2 and Discussion of Mullapudi).
Regarding claim 10,
Yan teaches …an environment in which the neural network device is used… (Paragraph 9 of Yan, “FIG. 3 is a block diagram illustrating components of a device suitable for image classification using hierarchical deep CNN techniques…”)
Yan does not teach select the class combination based on at least one of an external situation ... or an internal situation according to an output of the neural network module.
Mullapudi, in the same field of endeavor, teaches select the class combination based on at least one of an external situation ... or an internal situation according to an output of the neural network module (Page 2 Section 3, "The gating mechanism which decides what branches to execute at inference by using features from the stem."
Mullapudi's gate selects the active combination from the network's own stem features. Selecting from the network's internal features satisfies the internal situation.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan's coarse to-fine subset classification with Mullapudi's dynamic selection of specialized components in order to reduce computation cost and improve accuracy while specializing capacity to the selected classes (Section 4.2 and Discussion of Mullapudi).
Regarding claim 11,
Yan teaches perform learning based on learning data for the N classes to determine a weight of each layer provided in the neural network module (Paragraph 31 of Yan, " The pretrain module 230 pretrains the coarse category CNN and the fine category CNNs to reduce overlap between the fine category CNNs.", Paragraph 49, "A deep CNN model is trained based on train_train in operation 620 by the pretrain module 230 using standard training techniques. For example, the back-propagation training algorithm is one option for training the deep CNN model."
Yan pretrains the coarse and fine components over the category set before specialization.).
and then perform additional learning based on learning data including classes according to at least one class combination among the learning data, thereby determining the variable layer set according to each class combination and the weight changed in the variable layer (Paragraph 61 of Yan, “The fine category component is further trained on the portion of the dataset corresponding to the coarse category of the fine category component.”, Paragraph 62, “The shallow layers of the CNN for the fine category component may be kept fixed while the deep layers are allowed to change during training… the deep layers conv2, pool2, norm2, conv3, pool3, ip1, and prob are modified during training of each fine category component.”
After full-set pretraining, Yan performs additional training of each fine category component on the subset of data corresponding to its coarse category which is a class combination drawn from the N classes keeping the shallow layers fixed and modifying the deep layers. This determines the per-combination variable layer and its changed weights.)
Claims 9 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018), in further view of Grauman (US 11423548 B2).
Regarding claim 9,
Yan in view of Mullapudi does not teach the sub-feature extraction layer is configured so that the output is not transmitted to other layers of the neural network module.
Grauman, in the same field of endeavor, teaches the sub-feature extraction layer is configured so that the output is not transmitted to other layers of the neural network module (Col 11, Lines 54-56, "parallel streams for appearance 601 and motion 602 ... then join in a fusion layer 603", Col. 11 Lines 52-55, "the two streams are not fused in an early stage of the networks so that both of the steams have strong independent predictions."
Grauman runs two parallel streams (appearance and motion) that develop independent predictions and meet only at a downstream fusion layer, so neither stream’s output is passed into the other stream’s layers. A branch whose output reaches only the fusion/decision stage and no other layer corresponds to a sub-feature extraction layer whose output is not transmitted elsewhere in the module.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Yan in view of Mullapudi's teaching with Grauman’s separately-trained branch as an independent path joined only at the fusion stage in order to develop strong independent features before fusion (Col. 11 Lines 52-55 of Grauman).
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018), and in further view of Rebuffi ("Learning Multiple Visual Domains with Residual Adapters", 2017).
Regarding claim 3,
Yan in view of Mullapudi does not teach outputs a likelihood for each class in the number of variable layers corresponding to the number of classes included in each class combination.
Rebuffi, in the same field of endeavor, teaches outputs a likelihood for each class in the number of variable layers corresponding to the number of classes included in each class combination (Page 3 Section 3, "The parametric feature extractors φα is then used to construct predictors for each domain d as Φd = ψd ◦ φαd , where αd are domain-specific parameters and ψd(v) = softmax(Wdv) is a domain-specific linear classifier V → Yd mapping features to image labels.", Page 6 Section 4, "Each dataset Dd, d = 1,...,10 is formed of pairs (x,y) ∈ Dd where x is an image and y ∈ {1,...,Cd} = Yd is a label."
Rebuffi builds a domain-specific linear classifier head whose output dimension equals the selected domain’s class set Yd. A head sized to the selected subset’s class count corresponds to outputting a likelihood for the number of classes in the combination.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan in view of Mullapudi’s FC layer the per-combination with Rebuffi’s head sized to the subset in order to reconfigure one shared network to each combination (Page 2 Introduction of Rebuffi).
Regarding claim 4,
Yan teaches set a final fully-connected (FC) layer ... configured to output a likelihood for each of the N classes as the variable layer (Paragraph 57, "In an example embodiment, a network consisting of three convolutional layers, one fully-connected layer, and one SOFTMAX layer is used.", Paragraph 62, "The shallow layers of the CNN for the fine category component may be kept fixed while the deep layers are allowed to change during training...the shallow layers conv1, pool1, and norm1 may be kept the same for each fine category component while the deep layers conv2, pool2, norm2, conv3, pool3, ip1, and prob are modified during training of each fine category component."
Yan’s example network terminates in a fully-connected layer feeding a SOFTMAX output, and Yan’s training scheme modifies the deep layers including the final FC/output layer per fine-category component.).
Yan in view of Mullapudi does not teach change the structures and weights of the variable layers to output a likelihood for a number of classes smaller than N according to the class combination.
Rebuffi teaches change the structures and weights of the variable layers to output a likelihood for a number of classes smaller than N according to the class combination (Page 7 Section 5.1, "we also report the total number of parameters used, where 1× is the size of a single ResNet (excluding the last classification layer, which can never be shared). ", Page 2 Introduction, " The key idea is reconfigure a deep neural network on the fly to work on different domains as needed. Our construction is based on recent learning-to-learn methods that showed how the parameters of a deep network can be predicted from another [2, 16]."
Rebuffi’s final classifier is domain-specific and sized to each domain’s class count, which is fewer than the priming task’s N classes. Rebuffi reconfigures the network to the subset on the fly per domain, which corresponds to changing its structure and weights to emit fewer than N likelihoods.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan in view of Mullapudi’s FC layer the per-combination with Rebuffi’s head sized to the subset in order to reconfigure one shared network to each combination (Page 2 Introduction of Rebuffi).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018) in view of Dabiri (US 20240104160 A1), and in further view of Mallya ("Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights", 2018).
Regarding claim 2,
Yan in view of Mullapudi does not teach pre-designate and store variable layers whose weights is to be changed according to each class combination, and the weights, and change the weights of the designated variable layers to the stored weights according to the selected class combination.
Mallya, in the same field of endeavor, teaches pre-designate and store variable layers whose weights is to be changed according to each class combination, and the weights (Page 4 Section 2 of Mallya, "Our learned binary masks incur an overhead of 1 bit per network parameter, smaller than all of the prior work", Page 6 Section 3, "…only the thresholded masks associated with the backbone network layers are stored."
Mallya stores a per-task mask over the fixed backbone layers. The stored per-task mask over designated layers corresponds to the pre-designated, stored variable layers and their weights.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan in view of Mullapudi’s per-combination layers with Mallya’s per-task weight masking to the combination layers in order to provide a predictable use of a known masking technique to improve a similar network (Introduction of Mallya).
Yan in view of Mullapudi in further view of Dabiri does not teach change the weights of the designated variable layers to the stored weights …
Dabiri, in the same field of endeavor, teaches change the weights of the designated variable layers to the stored weights … (Paragraph 59 of Dabiri, “The transpose kernels may be configured to retrieve one or more particular weights from a weight matrix stored in a global memory (e.g., one or more rows of weights) such that the update kernel 714 may access the particular weights without communicating with the global memory.”
Dabiri retrieves weights from memory (preset and stored) which is equivalent to updating the weights to the retrieved weights.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Yan in view of Mullapudi in further view of Mallya’s teaching with Dabiri’s weight retrieval from memory in order to reduce the runtime of the computer (Paragraph 57 of Dabiri).
Claims 7 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018) in further view of Wang (US 12260556 B2).
Regarding claim 7,
Yan in view of Mullapudi does not teach concatenate some outputs designated among the outputs of the selection feature extraction layer with the input of the variable layer according to the class combination.
Wang, in the same field of endeavor, teaches concatenate some outputs designated among the outputs of the selection feature extraction layer with the input of the variable layer according to the class combination (Col. 7 Lines 59-64, " A sixth layer is a wise select layer. This layer is for selecting C/2 best channels from the output of the previous layer by the following method: performing sorting according to the output result of the activation function layer, selecting and outputting C/2 weight values making an output value of the activation function maximum, and discarding the other small weight values", Col. 2 Lines 22-24, "sorting each weight value from large to small, and selecting a preset proportion of top-ranking weight values as enhanced weight values", Col 9 Lines 36-39, “The next layer is a concatenation (concat) layer. This layer superimposes the result of the previous layer and a result of a wise channel select branch according to channels.”
Wang selects only a designated proportion of the channels (outputs) of a feature layer and discards the rest, then concatenates the selected channels. Selecting and concatenating only the designated subset of a layer's outputs corresponds to concatenating some designated outputs of the selection feature extraction layer with the variable-layer input.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan in view of Mullapudi’s teaching with Wang’s concatenation procedure of concatenating only a designated subset of the layer's outputs in order to reduce computation while retaining the channels of the task (Col 1 Lines 48-51 Summary of Wang).
Claims 5 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018) and in further view of Mao (US 11842282 B2).
Regarding claim 5,
Yan teaches a final fully-connected (FC) layer of the neural network module configured to output a likelihood for each of the N classes (Paragraph 57 of Yan, "In an example embodiment, a network consisting of three convolutional layers, one fully-connected layer, and one SOFTMAX layer is used.", Paragraph 62, “the deep layers conv2, pool2, norm2, conv3, pool3, ip1, and prob are modified during training of each fine category component.”
Yan’s network terminates in a fully-connected layer feeding a SoftMax output that produces a likelihood for each class the network identifies.)
Yan in view of Mullapudi does not teach set, as the variable layer, an adaptive decision layer additionally arranged to receive an output of a … layer of the neural network module configured to output a likelihood for each of the N classes, and change the structures and weights of the variable layers to output a likelihood for a number of classes smaller than N according to the class combination.
Mao, in the same field of endeavor, teaches set, as the variable layer, an adaptive decision layer additionally arranged to receive an output of a final fully-connected (FC) layer of the neural network module… (Col 9 Lines 37-45 of Mao, "the system can further include one or more external fine-object classifiers 222a-n. The external fine-object classifiers 222a-n can be separate from the object classifier neural network system 202.", Col. 10 Lineas 23-28, "a third fine object classifier 214c or 222c can indicate whether the object is or is not an adult pedestrian, and so on. In some implementations, the coarse-object classification 218 is provided as input to the fine object classifiers 214a-n or 222a-n in addition to the alternative representations 217a-n for use in predicting the fine-object classification 220 or 224.", Col. 8 Lines 60-64, "the classifications 218 and 220 may be represented as distributions (e.g. of confidence or probability scores) representing the relative likelihoods of the object of interest being within each of the possible object-type classifications."
Mao adds a separate fine-object classifier arranged downstream to receive the upstream classification output. The added fine classifier corresponds to the adaptive decision layer arranged to receive the output of the final FC layer.).
Mao further teaches change the structures and weights of the variable layers to output a likelihood for a number of classes smaller than N according to the class combination (Col. 10 Lines 7-10, " a different fine object classifier 214a-n or 222a-n is provided for each major object category for which minor object categories have been defined.", Col. 11 Lines 29-33, "The system saves computational expense by only generating fine-object classifications with the fine object classifiers that … most likely coarse classifications ", Col. 11. Lines 29-33, " the fine-object classification 220 indicates a minor category of the object of interest"
Mao provides a distinct fine object classifier having separately-trained weights for each coarse category and selects which fine classifier is applied based on the coarse classification result. The selected classifier produces a likelihood over a reduced set of minor categories. The selected fine object classifier corresponds to the recited variable layer changed according to the class combination, and its reduced minor-category output corresponds to the recited output for a number of classes smaller than N.).
It would have been obvious to one of ordinary skill in the art before the effective filing date to provide the variable layer of Yan in view of Mullapudi’s teaching with Mao's added fine-object decision stage in order to obtain a more accurate prediction over a reduced situation-appropriate class and reduce computational expense (Col. 14 Lines 51-55 of Mao).
Claims 12, 16, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018) in further view of Dabiri (US 20240104160 A1).
Regarding claim 12,
Yan teaches An output handling method of a neural network device, performed by a computing device having one or more processors and a memory that stores one or more programs to be executed by the one or more processors, the method comprising the steps of (Paragraph 73 of Yan, "The machine 900 may include processors 910, memory 930, and I/O components 950", Paragraph 74, "a machine-readable medium 938 on which is stored the instructions 916", Paragraph 65, “one or more of the methodologies described herein may facilitate generating an HD-CNN for image classification.”
selecting at least one class combination of different numbers and types from N (where N is a natural number) classes designated for input data (Paragraph 39, "the result of each branching CNN 540-550 is only valid for a subset of categories", Paragraph 17, "the challenging classes are routed to downstream fine CNNs that focus solely on confusing classes… the structure of an HD-CNN (e.g., the structure of each component CNN, the number of fine classes, and so on) may be determined by a designer...", Paragraph 54, “the 100 categories of the CIFAR100 dataset can be divided into coarse categories by training a deep CNN model based on the 50,000 training images and 10,000 testing images of the dataset. The number of coarse categories may be provided as an input (e.g., four coarse categories may be selected) and the process 600 used to divide the fine categories into the coarse categories.”
Yan’s HD-CNN routes each input to a downstream fine CNN whose prediction is valid only for a subset of the full category set, and Yan forms those subsets by clustering the N fine categories into coarse groups of differing membership. Each coarse group is a class combination of a particular number and type.)
Yan does not teach setting, according to each class combination selected in a neural network module including a plurality of layers, at least one layer among the layers of the neural network module as a changeable variable layer; and outputting a result of performing a neural network operation on the input data by changing a weight of the variable layer set according to the selected class combination to a preset and stored weight.
Mullapudi, in the same field of endeavor, teaches setting, according to each class combination selected in a neural network module including a plurality of layers, at least one layer among the layers of the neural network module as a changeable variable layer (Page 2 Section 3 of Mullapudi, "Branches which are specialized for computing features on visually similar classes. We view computing features relevant to a subset of the network inputs as a subtask of the larger classification task.", Page 3 Section 3.2, "…the job of the gating function is to choose which features (which branches) to compute. Since the gating function only needs to narrow down the final classification problem to determining which k subtasks to compute"
Mullapudi provides branches specialized to class clusters and a gate that selects which branches are active per input. Each branch is a replicated 4th block where the 4th block corresponds to a convolutional 3x3 nn (Fig. 2b). When the gating function is activated, the branch gets assigned certain layers pertaining to the subtask for computation. The branches that get activated for a given class cluster are the layer(s) treated as the changeable variable layer for that combination.).
and outputting a result of performing a neural network operation on the input data (Page 1 Introduction, “We evaluate HydraNets in the context of image classification where we specialize network components for visually simi lar classes. The gating mechanism in the HydraNet template enables a simple training process which effectively specializes the components to similar classes and enables conditional execution for faster inference.", Page 3 Section 3.3, "We evaluate all branches on all inputs and mask out features from branches not picked by the gating function. In effect, the branch outputs from the branches other than the top-k for each input are ignored and not seen by the combiner. Masking ensures that weights of the branches not chosen by the gating remain unchanged", Page 3 Section 3.2, “the job of the gating function is to choose which features (which branches) to compute.”
Mullapudi computes the output from only the top-k selected branches while masking the rest.).
Therefore, it would have been obvious before the effective filing date to one of ordinary skill in the art to combine Yan's coarse to-fine subset classification with Mullapudi's dynamic selection of specialized components in order to reduce computation cost and improve accuracy while specializing capacity to the selected classes (Section 4.2 and Discussion of Mullapudi).
Yan in view of Mullapudi does not teach by changing a weight … to a preset and stored weight.
Dabiri, in the same field of endeavor, teaches by changing a weight … to a preset and stored weight (Paragraph 59 of Dabiri, “The transpose kernels may be configured to retrieve one or more particular weights from a weight matrix stored in a global memory (e.g., one or more rows of weights) such that the update kernel 714 may access the particular weights without communicating with the global memory.”
Dabiri retrieves weights from memory (preset and stored) which is equivalent to updating the weights to the retrieved weights.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine Yan in view of Mullapudi’s teaching with Dabiri’s weight retrieval from memory in order to reduce the runtime of the computer (Paragraph 57 of Dabiri).
Claim 16 recites limitations similar to claim 6. Therefore, claim 16 is using the same rationale as claim 6.
Claim 19 recites similar limitations to claim 10. Therefore, claim 19 is using the same rationale as claim 10.
Claim 20 recites similar limitations to claim 11. Therefore, claim 20 is rejected using the same rationale as claim 11.
Claims 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018), in view of Dabiri (US 20240104160 A1), and in further view of Rebuffi ("Learning Multiple Visual Domains with Residual Adapters", 2017).
Claim 13 recites similar limitations to claim 3. Therefore, claim 13 is rejected using the same rationale as claim 3.
Claim 14 recites similar limitations to claim 4. Therefore, claim 14 is rejected using the same rationale as claim 4.
Claims 15 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018), in view of Dabiri (US 20240104160 A1), and in further view of Mao (US 11842282 B2).
Claim 15 recites similar limitations to claim 5. Therefore, claim 15 is rejected using the same rationale as claim 5.
Claims 17 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018), in view of Dabiri (US 20240104160 A1), in further view of Wang (US 12260556 B2).
Claim 17 recites similar limitations to claim 7. Therefore, claim 17 is rejected using the same rationale as claim 7.
Claims 18 is rejected under 35 U.S.C. 103 as being unpatentable over Yan (US 20160117587 A1) in view of Mullapudi ("HydraNets: Specialized Dynamic Architectures for Efficient Inference", 2018), in view of Dabiri (US 20240104160 A1), and in further view of Grauman (US 11423548 B2).
Claim 18 recites similar limitations to claims 8 and 9. Therefore, claim 18 is rejected using the same rationale as claims 8 and 9.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAJD MAHER HADDAD whose telephone number is (571)272-2265. The examiner can normally be reached Mon-Friday 8-5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar, can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.M.H./Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
Prosecution Insights