Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to the application filed 30 December 2024. Claims 1-9 are pending and have been examined.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 30 December 2024 is being considered by the examiner.
Claim Objections
Claims 1, 3, 4, 6, 7, and 9 objected to because of the following informalities.
Claim 1 recites "freezed" in "weights of the set of convolutional layers are freezed." Examiner believes "freezed" represents a grammatical or typographic error. For the purposes of examination, Claim 1 has been interpreted to read "weights of the set of convolutional layers are frozen." Claims 4 and 7 are objected to under the same rationales. Appropriate correction or clarification is required.
Claim 3 recites "freezed" in "weights ... are freezed to prevent aligning." Examiner believes "freezed" represents a grammatical or typographic error. For the purposes of examination, Claim 3 has been interpreted to read "weights ... are frozen to prevent aligning." Claims 4 and 7 are objected to under the same rationales. Appropriate correction or clarification is required.
Claim 3 recites "weight" in "freezed [sic] to prevent aligning weight to a new task." Examiner believes "weight" represents a grammatical or typographic error. For the purposes of examination, Claim 3 has been interpreted to read "frozen to prevent aligning the weights to a new task." Claims 4 and 7 are objected to under the same rationales. Appropriate correction or clarification is required.
Double Patenting
A rejection based on double patenting of the “same invention” type finds its support in the language of 35 U.S.C. 101 which states that “whoever invents or discovers any new and useful process... may obtain a patent therefor...” (Emphasis added). Thus, the term “same invention,” in this context, means an invention drawn to identical subject matter. See Miller v. Eagle Mfg. Co., 151 U.S. 186 (1894); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Ockert, 245 F.2d 467, 114 USPQ 330 (CCPA 1957).
A statutory type (35 U.S.C. 101) double patenting rejection can be overcome by canceling or amending the claims that are directed to the same invention so they are no longer coextensive in scope. The filing of a terminal disclaimer cannot overcome a double patenting rejection based upon 35 U.S.C. 101.
Claims 1-9 are provisionally rejected under 35 U.S.C. 101 as claiming the same invention as that of Claims 1-9 of copending Application No. 19/004,566 (reference application). This is a provisional statutory double patenting rejection since the claims directed to the same invention have not in fact been patented.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the set of convolutional layers" in "wherein weights of the set of convolutional layers are freezed for each hierarchical level." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited "one set of convolutional and pooling layers" and "a set of fully connected layers of a convolution neural network" but has previously recited no set of convolutional layers. For the purposes of examination, the limitation has been interpreted to read "wherein weights of the subset of convolutional layers of the set of convolutional and pooling layers are freezed for each hierarchical level." Claims 4 and 7 are rejected under the same rationale.
Claim 1 recites the limitation "the first level visual information" in "passing ... the first level visual information." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited "the extracted first set of visual information" and "a first level hierarchical information" but has previously recited no first level visual information. For the purposes of examination, the limitation has been interpreted to read "the first level hierarchical information." Claims 4 and 7 are rejected under the same rationale.
Claim 1 recites the limitation "the set of fully connected layers of a convolution neural network" in "passing ... the first level visual information and the second visual information ... to the set of fully connected layers of a convolution neural network." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited a set of fully connected layers in "classifying ... using a set of fully connected layers of a convolution neural network," so it is unclear whether "the set of fully connected layers" of the "passing" step refers to the previously recited fully connected layers or an undeclared set. For the purposes of examination, the present limitation has been interpreted to read "passing ... to the set of fully connected layers of a convolution neural network comprising a set of fully connected layers." Claims 4 and 7 are rejected under the same rationale.
Claim 1 recites the limitation "the second hierarchical level information" in "hiding ... to obtain the second hierarchical level information." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited "passing ... to obtain a second level hierarchical information", but has previously recited no second hierarchical level information. For the purposes of examination, the limitation has been interpreted to read " the second level hierarchical information." Claims 4 and 7 are rejected under the same rationale.
Claim 1 recites the limitation "the second visual information" in "passing ... the first level visual information, the second visual information and the third set of visual information." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited "a second set of visual information" but has not recited a second visual information. For the purposes of examination, the limitation has been interpreted to read "passing ... the first level visual information, the second set of visual information and the third set of visual information." Claims 4 and 7 are rejected under the same rationale.
Claim 1 recites the limitation "the set of fully connected layers of a convolution neural network" in "passing ... the first level visual information, the second visual information and the third set of visual information... to the set of fully connected layers of a convolution neural network." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited a set of fully connected layers in "classifying ... using a set of fully connected layers of a convolution neural network," so it is unclear whether "the set of fully connected layers" of the "passing" step refers to the previously recited fully connected layers or an undeclared set. For the purposes of examination, the present limitation has been interpreted to read "passing ... to the set of fully connected layers of a convolution neural network comprising a set of fully connected layers." Claims 4 and 7 are rejected under the same rationale.
Claim 2 recites the limitation "the convolution neural network" in "wherein the convolution neural network is trained progressively." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited "a convolution neural network" separately in the one step of classifying and the two steps of passing. For the purposes of examination, the limitation has been interpreted to read "wherein layers of a neural network comprising convolutional layers are trained progressively." Claims 5 and 8 are rejected under the same rationale.
Claim 3 recites the limitation "the convolution neural network" in "the convolution neural network is trained independently." There is insufficient antecedent basis for this limitation in the claim. Claim 1 has previously recited "a convolution neural network" separately in the one step of classifying and the two steps of passing. For the purposes of examination, the limitation has been interpreted to read "layers of a neural network comprising convolutional layers are trained independently." Claims 6 and 9 are rejected under the same rationale.
Claim 3 recites the limitation "the set of convolutional layers of the predecessor taxonomy level" in "one or more weights of the set of convolutional layers of the predecessor taxonomy level are freezed." There is insufficient antecedent basis for this limitation in the claim. Claim 3 has previously recited "a predecessor taxonomy level" but has not recited convolutional layers of the predecessor taxonomy level. For the purposes of examination, the limitation has been interpreted to read "fusing a visual information of a predecessor taxonomy level comprising convolutional layers ..., wherein ... the set of convolutional layers of the predecessor taxonomy level are freezed." Claims 6 and 9 are rejected under the same rationale.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1
Step 1
Claim 1 recites a ... method, and thus the claimed process falls within a statutory category of invention.
Step 2A Prong 1
The claim recites extracting ... a first set of ... information ..., wherein the first set of ... information pertains to a first hierarchical level information, which is a mental process. The claim recites classifying ... the extracted first set of ... information ... into a first level hierarchical information, which is a mental process. The claim recites extracting ... a second set of ... information ..., wherein the second set of ... information pertains to a second hierarchical level information, which is a mental process. The claim recites address[] a label inconsistency by hiding one or more irrelevant classes based on the first level hierarchical information to obtain the second hierarchical level information, which is a mental process. The claim recites extracting ... a third set of ... information, which is a mental process. The claim recites creating ... a taxonomy using the first level hierarchical information, the second level hierarchical information and the third level hierarchical information to facilitate hierarchical labelling, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element processor-implemented invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element collecting, via an Input/Output (I/O) interface, at least one image amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering"). The additional element image of a predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element training progressively at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information of the at least one image does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element using the trained at least one set of convolutional and pooling layers invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element using a set of fully connected layers of a convolution neural network, wherein weights of the set of convolutional layers are freezed for each hierarchical level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element passing, via the one or more hardware processors, the first level ... information and the second set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a second level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element passing, via the one or more hardware processors, the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element labelling of the predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
Step 2B
The additional element processor-implemented invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element collecting, via an Input/Output (I/O) interface, at least one image is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network"). The additional element image of a predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element training progressively at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information of the at least one image does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element using the trained at least one set of convolutional and pooling layers invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element using a set of fully connected layers of a convolution neural network, wherein weights of the set of convolutional layers are freezed for each hierarchical level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element passing the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element passing, via the one or more hardware processors, the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element labelling of the predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 2
Step 1
Regarding Claim 2, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites considering ... information from a first level to (n-1) levels, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element training progressively, via one or more hardware processors, at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique (as recited by Claim 1), wherein the convolution neural network is trained progressively to nth level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 3
Step 1
Regarding Claim 3, the rejection of Claim 2 is incorporated.
Step 2A Prong 1
The claim recites fusing a ... information of a predecessor taxonomy level to the ... information at that level, wherein during fusing the ... information of the predecessor taxonomy level, one or more weights of the set of convolutional layers of the predecessor taxonomy level are freezed to prevent aligning weight to a new task using labels of a new level, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element a root node training, wherein the convolution neural network is trained independently using a cross-entropy loss invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 4
Step 1
Claim 4 recites a system, and thus the claimed machine falls within a statutory category of invention.
Step 2A Prong 1
The claim recites extract a first set of ... information ..., wherein the first set of ... information pertains to a first hierarchical level information, which is a mental process. The claim recites classify the extracted first set of ... information ... into a first level hierarchical information, which is a mental process. The claim recites extract a second set of ... information ..., wherein the second set of ... information pertains to a second hierarchical level information, which is a mental process. The claim recites address[] a label inconsistency by hiding one or more irrelevant classes based on the first level hierarchical information to obtain the second hierarchical level information, which is a mental process. The claim recites extract a third set of ... information, which is a mental process. The claim recites create a taxonomy using the first level hierarchical information, the second level hierarchical information and the third level hierarchical information to facilitate hierarchical labelling, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element collect at least one image amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering"). The additional element image of a predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element train at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information of the at least one image does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element using the trained at least one set of convolutional and pooling layers invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element using a set of fully connected layers of a convolution neural network, wherein weights of the set of convolutional layers are freezed for each hierarchical level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element passing ... the first level ... information and the second set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a second level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element passing the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element labelling of the predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
Step 2B
The additional element collect at least one image is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network"). The additional element image of a predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element train at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element via the one or more hardware processors invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information of the at least one image does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element using the trained at least one set of convolutional and pooling layers invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element using a set of fully connected layers of a convolution neural network, wherein weights of the set of convolutional layers are freezed for each hierarchical level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element passing the first level ... information and the second set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a second level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element passing the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element labelling of the predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claim(s) 5 and 6, dependent on Claim 4, incorporate(s) the rejection of Claim 4. Claim(s) 5 and 6 incorporate(s) substantively all the limitations of Claim(s) 2 and 3, respectively, in system form and are rejected under the same rationale(s).
Regarding Claim 7
Step 1
Claim 7 recites one or more non-transitory machine-readable information storage mediums comprising one or more instructions ... executed by one or more hardware processors, and thus the claimed manufacture falls within a statutory category of invention.
Step 2A Prong 1
The claim recites extracting a first set of ... information ..., wherein the first set of ... information pertains to a first hierarchical level information, which is a mental process. The claim recites classifying the extracted first set of ... information ... into a first level hierarchical information, which is a mental process. The claim recites extracting a second set of ... information ..., wherein the second set of ... information pertains to a second hierarchical level information, which is a mental process. The claim recites address[] a label inconsistency by hiding one or more irrelevant classes based on the first level hierarchical information to obtain the second hierarchical level information, which is a mental process. The claim recites extracting a third set of ... information, which is a mental process. The claim recites creating a taxonomy using the first level hierarchical information, the second level hierarchical information and the third level hierarchical information to facilitate hierarchical labelling, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element collecting, via an Input/Output (I/O) interface, at least one image amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering"). The additional element image of a predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element training progressively at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information of the at least one image does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element using the trained at least one set of convolutional and pooling layers invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element using a set of fully connected layers of a convolution neural network, wherein weights of the set of convolutional layers are freezed for each hierarchical level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element passing the first level ... information and the second set of ... information of the at least one image to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a second level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element passing the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element labelling of the predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
Step 2B
The additional element collecting, via an Input/Output (I/O) interface, at least one image is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network"). The additional element image of a predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element training progressively, via one or more hardware processors, at least one set of convolutional and pooling layers using the collected at least one image based on a back propagation technique invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element via the one or more hardware processors invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information of the at least one image does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element using the trained at least one set of convolutional and pooling layers invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element using a set of fully connected layers of a convolution neural network, wherein weights of the set of convolutional layers are freezed for each hierarchical level invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element passing the first level ... information and the second set of ... information of the at least one image to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a second level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element visual information does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment"). The additional element passing the first level ... information, the second ... information and the third set of ... information ... to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a third level hierarchical information invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element labelling of the predefined article does not amount to more than generally linking the use of a judicial exception to a particular field of use (see MPEP 2106.05(h), "limit the use of the abstract idea to a particular technological environment").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claim(s) 8 and 9, dependent on Claim 7, incorporate(s) the rejection of Claim 7. Claim(s) 8 and 9 incorporate(s) substantively all the limitations of Claim(s) 2 and 3, respectively, in system form and are rejected under the same rationale(s).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4, 5, 7, and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Boone-Sifuentes, et al., "A Mask-based Output Layer for Multi-level Hierarchical Classification" (hereinafter "Boone-Sifuentes") in view of Cho, et al., "Leveraging Class Hierarchy in Fashion Classification" (hereinafter "Cho") in view of Zhang, et al., "Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework" (hereinafter "Zhang").
Regarding Claim 1, Boone-Sifuentes teaches:
A processor-implemented method (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details: "All models used in our experiments are based on the VGG19 [31] backbone neural network pretrained on ImageNet [32]" and Footnote 1: "Code repository: https://github.com/rbouadjenek/Mask-MLHC," where a processor is inherent in executing the Python language implementation of the repository's Mask-MLHC / Model_Training.ipynb file) comprising:
collecting, via an Input/Output (I/O) interface, at least one image ... (Boone-Sifuentes, source code repository Mask-MLHC, file data.py:
class Cifar100:
...
(X_c_train, y_c_train),(X_c_test, y_c_test)=cifar100.load_data(label_mode='coarse')
(X_f_train, y_f_train),(X_f_test, y_f_test)=cifar100.load_data(label_mode='fine')
and p. 3835, 3.1 Experimental setup, Implementation details: "For CIFAR-100 we used an image size of 32x32");
training progressively ... based on a back propagation technique ... (Boone-Sifuentes, p. 3835, 2.2 Proposed Mask-MLHC output layer: "The model is trained by minimizing the following objective function [Eq. 4] .... The loss function takes all levels’ loss into account to make sure the structure prior can play a role of internal guide to the whole model and make it easier to flow the gradients back to all layers," where Boone's backward flowing gradients corresponds to the instant training progressively using back propagation) at least one set of convolutional and pooling layers (Boone-Sifuentes, source code repository Mask-MLHC, file models.py:
x = self.Conv2D3(x)
x = layers.MaxPool2D(2, strides=2, padding='same')(x)
, where the keras Conv2D3 and MaxPool correspond to the instant layers) using the collected at least one image (Boone-Sifuentes, source code repository Mask-MLHC, file Model_Training.ipynb:
dataset = data.Cifar100()
...
history_mnets_model = mnets_model.fit(dataset.train_dataset,
and p. 3835, 3.1 Experimental setup, Implementation details: "For CIFAR-100 we used an image size of 32x32") ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation);
extracting ... a first set of visual information of the at least one image using the trained at least one set of convolutional and pooling layers (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting "Backbone CNN model" comprising convolutional and pooling layers), wherein the first set of visual information pertains to a first hierarchical level information (Boone-Sifuentes, p. 3835, 2.2 Proposed Mask-MLHC output layer: "the layer computes an embedding for every level of the taxonomy as follows:
z
l
i
=
W
l
i
×
a
+
b
l
i
(1)
where
a
is the embedding of the input, and
W
l
i
,
b
l
i
are parameters learnt during training that are associated with every level
l
i
of the taxonomy") ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation);
classifying ... the extracted first set of visual information of the at least one image into a first level hierarchical information (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting output of layer "Dense 1" resulting in classification result "Output
l
1
") using a set of fully connected layers of a convolution neural network (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting "Backbone CNN model" comprising convolutional and pooling layers) ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation);
extracting ... a second set of visual information of the at least one image using the trained set of convolutional and pooling layers, wherein the second set of visual information pertains to a second hierarchical level information (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting image input "Flatten
a
" and output of "Dense 2" corresponding to the instant second extracted information) ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation);
passing ... the first level visual information and the second set of visual information of the at least one image to the set of fully connected layers of a convolution neural network and a predefined logit masking strategy to obtain a second level hierarchical information (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting image input "Flatten
a
", output of "Dense 2" and output of "Mask Layer M1" resulting in classification result "Output
l
2
," where the mask
m
l
i
+
1
is applied to layer outputs prior to classification and correspond to the instant logits, per equations:
z
l
i
=
W
l
i
×
a
+
b
l
i
(1)
y
^
l
i
+
1
=
s
o
f
t
m
a
x
τ
z
l
i
+
1
∘
m
l
i
+
1
(3)
), wherein the logit masking strategy addresses a label inconsistency by hiding one or more irrelevant classes (Boone-Sifuentes, p. 3835, 2.2 Proposed Mask-MLHC output layer: "for each remaining level, a mask is first computed to enforce that the prediction complies with the taxonomy ... by projecting predictions of the upper level") based on the first level hierarchical information to obtain the second hierarchical level information (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting output of layer "Dense 1" being processed by "Mask Layer
M
l
1
,
l
2
" then being processed with the output of layer "Dense 2," resulting in classification result "Output
l
2
") ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation);
extracting ... a third set of visual information of the at least one image using the trained set of convolutional and pooling layers (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting image input "Flatten
a
" and output of "Dense 3" corresponding to the instant third extracted information) ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation);
passing ... the first level visual information, the second visual information and the third set of visual information of the at least one image to the set of fully connected layers of a convolution neural network and a predefined ... masking strategy to obtain a third level hierarchical information (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting image input "Flatten
a
", output of "Dense 3" and output of "Mask Layer M2" resulting in classification result "Output
l
3
") ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation); and
creating ... a taxonomy using the first level hierarchical information, the second level hierarchical information and the third level hierarchical information (Boone-Sifuentes, p. 3834, 2.1 Notation and the MLHC problem, Hierarchical classification: "In this paper, we consider only tree taxonomies, which are organized with a hierarchy structure of
n
levels
l
i
... Finally, we encode the relationship between two successive levels
l
i
and
l
i
+
1
in a taxonomy using ... matrix
M
l
i
,
l
i
+
1
") to facilitate hierarchical labelling ... (Boone-Sifuentes, p. 3834, Abstract: "Focusing on image classification, we evaluate the performance of our proposed output layer on three different datasets, each with a three-level hierarchical structure") ... via one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository Mask-MLHC," where a hardware processor is inherent in executing the Python language implementation).
Boone-Sifuentes teaches collecting at least one image and creating a taxonomy using hierarchical information to facilitate hierarchical labelling.
Boone-Sifuentes does not explicitly teach collecting ... at least one image of a predefined article and hierarchical labelling of the predefined article.
However, Cho teaches:
collecting ... at least one image of a predefined article (Cho, p. 3198, Figure 1, "Overall Structure of our model," depicting an image of a shirt as a network input, and p. 3199, 4.1. Experimental Settings: "Our experiments were conducted on the DeepFashion dataset")
hierarchical labelling of the predefined article (Cho, p. 3199, 4.1. Experimental Settings: "Our experiments were conducted on the DeepFashion dataset due to its extensive coverage. All data is classified into 50 categories and consists of total of 289,222 images, of which 40,000 are test images. ¶ Since DeepFashion dataset does not provide any hierarchical information, we additionally annotate the dataset by grouping 50 categories into certain number of parent categories").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Boone-Sifuentes regarding collecting at least one image and creating a taxonomy using hierarchical information to facilitate hierarchical labelling
with those of Cho regarding collecting at least one image of a predefined article and hierarchical labelling of the predefined article.
The motivation to do so would be to facilitate improved classification that captures the hierarchical nature of fashion annotations in a domain with industrial value (Cho, p. 3197, Abstract: "Among various tasks in the fashion domain, the classification problem is the vital one, because metadata extraction through fashion classification has tremendous industrial value. A flurry of recent deeplearning based models have been proposed for the task and have showed great performances but they fail to capture the hierarchical nature of fashion annotations.... [W]e propose a novel fashion classification model that works in a hierarchical manner. ... [T]aking into account hierarchical dependencies between class labels, can help improve performance").
The Boone-Sifuentes/Cho combination teaches classifying the extracted first set of visual information of the at least one image into a first level hierarchical information using a set of fully connected layers of a convolution neural network.
The Boone-Sifuentes/Cho combination does not explicitly teach wherein weights of the set of convolutional layers are freezed for each hierarchical level.
However, Zhang teaches:
wherein weights of the set of convolutional layers are freezed for each hierarchical level (Zhang, p. 4, 3. Methodology: "First, an encoder network and a projection head is learned using all of the available hierarchical labels. The encoder networks weights are then frozen, and no finetuning is done on the encoder network for downstream tasks," where Zhang's encoder is a convolutional network per p. 6, 4.2. Implementation Details: "We adopt a pre-trained ResNet-50 [14], which was trained on ImageNet [7], as the encoder network. For the ImageNet experiments, we train the model from scratch. We finetune on our datasets for 100 epochs. ... The encoder model weights are frozen after the network is trained in this fashion").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Boone-Sifuentes/Cho combination regarding classifying the extracted first set of visual information of the at least one image into a first level hierarchical information using a set of fully connected layers of a convolution neural network with those of Zhang regarding wherein weights of the set of convolutional layers are freezed for each hierarchical level.
The motivation to do so would be to facilitate training an encoder network for a given hierarchy that generalizes well for unseen, downstream tasks (Zhang, p. 1, 1. Introduction: "In order to generalize to unknown downstream tasks and unseen data, the embedding function must represent the data concisely and accurately, which includes preserving the hierarchical categorization in the embedding space" and "Although our experiments include image classification and image retrieval, our goal is different from multi-task methods [1, 12, 41]. The proposed algorithm is a general hierarchical multi-label representation learning framework that can be applied to any downstream task. Our approach is agnostic to downstream tasks and is not directly optimized for them" and p. 8, 6. Conclusion: "Hierarchically categorized data is common in the real world, and our novel approach provides a general framework for utilizing all available label data.... Our approach generalizes well on a variety of downstream tasks and unseen data, and significantly outperforms the evaluated baselines").
Regarding Claim 4, Boone-Sifuentes teaches:
A system comprising: a memory storing instructions; one or more Input/Output (I/O) interfaces; and one or more hardware processors coupled to the memory via the one or more I/O interfaces (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository: https://github.com/rbouadjenek/Mask-MLHC," where a system comprising instruction memory, an I/O interface, and a hardware processor is inherent in executing the Python language implementation of the repository's Mask-MLHC / Model_Training.ipynb file), wherein the one or more hardware processors are configured by the instructions to: execute the steps of the method of Claim 1. Claim 4 is rejected under the same rationale as Claim 1.
Regarding Claim 7, Boone-Sifuentes teaches:
One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors (Boone-Sifuentes, p. 3835, 3.1 Experimental setup, Implementation details, Footnote 1: "Code repository: https://github.com/rbouadjenek/Mask-MLHC," where a system comprising a hardware processor executing instructions stored on a non-transitory machine-readable storage medium is inherent in executing the Python language implementation of the repository's Mask-MLHC / Model_Training.ipynb file) cause: execution of the method steps of Claim 1. Claim 7 is rejected under the same rationale as Claim 1.
Regarding Claim 2, the rejection of Claim 1 is incorporated. The Boone-Sifuentes/Cho/Zhang combination teaches:
wherein the convolution neural network is trained progressively to nth level by considering visual information from a first level to (n-1) levels (Boone-Sifuentes, p. 3835, 2.2 Proposed Mask-MLHC output layer: "The model is trained by minimizing the following objective function ... [Eq. 4] ... The loss function takes all levels' loss into account to make sure the structure prior can play a role of internal guide to the whole model and make it easier to flow the gradients back to all layers," where Eq. 4 depicts a sum that progressively aggregates model loss over
n
layers for
m
instances of image data).
Claims 5 and 8 incorporate substantively all limitations of Claim 2 in system and non-transitory machine-readable information storage media forms, respectively, and are rejected under the same rationale.
Claims 3, 6, and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Boone-Sifuentes, et al., "A Mask-based Output Layer for Multi-level Hierarchical Classification" (hereinafter "Boone-Sifuentes") in view of Cho, et al., "Leveraging Class Hierarchy in Fashion Classification" (hereinafter "Cho") in view of Zhang, et al., "Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework" (hereinafter "Zhang") in view of Kim, et al., "Revisiting Softmax Masking For Stability In Continual Learning" (hereinafter "Kim").
Regarding Claim 3, the rejection of Claim 2 is incorporated. The Boone-Sifuentes/Cho/Zhang combination teaches:
wherein progressive training comprises: a root node training, wherein the convolution neural network is trained independently using a cross-entropy loss (Boone-Sifuentes, source code repository Mask-MLHC, file models.py:
# Conv base
...
loss = [keras.losses.SparseCategoricalCrossentropy() for x in num_classes]
, where the "Conv base" model is a convolutional base model comprising convolutional layers); and
fusing a visual information of a predecessor taxonomy level to the visual information at that level (Boone-Sifuentes, p. 3834, Figure 2, "Overview of the Mask-based Output Layer," depicting masked output of layer "Dense 1" being fused to the output of layer "Dense 2" in the "Element-wise product" layer after "Dense 2").
The Boone-Sifuentes/Cho/Zhang combination teaches fusing a visual information of a predecessor taxonomy level to the visual information at that level.
The Boone-Sifuentes/Cho/Zhang combination does not explicitly teach during fusing the visual information of the predecessor taxonomy level, one or more weights of the set of convolutional layers of the predecessor taxonomy level are freezed to prevent aligning weight to a new task using labels of a new level.
However, Kim teaches:
during fusing the visual information of the predecessor taxonomy level, one or more weights of the set of convolutional layers of the predecessor taxonomy level are freezed to prevent aligning weight to a new task using labels of a new level (Kim, p. 3, 3 Confidence Holding Of Softmax Masking In Continual Learning, Inter-task Interference: "In classification models utilizing softmax and cross-entropy loss, the cost function and gradient are defined as ... [Eq. (3)] ... which indicates that all input samples invoke gradients which not only affect their own classifier, but also other classifiers. Thanks to these gradients, classifiers using softmax and cross-entropy loss can achieve a max-margin decision. However, this pull-and-push effect of softmax (Yang et al., 2022) impedes the maintenance of the classifier trained on previous tasks in continual learning. Masked softmax can alleviate this inter-task interference, i.e., ... [Eq. (4)] ... which means that classifiers except for those of the target classes have a gradient of 0 and are maintained through negative infinite masking ").
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of the Boone-Sifuentes/Cho/Zhang combination regarding fusing a visual information of a predecessor taxonomy level to the visual information at that level with those of Kim regarding during fusing the visual information of the predecessor taxonomy level, one or more weights of the set of convolutional layers of the predecessor taxonomy level are freezed to prevent aligning weight to a new task using labels of a new level.
The motivation to do so would be to facilitate alleviating the effects of inter-task interference when using a softmax function in the model and cross-entropy loss during training (Kim, p. 3, 3 Confidence Holding Of Softmax Masking In Continual Learning, Inter-task Interference: "classifiers using softmax and cross-entropy loss can achieve a max-margin decision. However, this pull-and-push effect of softmax (Yang et al., 2022) impedes the maintenance of the classifier trained on previous tasks in continual learning. Masked softmax can alleviate this inter-task interference").
Claims 6 and 9 incorporate substantively all limitations of Claim 3 in system and non-transitory machine-readable information storage media forms, respectively, and are rejected under the same rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Inoue, et al., "Semantic Hierarchy-based Convolutional Neural Networks for Image Classification," teach use of semantic hierarchies in a training strategy for hierarchical CNNs that produces a smooth loss and increased stability during training by maintaining characteristics like the transition from coarse to fine levels, providing results from Fashion-MNIST.
Pizarro, et al., "An Attention-Based Architecture for Hierarchical Classification With CNNs," teach an attention mechanism for hierarchical classification that dynamically determines how branches influence each other during training and inference, allowing the neural net decide how these branches must be connected, and include results from experiments on image classification benchmarks.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT N DAY whose telephone number is (703)756-1519. The examiner can normally be reached M-F 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.N.D./Examiner, Art Unit 2122
/MICHAEL H HOANG/PRIMARY EXAMINER, Art Unit 2122
Prosecution Insights