DETAILED ACTION
This action is responsive to the amendment filed on 11/21/2025. Claims 1-7 are pending in the case. Claims 1-7 are currently amended. Claims 1, 6, and 7 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application
filed in Japan on 11/24/2021. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1 Statutory Category: Claim 1 is directed to a device, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial Exception: Claim 1 recites, in part, “obtain a predicted output value with respect to input data by execution of a regression process”. This limitation, under the broadest reasonable interpretation, cover the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, claim 1 recites, “calculate a local degree of similarity relating to the predicted output value, between the known feature spectrum group and a feature spectrum, wherein the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer, and the feature spectrum is obtained from an output of the specified partial region of the specific layer when the input data is input to the machine learning model”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a Practical Application: This judicial exception is not integrated into a practical application. In particular, the claim recites: “A character processing device”, “a regression processing unit”, and “a memory”. These limitations are additional elements that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers” and “using the machine learning model”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Further, the claim recites: “the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters” and “the obtained predicted output value corresponds to the rotation angle of the plurality of characters”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use, specifically character processing. See MPEP §2106.05(h). Further, the claim recites: “store a known feature spectrum group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Further, the claim recites: “read out the known feature spectrum group from the memory”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Finally, the claim recites: “output the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity”. This limitation this limitation is insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g).
Step 2B Significantly More: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “A character processing device”, “a regression processing unit”, and “a memory” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the claim recites the additional elements: “the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers”, “using the machine learning model”, “the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters”, and “the obtained predicted output value corresponds to the rotation angle of the plurality of characters” that generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. Further, the claim recites the additional elements: “store a known feature spectrum group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model” and “read out the known feature spectrum group from the memory”, that amount to adding insignificant extra-solution activity to the judicial exception. Further, these limitations are directed to storing and retrieving information in memory which the courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). Finally, the claim recites the additional element: “output the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity which is insignificant extra-solution activity to the judicial exception. Further, this limitation is directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). The claim is not patent eligible.
Regarding claim 2, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the output of the predicted output value involves a process to output the local degree of similarity, together with the predicted output value”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Further, this limitation is directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). The claim is not patent eligible.
Regarding claim 3, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the output of the predicted output value involves a process to output a degree of reliability of the predicted output value according to the local degree of similarity, together with the predicted output value”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. Further, this limitation is directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). The claim is not patent eligible.
Regarding claim 4, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the output of the predicted output value involves a first process to determine that the predicted output value is valid when the local degree of similarity is equal to or greater than a predetermined threshold value and a second process to determine that the predicted output value is invalid when the local degree of similarity is less than the predetermined threshold value”. This limitation, under the broadest reasonable interpretation, covers the recitation of the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case judgment. See MPEP § 2106.04(a)(2)(III).
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 5, the rejection of claim 1 is incorporated, and further, the claim recites: “the feature spectrum is any one of: “a first type of the feature spectrum obtained by an arrangement of a plurality of element values of an output vector of the vector neuron at one plane position in the specific layer, over the plurality of channels along the third axis; a second type of the feature spectrum obtained by multiplication of each of the plurality of element values of the first type of the feature spectrum by an activation value corresponding to a vector length of the output vector; and a third type of the feature spectrum obtained by an arrangement of the activation value at the one plane position in the specific layer, over the plurality of channels along the third axis”. This limitation is a continuation of the “calculate a local degree of similarity relating to the predicted output value, between the known feature spectrum group and a feature spectrum, wherein the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer, and the feature spectrum is obtained from an output of the specified partial region of the specific layer when the input data is input to the machine learning model” limitation identified as an abstract idea in the rejection of the parent claim, thus recites a judicial exception.
Further, the claim recites: “the specific layer has a configuration in which a vector neuron, arranged in a plane defined with two axes including a first axis and a second axis, is arranged as a plurality of channels along a third axis which is a direction different from the two axes”. This limitation is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 6:
Step 1 Statutory Category: Claim 6 is directed to a method, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 6 recites, in part, “obtaining a predicted output value with respect to input data by execution of a regression process”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “calculating a local degree of similarity relating to the predicted output value, between the known feature spectrum group and a feature spectrum, wherein the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer, and the feature spectrum is obtained from an output of the specified partial region of the specific layer when the input data is input to the machine learning model”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Finally, the claim recites: “calculating a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers” and “using the machine learning model”. These limitations are additional elements that amount to generally linking the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Further, the claim recites: “the input data corresponds to a plurality of characters” and “the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters”. These limitations are additional elements that amount to generally linking the use of the judicial exception to a particular technological environment or field of use, particularly to character processing. See MPEP §2106.05(h). Further, the claim recites: “reading out, …, a known feature spectrum group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Further, the claim recites: “from a memory”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Finally, the claim recites: “outputting the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity”. This limitation is insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers”, “using the machine learning model”, “the input data corresponds to a plurality of characters”, and “the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters” are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely amount to generally linking the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. Further the claim recites the additional element: “from a memory” which amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the claim recites the additional element: “reading out, …, a known feature spectrum group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model” that amounts to adding insignificant extra-solution activity to the judicial exception. Further, the limitation is directed to storing and retrieving information in memory which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). Finally, the claim recites the additional element: “outputting the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity” which is insignificant extra-solution activity to the judicial exception. Further, the limitation is directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). The claim is not patent eligible.
Regarding claim 7:
Step 1 Statutory Category: Claim 7 is directed to a machine, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 7 recites, in part, “obtaining a predicted output value with respect to input data by execution of a regression process”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “calculating a local degree of similarity, relating to the predicted output value, between the known feature spectrum group and a feature spectrum, wherein the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer, and the feature spectrum is obtained from an output of the specified partial region of the specific layer when the input data is input to the machine learning model”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Finally, the claim recites: “calculating a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “A non-transitory computer-readable storage medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations”. This limitation is an additional element that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers” and “using the machine learning model”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h). Further, the claim recites: “the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters” and “the obtained predicted output value corresponds to the rotation angle of the plurality of characters”. These limitations are additional elements that generally link the use of the judicial exception to a particular technological environment or field of use, specifically character processing. See MPEP §2106.05(h). Further, the claim recites: “reading out, …, a known feature spectrum group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Further, the claim recites: “from a memory”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “outputting the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity”. This limitation is insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “A non-transitory computer-readable storage medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations” and “from a memory” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the claim recites the additional elements: “the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers”, “using the machine learning model”, “the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters”, and “the obtained predicted output value corresponds to the rotation angle of the plurality of characters” that generally link the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. Further, the claim recites the additional element: “reading out, …, a known feature spectrum group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model” that amounts to adding insignificant extra-solution activity to the judicial exception. Further, the limitation is directed to storing and retrieving information in memory which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). Finally, the claim recites the additional element: “outputting the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity” which is insignificant extra-solution activity to the judicial exception. Further, the limitation is directed to receiving or transmitting data over a network which courts have recognized as well-understood, routine, and conventional when they are claimed in a generic manner, see MPEP §2106.05(d)(II). The claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Figueroa-
Mata G, Mata-Montero E. Using a Convolutional Siamese Network for Image-Based Plant
Species Identification with Small Datasets. Biomimetics (Basel). 2020 Mar 1;5(1):8. doi:
10.3390/biomimetics5010008. PMID: 32121572; PMCID: PMC7148474, hereinafter referred to
as "Figueroa-Mata" in view of Z. Ma et al., "Fine-Grained Vehicle Classification With Channel
Max Pooling Modified CNNs," in IEEE Transactions on Vehicular Technology, vol. 68, no. 4,
pp. 3224-3233, April 2019, doi: 10.1109/TVT.2019.2899972, hereinafter referred to as "Ma" in further view of Nakakimura, U.S. Patent Application Publication No. 20170356889, hereinafter referred to as “Nakakimura in further view of Akhter, et al., "Improving Skew Detection and Correction in Different Document Images Using a Deep Learning Approach," 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kharagpur, India, 2020, pp. 1-6, doi: 10.1109/ICCCNT49239.2020.9225619, hereinafter referred to as “Akhter”.
Regarding claim 1, Figueroa-Mata teaches a regression processing unit configured to obtain a predicted output value with respect to input data by execution of a regression process (Figueroa-Mata, Page 10, Section 3.5, Lines 1-2, “All experiments were conducted on a desktop computer with an Nvidia GeForce GTX 1070 GPU with 8GB GDDR5 of memory and a Ryzen 7 2700X AMD CPU with 32 GB of memory”; Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”), wherein
the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers (Figueroa-Mata, Page 6, Section 3.3, “We use a CNN similar to the one described in [5,16,17]. Figure 4 shows the architecture. It consists of three convolutional blocks: a convolutional layer with 32 filters of 11 × 11, a ReLU activation function, and a maxpooling layer; a convolutional layer with 64 filters of 8 × 8, a ReLU activation function, and a maxpooling layer; and a convolutional layer with 128 filters of 5 × 5 and a ReLU activation function. The units of this convolutional layer are flattened into a single vector using a global average-pooling (GAP) layer; this vector is connected to a fully-connected layer (FCN) with 1024 neurons, a ReLU activation function, and to a softmax layer”; Figueroa-Mata, Pages 6-7, Section 3.4, Lines 1-4, “As indicated in Section 2.1, convolutional Siamese networks are a class of CNN-based architecture that usually contains two identical CNNs. The twin CNNs have the same configuration with the same parameters and shared weights. The CNN model that we use to build our CSN is the one shown in Figure 4”; See also Figure 4),
a memory configured to store a known feature … group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model (Figueroa-Mata, Page 10, Section 3.5, Lines 1-2, “All experiments were conducted on a desktop computer with an Nvidia GeForce GTX 1070 GPU with 8GB GDDR5 of memory and a Ryzen 7 2700X AMD CPU with 32 GB of memory”; Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 8, Section 3.4.2, Paragraph 1, “We initially extract at random and without replacement 10 images of each species, this set P of images is used to test our models and it does not change of experiment to experiment. This guarantees that the testing images will not be used in the training and validation phases. The remainder of the images are divided into two sets, one for training called T and the other for validation called V, such that T ∩ V = ∅. These sets are built in a proportion of 80% to 20%, respectively, for the experiments” The “training” images are considered to be the “teaching data”), wherein …
read out the known feature … group from the memory (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species” The reference set is “stored” and thus to be used as input to the CSN, it must have been read out of memory);
calculate a local degree of similarity, relating to the predicted output value, between the known feature … group and a feature … (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 10, Section 3.4.3, Paragraph 3, Lines 2-4 and Equation (3), “To classify a new image q in one of the n classes available, we calculate the average similarity of image q to each of the reference sets Ci , using the similarity function d learned by the convolutional Siamese network, as follows
S
i
-
q
,
C
i
=
1
C
i
∑
x
i
∈
C
i
d
(
q
,
x
i
)
i
=
1,2
,
…
,
n
(3)”), wherein …
the feature … is obtained from an output of … the specific layer when the input data is input to the machine learning model (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; see also Figueroa-Mata, Page 7, Figure 5);
…
output the predicted output value … using the degree of similarity (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 10, Section 3.4.3, Paragraph 4, “Then, the class C* predicted for image q is given by
C
*
=
a
r
g
m
i
n
(
S
1
-
q
,
C
1
,
S
2
-
q
,
C
2
,
…
,
S
n
-
q
,
C
n
)
(4)” The output of the model is the “class C*” which is considered to be the “predicted output value” and it is determined “using the degree of similarity” in equation 4).
Figueroa-Mata does not explicitly teach …a known feature spectrum group, …the known feature spectrum group, …a feature spectrum, …the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer nor a character processing device, the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters, the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions, and … which corresponds to the rotation angle of the plurality of characters.
Ma teaches …a known feature spectrum group (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer.” Here, the corresponding positions of the consecutive feature maps can be considered a feature spectrum group as they represent a continuous range of features)
…the known feature spectrum group (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features.”)
…a feature spectrum (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer.” Here, the consecutive feature maps can be considered a feature spectrum group as they represent a continuous range of features and the CMP (Channel Max Pooling) operation can be considered calculating a feature spectrum group)
…the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer… (Ma, Page 3226, Figure 2, see the red highlighted sections, the “specified partial region” is the highlighted region, but a person of ordinary skill in the art would recognize that any section the size of the highlighted region could be chosen).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the regression processing method of Figueroa-Mata to include the cross-channel analysis of Ma. The motivation for doing so would have been to improve classification accuracy, as stated in Ma, abstract, “Experimental results on two fine-grained vehicle datasets demonstrate that the CMP modified CNNs can improve the classification accuracies on the task of fine-grained vehicle classification while a massive amount of parameters are reduced”.
The proposed combination thus far does not explicitly teach a character processing device, the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters, the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions, and … which corresponds to the rotation angle of the plurality of characters.
Nakakimura teaches calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions (Nakakimura, Paragraph 30, “Normally, since a plurality of characteristic spectrums are obtained, the spectrum similarity calculation unit calculates the similarity between each spectrum at each measurement time extracted from the three-dimensional spectral data to a single sample and a characteristic spectrum for each of three-dimensional spectral data with respect to a plurality of samples for each characteristic spectrum. Therefore, in one sample, the similarity for a single characteristic spectrum is obtained by the number of spectrums. Therefore, from the plurality of similarities, a representative value of similarity related to a single characteristic spectrum is calculated in one sample. The representative value may be, for example, an average value, a median value, a mode value, a sum value, or a maximum value of a plurality of similarities. As a result, for each sample, the representative value of similarity is obtained by the number of characteristic spectrums for each sample”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the regression processing method of the proposed combination to include a method of combining local similarities to calculate a degree of similarity as taught by Nakakimura. The motivation to do so would have been to obtain a more representative value of similarity by combining the local similarity values (Nakakimura, Paragraphs 30-31).
The proposed combination thus far does not explicitly teach a character processing device, the input data corresponds to a plurality of characters, and the predicted output value corresponds to a rotation angle of the plurality of characters, the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, and … which corresponds to the rotation angle of the plurality of characters.
Akhter teaches A character processing device (Akhter, Page 2, Section 3, Lines 1-5, “Figure 1 shows the proposed deep learning architecture (also called a ConvNet [18]) used for skewed angle detection of the input scanned images. We have trained the network on the dataset containing skewed scanned images taken from different scripts”)
the input data corresponds to a plurality of characters (Akhter, Page 2, Section 3, Lines 3-5, “We have trained the network on the dataset containing skewed scanned images taken from different scripts”; “scripts” are considered to be “a plurality of characters”, thus the input of “scanned images” are considered to be corresponding to a plurality of characters), and
the predicted output value corresponds to a rotation angle of the plurality of characters (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”);
the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”; Akhter, Page 3, Section B, Paragraph 2, 12-15, “the FC layer classifies the features learned from the feature extractor in the respective class assigned. The output layer has 360 neurons with each neuron having the probabilities of the respective angle classes”);
… which corresponds to the rotation angle of the plurality of characters (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the regression processing method of the proposed combination to include using input corresponding to a plurality of characters and output to comprise the degree of rotation of the plurality of characters as taught by Akhter. The motivation to do so would have been that categorizing the degree of rotation of characters can aid in de-skewing documents and improving Optical Character Recognition devices (Akhter, Page 1, Abstract, Lines 1-7, “During the optical scanning of a document using any of the Optical Character Recognition (OCR) devices, it is difficult to ensure that the scanned document will not get skewed. The high-speed scanning of the skewed document affects the text recognition process. Therefore, it is essential to de-skew the document images to improve the accuracy of the text recognition task”).
Regarding claim 5, Regarding claim 5, the rejection of claim 1 is incorporated, and further, the proposed combination teaches the specific layer has a configuration in which a vector neuron arranged in a plane defined with two axes including a first axis and a second axis is arranged as a plurality of channels along a third axis which is a direction different from the two axes (Ma, section III. B, Paragraph 2, “Similar as the MP operation, we denote the input feature maps and output feature maps of a CMP layer as F ∈ RC×M×N and C ∈ Rc×M×N, respectively, where C and c are the channel numbers of the input and output feature maps, M and N are the widths and the height of the feature maps, respectively”), and
the feature spectrum is any one of:
PNG
media_image1.png
477
565
media_image1.png
Greyscale
a first type of a feature spectrum obtained by an arrangement of a plurality of element values of an output vector of the vector neuron at one plane position in the specific layer, over the plurality of channels along the third axis (Ma, Section III. B, Paragraph 1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer. To this end, we propose the CMP operation, which is illustrated in Fig. 2.”);
It is noted the claim recites alternative language, and Ma teaches at least one of the alternatives. Further, it would have been obvious to combine the teachings of Figueroa-Mata and Ma for the reasons set forth in connection with claim 1 above.
Regarding claim 6, Figueroa-Mata teaches obtaining a predicted output value with respect to input data by execution of a regression process (Figueroa-Mata, Page 10, Section 3.5, Lines 1-2, “All experiments were conducted on a desktop computer with an Nvidia GeForce GTX 1070 GPU with 8GB GDDR5 of memory and a Ryzen 7 2700X AMD CPU with 32 GB of memory”; Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”), wherein
the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers (Figueroa-Mata, Page 6, Section 3.3, “We use a CNN similar to the one described in [5,16,17]. Figure 4 shows the architecture. It consists of three convolutional blocks: a convolutional layer with 32 filters of 11 × 11, a ReLU activation function, and a maxpooling layer; a convolutional layer with 64 filters of 8 × 8, a ReLU activation function, and a maxpooling layer; and a convolutional layer with 128 filters of 5 × 5 and a ReLU activation function. The units of this convolutional layer are flattened into a single vector using a global average-pooling (GAP) layer; this vector is connected to a fully-connected layer (FCN) with 1024 neurons, a ReLU activation function, and to a softmax layer”; Figueroa-Mata, Pages 6-7, Section 3.4, Lines 1-4, “As indicated in Section 2.1, convolutional Siamese networks are a class of CNN-based architecture that usually contains two identical CNNs. The twin CNNs have the same configuration with the same parameters and shared weights. The CNN model that we use to build our CSN is the one shown in Figure 4”; See also Figure 4),
reading out, from a memory, a known feature … group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species” The reference set is “stored” and thus to be used as input to the CSN, it must have been read out of memory; Figueroa-Mata, Page 8, Section 3.4.2, Paragraph 1, “We initially extract at random and without replacement 10 images of each species, this set P of images is used to test our models and it does not change of experiment to experiment. This guarantees that the testing images will not be used in the training and validation phases. The remainder of the images are divided into two sets, one for training called T and the other for validation called V, such that T ∩ V = ∅. These sets are built in a proportion of 80% to 20%, respectively, for the experiments” The “training” images are considered to be the “teaching data”);
calculating a local degree of similarity, relating to the predicted output value, between the known feature … group and a feature … (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 10, Section 3.4.3, Paragraph 3, Lines 2-4 and Equation (3), “To classify a new image q in one of the n classes available, we calculate the average similarity of image q to each of the reference sets Ci , using the similarity function d learned by the convolutional Siamese network, as follows
S
i
-
q
,
C
i
=
1
C
i
∑
x
i
∈
C
i
d
(
q
,
x
i
)
i
=
1,2
,
…
,
n
(3)”), wherein
…
the feature … is obtained from an output of … the specific layer when the input data is input to the machine learning model (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; see also Figueroa-Mata, Page 7, Figure 5);
outputting the predicted output value … using the degree of similarity (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 10, Section 3.4.3, Paragraph 4, “Then, the class C* predicted for image q is given by
C
*
=
a
r
g
m
i
n
(
S
1
-
q
,
C
1
,
S
2
-
q
,
C
2
,
…
,
S
n
-
q
,
C
n
)
(4)” The output of the model is the “class C*” which is considered to be the “predicted output value” and it is determined “using the degree of similarity” in equation 4).
Figueroa-Mata does not explicitly teach …a known feature spectrum group, …the known feature spectrum group, …a feature spectrum, …the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer nor a method of executing character processing, the input data corresponds to a plurality of characters, the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions, and … which corresponds to the rotation angle of the plurality of characters.
Ma teaches …a known feature spectrum group (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer.” Here, the corresponding positions of the consecutive feature maps can be considered a feature spectrum group as they represent a continuous range of features)
…the known feature spectrum group (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features.”)
…a feature spectrum (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer.” Here, the consecutive feature maps can be considered a feature spectrum group as they represent a continuous range of features and the CMP (Channel Max Pooling) operation can be considered calculating a feature spectrum group)
…the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer… (Ma, Page 3226, Figure 2, see the red highlighted sections, the “specified partial region” is the highlighted region, but a person of ordinary skill in the art would recognize that any section the size of the highlighted region could be chosen).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the regression processing method of Figueroa-Mata to include the cross-channel analysis of Ma. The motivation for doing so would have been to improve classification accuracy, as stated in Ma, abstract, “Experimental results on two fine-grained vehicle datasets demonstrate that the CMP modified CNNs can improve the classification accuracies on the task of fine-grained vehicle classification while a massive amount of parameters are reduced”.
The proposed combination thus far does not explicitly teach a method of executing character processing, the input data corresponds to a plurality of characters, the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions, and … which corresponds to the rotation angle of the plurality of characters.
Nakakimura teaches calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions (Nakakimura, Paragraph 30, “Normally, since a plurality of characteristic spectrums are obtained, the spectrum similarity calculation unit calculates the similarity between each spectrum at each measurement time extracted from the three-dimensional spectral data to a single sample and a characteristic spectrum for each of three-dimensional spectral data with respect to a plurality of samples for each characteristic spectrum. Therefore, in one sample, the similarity for a single characteristic spectrum is obtained by the number of spectrums. Therefore, from the plurality of similarities, a representative value of similarity related to a single characteristic spectrum is calculated in one sample. The representative value may be, for example, an average value, a median value, a mode value, a sum value, or a maximum value of a plurality of similarities. As a result, for each sample, the representative value of similarity is obtained by the number of characteristic spectrums for each sample”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the regression processing method of the proposed combination to include a method of combining local similarities to calculate a degree of similarity as taught by Nakakimura. The motivation to do so would have been to obtain a more representative value of similarity by combining the local similarity values (Nakakimura, Paragraphs 30-31).
The proposed combination thus far does not explicitly teach a method of executing character processing, the input data corresponds to a plurality of characters, the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, and … which corresponds to the rotation angle of the plurality of characters.
Akhter teaches A method of executing character processing (Akhter, Page 2, Section 3, Lines 1-5, “Figure 1 shows the proposed deep learning architecture (also called a ConvNet [18]) used for skewed angle detection of the input scanned images. We have trained the network on the dataset containing skewed scanned images taken from different scripts”)
the input data corresponds to a plurality of characters (Akhter, Page 2, Section 3, Lines 3-5, “We have trained the network on the dataset containing skewed scanned images taken from different scripts”; “scripts” are considered to be “a plurality of characters”, thus the input of “scanned images” are considered to be corresponding to a plurality of characters), and
the predicted output value corresponds to a rotation angle of the plurality of characters (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”);
the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”; Akhter, Page 3, Section B, Paragraph 2, 12-15, “the FC layer classifies the features learned from the feature extractor in the respective class assigned. The output layer has 360 neurons with each neuron having the probabilities of the respective angle classes”);
… which corresponds to the rotation angle of the plurality of characters (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the regression processing method of the proposed combination to include using input corresponding to a plurality of characters and output to comprise the degree of rotation of the plurality of characters as taught by Akhter. The motivation to do so would have been that categorizing the degree of rotation of characters can aid in de-skewing documents and improving Optical Character Recognition devices (Akhter, Page 1, Abstract, Lines 1-7, “During the optical scanning of a document using any of the Optical Character Recognition (OCR) devices, it is difficult to ensure that the scanned document will not get skewed. The high-speed scanning of the skewed document affects the text recognition process. Therefore, it is essential to de-skew the document images to improve the accuracy of the text recognition task”).
Regarding claim 7, Regarding claim 7, Figueroa-Mata teaches A non-transitory computer-readable storage medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations (Figueroa-Mata, Page 10, Section 3.5, Lines 1-2, “All experiments were conducted on a desktop computer with an Nvidia GeForce GTX 1070 GPU with 8GB GDDR5 of memory and a Ryzen 7 2700X AMD CPU with 32 GB of memory”), the operations comprising:
obtaining a predicted output value with respect to input data by execution of a regression process (Figueroa-Mata, Page 10, Section 3.5, Lines 1-2, “All experiments were conducted on a desktop computer with an Nvidia GeForce GTX 1070 GPU with 8GB GDDR5 of memory and a Ryzen 7 2700X AMD CPU with 32 GB of memory”; Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”), wherein
the regression process is executed based on a machine learning model, the machine learning model includes a vector neural network, the vector neural network includes a plurality of vector neuron layers (Figueroa-Mata, Page 6, Section 3.3, “We use a CNN similar to the one described in [5,16,17]. Figure 4 shows the architecture. It consists of three convolutional blocks: a convolutional layer with 32 filters of 11 × 11, a ReLU activation function, and a maxpooling layer; a convolutional layer with 64 filters of 8 × 8, a ReLU activation function, and a maxpooling layer; and a convolutional layer with 128 filters of 5 × 5 and a ReLU activation function. The units of this convolutional layer are flattened into a single vector using a global average-pooling (GAP) layer; this vector is connected to a fully-connected layer (FCN) with 1024 neurons, a ReLU activation function, and to a softmax layer”; Figueroa-Mata, Pages 6-7, Section 3.4, Lines 1-4, “As indicated in Section 2.1, convolutional Siamese networks are a class of CNN-based architecture that usually contains two identical CNNs. The twin CNNs have the same configuration with the same parameters and shared weights. The CNN model that we use to build our CSN is the one shown in Figure 4”; See also Figure 4),
reading out, from a memory, a known feature … group obtained from an output of a specific layer of the machine learning model when a plurality of pieces of teaching data is input to the machine learning model (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species” The reference set is “stored” and thus to be used as input to the CSN, it must have been read out of memory; Figueroa-Mata, Page 8, Section 3.4.2, Paragraph 1, “We initially extract at random and without replacement 10 images of each species, this set P of images is used to test our models and it does not change of experiment to experiment. This guarantees that the testing images will not be used in the training and validation phases. The remainder of the images are divided into two sets, one for training called T and the other for validation called V, such that T ∩ V = ∅. These sets are built in a proportion of 80% to 20%, respectively, for the experiments” The “training” images are considered to be the “teaching data”);
calculating a local degree of similarity, relating to the predicted output value, between the known feature … group and a feature … (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 10, Section 3.4.3, Paragraph 3, Lines 2-4 and Equation (3), “To classify a new image q in one of the n classes available, we calculate the average similarity of image q to each of the reference sets Ci , using the similarity function d learned by the convolutional Siamese network, as follows
S
i
-
q
,
C
i
=
1
C
i
∑
x
i
∈
C
i
d
(
q
,
x
i
)
i
=
1,2
,
…
,
n
(3)”), wherein
…
the feature … is obtained from an output of … the specific layer when the input data is input to the machine learning model (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; see also Figueroa-Mata, Page 7, Figure 5);
outputting the predicted output value … using the degree of similarity (Figueroa-Mata, Page 10, Section 3.4.3, Lines 1-3, “Once the CSN has been fine-tuned, we can identify the species of a new specimen picture by comparing its extracted features vector with the features vector of a reference set stored for each of the species”; Figueroa-Mata, Page 10, Section 3.4.3, Paragraph 4, “Then, the class C* predicted for image q is given by
C
*
=
a
r
g
m
i
n
(
S
1
-
q
,
C
1
,
S
2
-
q
,
C
2
,
…
,
S
n
-
q
,
C
n
)
(4)” The output of the model is the “class C*” which is considered to be the “predicted output value” and it is determined “using the degree of similarity” in equation 4).
Figueroa-Mata does not explicitly teach …a known feature spectrum group, …the known feature spectrum group, …a feature spectrum, …the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer nor the input data corresponds to a plurality of characters, the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions, and … which corresponds to the rotation angle of the plurality of characters.
Ma teaches …a known feature spectrum group (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer.” Here, the corresponding positions of the consecutive feature maps can be considered a feature spectrum group as they represent a continuous range of features)
…the known feature spectrum group (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features.”)
…a feature spectrum (Ma, Section III. B, ¶1, “A CMP operation layer conducts the MP operation along the channel side among the corresponding positions of the consecutive feature maps for the purpose of redundant information elimination. The CMP makes the significant features gathering together within less channels, which is important for fine-grained image classification that needs more discriminative features. Meanwhile, another advantage of the CMP operation is to make the channel number of feature maps smaller, before it connects to the first fully connected (FC) layer.” Here, the consecutive feature maps can be considered a feature spectrum group as they represent a continuous range of features and the CMP (Channel Max Pooling) operation can be considered calculating a feature spectrum group)
…the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer… (Ma, Page 3226, Figure 2, see the red highlighted sections, the “specified partial region” is the highlighted region, but a person of ordinary skill in the art would recognize that any section the size of the highlighted region could be chosen).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the regression processing method of Figueroa-Mata to include the cross-channel analysis of Ma. The motivation for doing so would have been to improve classification accuracy, as stated in Ma, abstract, “Experimental results on two fine-grained vehicle datasets demonstrate that the CMP modified CNNs can improve the classification accuracies on the task of fine-grained vehicle classification while a massive amount of parameters are reduced”.
The proposed combination does not explicitly teach the input data corresponds to a plurality of characters, the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions, and … which corresponds to the rotation angle of the plurality of characters.
Nakakimura teaches calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions (Nakakimura, Paragraph 30, “Normally, since a plurality of characteristic spectrums are obtained, the spectrum similarity calculation unit calculates the similarity between each spectrum at each measurement time extracted from the three-dimensional spectral data to a single sample and a characteristic spectrum for each of three-dimensional spectral data with respect to a plurality of samples for each characteristic spectrum. Therefore, in one sample, the similarity for a single characteristic spectrum is obtained by the number of spectrums. Therefore, from the plurality of similarities, a representative value of similarity related to a single characteristic spectrum is calculated in one sample. The representative value may be, for example, an average value, a median value, a mode value, a sum value, or a maximum value of a plurality of similarities. As a result, for each sample, the representative value of similarity is obtained by the number of characteristic spectrums for each sample”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the regression processing method of the proposed combination to include a method of combining local similarities to calculate a degree of similarity as taught by Nakakimura. The motivation to do so would have been to obtain a more representative value of similarity by combining the local similarity values (Nakakimura, Paragraphs 30-31).
The proposed combination does not explicitly teach the input data corresponds to a plurality of characters, the predicted output value corresponds to a rotation angle of the plurality of characters, and the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model, and … which corresponds to the rotation angle of the plurality of characters.
Akhter teaches the input data corresponds to a plurality of characters (Akhter, Page 2, Section 3, Lines 3-5, “We have trained the network on the dataset containing skewed scanned images taken from different scripts”; “scripts” are considered to be “a plurality of characters”, thus the input of “scanned images” are considered to be corresponding to a plurality of characters), and
the predicted output value corresponds to a rotation angle of the plurality of characters (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”);
the obtained predicted output value corresponds to the rotation angle of the plurality of characters using the machine learning model (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”; Akhter, Page 3, Section B, Paragraph 2, 12-15, “the FC layer classifies the features learned from the feature extractor in the respective class assigned. The output layer has 360 neurons with each neuron having the probabilities of the respective angle classes”);
… which corresponds to the rotation angle of the plurality of characters (Akhter, Page 2, Section 3, Lines 5-6, “The initial task is to detect the angle through which the document has been rotated using a deep learning model”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the regression processing method of the proposed combination to include using input corresponding to a plurality of characters and output to comprise the degree of rotation of the plurality of characters as taught by Akhter. The motivation to do so would have been that categorizing the degree of rotation of characters can aid in de-skewing documents and improving Optical Character Recognition devices (Akhter, Page 1, Abstract, Lines 1-7, “During the optical scanning of a document using any of the Optical Character Recognition (OCR) devices, it is difficult to ensure that the scanned document will not get skewed. The high-speed scanning of the skewed document affects the text recognition process. Therefore, it is essential to de-skew the document images to improve the accuracy of the text recognition task”).
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Figueroa-Mata in view of Ma, in further view of Nakakimura, in further view of Akhter, in further view of Zhang, et al., Learning regression and verification networks for long-term visual tracking, 11/19/2018, https://arxiv.org/pdf/1809.04320, hereinafter referred to as “Zhang”.
Regarding claim 2, the rejection of claim 1 is incorporated.
The proposed combination thus far does not explicitly teach the output of the predicted output value involves a process to output the local degree of similarity, together with the predicted output value.
Zhang teaches the output of the predicted output value involves a process to output the local degree of similarity, together with the predicted output value (Zhang, Page 3, Figure 2, The similarity scores can been seen as an output of the “Regression Network”; Zhang, Page 1, Abstract, Lines 12-14, “The verification network evaluates these candidates and outputs the optimal one as the tracked object with its classification score”; The optimal candidate is considered the “predicted output value” and is output by the verification network, they come together to form the “confidence score” and are thus considered to be output together).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the regression processing method of the proposed combination to include outputting the degree of similarity with the predicted output value as taught by Zhang. The motivation for doing so would have been that outputting the similarity score allows the model to calculate a confidence value and determine if the model succeeded or failed (Zhang, Page 1, Abstract, Lines 16-20, “The similarity and classification scores are combined to obtain the final confidence value, based on which our tracker can determine the absence of the target accurately and conduct image-wide re-detection to capture the target successfully when it reappears”).
Regarding claim 3, the rejection of claim 1 is incorporated.
The proposed combination thus far does not explicitly teach the output of the predicted output value involves a process to output of a degree of reliability of the predicted output value according to the local degree of similarity, together with the predicted output value.
Zhang teaches the output of the predicted output value involves a process to output of a degree of reliability of the predicted output value according to the local degree of similarity, together with the predicted output value (Zhang, Page 1, Abstract, Lines 12-14, “The verification network evaluates these candidates and outputs the optimal one as the tracked object with its classification score”; Zhang, Page 1, Abstract, Lines 16-17, “The similarity and classification scores are combined to obtain the final confidence value”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the regression processing method of the proposed combination to include outputting the degree of reliability with the predicted output value as taught by Zhang. The motivation for doing so would have been that outputting the reliability score allows the model to determine if the model succeeded or failed (Zhang, Page 1, Abstract, Lines 16-20, “The similarity and classification scores are combined to obtain the final confidence value, based on which our tracker can determine the absence of the target accurately and conduct image-wide re-detection to capture the target successfully when it reappears”).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Figueroa-Mata in view of Ma, in further view of Nakakimura, in further view of Akhter, in further view of Price, et al., U.S. Patent No. 10650265, hereinafter referred to as “Price”.
Regarding claim 4, the rejection of claim 1 is incorporated.
The proposed combination thus far does not explicitly teach the output of the predicted output value involves a first process to determine that the predicted output value is valid when the local degree of similarity is equal to or greater than a predetermined threshold value and a second process to determine that the predicted output value is invalid when the local degree of similarity is less than the predetermined threshold value.
Price teaches the output of the predicted output value involves a first process to determine that the predicted output value is valid when the local degree of similarity is equal to or greater than a predetermined threshold value and a second process to determine that the predicted output value is invalid when the local degree of similarity is less than the predetermined threshold value (Price, Col 5, Lines 55-58, “The image may be analyzed and first predicted identity and first confidence values may be determined. The first confidence value may be compared to a predetermined threshold”; Price, Col 5, Line 67 – Col 6, Line 3, “Alternatively, the system may determine that a processing technique failed when the first or second confidence value is not within the predetermined threshold” The “confidence value” is considered to be the “local degree of similarity”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the regression processing method of the proposed combination to include thresholding the degree of similarity as taught by Price. The motivation for doing so would have been the ability to attempt further processing to obtain a better similarity score, or notify a user of the failure (Price, Col 6, Lines 3-7, “Thus, the system may attempt another processing technique resulting in a third confidence value, notify the user of identification failure, and/or conduct any other process consistent with the disclosed embodiments”).
Response to Arguments
Applicant’s arguments regarding the 35 U.S.C. 101 rejections of the claims have been fully considered but are unpersuasive.
Applicant first argues, on page 11 – page 12, paragraph 1 of the response, that amended claim 1 are “essentially tied to a machine and do not represent mental process or mathematical concepts for mathematical relationships”. Examiner respectfully disagrees. It is important to note that a claim that requires a computer may still recite an abstract idea, see MPEP 2106.04(a)(2)(III)(C). Further, it is important to note, mere physical or tangible implementation of an exception is not in itself an inventive concept and does not guarantee eligibility, see MPEP 2106.05(I)(A).
Applicant next argues, on page 12, paragraph 2 – page 13, paragraph 2 of the response, that the abstract idea is integrated into a practical implementation. Applicant specifically points to “computing local similarity values across partial regions and aggregating them using statistical measures such as maximum, average, or minimum and thus allowing efficient and simplified computation”. The step of “calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions” has been identified as a mathematical calculation, falling into the mathematical concepts grouping of abstract ideas. It is important to note that an improvement in the abstract idea itself (e.g. a recited mathematical concept) is not an improvement in technology, see MPEP 2106.05(a)(II).
Applicant next argues, on page 13, final paragraph – page 15 first paragraph of the response, that claim 1 “amounts to significantly more than a conventional activity and provides an inventive step”. Applicant specifically points out “generating a highly reliable predicted output value by utilizing a degree of similarity calculated from feature spectra of specific neural network layers” and “deriving the degree of similarity by computing local similarity values across partial regions and aggregating them using statistical measures such as maximum, average, or minimum, allows efficient and simplified computation”. Examiner respectfully disagrees. The features mentioned have been identified as abstract ideas in the updated 101 rejections seen above, and it is important to note that the judicial exception alone cannot provide the improvement, see MPEP 2106.05(a). Further, it is important to note that claiming the improved speed or efficiency inherent with applying the abstract idea on a computer does not integrate the judicial exception into a practical application or provide an inventive concept, see MPEP 2106.05(f).
Applicant's arguments regarding the remainder of the claims rely upon the arguments asserted with respect to the independent claims, and are thus unpersuasive.
Applicant’s arguments regarding the 35 U.S.C. 103 rejections of the claims have been fully considered but are unpersuasive.
Applicant first argues, on page 15, final paragraph – page 16, paragraph 2 of the response, that the combination of Figueroa-Mata does not teach "calculate a local degree of similarity, relating to the predicted output value, between the known feature spectrum group and a feature spectrum, wherein the known feature spectrum group is associated with a specified partial region of a plurality of partial regions of the specific layer, and the feature spectrum is obtained from an output of the specified partial region of the specific layer when the input data is input to the machine learning model calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions and output the predicted output value which corresponds to the rotation angle of the plurality of characters using the degree of similarity”. Examiner respectfully disagrees. With regard to “calculate a degree of similarity between the known feature spectrum group and the feature spectrum, based on one of a maximum value, an average value, or a minimum value of the local degree of similarity for the plurality of partial regions”, the remarks are moot because the new grounds of rejection do not rely upon the references applied in the prior rejection of record. With regard to the remaining newly presented limitations, please refer to the updated 35 U.S.C. 103 rejection seen above for an in depth mapping of the newly presented claim limitations.
Applicant's arguments regarding the remainder of the claims rely upon the arguments asserted with respect to the independent claims, and are thus unpersuasive. While applicant argues each dependent claim “recites subject matter not described or suggested by any of the cited references, whether taken individually or in combination”, applicant does not clearly point out the patentable novelty which he or she thinks the claims present in view of the state of the art disclosed by the references cited. Please refer to the updated 35 U.S.C. 103 rejection seen above for an in depth mapping.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOLLY CLARKE SIPPEL whose telephone number is (571)272-3270. The examiner can normally be reached Monday - Friday, 7:30 a.m. - 4:30 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571)272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.C.S./ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122