DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP-2023-091032, filed on 06/21/2024.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/22/2024, 06/17/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: a first processing unit, a second processing unit, an update unit, a third processing unit in claims 1-8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification in Paragraphs 138: “the convolution operation may be realized as a result of execution of a computer program by a processor such as a CPU, a graphics processing unit (GPU), and a digital signal processing unit (DSP).” as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2 and 4-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yoshinaga et al (U.S. 20200293885 A1; Yoshinaga), in view of Chen et al (U.S. 20200117519 A1; Chen)
Regarding claim 1, Yoshinaga discloses a computation apparatus, (Paragraph 26: “The data compression apparatus according to the first embodiment may be included in, for example, a data processing apparatus that performs calculation processing using a neural network. FIG. 3 is a block diagram showing an example of the arrangement of the data processing apparatus.”) comprising:
a first processing unit (Fig.1 : Calculation unit 106) configured to obtain a first feature by executing computation of a neural network with use of a first coefficient that is not to be updated in online learning of the neural network; (Paragraph 49: “if the compressed feature planes are stored in the storage unit 103, the decompression processing unit 107 can decompress the feature planes of the preceding layer stored in the storage unit 103, and transfer them to the calculation unit 106. In this case, the calculation unit 106 can perform calculation processing for calculating feature planes of the next layer using the received decompressed feature planes”; Paragraph 57; Paragraph 126: “Note that the filter coefficient used for the calculation processing using the neural network can be determined in advance by training. That is, training of the neural network can be performed using output data obtained by performing calculating processing using the neural network for input data for training and supervisory data corresponding to the input data for training”, it shows that filter coefficient is interpreted as “first coefficient that is not to be updated”.)
a second processing unit (Fig.1 compression processing unit 101) configured to obtain a second feature by executing the computation of the neural network with use of the first feature and a second coefficient (the first control signal) that is to be updated in the online learning; (Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 61: “the compression processing unit 101 performs fixed-length compression by executing quantization processing as the first compression processing. At this time, the compression processing unit 101 performs quantization with a bit width designated by the control signal.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”) and
an update unit(Fig.1 store unit 103) configured to update the second coefficient by executing the online learning with use of the second coefficient and a second feature that has been obtained by the second processing unit in a past, (Paragraph 60: “In step S207 of the first loop, the compression processing unit 101 performs the first compression processing corresponding to the first control signal for the feature plane block”; Paragraph 72: “In step S209, the compression processing unit (for example, the compression processing unit 102) of the final stage stores the generated feature plane block 609 in the storage unit 103.”; Paragraph 84: “Therefore, when processing the network shown in FIG. 4A using the storage unit 103, the compressed feature planes 403 and 404 are stored in the storage unit 103. The compression ratios by the compression processing units 101 and 102 are determined based on the memory capacity of the storage unit 103 and the network information of the network shown in FIG. 4B.”)
However, Yoshinaga does not disclose wherein processing of the first processing unit and processing of the update unit are executed in parallel.
Chen disclose wherein processing of the first processing unit and processing of the update unit are executed in parallel. (Paragraph 545: “a training process and a test process are included, and each forward path in a batch may be parallel performed, where parallel performed computation for each forward path is independent (particularly, weights may be may shared and may also not be may shared). … In case of the test process, the device may compute an optimal configuration and complete configuration off line, … it is necessary in the training process to reversely compute a gradient and update weights in the network. At this moment, the device may be divided into multiple groups to complete computation of gradients corresponding to different input samples in the batch, and the device may be configured into a group on line, thereby rapidly updating and computing the weights (particularly, the device may also be configured into a group on line to complete computation of the gradients corresponding to different input samples in the batch).”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga by including perform batch computation on a neural network that is taught by Chen, to make the invention that a data sharing system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the recognition and prediction accuracy of the device as well as the delay and the power consumption are reduced, the computational speed is further increased, and the computational energy consumption is reduced . (Chen: Paragraphs 285 and 592)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 2, Yoshinaga, as modified by Chen, discloses all the claims invention. wherein the first processing unit obtains a first feature of a frame with use of the frame and the first coefficient, (Paragraphs 47: “The calculation unit 106 can perform calculation processing for calculating feature planes. That is, the calculation unit 106 can perform calculation processing of generating feature plane data of the next layer using feature plane data of the preceding layer stored in the storage unit 103. In this embodiment, the calculation unit 106 performs calculation processing based on equation (1) above. Furthermore, in this embodiment, the calculation unit 106 receives, as an input, the feature planes of the preceding layer whose bit width is 8 or less, and outputs feature planes of the next layer whose bit width is 8.”) and the second processing unit obtains a second feature of the frame with use of the first feature of the frame and the second coefficient. (Paragraph 41-42: “the compression processing unit 101 performs the first compression processing corresponding to the first control signal for feature plane data of a layer included in the neural network. … the compression processing unit 101 performs quantization processing on values forming each feature plane.”; Paragraph 33: “the CPU 306 can perform image processing or image recognition processing on the image data, and save a processing result in the RAM 308. The image data may be a moving image formed by a plurality of frames.”)
Regarding claim 3, The computation apparatus according to claim 2, wherein the first processing unit obtains a first feature of a second frame, the update unit updates the second coefficient by executing the online learning with use of the second coefficient and a second feature of a first frame that has been input earlier than the second frame, and
the second processing unit obtains a second feature of the second frame with use of the first feature of the second frame and the second coefficient updated by the update unit.
Regarding claim 4, Yoshinaga, as modified by Chen, discloses all the claims invention. Chen further discloses a first memory (a storage unit 1) configured to hold the first coefficient; and a second memory (The storage unit 2) configured to hold the second coefficient, (The storage unit 2) wherein the first processing unit (the computation unit 1) stores the first feature into the first memory (a storage unit 1), and the second processing unit (the computation unit 2)stores the second feature into the second memory. (The storage unit 2) (Figs. 1, 8-9 Paragraphs 284-285: “. If neural network computation is completed, if a computation unit 1 computes an output value 1, the result of the output value 1 is represented with out1. … the computation unit 1 reads out n and w from the core internal storage module at first and directly may perform computation to obtain out1. The computation unit 2 reads out m from the core internal storage module, may receive the synapse values w from the computation unit 1 and may perform corresponding computation to obtain out2. The computation unit 3 reads out q from the core internal storage module, may receive the synapse values w from the computation unit 1 and may perform corresponding computation to obtain out3. …. As illustrated in FIG. 9, assuming that the sharing system may include two storage units, a storage unit 1 may be shared by the computation unit 1 and the computation unit 2. … The storage unit 2 may be exclusive to the computation unit 3, and may be directly accessed by the computation unit 3 and may not be directly accessed by the computation unit 1 and the computation unit 2.”)
Regarding claim 5, Yoshinaga, as modified by Chen, discloses all the claims invention. Yoshinaga further discloses a third processing unit (Fig.1 compression processing unit 102) configured to obtain a third feature by executing the computation of the neural network with use of the second feature and a third coefficient of the neural network. ( Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 65: “In step S207 of the second loop, the compression processing unit 102 performs the second compression processing corresponding to the second control signal for the feature plane block after the first compression processing. Information necessary for compression is provided in the form of the control signal from the control unit 104.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”)
Regarding claim 6, Yoshinaga discloses a computation apparatus, (Paragraph 26: “The data compression apparatus according to the first embodiment may be included in, for example, a data processing apparatus that performs calculation processing using a neural network. FIG. 3 is a block diagram showing an example of the arrangement of the data processing apparatus.”) comprising:
a first processing unit (Fig.1 : Calculation unit 106) configured to obtain a first feature by executing computation of a neural network with use of a first coefficient that is not to be updated in online learning of the neural network; (Paragraph 49: “if the compressed feature planes are stored in the storage unit 103, the decompression processing unit 107 can decompress the feature planes of the preceding layer stored in the storage unit 103, and transfer them to the calculation unit 106. In this case, the calculation unit 106 can perform calculation processing for calculating feature planes of the next layer using the received decompressed feature planes”; Paragraph 57; Paragraph 126: “Note that the filter coefficient used for the calculation processing using the neural network can be determined in advance by training. That is, training of the neural network can be performed using output data obtained by performing calculating processing using the neural network for input data for training and supervisory data corresponding to the input data for training”, it shows that filter coefficient is interpreted as “first coefficient that is not to be updated”.)
a second processing unit (Fig.1 compression processing unit 101) configured to obtain a second feature by executing the computation of the neural network with use of the first feature and a second coefficient (the first control signal) that is to be updated in the online learning; (Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 61: “the compression processing unit 101 performs fixed-length compression by executing quantization processing as the first compression processing. At this time, the compression processing unit 101 performs quantization with a bit width designated by the control signal.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”)
a third processing unit (Fig.1 compression processing unit 102) configured to obtain a third feature by executing the computation of the neural network with use of the second feature and a third coefficient (the second control signal) of the neural network; Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 65: “In step S207 of the second loop, the compression processing unit 102 performs the second compression processing corresponding to the second control signal for the feature plane block after the first compression processing. Information necessary for compression is provided in the form of the control signal from the control unit 104.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”) and
an update unit (Fig.1 store unit 103) configured to update the second coefficient (the first control signal) by executing the online learning based on the second coefficient and the first feature, ( Paragraph 60: “In step S207 of the first loop, the compression processing unit 101 performs the first compression processing corresponding to the first control signal for the feature plane block”; Paragraph 72: “In step S209, the compression processing unit (for example, the compression processing unit 102) of the final stage stores the generated feature plane block 609 in the storage unit 103.”; Paragraph 84: “Therefore, when processing the network shown in FIG. 4A using the storage unit 103, the compressed feature planes 403 and 404 are stored in the storage unit 103. The compression ratios by the compression processing units 101 and 102 are determined based on the memory capacity of the storage unit 103 and the network information of the network shown in FIG. 4B.”) wherein processing of the third processing unit and processing of the update unit are executed in parallel.
However, Yoshinaga does not discloses wherein processing of the third processing unit and processing of the update unit are executed in parallel.
Chen disclose processing of the third processing unit and processing of the update unit are executed in parallel. (Paragraph 545: “a training process and a test process are included, and each forward path in a batch may be parallel performed, where parallel performed computation for each forward path is independent (particularly, weights may be may shared and may also not be may shared). … In case of the test process, the device may compute an optimal configuration and complete configuration off line, … it is necessary in the training process to reversely compute a gradient and update weights in the network. At this moment, the device may be divided into multiple groups to complete computation of gradients corresponding to different input samples in the batch, and the device may be configured into a group on line, thereby rapidly updating and computing the weights (particularly, the device may also be configured into a group on line to complete computation of the gradients corresponding to different input samples in the batch).”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga by including perform batch computation on a neural network that is taught by Chen, to make the invention that a data sharing system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the recognition and prediction accuracy of the device as well as the delay and the power consumption are reduced, the computational speed is further increased, and the computational energy consumption is reduced . (Chen: Paragraphs 285 and 592)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 7, Yoshinaga, as modified by Chen, discloses all the claims invention. Yoshinaga further discloses the update unit updates the second coefficient with use of the second coefficient and a feature that is obtained by executing computation equivalent to the computation that is executed by the second processing unit with use of the second coefficient and the first feature. (Paragraph 82-83: “ith reference to the thus set control parameters 105, the control unit 104 can output the first and second control signals as signals determined based on the capacity of the storage unit 103. Then, by compressing the feature planes based on the control signals, the data processing apparatus can process the network shown in FIG. 4A using the storage unit 103 having the limited capacity. … the upper limit of the memory use amount of the storage unit 103 corresponds to the maximum value of the total data size of the feature planes of successive two layers.”)
Regarding claim 8, Yoshinaga, as modified by Chen, discloses all the claims invention. Yoshinaga further discloses the computation apparatus is an embedded device. (Paragraph 125: “the processing shown in FIG. 2 may be implemented by a computer including a processor and a memory, such as the data processing apparatus shown in FIG. 3.”)
Regarding claim 9, Yoshinaga discloses a computation method implemented by a computation apparatus, (Paragraph 129: “a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s)”) comprising:
obtaining a first feature by executing computation of a neural network with use of a first coefficient that is not to be updated in online learning of the neural network; (Paragraph 49: “if the compressed feature planes are stored in the storage unit 103, the decompression processing unit 107 can decompress the feature planes of the preceding layer stored in the storage unit 103, and transfer them to the calculation unit 106. In this case, the calculation unit 106 can perform calculation processing for calculating feature planes of the next layer using the received decompressed feature planes”; Paragraph 57; Paragraph 126: “Note that the filter coefficient used for the calculation processing using the neural network can be determined in advance by training. That is, training of the neural network can be performed using output data obtained by performing calculating processing using the neural network for input data for training and supervisory data corresponding to the input data for training”, it shows that filter coefficient is interpreted as “first coefficient that is not to be updated”.)
obtaining a second feature by executing the computation of the neural network with use of the first feature and a second coefficient (the first control signal) that is to be updated in the online learning; (Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 61: “the compression processing unit 101 performs fixed-length compression by executing quantization processing as the first compression processing. At this time, the compression processing unit 101 performs quantization with a bit width designated by the control signal.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”) and
updating the second coefficient by executing the online learning with use of the second coefficient and a second feature that has been obtained by the second processing unit in a past, (Paragraph 60: “In step S207 of the first loop, the compression processing unit 101 performs the first compression processing corresponding to the first control signal for the feature plane block”; Paragraph 72: “In step S209, the compression processing unit (for example, the compression processing unit 102) of the final stage stores the generated feature plane block 609 in the storage unit 103.”; Paragraph 84: “Therefore, when processing the network shown in FIG. 4A using the storage unit 103, the compressed feature planes 403 and 404 are stored in the storage unit 103. The compression ratios by the compression processing units 101 and 102 are determined based on the memory capacity of the storage unit 103 and the network information of the network shown in FIG. 4B.”)
However, Yoshinaga does not disclose wherein the obtainment of the first feature and the updating are executed in parallel.
Chen disclose wherein the obtainment of the first feature and the updating are executed in parallel. (Paragraph 545: “a training process and a test process are included, and each forward path in a batch may be parallel performed, where parallel performed computation for each forward path is independent (particularly, weights may be may shared and may also not be may shared). … In case of the test process, the device may compute an optimal configuration and complete configuration off line, … it is necessary in the training process to reversely compute a gradient and update weights in the network. At this moment, the device may be divided into multiple groups to complete computation of gradients corresponding to different input samples in the batch, and the device may be configured into a group on line, thereby rapidly updating and computing the weights (particularly, the device may also be configured into a group on line to complete computation of the gradients corresponding to different input samples in the batch).”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga by including perform batch computation on a neural network that is taught by Chen, to make the invention that a data sharing system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the recognition and prediction accuracy of the device as well as the delay and the power consumption are reduced, the computational speed is further increased, and the computational energy consumption is reduced . (Chen: Paragraphs 285 and 592)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 10 Yoshinaga discloses a computation method implemented by a computation apparatus, (Paragraph 129: “a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s)”), comprising:
obtaining a first feature by executing computation of a neural network with use of a first coefficient that is not to be updated in online learning of the neural network; (Paragraph 49: “if the compressed feature planes are stored in the storage unit 103, the decompression processing unit 107 can decompress the feature planes of the preceding layer stored in the storage unit 103, and transfer them to the calculation unit 106. In this case, the calculation unit 106 can perform calculation processing for calculating feature planes of the next layer using the received decompressed feature planes”; Paragraph 57; Paragraph 126: “Note that the filter coefficient used for the calculation processing using the neural network can be determined in advance by training. That is, training of the neural network can be performed using output data obtained by performing calculating processing using the neural network for input data for training and supervisory data corresponding to the input data for training”, it shows that filter coefficient is interpreted as “first coefficient that is not to be updated”.)
obtaining a second feature by executing the computation of the neural network with use of the first feature and a second coefficient (the first control signal) that is to be updated in the online learning; (Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 61: “the compression processing unit 101 performs fixed-length compression by executing quantization processing as the first compression processing. At this time, the compression processing unit 101 performs quantization with a bit width designated by the control signal.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”)
obtaining a third feature by executing the computation of the neural network with use of the second feature and a third coefficient (the second control signal) of the neural network; Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 65: “In step S207 of the second loop, the compression processing unit 102 performs the second compression processing corresponding to the second control signal for the feature plane block after the first compression processing. Information necessary for compression is provided in the form of the control signal from the control unit 104.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”) and
updating the second coefficient (the first control signal) by executing the online learning based on the second coefficient and the first feature, ( Paragraph 60: “In step S207 of the first loop, the compression processing unit 101 performs the first compression processing corresponding to the first control signal for the feature plane block”; Paragraph 72: “In step S209, the compression processing unit (for example, the compression processing unit 102) of the final stage stores the generated feature plane block 609 in the storage unit 103.”; Paragraph 84: “Therefore, when processing the network shown in FIG. 4A using the storage unit 103, the compressed feature planes 403 and 404 are stored in the storage unit 103. The compression ratios by the compression processing units 101 and 102 are determined based on the memory capacity of the storage unit 103 and the network information of the network shown in FIG. 4B.”) wherein processing of the third processing unit and processing of the update unit are executed in parallel.
However, Yoshinaga does not discloses wherein the obtainment of the third feature and the updating are executed in parallel.
Chen disclose wherein the obtainment of the third feature and the updating are executed in parallel. (Paragraph 545: “a training process and a test process are included, and each forward path in a batch may be parallel performed, where parallel performed computation for each forward path is independent (particularly, weights may be may shared and may also not be may shared). … In case of the test process, the device may compute an optimal configuration and complete configuration off line, … it is necessary in the training process to reversely compute a gradient and update weights in the network. At this moment, the device may be divided into multiple groups to complete computation of gradients corresponding to different input samples in the batch, and the device may be configured into a group on line, thereby rapidly updating and computing the weights (particularly, the device may also be configured into a group on line to complete computation of the gradients corresponding to different input samples in the batch).”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga by including perform batch computation on a neural network that is taught by Chen, to make the invention that a data sharing system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the recognition and prediction accuracy of the device as well as the delay and the power consumption are reduced, the computational speed is further increased, and the computational energy consumption is reduced . (Chen: Paragraphs 285 and 592)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 11, Yoshinaga discloses A non-transitory computer-readable storage medium storing a computer program that causes a computer to function(Paragraph 7: “a non-transitory computer-readable medium stores a program which, when executed by a computer comprising a processor and a memory, causes the computer to”) as:
a first processing unit (Fig.1 : Calculation unit 106) configured to obtain a first feature by executing computation of a neural network with use of a first coefficient that is not to be updated in online learning of the neural network; (Paragraph 49: “if the compressed feature planes are stored in the storage unit 103, the decompression processing unit 107 can decompress the feature planes of the preceding layer stored in the storage unit 103, and transfer them to the calculation unit 106. In this case, the calculation unit 106 can perform calculation processing for calculating feature planes of the next layer using the received decompressed feature planes”; Paragraph 57; Paragraph 126: “Note that the filter coefficient used for the calculation processing using the neural network can be determined in advance by training. That is, training of the neural network can be performed using output data obtained by performing calculating processing using the neural network for input data for training and supervisory data corresponding to the input data for training”, it shows that filter coefficient is interpreted as “first coefficient that is not to be updated”.)
a second processing unit (Fig.1 compression processing unit 101) configured to obtain a second feature by executing the computation of the neural network with use of the first feature and a second coefficient (the first control signal) that is to be updated in the online learning; (Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 61: “the compression processing unit 101 performs fixed-length compression by executing quantization processing as the first compression processing. At this time, the compression processing unit 101 performs quantization with a bit width designated by the control signal.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”) and
an update unit(Fig.1 store unit 103) configured to update the second coefficient by executing the online learning with use of the second coefficient and a second feature that has been obtained by the second processing unit in a past, (Paragraph 60: “In step S207 of the first loop, the compression processing unit 101 performs the first compression processing corresponding to the first control signal for the feature plane block”; Paragraph 72: “In step S209, the compression processing unit (for example, the compression processing unit 102) of the final stage stores the generated feature plane block 609 in the storage unit 103.”; Paragraph 84: “Therefore, when processing the network shown in FIG. 4A using the storage unit 103, the compressed feature planes 403 and 404 are stored in the storage unit 103. The compression ratios by the compression processing units 101 and 102 are determined based on the memory capacity of the storage unit 103 and the network information of the network shown in FIG. 4B.”)
However, Yoshinaga does not disclose wherein processing of the first processing unit and processing of the update unit are executed in parallel.
Chen disclose wherein processing of the first processing unit and processing of the update unit are executed in parallel. (Paragraph 545: “a training process and a test process are included, and each forward path in a batch may be parallel performed, where parallel performed computation for each forward path is independent (particularly, weights may be may shared and may also not be may shared). … In case of the test process, the device may compute an optimal configuration and complete configuration off line, … it is necessary in the training process to reversely compute a gradient and update weights in the network. At this moment, the device may be divided into multiple groups to complete computation of gradients corresponding to different input samples in the batch, and the device may be configured into a group on line, thereby rapidly updating and computing the weights (particularly, the device may also be configured into a group on line to complete computation of the gradients corresponding to different input samples in the batch).”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga by including perform batch computation on a neural network that is taught by Chen, to make the invention that a data sharing system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the recognition and prediction accuracy of the device as well as the delay and the power consumption are reduced, the computational speed is further increased, and the computational energy consumption is reduced . (Chen: Paragraphs 285 and 592)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 12, Yoshinaga discloses A non-transitory computer-readable storage medium storing a computer program that causes a computer to function (Paragraph 7: “a non-transitory computer-readable medium stores a program which, when executed by a computer comprising a processor and a memory, causes the computer to”) as:
a first processing unit (Fig.1 : Calculation unit 106) configured to obtain a first feature by executing computation of a neural network with use of a first coefficient that is not to be updated in online learning of the neural network; (Paragraph 49: “if the compressed feature planes are stored in the storage unit 103, the decompression processing unit 107 can decompress the feature planes of the preceding layer stored in the storage unit 103, and transfer them to the calculation unit 106. In this case, the calculation unit 106 can perform calculation processing for calculating feature planes of the next layer using the received decompressed feature planes”; Paragraph 57; Paragraph 126: “Note that the filter coefficient used for the calculation processing using the neural network can be determined in advance by training. That is, training of the neural network can be performed using output data obtained by performing calculating processing using the neural network for input data for training and supervisory data corresponding to the input data for training”, it shows that filter coefficient is interpreted as “first coefficient that is not to be updated”.)
a second processing unit (Fig.1 compression processing unit 101) configured to obtain a second feature by executing the computation of the neural network with use of the first feature and a second coefficient (the first control signal) that is to be updated in the online learning; (Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 61: “the compression processing unit 101 performs fixed-length compression by executing quantization processing as the first compression processing. At this time, the compression processing unit 101 performs quantization with a bit width designated by the control signal.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”)
a third processing unit (Fig.1 compression processing unit 102) configured to obtain a third feature by executing the computation of the neural network with use of the second feature and a third coefficient (the second control signal) of the neural network; Paragraph 49: “When compressing the feature planes of the next layer, the compression processing units 101 and 102 perform compression processes for the feature planes of the next layer received from the calculation unit 106 before storing the feature planes of the next layer in the storage unit 103, and stores the compressed feature planes in the storage unit 103.”; Paragraph 65: “In step S207 of the second loop, the compression processing unit 102 performs the second compression processing corresponding to the second control signal for the feature plane block after the first compression processing. Information necessary for compression is provided in the form of the control signal from the control unit 104.”; Paragraph 127: “there is provided an arrangement of performing the first and second compression processes for the feature plane data stored in the storage unit 103 or 803, training can be performed as follows. That is, the training unit can acquire output data obtained by performing calculation processing using the neural network for the input data for training while performing the first and second compression processes corresponding to the layer. The training unit can perform training of the neural network using the output data and the supervisory data corresponding to the input data for training”) and
an update unit (Fig.1 store unit 103) configured to update the second coefficient (the first control signal) by executing the online learning based on the second coefficient and the first feature, ( Paragraph 60: “In step S207 of the first loop, the compression processing unit 101 performs the first compression processing corresponding to the first control signal for the feature plane block”; Paragraph 72: “In step S209, the compression processing unit (for example, the compression processing unit 102) of the final stage stores the generated feature plane block 609 in the storage unit 103.”; Paragraph 84: “Therefore, when processing the network shown in FIG. 4A using the storage unit 103, the compressed feature planes 403 and 404 are stored in the storage unit 103. The compression ratios by the compression processing units 101 and 102 are determined based on the memory capacity of the storage unit 103 and the network information of the network shown in FIG. 4B.”) wherein processing of the third processing unit and processing of the update unit are executed in parallel.
However, Yoshinaga does not discloses wherein processing of the third processing unit and processing of the update unit are executed in parallel.
Chen disclose processing of the third processing unit and processing of the update unit are executed in parallel. (Paragraph 545: “a training process and a test process are included, and each forward path in a batch may be parallel performed, where parallel performed computation for each forward path is independent (particularly, weights may be may shared and may also not be may shared). … In case of the test process, the device may compute an optimal configuration and complete configuration off line, … it is necessary in the training process to reversely compute a gradient and update weights in the network. At this moment, the device may be divided into multiple groups to complete computation of gradients corresponding to different input samples in the batch, and the device may be configured into a group on line, thereby rapidly updating and computing the weights (particularly, the device may also be configured into a group on line to complete computation of the gradients corresponding to different input samples in the batch).”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga by including perform batch computation on a neural network that is taught by Chen, to make the invention that a data sharing system; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the recognition and prediction accuracy of the device as well as the delay and the power consumption are reduced, the computational speed is further increased, and the computational energy consumption is reduced . (Chen: Paragraphs 285 and 592)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yoshinaga et al (U.S. 20200293885 A1; Yoshinaga), in view of Chen et al (U.S. 20200117519 A1; Chen), and in further view of Zhou et al (“Discriminative and Robust Online Learning for Siamese Visual Tracking”; Zhou).
Regarding claim 3, Yoshinaga, as modified by Chen, discloses all the claims invention. except wherein the first processing unit obtains a first feature of a second frame, the update unit updates the second coefficient by executing the online learning with use of the second coefficient and a second feature of a first frame that has been input earlier than the second frame, the second processing unit obtains a second feature of the second frame with use of the first feature of the second frame and the second coefficient updated by the update unit. (3.2 Target-specific Features: “As shown in Figure 3, the compression module, and the attention module together form a target-specific feature extractor. Note that these 2 modules (gray area) are only fine-tuned with the first frame of given sequences and are kept fixed during tracking to ensure stability. The harnessed target specific features are then leveraged to optimize the filter module (white area) in subsequent frames.”)
Zhou discloses the first processing unit obtains a first feature of a second frame, the update unit updates the second coefficient by executing the online learning with use of the second coefficient and a second feature of a first frame that has been input earlier than the second frame, the second processing unit obtains a second feature of the second frame with use of the first feature of the second frame and the second coefficient updated by the update unit. (3.2 Target-specific Features: “As shown in Figure 3, the compression module, and the attention module together form a target-specific feature extractor. Note that these 2 modules (gray area) are only fine-tuned with the first frame of given sequences and are kept fixed during tracking to ensure stability. The harnessed target specific features are then leveraged to optimize the filter module (white area) in subsequent frames.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Yoshinaga and Chen by including the visual object tracking task that is taught by Zhou, to make the invention that Discriminative and Robust Online Learning for Siamese Visual Tracking; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving extract target-specific features via template update to improve their robustness handling deformation, rotation, and illumination, etc.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Grill et al (U.S. A1), “SELF-SUPERVISED REPRESENTATION LEARNING USING BOOTSTRAPPED LATENT REPRESENTATIONS”, teaches about The method comprises processing a first transformed view of a training data item, e.g. an image, with a target neural network to generate a target output, processing a second transformed view of the training data item, e.g. image, with an online neural network to generate a prediction of the target output, updating parameters of the online neural network to minimize an error between the prediction of the target output and the target output, and updating parameters of the target neural network based on the parameters of the online neural network. The method can effectively train an encoder neural network without using labelled training data items, and without using a contrastive loss, i.e. without needing “negative examples” which comprise transformed views of different data items.”.
Roth et al (U.S. 20210374502 A1), “ TECHNIQUE TO PERFORM NEURAL NETWORK ARCHITECTURE SEARCH WITH FEDERATED LEARNING”, teaches about using different computing systems to train a portion of a neural network in a federated learning (FL) setting. For example, at least one embodiment pertains to causing different portions of a neural network to be trained at each different computing system and results from each of these different computing systems training different portions are combined to train the neural network.
Sun et al (U.S. 20210158512 A1), “Online adaption of Neural Network”, teaches about e neural networks may be pre-trained to learn parameters or models for processing the imagery data and upon deployment the neural networks may automatically perform further optimization of the learned parameters or models based on a small set of online data samples. The online optimization may be facilitated via offline meta-learning so that the optimization may be accomplished quickly in a few optimization steps.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Duy A Tran whose telephone number is (571)272-4887. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL R MISTRY can be reached at (313)-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY TRAN/ Examiner, Art Unit 2674
/ONEAL R MISTRY/ Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights