DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 03/28/2026 have been fully considered but they are not persuasive.
Applicant argues: The limitation “wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data”, cannot be performed in the human mind. (pages 6-7 of remarks).
Examiner response: Examiner respectfully disagrees. Applicant argues a human cannot simultaneously process time-division multiplexed data in parallel however, the claims do not recite parallel time-division multiplexing. Furthermore, a research studies have shown that a human can perform parallel processing (Springer Parallel Processing | Springer Nature Link). Moreover, receiving and storing data is insignificant extra-solution activity that is well-understood, routine, and conventional (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim) and do not integrate the claims into a practical applicaiton. Arguments are not persuasive.
Applicant argues: Cited references do not teach “wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data.”
Examiner response: Examiner respectfully disagrees. In particular, Applicant argues the operation of “receiving and storing” the mth piece of point data overlaps in time with the pooling operation” performed on the (m-1)th piece of point data (pages 8-11). Wu discloses tensor data stored in a tensor buffer 405 (See Wu fig. 4A, col. 5 lines 58-67;) Wu further discloses retrieving a piece of data from the tensor data and storing the data in cache 410 (See Wu col. 6 lines 1-3;). Wu further discloses that the pre-pooler may perform a pooling operation on a previous piece of data (i.e. (m-1)th piece of data) while the cache 410 may receive and store the next piece of data to be processed (i.e., mth piece of data) (See Wu col. 8 lines 52-62; “The tensor data generated by pre-pooling can be stored in the dedicate memory page in the tensor buffer 405—e.g., the P.sub.0 memory page. For example, the pre-pooler 415 can use write port 1 to write the tensor data into the P.sub.0 memory page. In parallel, the cache 410 can continue to read tensor data stored in one of the X.sub.0, X.sub.1, B.sub.0, and B.sub.1 memory pages. That is, while the pre-pooler 415 is processing and storing data in the tensor buffer 405, the cache 410 can continue to read more tensor data from the tensor buffer 405 for the pre-pooler 415 to process, thereby acting like a streaming memory device.”). In other words, the pooling operation being performed on the previous tensor data occurs in parallel at the same time the next piece of tensor data is being retrieved and stored in the cache. Arguments are not persuasive.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites:
Step 2A, Prong 1
“performing a preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on point data. For example, a human can determine a maximum or average value of a 4x4 array comprising numeric values (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on point data at the same time when the mth piece of data is received. For example, a human can determine a maximum or average value of a 4x4 array comprising numeric values at the same time when an additional piece of data is received and stored (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.)
Step 2A, Prong 2
“A global pooling method for a neural network applied to a many-core system” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
“receiving and storing point data of to-be-processed data sequentially input by a previous network layer” (Insignificant extra-solution activity)
“wherein an operation of receiving and storing an mth piece of point data” (Insignificant extra-solution activity)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“A global pooling method for a neural network applied to a many-core system” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
“receiving and storing point data of to-be-processed data sequentially input by a previous network layer” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
“wherein an operation of receiving and storing an mth piece of point data” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 2 recites:
Step 2A, Prong 1
“performing the preset pooling operation on the first piece of point data to obtain a first pooling result” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data. For example, a human can determine a maximum or average value for a 2x2 window of a 4x4 array comprising numeric values (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain a final pooling result” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data. For example, a human can determine a maximum or average value for a 2x2 window of a 4x4 array comprising numeric values to determine a final result (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
Step 2A, Prong 2
“receiving and storing a first piece of point data input by the previous network layer” (Insignificant extra-solution activity).
“sequentially receiving and storing other pieces of point data of the to-be-processed data except the first piece of point data” (Insignificant extra-solution activity).
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“receiving and storing a first piece of point data input by the previous network layer” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. ).
“sequentially receiving and storing other pieces of point data of the to-be-processed data except the first piece of point data” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 3 recites:
Step 2A, Prong 1
“performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain the final pooling result comprises” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data. For example, a human can determine a maximum or average value for a 2x2 window of a 4x4 array comprising numeric values to determine a final result (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“performing the preset pooling operation on the nth piece of point data based on a pooling result of an (n- 1)th piece of point data to obtain an nth pooling result” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data to obtain a result based on the previous pooling result (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“performing the preset pooling operation on the Nth piece of point data based on a pooling result of an (N-1)th piece of point data to obtain an Nth pooling result” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data to obtain a final result based on the previous pooling result (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“wherein the Nth pooling result is the final pooling result of the to-be-processed data” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data to obtain a final result (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“n is a positive integer, and 1 < n <N” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can determine a number of pieces of data such that the number is greater than 1 but less than a threshold value (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
Step 2A, Prong 2
“wherein sequentially receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data” (Insignificant extra-solution activity)
“receiving and storing an nth piece of point data of the to-be-processed data” (Insignificant extra-solution activity).
“receiving and storing an Nth piece of point data of the to-be-processed data” (Insignificant extra-solution activity)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“wherein sequentially receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“receiving and storing an nth piece of point data of the to-be-processed data” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“receiving and storing an Nth piece of point data of the to-be-processed data” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 4 recites:
Step 2A, Prong 1
“wherein the preset pooling operation comprises an average pooling operation or a maximum pooling operation” (This step is directed to a mathematical concept. See MPEP § 2106.04(a)(2), subsection I.)
Step 2A, Prong 2 & 2B
This claim does not recite any additional elements.
Claim 5 recites:
Step 2A, Prong 1
“in a case where the preset pooling operation is an average pooling operation, performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed comprises” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform an average pooling operation on pieces of data (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“wherein n is a positive integer, and 1 < n <N” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can determine a number of pieces of data such that the number is greater than 1 but less than a threshold value (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
Step 2A, Prong 2
“wherein a storage space of the many-core system comprises a first storage space and a second storage space” (Generic computer component. See 2106.05(f).)
“receiving a first piece of point data and storing the first piece of point data in the first storage space as data A1” (Insignificant extra-solution activity)
“initializing data in the second storage space to be 0, and storing data B1=Ai*(1/N) in the second storage space” (Insignificant extra-solution activity)
“receiving an nth piece of point data and storing in the first storage space as data An” (Insignificant extra-solution activity)
“outputting An to the second storage space through a multiplier accumulator to obtain Bn=Bn-1+An*(1/N),
PNG
media_image1.png
43
226
media_image1.png
Greyscale
” (Insignificant extra-solution activity)
“receiving an Nth piece of point data and storing in the first storage space as data AN” (Insignificant extra-solution activity)
“outputting AN to the second storage space through the multiplier accumulator to obtain BN=BN-1+AN*(1/N),
PNG
media_image2.png
50
241
media_image2.png
Greyscale
” (Insignificant extra-solution activity)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“wherein a storage space of the many-core system comprises a first storage space and a second storage space” (Generic computer component. See 2106.05(f).)
“receiving a first piece of point data and storing the first piece of point data in the first storage space as data A1” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“initializing data in the second storage space to be 0, and storing data B1=Ai*(1/N) in the second storage space” (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
“receiving an nth piece of point data and storing in the first storage space as data An” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“outputting An to the second storage space through a multiplier accumulator to obtain Bn=Bn-1+An*(1/N),
PNG
media_image1.png
43
226
media_image1.png
Greyscale
” (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
“receiving an Nth piece of point data and storing in the first storage space as data AN” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“outputting AN to the second storage space through the multiplier accumulator to obtain BN=BN-1+AN*(1/N),
PNG
media_image2.png
50
241
media_image2.png
Greyscale
” (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 6 recites:
Step 2A, Prong 1
“in a case where the preset pooling operation is the maximum pooling operation, performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be- processed data are completed comprises” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a maximum pooling operation on pieces of data (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“wherein n is a positive integer, and 1 < n <N” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can determine a number of pieces of data such that the number is greater than 1 but less than a threshold value (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
Step 2A, Prong 2
“wherein a storage space of the many-core system comprises a first storage space and a second storage space” (Generic computer component. See 2106.05(f).)
“receiving a first piece of point data and storing in the first storage space as data A1” (Insignificant extra-solution activity)
“initializing data Bo in the second storage space to be negative infinity, and storing a maximum value Bi=Max(Ai,Bo) in the second storage space” (Insignificant extra-solution activity)
“receiving an nth piece of point data and storing in the first storage space as data An” (Insignificant extra-solution activity)
“storing a maximum value Bn=Max(A,Bn-1) in the second storage space, wherein Bn-1=Max(A1,...,An-1);” (Insignificant extra-solution activity)
“receiving an Nh piece of point data and storing in the first storage space as data AN” (Insignificant extra-solution activity)
“storing a maximum value BN=Max(AN,BN-1) in the second storage space, wherein BN-1=Max(A1,...,AN-1)” (Insignificant extra-solution activity)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“wherein a storage space of the many-core system comprises a first storage space and a second storage space” (Generic computer component. See 2106.05(f).)
“receiving a first piece of point data and storing in the first storage space as data A1” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“initializing data Bo in the second storage space to be negative infinity, and storing a maximum value Bi=Max(Ai,Bo) in the second storage space” (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed strong step is well-understood, routine, conventional activity is supported under Berkheimer.)
“receiving an nth piece of point data and storing in the first storage space as data An” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“storing a maximum value Bn=Max(A,Bn-1) in the second storage space, wherein Bn-1=Max(A1,...,An-1); (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
“receiving an Nh piece of point data and storing in the first storage space as data AN” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
“storing a maximum value BN=Max(AN,BN-1) in the second storage space, wherein BN-1=Max(A1,...,AN-1)” (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 7 recites:
Step 2A, Prong 1
“performing a preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on point data. For example, a human can determine a maximum or average value of a 4x4 array comprising numeric values (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
“wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on point data at the same time when the mth piece of data is received. For example, a human can determine a maximum or average value of a 4x4 array comprising numeric values at the same time when an additional piece of data is received and stored (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.)
Step 2A, Prong 2
“a plurality of processing cores, at least one of the plurality of processing cores performs the following operations” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
“receiving and storing point data of to-be-processed data sequentially input by a previous network layer” (Insignificant extra-solution activity)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“a plurality of processing cores, at least one of the plurality of processing cores performs the following operations” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
“receiving and storing point data of to-be-processed data sequentially input by a previous network layer” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.).
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 8 recites:
Step 2A, Prong 1
“…perform the preset pooling operation on the point data…” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data. For example, a human can determine a maximum or average value for a 2x2 window of a 4x4 array comprising numeric values (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
Step 2A, Prong 2
“a controller configured to control reception and storage of the point data input by the previous network layer” (This step is directed to receiving and storing data which is insignificant extra-solution activity and mere data gathering. See MPEP 2106.05(g). The controller is understood as a generic computer component. See 2106.05(f).)
“a memory configured to store the point data” (This step is directed to storing data which is insignificant extra-solution activity and mere data gathering. See MPEP 2106.05(g).)
“operation unit configured to perform the preset pooling operation on the point data under the control of the controller” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“a controller configured to control reception and storage of the point data input by the previous network layer” (MPEP 2106.05(d)(II) indicates that receiving and transmitting data “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data,” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed receiving step is well-understood, routine, conventional activity is supported under Berkheimer. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer. The controller is understood as a generic computer component. See 2106.05(f).)
“a memory configured to store the point data” (This step is directed to storing data. Storing data in a computer memory is well-understood, routine, and conventional as evidenced by the court cases cited at MPEP 2106.05(d), section II, example iv. Storing and retrieving information in memory. Thereby, a conclusion that the claimed storing step is well-understood, routine, conventional activity is supported under Berkheimer.)
“operation unit configured to perform the preset pooling operation on the point data under the control of the controller” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim 9 recites:
Step 2A, Prong 1
“… implement the global pooling method for a neural network” (This step is a recitation of a mental process that is practical to perform in the human mind. A human can perform a pooling operation on a piece of point data. For example, a human can determine a maximum or average value for a 2x2 window of a 4x4 array comprising numeric values (i.e. observation, evaluation, judgement, opinion). See MPEP § 2106.04(a)(2), subsection III.).
Step 2A, Prong 2
“A non-transitory computer-readable storage medium having a computer program stored therein, wherein the program is executed by a processor to implement the global pooling method for a neural network of claim 1” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
The additional elements do not integrate the judicial exception into a practical application.
Step 2B
“A non-transitory computer-readable storage medium having a computer program stored therein, wherein the program is executed by a processor to implement the global pooling method for a neural network of claim 1” (Mere instructions to apply the exception using a generic computer component. See 2106.05(f).)
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, and 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (US-20200410337-A1) in view of Wu et al. (US-11531869-B1).
Regarding Claim 1,
Huang (US 20200410337 A1) teaches a global pooling method for a neural network applied to a many-core system, comprising:
receiving and storing point data of to-be-processed data sequentially input (para [0036] “Each processing node on a layer (e.g., an input layer, an intermediate layer, etc.) may receive a sequential stream of input data elements, multiply each input data element with a weight, compute a weighted sum of the input data elements, and forward the weighted sum to the next layer. And para [0102] “Each PE may also include sequential logic circuitries (e.g., registers, latches, flip-flops, state machines, etc.) to store input data, weights, and output data for the adder and multiplier circuitry, and to synchronize the flow of the data into and out of the circuitry.”) by a previous network layer (para [0053] “First convolution layer 215 may also perform a non-linear activation function (e.g., ReLU). An output matrix 220 from first convolution layer 215 may have smaller dimensions than the input image, and may be referred to as a convolved feature, activation map, or feature map… Each output matrix 220 (e.g., an output feature map) may be passed to a pooling layer 225, where each output matrix 220 may be subsampled or down-sampled to generate a matrix 230.” Convolutional layer 215 (i.e. previous layer). Output matrix 220 (i.e. point data of to-be-processed data). The pooling layer receives the output matrix 220. para [0071] A non-linear activation function (e.g., ReLU, sigmoid, tan h, etc.) may then be applied to output matrix 430 to generate a matrix 440 as shown in FIG. 4D. In the example shown in FIG. 4D, the ReLU function is used, and thus all negative values in output matrix 430 are replaced by Os in matrix 440. A pooling operation (e.g., a max, average, or sum pooling operation) may be applied to matrix 440 to sub-sample or down-sample data in matrix 440.” Matrix 440 is another embodiment (i.e. point data of to-be-processed data) that is received as input.); and
performing a preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed (para [0071] “A pooling operation (e.g., a max, average, or sum pooling operation) may be applied to matrix 440 to sub-sample or down-sample data in matrix 440. In the example shown in FIGS. 4D and 4E, a max pooling operation may be applied to matrix 440, where the 4×4 matrix 440 may be divided into four 2×2 regions 442, 444, 446, and 448. The maximum value of each region may be selected as a subsample representing each region.” The output matrix 440 (i.e. point data) is divided into four regions 442, 444, 446, and 448 (i.e. pieces of point data). A pooling operation is performed on each of the four regions resulting in feature matrix 450 when the pooling operations are completed.),
Huang does not explicitly disclose
wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data.
However, Wu (US 11531869 B1) teaches
wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data (col. 8 lines 52-62; “The tensor data generated by pre-pooling can be stored in the dedicate memory page in the tensor buffer 405—e.g., the P.sub.0 memory page. For example, the pre-pooler 415 can use write port 1 to write the tensor data into the P.sub.0 memory page. In parallel, the cache 410 can continue to read tensor data stored in one of the X.sub.0, X.sub.1, B.sub.0, and B.sub.1 memory pages. That is, while the pre-pooler 415 is processing and storing data in the tensor buffer 405, the cache 410 can continue to read more tensor data from the tensor buffer 405 for the pre-pooler 415 to process, thereby acting like a streaming memory device.”).
Huang and Wu are analogous because they are directed towards implementing pooling operation for neural networks.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the method of performing pooling operations in parallel with other operations of Wu.
Doing so would allow for eliminating the wait time for the multiply accumulate operation (Wu Abs.)
Regarding Claim 4,
Huang and Wu teach the method of claim 1. Huang further teaches wherein the preset pooling operation comprises an average pooling operation or maximum pooling operation (para [0071] A pooling operation (e.g., a max, average, or sum pooling operation) may be applied to matrix 440 to sub-sample or down-sample data in matrix 440.”).
Regarding Claim 7,
Huang teaches a many-core system, comprising:
a plurality of processing cores (para [0080] Special-purpose or domain-specific neural network processors can achieve better performance than both CPUs and GPUs when executing a neural network. Neural network processors can employ a spatial architecture including a processing element (PE) array, in which the processing elements may form processing chains and can pass data directly from one processing element to another.), at least one of the plurality of processing cores performs the following operations:
receiving and storing point data of to-be-processed data sequentially input (para [0036] “Each processing node on a layer (e.g., an input layer, an intermediate layer, etc.) may receive a sequential stream of input data elements, multiply each input data element with a weight, compute a weighted sum of the input data elements, and forward the weighted sum to the next layer. And para [0102] “Each PE may also include sequential logic circuitries (e.g., registers, latches, flip-flops, state machines, etc.) to store input data, weights, and output data for the adder and multiplier circuitry, and to synchronize the flow of the data into and out of the circuitry.”) by a previous network layer (para [0053] “First convolution layer 215 may also perform a non-linear activation function (e.g., ReLU). An output matrix 220 from first convolution layer 215 may have smaller dimensions than the input image, and may be referred to as a convolved feature, activation map, or feature map… Each output matrix 220 (e.g., an output feature map) may be passed to a pooling layer 225, where each output matrix 220 may be subsampled or down-sampled to generate a matrix 230.” Convolutional layer 215 (i.e. previous layer). Output matrix 220 (i.e. point data of to-be-processed data). The pooling layer receives the output matrix 220. para [0071] A non-linear activation function (e.g., ReLU, sigmoid, tan h, etc.) may then be applied to output matrix 430 to generate a matrix 440 as shown in FIG. 4D. In the example shown in FIG. 4D, the ReLU function is used, and thus all negative values in output matrix 430 are replaced by Os in matrix 440. A pooling operation (e.g., a max, average, or sum pooling operation) may be applied to matrix 440 to sub-sample or down-sample data in matrix 440.” Matrix 440 is another embodiment (i.e. point data of to-be-processed data) that is received as input.); and
performing a preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed (para [0071] “A pooling operation (e.g., a max, average, or sum pooling operation) may be applied to matrix 440 to sub-sample or down-sample data in matrix 440. In the example shown in FIGS. 4D and 4E, a max pooling operation may be applied to matrix 440, where the 4×4 matrix 440 may be divided into four 2×2 regions 442, 444, 446, and 448. The maximum value of each region may be selected as a subsample representing each region.” The output matrix 440 (i.e. point data) is divided into four regions 442, 444, 446, and 448 (i.e. pieces of point data). A pooling operation is performed on each of the four regions resulting in feature matrix 450 when the pooling operations are completed.).
Huang does not explicitly disclose
wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data.
However, Wu (US 11531869 B1) teaches
wherein an operation of receiving and storing an mth piece of point data and the preset pooling operation performed on an (m-1)th piece of point data are performed in parallel, m is a positive integer, 1<m<=N, and N represents a number of the pieces of point data of the to-be-processed data (col. 8 lines 52-62; “The tensor data generated by pre-pooling can be stored in the dedicate memory page in the tensor buffer 405—e.g., the P.sub.0 memory page. For example, the pre-pooler 415 can use write port 1 to write the tensor data into the P.sub.0 memory page. In parallel, the cache 410 can continue to read tensor data stored in one of the X.sub.0, X.sub.1, B.sub.0, and B.sub.1 memory pages. That is, while the pre-pooler 415 is processing and storing data in the tensor buffer 405, the cache 410 can continue to read more tensor data from the tensor buffer 405 for the pre-pooler 415 to process, thereby acting like a streaming memory device.”).
Huang and Wu are analogous because they are directed towards implementing pooling operation for neural networks.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the method of performing pooling operations in parallel with other operations of Wu.
Doing so would allow for eliminating the wait time for the multiply accumulate operation (Wu Abs.)
Regarding Claim 8,
Huang and Wu teach the many-core system of claim 7. Huang further teaches wherein each processing core comprises:
a controller configured to control reception and storage of the point data input by the previous network layer (para [0080] In some examples, the weights or inputs can be pre-loaded into the processing element array. In some examples, neural network processors can also include an on-chip buffer that can store values read from processor memory, and that can distribute values to multiple computation engines in the processor. And para [0081] “The example shown in FIG. 7 includes an accelerator 702. In various examples, accelerator 702 can execute computations for a set of input data (e.g., input data 750) using a processing element array 710, an activation engine 716, and/or a pooling engine 718.”);
a memory configured to store the point data (para [0091] “Outputs from the last row in processing element array 710 can be temporarily stored in a results buffer 712 (e.g., partial sum (PSUM) buffer). The results can be intermediate results, which can be written to memory banks 714 to be provided to processing element array 710 for additional computation.”); and
an operation unit configured to perform the preset pooling operation on the point data under the control of the controller (para [0094] “In some implementations, accelerator 702 can include a pooling engine 718. Pooling is the combining of outputs of the columns of processing element array 710. Combining can include for example, computing a maximum value, a minimum value, an average value, a median value, a summation, a multiplication, or another logical or mathematical combination. In various examples, pooling engine 718 can include multiple execution channels that can operating on values from corresponding columns of processing element array 710.”).
Regarding Claim 9,
Huang and Wu teach a non-transitory computer-readable storage medium having a computer program stored therein. Huang further teaches wherein the program is executed by a processor to implement the global pooling method for a neural network of claim 1 (para [0206] “If the modules are software modules, the modules can be embodied on a non-transitory computer readable medium and processed by a processor in any of the computer systems described herein.”).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Huang/Wu, as applied above, and further in view of Zhang et al. (US-20200242353-A1).
Regarding Claim 2,
Huang and Wu teach the method of claim 1. Huang further teaches wherein performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed comprises:
receiving and storing a first piece of point data input by the previous network layer, and performing the preset pooling operation on the first piece of point data to obtain a first pooling result (para [0071] “For example, a maximum value of 9 is selected from region 442,”. Region 442 (i.e. first piece of point data).); and
receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data, and performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain a final pooling result (para [0071] “The maximum value of each region may be selected as a subsample representing each region. For example, a maximum value of 9 is selected from region 442, a maximum value of 2 is selected from region 444, a maximum value of 5 is selected from region 446, and a maximum value of 6 is selected from region 448. Thus, a feature map 450 with four elements 9, 2, 6, and 5 may be generated from the 6×6 input matrix 410 after the convolution, non-linear activation, and pooling operations.” Pooling operations are performed on regions 444 and 446 (i.e. other pieces of point data). Feature map 450 (i.e. final result).).
While Huang discloses performing pooling operations on pieces of point data Huang does not explicitly disclose that the pieces of point data are sequentially received and processed. In other words, Zhang does not explicitly disclose
sequentially receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data, and performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain a final pooling result.
However, Zhang (US 20200242353 A1) teaches
sequentially receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data, and performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain a final pooling result (para [0056] “As shown in FIG. 2, the strided pooling illustrated by the transition from the feature map 200 to the downsampled, pooled feature map 202 is functionally equivalent to a two-step process of dense pooling and naive downsampling, though conventional image classification systems do not perform the two steps separately (e.g., via independent layers of a neural network) and instead utilize a single strided pooling layer.” Figure 2 shows a strided pooling operation where a 2x2 grid (i.e. piece of point data) is sequentially shifted over the feature map 200 to produce pooling results in 202.).
Huang, Wu, and Zhang are analogous because they are both directed towards the same field of endeavor of convolutional neural networks.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the pooling layer of Zhang.
Doing so would allow for applying the dense pooling, low-pass filtering, and down-sampling operations to improve the accuracy of the neural network (Zhang para [0030]).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Huang/Wu/Zhang, as applied above, and further in view of Ko et al. (US-11783167-B1).
Regarding Claim 3,
Huang, Wu, and Zhang teach the method of claim 2.
Huang further teaches
n is a positive integer, and 1 < n <N (para [0071] In the example shown in FIGS. 4D and 4E, a max pooling operation may be applied to matrix 440, where the 4×4 matrix 440 may be divided into four 2×2 regions 442, 444, 446, and 448. The maximum value of each region may be selected as a subsample representing each region. For example, a maximum value of 9 is selected from region 442, a maximum value of 2 is selected from region 444, a maximum value of 5 is selected from region 446, and a maximum value of 6 is selected from region 448. Figure 4D shows that there are four pieces of point data wherein a maximum pooling operation is being performed on each piece of data and Figure 4E shows four pooling results therefore N = 4 in this instance.).
Huang, Wu, and Zhang do not explicitly disclose
wherein sequentially receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data, and performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain the final pooling result comprises:
receiving and storing an nth piece of point data of the to-be-processed data, and performing the preset pooling operation on the nth piece of point data based on a pooling result of an (n- 1)th piece of point data to obtain an nth pooling result; and
receiving and storing an Nth piece of point data of the to-be-processed data, and performing the preset pooling operation on the Nth piece of point data based on a pooling result of an (N-1)th piece of point data to obtain an Nth pooling result
wherein the Nth pooling result is the final pooling result of the to-be-processed data; and
However, Ko teaches
wherein sequentially receiving and storing the other pieces of point data of the to-be-processed data except the first piece of point data, and performing the preset pooling operation after each of the other pieces of point data is received until the preset pooling operations of all the point data of the to-be-processed data are completed to obtain the final pooling result comprises:
receiving and storing an nth piece of point data of the to-be-processed data (col. 30 lines 12-14; “In the third cycle 1415, the circuit receives the next input A2 as the first operand.” A2 (i.e. nth piece of point data).), and performing the preset pooling operation on the nth piece of point data based on a pooling result of an (n- 1)th piece of point data to obtain an nth pooling result (col. 30 lines 12-19; “The accumulate signal input is on, so the second operand is read from the register 1220, currently storing the result of the previous output, max(A1,A0). The math circuit therefore outputs the maximum of A2 and max(A1,A0) to the register.” Max (A1,A0) (i.e. pooling result of an (n- 1)th piece of point data). Max of A2 and max(A1,A0) (i.e. nth pooling result).); and receiving and storing an Nth piece of point data of the to-be-processed data, and performing the preset pooling operation on the Nth piece of point data based on a pooling result of an (N-1)th piece of point data to obtain an Nth pooling result (col. 30 lines 20-28;);
n is a positive integer, and 1 < n <N (col. 13 lines 27-28; “For example, if A3>A2>A1>A0, then the math circuit outputs A3.”).
Huang, Wu, Zhang, and Ko are analogous because they are both directed towards the same field of endeavor of convolutional neural networks performing pooling operations.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the pooling calculation of Ko.
Doing so would allow for computing neural network operations in an efficient and low-power manner in accordance to configuration data provided by control circuits (Ko col. 14 lines 1-3;)
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Huang/Wu, as applied above, and further in view of Ko et al. (US-11783167-B1) and Lyuh et al. (US-20190079801-A1).
Regarding Claim 5,
Huang and Wu teach the method of claim 4.
Huang further teaches
wherein n is a positive integer, and 1 < n <N (para [0071] In the example shown in FIGS. 4D and 4E, a max pooling operation may be applied to matrix 440, where the 4×4 matrix 440 may be divided into four 2×2 regions 442, 444, 446, and 448. The maximum value of each region may be selected as a subsample representing each region. For example, a maximum value of 9 is selected from region 442, a maximum value of 2 is selected from region 444, a maximum value of 5 is selected from region 446, and a maximum value of 6 is selected from region 448. Figure 4D shows that there are four pieces of point data wherein a maximum pooling operation is being performed on each piece of data and Figure 4E shows four pooling results therefore N = 4 in this instance.).
Huang and Wu do not explicitly disclose
wherein a storage space of the many-core system comprises a first storage space and a second storage space;
in a case where the preset pooling operation is the average pooling operation, performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be- processed data are completed comprises:
receiving a first piece of point data and storing the first piece of point data in the first storage space as data A;
initializing data in the second storage space to be 0, and storing data Bi=Ai*(1/N) in the second storage space;
receiving an nth piece of point data and storing in the first storage space as data An;
outputting An to the second storage space through a multiplier accumulator to obtain Bn=Bn-1+An*(1/N),
PNG
media_image3.png
50
233
media_image3.png
Greyscale
;
receiving an Nth piece of point data and storing in the first storage space as data AN; and
outputting AN to the second storage space through the multiplier accumulator to obtain BN=BN-1+AN*(1/N),
PNG
media_image4.png
58
234
media_image4.png
Greyscale
;
However, Ko (US 11783167 B1) teaches
wherein a storage space of the many-core system comprises a first storage space and a second storage space (col. 28 lines 25-30; “The register 1220 holds the previous output of the math circuit 1215 until that output is required as an operand for the next operation of the math circuit 1215, or the operation is complete and the value is sent to the other post-processing operations (as shown in FIG. 10).” Math circuit 1215 (i.e., first storage space). Register 1220 (i.e., second storage space).);
in a case where the preset pooling operation is the average pooling operation, performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be- processed data are completed comprises:
receiving a first piece of point data and storing the first piece of point data in the first storage space as data A1 (col. 28 lines 65-67; “Accordingly, the math circuit 1215 receives input value A0 and outputs value A0 to the register.” A0 (i.e., A1) is stored in math circuit 1215 (i.e., first storage space).);
initializing data in the second storage space to be 0, and storing data Bi=Ai*(1/N) in the second storage space (col. 28 lines 61-67; “In the first (initial) cycle 1305, the activation input A0 is received at the first operand input. The accumulate signal input is off, indicating that the secondary operand input should be a preset value (zero, in this case defined as the preset based on the operation type). Accordingly, the math circuit 1215 receives input value A0 and outputs value A0 to the register.” Cycle 1305 of figure 13 shows ‘0’ stored in register 1220 (i.e., second storage space). An average pooling operation is performed on A0 and ‘0’ and the result of the average pooling operation (i.e., B1) is stored in register 1220 (i.e., second storage space).);
receiving an nth piece of point data and storing in the first storage space as data An; outputting An to the second storage space through a multiplier accumulator (col. 28 lines 31-40; The multiplier accumulator comprises circuit 1215 and register 1220 and shown in figure 12.) to obtain Bn=Bn-1+An*(1/N),
PNG
media_image3.png
50
233
media_image3.png
Greyscale
; receiving an Nth piece of point data and storing in the first storage space as data AN; and outputting AN to the second storage space through the multiplier accumulator to obtain BN=BN-1+AN*(1/N),
PNG
media_image4.png
58
234
media_image4.png
Greyscale
(col. 29 lines 1-16; This section describes receiving activation data (i.e. nth piece of data) that is stored in circuit 1215 (i.e., first storage space) and summing it with the previous average value stored in register 1220 (i.e., second storage space).);
wherein n is a positive integer, and 1 < n <N, and 1 < n <N (col. 29 lines 17-29; This section shows there are at least 4 pieces of data therefore N = 4.).
Huang, Wu, and Ko are analogous because they are both directed towards the same field of endeavor of convolutional neural networks performing pooling operations.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the pooling calculation of Ko.
Doing so would allow for computing neural network operations in an efficient and low-power manner in accordance to configuration data provided by control circuits (Ko col. 14 lines 1-3;)
While Ko suggests an embodiment wherein a scaling operation (i.e., division) is performed on the output of math circuit 1215 after each cycle (col. 28 lines 52-60; “The four inputs (A0, A1, A2, A3) are received one at a time (e.g., in subsequent clock cycles) as primary input, added using the 18-bit adder, and scaled appropriately to calculate the average value. In this example, the operation type is seven bits (six bits to specify a pooling operation and one extra bit to specify that it is an average pooling operation). The math circuit also receives shift/scale factors (not shown) as configuration data in some embodiments.” And col. 29 lines 30-35; “In some embodiments, the math circuit output is scaled after each cycle,”), it is not explicitly shown in Ko.
However, Lyuh teaches
receiving an nth piece of point data and storing in the first storage space as data An; outputting An to the second storage space through a multiplier accumulator to obtain Bn=Bn-1+An*(1/N) (para [0057] For the average pooling, the adder 1115 may perform an addition and division calculation (e.g., a shift calculation) on the previous and new calculation results. For example, the output register 1117 may be updated with the division calculation result. Register 1117 holds previous calculation results.);
Huang, Ko, and Lyuh are analogous because they are both directed towards the same field of endeavor of convolutional neural networks performing pooling operations.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang and Ko with the pooling calculation of Lyuh.
Doing so would allow for minimizing the number of memory accesses to the feature map resulting in improved power consumption (Lyuh para [0058]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Huang/Wu, as applied above, and further in view of Ko et al. (US-11783167-B1) and Dimant et al. (US-11188302-B1).
Regarding Claim 6,
Huang and Wu teach the method of claim 4. Huang further teaches wherein n is a positive integer, and 1 < n <N (para [0071] In the example shown in FIGS. 4D and 4E, a max pooling operation may be applied to matrix 440, where the 4×4 matrix 440 may be divided into four 2×2 regions 442, 444, 446, and 448. The maximum value of each region may be selected as a subsample representing each region. For example, a maximum value of 9 is selected from region 442, a maximum value of 2 is selected from region 444, a maximum value of 5 is selected from region 446, and a maximum value of 6 is selected from region 448. Figure 4D shows that there are four pieces of point data wherein a maximum pooling operation is being performed on each piece of data and Figure 4E shows four pooling results therefore N = 4 in this instance.).
Huang and Wu do not explicitly disclose
wherein a storage space of the many-core system comprises a first storage space and a second storage space;
in a case where the preset pooling operation is the maximum pooling operation, performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be- processed data are completed comprises:
receiving a first piece of point data and storing in the first storage space as data A1;
initializing data Bo in the second storage space to be negative infinity, and storing a maximum value Bi=Max(Ai,Bo) in the second storage space;
receiving an nth piece of point data and storing in the first storage space as data An;
storing a maximum value Bn=Max(A,Bn-1) in the second storage space, wherein Bn-1=Max(A1,...,An-1); and
receiving an Nth piece of point data and storing in the first storage space as data AN; and
storing a maximum value BN=Max(AN,BN-1) in the second storage space, wherein BN-1=Max(A1,...,AN-1);
However, Ko (US 11783167 B1) teaches
wherein a storage space of the many-core system comprises a first storage space and a second storage space (col. 28 lines 25-30; “The register 1220 holds the previous output of the math circuit 1215 until that output is required as an operand for the next operation of the math circuit 1215, or the operation is complete and the value is sent to the other post-processing operations (as shown in FIG. 10).” Math circuit 1215 (i.e., first storage space). Register 1220 (i.e., second storage space).);
in a case where the preset pooling operation is the maximum pooling operation, performing the preset pooling operation on the received point data after each piece of point data is received until the preset pooling operations of all the point data of the to-be- processed data are completed comprises:
receiving a first piece of point data and storing in the first storage space as data A1 (col. 29 lines 62-53; “In the first (initial) cycle 1405, the activation input A0 is received at the first operand input.” Cycle 1405 of figure 14 shows A0 (i.e. A1) being received and stored in the circuit 1215 (i.e. first storage space).);
storing a maximum value Bi=Max(Ai,Bo) in the second storage space (col. 29 lines 62-66; “In the first (initial) cycle 1405, the activation input A0 is received at the first operand input. The accumulate signal input is off, indicating that the secondary operand input should be a preset value (−16, in this case defined as the preset based on the operation type). The preset value is chosen as the lowest value possible, which is −16 for 5-bit signed integers. This ensures that the output of this first comparison will always be the value received at the primary input, so that the initial value A0 is the one stored at the register 1220 instead of a result from a previous computation.” A max pooling operation is performed comparing the activation value A0 (i.e., A1) with the initial preset value -16 (i.e., B0) stored in register 1220 (i.e. second memory). The larger value A0 (i.e. A1) is stored in register 1220 (i.e. second storage space).);
receiving an nth piece of point data and storing in the first storage space as data An; storing a maximum value Bn=Max(A,Bn-1) in the second storage space, wherein Bn-1=Max(A1,...,An-1);; and receiving an Nth piece of point data and storing in the first storage space as data AN; and storing a maximum value BN=Max(AN,BN-1) in the second storage space, wherein BN-1=Max(A1,...,AN-1) (fig. 15; col. 30 lines 6-32; This section describes receiving an input activation (i.e. nth piece of point data) that is stored in circuit 1215 (i.e. first storage space). A previous pooling result (i.e. Bn-1) is stored in register 1220 (i.e. second storage space). A max pooling operation is performed on the input activation and the previous pooling result and the result is stored in register 1220 (i.e. second storage space).);
wherein n is a positive integer, and 1 < n <N (col. 13 lines 27-28; “For example, if A3>A2>A1>A0, then the math circuit outputs A3.”).
Huang, Wu, and Ko are analogous because they are both directed towards the same field of endeavor of convolutional neural networks performing pooling operations.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the pooling calculation of Ko.
Doing so would allow for computing neural network operations in an efficient and low-power manner in accordance to configuration data provided by control circuits (Ko col. 14 lines 1-3;)
Diamant (US 11188302 B1) teaches
initializing data Bo in the second storage space to be negative infinity (col. 9 lines 56-58; When Xv[index]==the maximum value, the process 300 sets Xv[index] equal to a value representing negative infinity. Doing so makes Xv[index] now the smallest value among the values in Xv.), and storing a maximum value Bi=Max(Ai,Bo) in the second storage space (col. 9 lines 19-23; “At step 314, the process 300 finds the maximum value among the values in Xv. To find the maximum value, the process 300 can, for example, instruct the accelerator to compare each value in Xv to each other value in Xv until the largest is found.”);
Huang and Diamant are analogous because they are both directed towards the same field of endeavor of convolutional neural networks performing pooling operations.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the convolutional neural network of Huang with the pooling operation of Diamant.
Doing so would allow for resetting the maximum values by removing the maximum values so that the next set of maximum values can be calculated (Diamant col. 9 lines 45-57;).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HENRY K NGUYEN whose telephone number is (571)272-0217. The examiner can normally be reached Mon - Fri 7:00am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at 5712723768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HENRY NGUYEN/Examiner, Art Unit 2121
Prosecution Insights