Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1-19, 22, 25 and 26 have been considered but are mostly moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The arguments that still apply are rebutted below.
Applicant argues, “claim 1 is directed to a non-transitory digital medium storing a specifically structured data stream…” remarks 15. Products that do not have a physical or tangible form, such as information (often referred to as "data per se") when claimed as a product without any structural recitations. The product is the digital medium. There is not structural recitation after that. This is a claim to data per se in claim 1.
Applicant argues that their claims are directed to an improvement, and therefore are patent eligible. DesJardins allowed a claimed improvement where “the specification identified improvements as to how the machine learning model itself operates…” The model is the neural network. It is unclear how encoding a neural network into a data stream improves the neural network. Sending data or a model somewhere doesn’t necessarily improve the data or the model.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to data per se which is not a statutory category. There is no structure recitation in the body of the claim.
Claims 1-19, 22, 25 and 26 are rejected under 35 U.S.C. 101 because the claimed invention is directed to the abstract idea of a mental concept and mathematical relationship for encoding/decoding a representation of a neural network onto or from a data stream; placing pointers in a data stream with the encoded neural network; placing a data length parameter in the data stream; and structuring the data stream into separate portions and sub-portions. This judicial exception is not integrated into a practical application because Applicant providing and receiving a data stream is “mere data gathering”, which is insignificant extra solution activity. MPEP 2106.05(g). The claimed “efficient execution” is not realistic. If anything, encoding a neural network makes the execution of the neural network less efficient because the network has to be decoded before execution. Therefore, this claim element does not integrate the abstract idea into a practical application. The additional element of picture and video processing merely link the abstract idea to picture and video processing. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the computer and computer readable media are generic computer parts.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-19, 22, 25 and 26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as failing to set forth the subject matter which the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the applicant regards as the invention.
The term “improved efficient execution” in claims 1, 2, 3 and 22 is a relative term which renders the claim indefinite. The term “improved efficient execution” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. There is no definition in the claims or the rest of the specification, that describes how the algorithm is efficient or what the efficiency is.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-19, 22, 25 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over US20180082181A1 to Brothers et al, US20020102026A1 to Rijavec et al and US20140016785A1 to Neuendorf et al.
Brothers teaches claims 1-3 and 22. (Currently Amended) A non-transitory digital storage medium storing a data stream having a representation of a neural network encoded thereinto for a transmission to an apparatus for neural network processing, (Brothers encoding is in fig. 4, the system is in fig. 3, and decoding is taught in fig. 5.) wherein the data stream is structured (Brothers fig. 4 compress weights 407, store compressed weights 409. Brothers para 40 “In one embodiment, the manner in which zero values are handled depends in part on the layer type (e.g., convolutional layer vs. fully connected layer).”) (Brothers para 25 “ network pruning and weight clustering of selected weights may be performed after network training.” Clustes are sub-portions.) wherein the apparatus is configured to decode from the data stream, for each of one or more predetermined individually accessible (Encoding and decoding are taught by Brothers fig. 4 and 5 409 and 505.)
wherein the neural network is configured for a picture and/or video analysis and the structuring of the data stream into the one or more(Brothers teaches that weights “will also typically be arbitrarily organized in memory… [which] impacts… execution efficiency.” Brothers solves this with reordering, fig. 4 step 405, before streaming the reordered weights to the memory in step 409. Brothers para 3 “Example applications of NNs include image processing, speech recognition…”)
Brothers doesn’t teach a portion in the data stream.
However, Rijavec teaches so that the data stream is structured into one or more individually accessible portions, each portion representing a corresponding… (Rijavec abs “a compressed data set, at least one pointer to a location in the compressed data stream…” Rijavec para 8 “a user may only want to access a section of an image…” The section is the portion.)
and wherein the data stream is, within a predetermined portion, further structured into individually accessible sub-portions…(Rijavec para 10 “at least one pointer to a location in the compressed data stream whose decoded output comprises a location on a line of data…” The line is the sub-portion.)
a pointer pointing to a beginning of the respective predetermined individually accessible sub-portion, and (Rijavec para 10 “at least one pointer to a location in the compressed data stream whose decoded output comprises a location on a line of data…”)
a data stream length parameter indicating a data stream length of the respective predetermined individually accessible sub-portion (Rijavec para 21 “The reentry data set includes a pointer into the compressed data at the location whose decoded output is the start of the image section in the line and an indication of the location in the compressed data stream whose decoded output is the end of the image section on the line, such as a length of the image section intersecting the line or an end pointer to that location in the compressed data stream whose output is the end of the image section on the line.” Length of image section intersecting line is the length parameters for the sub portion.)
individually accessible portions and the individually accessible sub-portions…(Rijavec abs “a compressed data set, at least one pointer to a location in the compressed data stream…”)
Rijavec, the claims, and Brothers all encode compressed date onto a data stream. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to use Rijavec’s portions to allow a user to decode “only a section… that a user wants access[ to].” Rijavec para 10.
Rijavec doesn’t teach skipping.
However, Neuendorf teaches a parameter for skipping the respective predetermined individually accessible sub-portion in parsing the data stream,… (Neuendorf para 178 “The extension payload length is signaled for nescient decoders to skip over it.”)
Rijavec, Brothers, Neuendorf and the claims are all directed to decoding/encoding data streams. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to skip based on a length parameter “in order to be able to skip over configuration extensions unknown to the decoder.” Neuendorf para 209.
Brothers teaches claim 4. (Currently Amended) The apparatus of claim 3, wherein the neural network layer type parameter discriminates, at least, between a fully-connected and a convolutional layer type. (Brothers para 40 “In one embodiment, the manner in which zero values are handled depends in part on the layer type (e.g., convolutional layer vs. fully connected layer).”)
Rijavec teaches claim 5. (Currently Amended) The apparatus of claim 3, wherein the apparatus is configured to decode, from the data stream, for each of the one or more predetermined individually accessible portions, a pointer pointing to a beginning of each respective individually accessible portion. (Rijavec para 21 “The reentry data set includes a pointer into the compressed data at the location whose decoded output is the start of the image section…”)
Brothers teaches claim 6. (Currently Amended) The apparatus of claim 5, wherein each individually accessible portion represents a corresponding neural network layer of the neural network or a neural network portion of a corresponding neural network layer of the neural network. (Brothers para 40 “In one embodiment, the manner in which zero values are handled depends in part on the layer type (e.g., convolutional layer vs. fully connected layer).”)
Rijavec teaches claim 7. (Currently Amended) The apparatus of claim 3, wherein the apparatus is configured to decode from the data stream, for each of one or more predetermined individually accessible sub- portions, a start code at which the respective predetermined individually accessible sub- portion begins. (Rijavec para 21 “The reentry data set includes a pointer into the compressed data at the location whose decoded output is the start of the image section in the line and an indication of the location in the compressed data stream….”)
Rijavec teaches claim 8. (Currently Amended) The apparatus of claim 7, wherein the apparatus is configured to decode, from the data stream, the representation of the neural network using context-adaptive arithmetic decoding and using context initialization at a start of each individually accessible portion and each individually accessible sub-portion. (Rijavec para 10 “ at least one pointer to a location in the compressed data stream whose decoded output comprises a location on a line of data, and decoding information for each received pointer that enables decoding from a point within the compressed data stream addressed by the pointer.”)
Brothers teaches claim 13. (Currently Amended) The apparatus of claim 3, wherein the apparatus is configured to decode from the data stream, for each of one or more (Brothers para 22 “Additionally, the reordering may be selected to improve prediction accuracy in the encoding. As another example, network feature maps can be reordered so that weight values tend to increase or the number of zero value weights increase.” The supplemental data is the improved accuracy or increase of the number of zero weights.)
Brothers doesn’t teach portions.
Rijavec teaches predetermined individually accessible portions. (Rijavec para 10 “ at least one pointer to a location in the compressed data stream whose decoded output comprises a location on a line of data…”)
Brothers teaches claim 14. (Currently Amended) The apparatus of claim 13, wherein the data stream indicates the supplemental data as being dispensable for inference based on the neural network. (Brothers para 22 “As another example, network feature maps can be reordered so that weight values tend to increase or the number of zero value weights increase.” Zero weights are dispensable because they can be skipped or pruned in the neural network. Brothers para 23 “Also, by redistributing non-zero weights, it is possible to more effectively skip over zero-value-weights during network execution.”)
Brothers teaches claims 25 and 26. (New) The apparatus of claim 2, wherein the apparatus is a picture and/or video processing device configured to derive the neural network for picture and/or video processing. (Brothers teaches that weights “will also typically be arbitrarily organized in memory… [which] impacts… execution efficiency.” Brothers solves this with reordering, fig. 4 step 405, before streaming the reordered weights to the memory in step 409. Brothers para 3 “Example applications of NNs include image processing, speech recognition…”)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Austin Hicks whose telephone number is (571)270-3377. The examiner can normally be reached Monday - Thursday 8-4 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AUSTIN HICKS/Primary Examiner, Art Unit 2142
Prosecution Insights