DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-6, 8-16, and 18-20 are rejected under 35 U.S.C. 102(a)(2) as being unpatentable by Rezazadegan (US 20210218997 A1).
Regarding Claim 1, representative of Claim 11, Rezazadegan teaches a method of joint optimization-training, the method comprising:
identifying, by a feature extractor, an input feature ([0135]: a video or a video segment 605 may be input to a machine-targeted neural encoder 610, which may be configured to perform generic feature extraction);
generating, by the feature extractor, a feature map for the input feature ([0135]: to perform generic feature extraction. The output spatio-temporal M-features 612), wherein generating the feature map further comprises:
receiving an initial parameter set ([0087]: training is an iterative process, where, at each iteration, the algorithm modifies the weights of the neural network to make a gradual improvement of the network's output, i.e., to gradually decrease the loss);
instantiating a loss function representing a feature representation precision and an encoding complexity level as a function of the input feature and the initial parameter set ([0142]: multiple losses may be considered in the training. For example, one, some, or all of the following losses may be included: [0143] Task loss(es), which may be derived by using the task NNs 634, 636, and/or 638 (e.g., cross-entropy for classifier NN, MSE for regression NN, etc.)…[0145] Rate loss(es), which may include a loss for the machine-targeted bitstream (L.sub.Rm) and a loss for the human-targeted bitstream (L.sub.Rh); and
generating the feature map that minimizes the loss function ([0141] Training of the design for approach A (FIG. 5) or the inference pipeline for approach A (FIG. 6) may comprise minimizing Mean Squared Error (MSE) loss computed on the original video 505 and the decoded video 565, for example by computing gradients of this loss with respect to the neural encoder's 510 and decoder's 560 parameters by backpropagation and computing a weight-update based on a certain optimizer); and
transmitting, by the feature extractor, the feature map to an encoder (Fig. 6, element 610 feature extractor, and downstream element 616 encoder, [0135]: neural encoder 610, which may be configured to perform generic feature extraction. The output spatio-temporal M-features 612 may undergo quantization 614. The quantized output may undergo lossless encoding 616).
Regarding Claim 2, representative of Claim 12, Rezazadegan teaches the method of claim 1. In addition, Rezazadegan teaches wherein identifying the input feature further comprises receiving an input video ([0135] In the machine-targeted instance, a video or a video segment 605 may be input to a machine-targeted neural encoder 610).
Regarding Claim 3, representative of Claim 13, Rezazadegan teaches the method of claim 1. In addition, Rezazadegan teaches wherein identifying the input feature further comprises determining a relevant feature ([0084]: After the feature extraction layers there may be one or more layers performing a certain task, such as classification, semantic segmentation, object detection).
Regarding Claim 4, representative of Claim 14, Rezazadegan teaches the method of claim 3. In addition, Rezazadegan teaches wherein the determining the relevant feature further comprises:
receiving an input training set that correlates a plurality of input features to a plurality of relevant features ([0089]: training set may be used for training the network, i.e., for modification of its learnable parameters in order to minimize the loss, [0132] neural encoder 510 may be a convolutional neural network. Examiner interpreting relevant features to be the features extracted); and
determine the relevant feature as a function of the input feature using a machine learning model, wherein the machine learning model is trained as a function of the input training set ([0132] neural encoder 510 may be a convolutional neural network).
Regarding Claim 5, representative of Claim 15, Rezazadegan teaches the method of claim 4. In addition, Rezazadegan teaches wherein the machine learning model includes a convolutional neural network ([0132] Neural encoder 510 may be a convolutional neural network).
Regarding Claim 6, representative of Claim 16, Rezazadegan teaches the method of claim 4. In addition, Rezazadegan teaches wherein the machine learning model includes a deep neural network ([0132] Neural encoder 510 may be a convolutional neural network where the convolutional layers may perform one or more of the following convolutions: 1D convolution, 2D convolution, 3D convolution, 4D convolution, etc. Alternatively, the neural encoder may comprise a recurrent neural network).
Regarding Claim 8, representative of Claim 18, Rezazadegan teaches the method of claim 1. In addition, Rezazadegan teaches wherein the loss function is configured to reduce a bitstream size ([0142]: the rate loss may be used to train the codec to encode data into a short bitstream, [0152] The rate loss computed on the machine-targeted data may be used to train one or more of the NNs in the encoder part of the machine-targeted structure, such as any NNs used for lossless encoding (e.g. 616), and the neural encoder 610).
Regarding Claim 9, representative of Claim 19, Rezazadegan teaches the method of claim 1. In addition, Rezazadegan teaches wherein the loss function is configured to enhance the feature representation precision ([0142]: one, some, or all of the following losses may be included: [0143] Task loss(es), which may be derived by using the task NNs 634, 636, and/or 638 (e.g., cross-entropy for classifier NN, MSE for regression NN, etc.), [0150]: the task losses may be used to train one or more of the NNs in the machine-targeted structure (e.g. 610,…).
Regarding Claim 10, representative of Claim 20, Rezazadegan teaches the method of claim 1. In addition, Rezazadegan teaches transmitting the feature map further comprises producing a bitstream as a function of the feature map ([0135]: The output machine-targeted encoded data 618 may be stored in memory or otherwise made available for analysis and/or consumption, [0110] Device D1 may be a device that contains an encoder of data, i.e., that is able to encode data to a bitstream).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Rezazadegan (US 20210218997 A1) in view of Duan (Ling-Yu Duan et al. “Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics”, arxiv.org, Cornell University Library, 13 January 2020, arXiv:2001.03569v2).
Regarding Claim 7, representative of Claim 17, Rezazadegan teaches the method of claim 4. However, Rezazadegan does not explicitly teach the remaining limitations of Claim 7.
Duan teaches wherein the machine learning model is configured to perform a pooling function ([Section III. B]: NIP method produces compact global descriptors from a CNN model by progressive pooling operations to improve the translation, scale and rotation invariance over the feature maps).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present invention to have modified the teachings of Rezazadegan by Duan to include a CNN with a pooling operation. Doing so would improve the invariance over feature maps extracted thereby improving the quality of the feature extraction and improving the accuracy of downstream encoding and decoding.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANICE VAZ whose telephone number is (703)756-4685. The examiner can normally be reached Monday-Friday 9:00-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571) 272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANICE E. VAZ/Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667
Prosecution Insights