DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 8, 9-11, 14, and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yu (EP 4199512 A1) in view of Lee (KR 102119687 B1).
Regarding claims 1 and 11. Yu teaches a device for video decoding, the device comprising:
a processor configured to:
obtain a neural network (NN) feature restorer type indication in video data (see Fig. 4, item 402, decoding the bitstream to obtain third feature data; see paragraph 107, the apparatus for decoding feature data obtains the bitstream, decodes it, and then performs inverse expression to reconstruct the original feature data, and inputs the reconstructed original feature data back to the neural network model for task processing and analysis),
wherein the NN feature restorer type indication is configured to indicate an NN feature restorer type that is associated with an NN model (see paragraph 109, data type conversion processing is performed on the second feature data corresponding to each channel to obtain first feature data corresponding to each channel. That is to say, the third feature data obtained by decoding is processed and restored to the second feature data corresponding to each channel through the spatial inverse expression technology, and the data type conversion processing is performed on the second feature data corresponding to each channel to obtain the first feature data corresponding to each channel output by the intermediate layer of the neural network model, thereby reducing the interaction of feature data among different channels during encoding, and avoiding the occurrence of feature data of two channels in the same coding block when encoding the feature data, so as to efficiently utilize the existing video encoding and decoding standards); and
based on the feature parameter associated with the NN feature restorer type, decode the NN model (see paragraph 126, the decoding apparatus decodes and performs spatial inverse expression on the bitstream, and performs feature data floating-point to reconstruct the feature data. The reconstructed feature data is input back to the neural network model for task processing and analysis).
However, Yu does not expressly teach based on the NN feature restorer type indication, obtain a feature parameter associated with the NN feature restorer type.
Lee teaches that the loss calculation unit 30 is the first loss value (ie, the restoration loss value of the encoder and decoder) generated when the input data through the neural network passes through the encoder 12 and the decoder 14 to obtain a reconstructed improved image. A second loss value for essential preservation information to be preserved when the input data is restored through the encoder 12 and decoder 14 (that is, an information theoretical optimization value for preserving image information), the input data is an encoder (12) and the third loss value generated when passing through the classifier network 20 (that is, the classification loss value of the classifier network 20) and the essential preservation information to be preserved when classifying in the classifier network 20 The fourth loss value (that is, an information-theoretic optimization value for eliminating bias influence) is calculated.
At this time, the first to fourth loss values are as shown in Equation 2.
Here, θ .sub.f is a feature parameter of the encoder function, θ .sub.g is a feature parameter of the decoder function, θ .sub.h is a feature parameter of the classifier network function, Is the expected function for the input data probability distribution p(x), P .sub.x (X) is the probability distribution of low quality data, L .sub.c is the cross entropy loss function, The Clear image (ie, target image), Is the value sampled from the input data probability distribution (for example, the input image). The bias level of (i.e. the label of the weather condition class), Is the bias prediction value from the auxiliary distribution called Q, L .sub.B is the cross entropy for bias classification, and α, β, and γ are values of 0.05 to 0.2 as a balance function (see page 5 and 6, lines 22-31 and 1-12, and equation 2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yu by Lee to obtain , A second loss value for essential preservation information to be preserved when the input data is restored through the encoder 12 and decoder 14 (that is, an information theoretical optimization value for preserving image information), the input data is an encoder (12) and the third loss value generated when passing through the classifier network 20 (that is, the classification loss value of the classifier network 20) and the essential preservation information to be preserved when classifying in the classifier network 20 The fourth loss value (that is, an information-theoretic optimization value for eliminating bias influence) is calculated; and θ .sub.f is a feature parameter of the encoder function, θ .sub.g is a feature parameter of the decoder function, θ .sub.h is a feature parameter of the classifier network function, in order to provide based on the NN feature restorer type indication, obtain a feature parameter associated with the NN feature restorer type. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 4 and 14. The device of claim 1, wherein the NN feature restorer type indication is obtained in a supplemental enhancement information (SEI) message (see Yu, paragraph 32).
Regarding claim 8. The device of claim 1,
wherein the feature parameter associated with the NN feature restorer type is obtained in neural network coding (NNC) video data (see Yu, paragraph 55, the encoded feature data in this technology is feature data output from the intermediate layer of the neural network model. The general flow of the present disclosure is as follows. Original feature data is output from the intermediate layer of the neural network model, in general form (C, H, W), where C is the channel number of feature data, H is the height, and W is the width. The original feature data is re-expressed in spatial domain and encoded by an encoder to generate a bitstream; then the bitstream is decoded at the decoding end and subjected to an inverse expression to reconstruct and generate the original feature data, and the reconstructed original feature data is input back to the neural network model for task processing and analysis),
wherein the NNC video data comprises at least one of a uniform resource identifier (URI) pointer or a payload (see Yu, paragraph 32).
Regarding claims 9 and 18. The device of claim 1, wherein an NN feature associated with the NN model comprises a number of tensors (see Yu, paragraph 41),
wherein the feature parameter associated with the NN feature restorer type comprises a number of NNC streams (see Lee page 5, lines 1-5, The auto-encoder 10 is a place for learning data input through a neural network. When a video image (hereinafter referred to as "input data") is input through a neural network, important features in the input data are input through the encoder 12. The extracted and extracted important features are restored through the decoder 14 and output), and
wherein each of the number of tensors is associated with at least one of the number of NNC streams (see Yu, paragraph 41 and 55, three-dimensional (3D) feature data tensor expression includes channel number, height and width, abbreviated as (C, H, W); original feature data is output from the intermediate layer of the neural network model, in general form (C, H, W), where C is the channel number of feature data, H is the height, and W is the width. The original feature data is re-expressed in spatial domain and encoded by an encoder to generate a bitstream; then the bitstream is decoded at the decoding end and subjected to an inverse expression to reconstruct and generate the original feature data, and the reconstructed original feature data is input back to the neural network model for task processing and analysis).
Regarding claims 10 and 19. The device of claim 1, wherein the processor is further configured to:
obtain an NN model indication, wherein the NN model indication is associated with a feature coding for machine (FCM) decoding model; and decode the NN model further based on the FCM decoding model (see paragraph 55, the encoded feature data in this technology is feature data output from the intermediate layer of the neural network model. The general flow of the present disclosure is as follows. Original feature data is output from the intermediate layer of the neural network model, in general form (C, H, W), where C is the channel number of feature data, H is the height, and W is the width. The original feature data is re-expressed in spatial domain and encoded by an encoder to generate a bitstream; then the bitstream is decoded at the decoding end and subjected to an inverse expression to reconstruct and generate the original feature data, and the reconstructed original feature data is input back to the neural network model for task processing and analysis).
Claim(s) 5-7 and 15-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yu (EP 4199512 A1) in view of Lee (KR 102119687 B1), and in view of Sun (CN 108734206 B).
Regarding claims 5 and15. The device of claim 1, wherein the NN feature restorer type comprises at least one of an NN-based restorer type (see Yu, paragraph 109, the third feature data obtained by decoding is processed and restored to the second feature data corresponding to each channel through the spatial inverse expression technology, and the data type conversion processing is performed on the second feature data corresponding to each channel to obtain the first feature data corresponding to each channel output by the intermediate layer of the neural network model).
However, the combination does not expressly teach or a principal component analysis (PCA) restorer type.
Sun teaches that multi-layer self-encoding can be regarded as linear PCA, it uses adaptive multi-coder "network the high-dimensional data into low-dimensional code, and the like of the" decoder " network, restoring data from low-dimensional code. from the target code is centralized learning of a representation from the data (code), typically in order to achieve the purpose of dimensionality reduction (see page 4, lines 10-14); using principal component analysis on the data dimensionality, so it can efficiently decrease the dimension for a high dimensional data of linear configuration.
Preferably, the method comprises the following steps:
(1) to provide the target function;
(2) optimizing the objective function.
Preferably, in the step (1):
For a given original data set, expressed as wherein m and respectively represents the dimension and the number of samples of the original data, the original data X (0) has centralized;
data flow of the defined formula (1)
wherein is a weight. is a deviation, diag (.) represents a vector mapped to a diagonal matrix, activation function and an element representing all 1 of the vector, using φ represents a linear transformation of the original data, X (2) = φ (X (0)), by maximizing the mapping after data X (2) of the covariance of the q-FyFan norm, the covariance matrix with low rank attribute (see page 5 and 6, lines 21-31 and 1-5).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination by Sun to obtain multi-layer self-encoding can be regarded as linear PCA, it uses adaptive multi-coder "network the high-dimensional data into low-dimensional code, and the like of the" decoder " network, restoring data from low-dimensional code and using principal component analysis on the data dimensionality, so it can efficiently decrease the dimension for a high dimensional data of linear configuration, in order to provide a principal component analysis (PCA) restorer type. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Regarding claims 6 and 16. The device of claim 5, wherein the feature parameter associated with the NN-based restorer type comprises at least one of a structure parameter associated with the NN model or a weight parameter associated with the NN model (see Sun, page 11, lines 1-3, for each video segment, using LBP-Top characteristic, and then extracting a linear-structure through SVD. adjusting the video fragment is a 177 * 1 matrix. The Gaussian kernel function, adopting the Grassmann geodesic distance, obtaining the nuclear matrix).
Regarding claims 7 and 17. The device of claim 5, wherein the feature parameter associated with the PCA restorer type comprises at least one of a PCA basis vector or a PCA mean value (see Sun, page 3, lines 9-15, in order to accurately and efficiently processing data of the real world, people always efforts from the original high-dimensional data to reduce data dimensionality and extracting the important features. wherein, principal component analysis (PCA) is a standard tool, can be removed from the band noise or high-dimensional data extracting relative feature information. by the least effort, PCA provides a basic method to reveal hidden or simplified structure, and the high-dimensional data is reduced to a lower dimension).
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yu (EP 4199512 A1) in view of Luo (CN 117576510 A).
Regarding claim 20. Yu teaches a device for video encoding, the device comprising:
a processor configured to:
perform feature reduction to reduce features associated with a neural network (NN) model into compressed features associated with the NN model (see paragraph 109, data type conversion processing is performed on the second feature data corresponding to each channel to obtain first feature data corresponding to each channel. That is to say, the third feature data obtained by decoding is processed and restored to the second feature data corresponding to each channel through the spatial inverse expression technology, and the data type conversion processing is performed on the second feature data corresponding to each channel to obtain the first feature data corresponding to each channel output by the intermediate layer of the neural network model, thereby reducing the interaction of feature data among different channels during encoding, and avoiding the occurrence of feature data of two channels in the same coding block when encoding the feature data, so as to efficiently utilize the existing video encoding and decoding standards);
determine an NN feature restorer type that is associated with the NN model (see paragraph 170, the third feature data obtained by decoding is processed and restored to the second feature data corresponding to each channel through the spatial inverse expression technology, and the data type conversion processing is performed on the second feature data corresponding to each channel to obtain the first feature data corresponding to each channel output by the intermediate layer of the neural network mode l, thereby reducing the interaction of feature data among different channels during encoding, and avoiding the occurrence of feature data of two channels in the same coding block when encoding the feature data, so as to efficiently utilize the existing video encoding and decoding standards. Therefore, on the basis of improving the encoding efficiency of feature data, the encoded data can be decoded quickly and accurately, and thus the original feature data corresponding to each channel output by the intermediate layer of the neural network model can be obtained).
However, Yu does not expressly teach to send an NN feature restorer type indication in video data, wherein the NN feature restorer type indication is configured to indicate the NN feature restorer type that is associated with the NN model.
Luo teaches that the first image of the first type without label can be input into the first neural network, the encoder in the first neural network can obtain the depth feature in the first image of the first type, according to the decoder in the first neural network, the obtained depth feature of the first type is restored, A restoration image similar to the input first image is obtained, and the first neural network is trained according to the difference between the restoration image and the first image. The trained first neural network can learn depth features in the first type of image data (see page 11 and 12, lines 34-35 and 1-5).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Yu by Luo to obtain the first image of the first type without label can be input into the first neural network, the encoder in the first neural network can obtain the depth feature in the first image of the first type, according to the decoder in the first neural network, the obtained depth feature of the first type is restored, A restoration image similar to the input first image is obtained, and the first neural network is trained according to the difference between the restoration image and the first image, in order to provide to send an NN feature restorer type indication in video data, wherein the NN feature restorer type indication is configured to indicate the NN feature restorer type that is associated with the NN model. Therefore, combining the elements from prior arts according to known methods and technique would yield predictable results.
Allowable Subject Matter
Claims 2-3 and 12-13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN JIA whose telephone number is (571)270-5536. The examiner can normally be reached 9:00 am-7:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIN JIA/Primary Examiner, Art Unit 2663
Prosecution Insights