DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments with respect to claims 1-10 and 21-32 have been considered but are moot in view of the new ground(s) of rejection.
Claim Objections
Claim 1 objected to because of the following informalities: incorrect grammar. The claim states in its last limitation “output a the feature map to a machine process”, however the examiner believes this to merely be a simple typo and will interpret it as “output the feature map to a machine process”. Appropriate correction is required.
Claim 21 objected to because of the following informalities: lack of antecedent basis. The claim states in “the feature encoder receiving the feature signal” but no feature signal was previously cited, nor defined. For the purposes of examining, this will be interpreted as “a feature signal”. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-4, 8, 10, 21-26, 29 and 30 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Karabutov et al. (“Kara”) (U.S. PG Publication No. 2023/0065862).
In regards to claim 1, Kara teaches a decoder, the decoder comprising circuitry (See ¶0291) configured to:
receive an MPEG encoded bitstream (See ¶0004), the bitstream including a header including a feature parameter set (Given the broadest reasonable interpretation consistent with applicant’s specification, the feature parameter set may be taught as at least machine-learning model weightings, coefficients, or supplemental data that describes parameters of the machine model as described in ¶0056-0057 of applicant’s specification, see ¶0108 of Kara wherein any machine-learning-based network, neural network [including convolutional neural network] with any pre-training creates parameters representing such training test data, with further examples of ¶0113-0119, 0226 and 0229 showing example parameters that are used, including those parameters which are included in the header and used in the base layer data), and at least a base feature layer including at least one feature map extracted at an encoder by a partial convolutional neural network (See ¶0113-0115, 0117-0118, 0126-0127 wherein feature maps are a result of a layer through CNN or neural network, and may also be termed as channels, activation map, with ¶0136 describing that base layer features are encoded within the respective base layer, with the base layer bitstream and base feature bitstream being used synonymously at times, additionally 0168-0169 further describing the trained network extracting and decoding the base layer features from the base layer bitstream; it is additionally noted that this is executed as a partial convolutional neural network through the use of a series of convolutional layers as described in at least ¶0113-0117, 0119, );
using information in the feature parameter set, decode the at least a base feature layer to decode the at least one feature map (See at least ¶0108, 0113-0119, 0168-0169, 0226 and 0229 as described above wherein features [including base layer features] are used to extract the base layer data; it is noted by the examiner that the term “feature” is such a broad term that it may encompass the features by which a neural network has been trained [thus features before any decoding has been executed], as well as features which are extracted out of the bitstream by the neural network [using its respective pre-defined features by which decoding has been executed]);
output a the feature map to a machine process (See at least ¶0113-0119; also see FIG. 1-3, 6 and 11-14).
In regards to claim 2, Kara teaches the decoder of claim 1, wherein decoding the at least a base feature layer further comprises inversely pre-processing the at least a decoded base feature layer (See ¶0084-0086 wherein the inverse pre-processing may be taught as inverse processing of the encoded data).
In regards to claim 3, Kara teaches the decoder of claim 2, wherein the at least a header includes at least a pre-processing parameter (See ¶0229-0230 wherein the header includes information associated with the base data as well as information related to the neural network, feature information, number of features, locations of features, as well as other parameters of feature data); and
decoding the at least a base feature layer further comprises inversely pre-processing the at least a decoded base feature layer as a function of the at least a pre-processing parameter (See ¶0084-0086 in view of 0229-0230, wherein it is further noted that parameters are a part of pre-training for neural networks as described in 0108).
In regards to claim 4, Kara teaches the decoder of claim 1, wherein the bitstream includes at least a residual visual layer, and decoding further comprises: decoding the residual layer (See at least FIG. 2 and 6 wherein the residual visual layer is taught as the enhancement layer); and
combining the decoded base feature layer and the decoded residual layer to form a human viewable video signal (See FIG. 2, 6 and 20A).
In regards to claim 8, Kara teaches the decoder of claim 1, wherein the circuitry is further configured to output at least a feature parameter, signaled in the at least a header, to the at least a machine (See ¶0229-0230 and FIG. 4 in view FIG. 5).
In regards to claim 10, Kara teaches the decoder of claim 1, wherein the MPEG encoded bitstream is one of an AVC compliant bitstream or VVC compliant bitstream (See ¶0004-0005, 0088 and 0199).
In regards to claim 21, Kara teaches video encoder for encoding a bit stream to be used by a machine video application, the encoder comprising:
a feature extractor, the feature extractor receiving an input video signal (See FIG. 1, 3 and 11) and extracting feature data including at least one feature map (See ¶0113-0115 and 0126-0127);
a feature encoder comprising a temporal predictor, a transformer, and a quantizer (See ¶0085-0086, 0204, 0208, 0212 and 0222), the feature encoder receiving the feature signal from the feature extractor and providing an encoded feature signal (See at least 0088-0094 0100-0108 wherein various features are taught; it is noted by the examiner that the “feature signal” is unclear and is not well-defined, as “features” can comprise of just about anything within the encoding process as per currently claimed and will thus be interpreted as such within prior art references); and
an MPEG encoder coupled to the output of the feature encoder and generating an MPEG coded bitstream (See ¶0004 in view of FIG. 1, 3 and 11), the MPEG bitstream including signaling information including a feature parameter set and compressed feature data (Given the broadest reasonable interpretation consistent with applicant’s specification, the feature parameter set may be taught as at least machine-learning model weightings, coefficients, or supplemental data that describes parameters of the machine model as described in ¶0056-0057 of applicant’s specification, see ¶0108 of Kara wherein any machine-learning-based network, neural network [including convolutional neural network] with any pre-training creates parameters representing such training test data, with further examples of ¶0113-0119, 0226 and 0229 showing example parameters that are used, including those parameters which are included in the header and used in the base layer data).
In regards to claim 22, Kara teaches the video encoder of claim 21 wherein the feature extractor is a convolutional neural network (See ¶0108, 0112 and 0117).
In regards to claim 23, Kara fails to teach the video encoder of claim 1 wherein the feature extractor is a partial convolutional neural network (See ¶0113-0117, 0119 and 0126).
In regards to claim 24, Kara teaches the encoder of claim 21 wherein the feature extractor uses a machine learning model to extract the features and the MPEG bitstream stream includes information about the model (See ¶0100 and 0113-0115 in view of 0007, 0090, 0108-0110).
In regards to claim 25, Kara teaches the encoder of claim 21 wherein the MPEG encoder is an AVC encoder, and the bitstream is an AVC compliant bit stream (See ¶0004-0005 and 0088).
In regards to claim 26, Kara teaches the encoder of claim 21, wherein the feature encoder forms a sequence of frames with each frame comprising multiple rectangular feature map patches (See ¶0119-0121).
In regards to claim 29, Kara teaches the encoder of claim 21 wherein the MPEG encoder is a VVC encoder and the bitstream is a VVC compliant bit stream (See ¶0004 and 0199).
In regards to claim 30, Kara teaches the decoder of claim 1, wherein the bitstream comprises a sequence of frames with at least one frame comprising multiple rectangular feature map patches (See ¶0006, 0084, 0086 and 0089, also see FIG. 9 and 10).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karabutov et al. (“Kara”) (U.S. PG Publication No. 2023/0065862) in view of Hendry et al. (“Hendry”) (U.S. PG Publication No. 2020/0252634).
In regards to claim 5, Kara fails to teach the decoder of claim 4, wherein the bitstream includes a plurality of residual layers and the number of residual visual layers is signaled within the at least a header.
In a similar endeavor Hendry teaches wherein the bitstream includes a plurality of residual layers and the number of residual visual layers is signaled within the at least a header (See ¶0113 and 0146-0147 wherein, for example, sps_max_sub_layers_minus1 is a parameter within the sps header that may signal a number of layers).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Hendry into Kara because it allows for the for the parsing of header information, as well as parameters further specifying how the encoding stream should be structured such as is described in ¶0146.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karabutov et al. (“Kara”) (U.S. PG Publication No. 2023/0065862) in view of Hendry et al. (“Hendry”) (U.S. PG Publication No. 2020/0252634) and Meardi et al. (“Meardi”) (WO 2020/188273).
In regards to claim 6, Kara fails to teach the decoder of claim 5, wherein the circuitry is further configured to combine the at least a decoded base feature layer with the first residual visual layer; and combine the at least a combined decoded base feature and first residual visual layer with the second residual visual layer.
In a similar endeavor Meardi teaches wherein the circuitry is further configured to combine the at least a decoded base feature layer with the first residual visual layer (See FIG. 2 with regards to 220, FIG. 5A-5C and 26 as examples); and
combine the at least a combined decoded base feature and first residual visual layer with the second residual visual layer (See FIG. 2 with regards to 220, FIG. 5A-5C and 26 as examples).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Meardi into Kara because it allows for the necessary inverse-processing step of the overall decoding process, which includes parameter data from the header as described by Meardi, thus allowing for proper decoding of data.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karabutov et al. (“Kara”) (U.S. PG Publication No. 2023/0065862) in view of Meardi et al. (“Meardi”) (WO 2020/188273).
In regards to claim 9, Kara fails to explicitly teach the decoder of claim 1, wherein the circuitry is further configured to inversely pre-process the at least a decoded base feature layer (See col. 270, li. 21 – col. 271, li. 4).
In a similar endeavor Meardi teaches wherein the circuitry is further configured to inversely pre-process the at least a decoded base feature layer (See col. 270, li. 21 – col. 271, li. 4).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Meardi into Kara because it allows for the necessary inverse-processing step of the overall decoding process, which includes parameter data from the header as described by Meardi, thus allowing for proper decoding of data.
Claim(s) 27, 28, 31 and 32 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karabutov et al. (“Kara”) (U.S. PG Publication No. 2023/0065862)in view of Kim et al. (“Kim”) (U.S. PG Publication No. 2020/0389658).
In regards to claim 27, Kara fails to teach the encoder of claim 26, wherein the multiple rectangular feature map patches include patches of different sizes.
In a similar endeavor Kim teaches wherein the multiple rectangular feature map patches include patches of different sizes (See FIG. 10, 16, 20 and 22).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Kim into Kara because it allows for a variety of shapes to make up an image as seen in FIG. 22, thus allowing for adaptability by the encoding system based on the amount of detail in specific areas of an image with regards to previously coded image, thus improving overall efficiency.
In regards to claim 28, Kara fails to teach the encoder of claim 27, wherein the multiple rectangular feature map patches comprise substantially all of a width and height of the frame.
In a similar endeavor Kim teaches wherein the multiple rectangular feature map patches comprise substantially all of a width and height of the frame (See FIG. 10, 16 and 20 in view of 22).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Kim into Kara because it allows for a variety of shapes to make up an image as seen in FIG. 22, thus allowing for adaptability by the encoding system based on the amount of detail in specific areas of an image with regards to previously coded image, thus improving overall efficiency.
In regards to claim 31, Kara fails to teach the decoder of claim 30, wherein the multiple rectangular feature map patches include patches of different sizes.
In a similar endeavor Kim teaches wherein the multiple rectangular feature map patches include patches of different sizes (See FIG. 10, 16 and 20 in view of 22).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Kim into Kara because it allows for a variety of shapes to make up an image as seen in FIG. 22, thus allowing for adaptability by the encoding system based on the amount of detail in specific areas of an image with regards to previously coded image, thus improving overall efficiency.
In regards to claim 32, Kara fails to teach the decoder of claim 31, wherein the multiple rectangular feature map patches comprises substantially all of a width and height of the frame.
In a similar endeavor Kim teaches wherein the multiple rectangular feature map patches comprises substantially all of a width and height of the frame (See FIG. 10, 16 and 20 in view of 22).
It would have been obvious to a person of ordinary skill in the art, and before the effective filing date of the claimed invention, to incorporate the teaching of Kim into Kara because it allows for a variety of shapes to make up an image as seen in FIG. 22, thus allowing for adaptability by the encoding system based on the amount of detail in specific areas of an image with regards to previously coded image, thus improving overall efficiency.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EDEMIO NAVAS JR whose telephone number is (571)270-1067. The examiner can normally be reached M-F, ~ 9 AM -6 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Ustaris can be reached at 5712727383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
EDEMIO NAVAS JR
Primary Examiner
Art Unit 2483
/EDEMIO NAVAS JR/Primary Examiner, Art Unit 2483