DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to pending claims 103-122 filed 10/13/2023.
Claim Objections
The following claim(s) are objected to for formality issues:
In claims 107, 117, “URI” is typically understood to be an abbreviation of “uniform resource identifier”, see Specs p.20 line 2. Hence, a reader may be confused as to whether some different convention is intended in specifying “universal resource indicator”. Hence, correction to “uniform resource identifier” is needed.
Appropriate correction(s) are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 103-113 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The 35 U.S.C. 101 subject matter eligibility analysis first asks whether the claim is directed to one of the four statutory categories (Step 1). It next asks whether the claim is directed to an abstract idea (Step 2A), via Prong 1, whether an abstract idea (e.g., mathematical concept, mental process, certain methods of organizing human activity) is recited, and Prong 2, whether it is integrated into a practical application. It finally asks whether the claim as a whole includes additional elements that amount to significantly more than the judicial exception (Step 2B). See MPEP 2106.
STEP 1: The claims falls within one of the four statutory categories:
All claims are directed to processing apparatuses and methods and hence fall within one of the four statutory categories.
STEP 2A PRONG 1: The claims recite a judicial exception:
Claim 103 is directed to finetuning or training a neural network based on a temporal persistence scope. Under BRI, this is merely the comparison and updating of weights of a neural network to one of an earlier time. For example, a neural network’s weights may be compared to and rolled back to earlier times. This is a mental process, as comparing and updating weights can be practically performed in the mind.
Claim 104 limits the temporal scopes that may be used. However, the limitation of the time range of the earlier neural network does not preclude performance of the technique in the mind.
Claim 105-106 limits the temporal scope range and the nature of the neural network being fine-tuned. However, the limitation of the time range of the earlier neural network or the state of the current network does not preclude performance of the technique in the mind.
Claim 107-112 limits the data on which the fine tuning operates, including encoding neural network elements via a URI, high-level syntax, unique identifier, or flag. However, associating various elements with a URI, high-level syntax, identifier, or flag may be performed in the mind.
Claim 113 is directed to receiving a weight error or difference, predicting a weight update, and combining the difference with the prediction to generate a final weight update. The weight update error may be received from an encoder-side, i.e., an apparatus for encoding objects with the neural network. However, all these steps may be performed in the mind.
STEP 2A PRONG 2: The claims do not integrate the exception into a practical application:
For claim 103, the additional elements include computer hardware elements (processor, memory, et.). However, this is mere instructions to implement the mental process on a general purpose computer and hence do not comprise an integration into a practical application.
Further, the additional elements include encoding media elements based on the neural network. However, this amounts to no more than generally linking the judicial exception to a particular field of use and does not impose meaningful limitations on the abstract idea. Hence, it does not constitute an integration into a practical application.
For claim 113, the additional elements include computer hardware elements (processor, memory, et.). However, this is mere instructions to implement the mental process on a general purpose computer and hence do not comprise an integration into a practical application.
STEP 2B: The claim as a whole do not include additional elements that amount to significantly more than the abstract idea:
For claim 103, the additional elements include computer hardware elements (processor, memory, et.). However, the use of computers for network tunning is well-understood, routine, and conventional in the field of machine learning and hence does not constitute significantly more.
Further, the additional elements include encoding media elements based on the neural network. However, the use of neural networks to encode media is well-understood, routine, and conventional in the field of machine learning and hence does not constitute significantly more.
For claim 113, the additional elements include computer hardware elements (processor, memory, et.). However, the use of computers for network tunning is well-understood, routine, and conventional in the field of machine learning and hence does not constitute significantly more.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 103-106, 109-116, 119-122 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Cricri-wo (WO 2020008104 A1).
For claim 103, Cricri discloses: an apparatus comprising:
at least one processor (fig.1, p.5¶1-2); and
at least one non-transitory memory including computer program code (ibid);
wherein the at least one non-transitory memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform (ibid):
train or finetune at least one neural network (NN) based at least on a temporal persistence scope (p.10-11¶2 gives overview of the algorithm, in particular, an encoder network and a decoder network, such as over a transmission channel (see p.11¶2) is progressively updated or overfitted to local data, such as image block data, this being achieved by the encoder-decoder being adjusted or overfitted at the transmitter end and the decoder weight differences being transmitted; see p.15¶2-p.17¶2 providing details of decoder overfitting or weight adjustments, in particular, p.16:5-15 disclosing the transmitting of decoder weight residuals, p.16¶4-5 disclosing transmission of residuals with respect to baseline network, neighboring frame networks, or any baseline applicable to the encoder overfitting (“first embodiment”); in particular, see p.13¶4 contemplating transmitting weight differences with respect to a combination (e.g., average) of neighboring block networks, p.14¶2 contemplating differences with respect to a “best” block; hence, a prediction of a future decoder weights based on prior blocks, the performance of prior blocks, combinations of prior blocks, spatially or temporally at the encoder side in order to generate weight updates and a prediction function, e.g., neighbor, best, average, space, time, baseline, etc., , hence, encoder and decoder neural networks are tuned based on temporal persistence scope including baseline, prior frame, prior block, etc.); and
encode or decode one or more media elements based at least on the trained or finetuned at least one neural network (figs.3-4 show encoding for transmission of media image data based on finetuned neural networks, see also p.9¶2).
For claim 104, Cricri discloses the apparatus of claim 103, as described above. Cricri further discloses: wherein the temporal persistence scope comprises one or more of following:
any test video, and wherein the at least one NN is used to encode or decode the any test video;
a first set of videos, and wherein the at least one NN is used to encode or decode a video in the first set of videos (p.10¶2: baseline network trained on a large corpus of data, with p.4:25-35 contemplating video data);
a first video, and wherein the at least one NN is used to encode or decode any frame or any patch of the first video;
one or more sets of consecutive video frames from a second video, and wherein the at least one NN is used to encode or decode any frame or any patch in the one or more sets of consecutive video frames from the second video (p.14¶3: current and prior frame);
one or more video frames from a third video, and wherein, the at least one NN is used to encode or decode any patch in the one or more video frames from the third video (p.14¶3); or
one or more patches from one or more video frames, and wherein the at least one NN is used to encode or decode the one or more patches from a video frame of the one or more video frames from a fourth video (p.14¶2-3).
For claim 105, Cricri discloses the apparatus of claim 104, as described above. Cricri further discloses: wherein when the temporal persistence scope comprises the first video, the at least one NN is trained based on a base NN by using content from the first video as training data (p.14¶2-3: temporal scope comprises prior frames of the first video, hence, comprising first video, the NN being trained on prior frame data).
For claim 106, Cricri discloses the apparatus of claim 105, as described above. Cricri further discloses: wherein the base NN comprises one of the following:
a randomly initialized NN;
an NN pretrained on a training dataset (p.10 ¶2: pre-trained on prior frame dataset); or
an NN pretrained or finetuned on a second set of videos comprising the first video (p.14¶2-3: second set of videos including first video and respective frames).
For claim 109, Cricri discloses the apparatus of claim 103, as described above. Cricri further discloses: wherein the apparatus is further caused to: signal a unique identifier for each NN (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc.).
For claim 110 Cricri discloses the apparatus of claim 103, as described above. Cricri further discloses: wherein the apparatus is further caused to signal a flag to indicate whether a NN comprises a base NN (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, signaling via the unique identifier identifying a base NN to update, the baseline potentially being a baseline NN (p.16:27), hence, the passing of the baseline neural network being a flag indicating a reference NN comprises a base NN).
For claim 111 Cricri discloses the apparatus of claim 103, as described above. Cricri further discloses: wherein the apparatus is further caused to associate the each of the one or more media elements an identifier of an associated NN (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, associating current media element block with a reference NN identifier).
For claim 112 Cricri discloses the apparatus of claim 111, as described above. Cricri further discloses: wherein the identifier comprises ref_nn_id, wherein the ref_nn_id comprises one of predetermined values of an nn_id (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, selected reference decoder constitutes a ref_nn_id identifier, the identifier being one of the various stored and predetermined nn_id’s).
For claim 113, Cricri discloses: an apparatus comprising:
at least one processor (fig.1, p.5¶1-2); and
at least one non-transitory memory including computer program code (ibid);
wherein the at least one non-transitory memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform (ibid):
receive a weight-update prediction error from an encoder-side (p.10-11¶2 gives overview of the algorithm, in particular, an encoder network and a decoder network, such as over a transmission channel (see p.11¶2) is progressively updated or overfitted to local data, such as image block data, this being achieved by the encoder-decoder being adjusted or overfitted at the transmitter end and the decoder weight differences being transmitted; see p.15¶2-p.17¶2 providing details of decoder overfitting or weight adjustments, in particular, p.16:5-15 disclosing the transmitting of decoder weight residuals, p.16¶4-5 disclosing transmission of residuals with respect to baseline network, neighboring frame networks, or any baseline applicable to the encoder overfitting (“first embodiment”); in particular, see p.13¶4 contemplating transmitting weight differences with respect to a combination (e.g., average) of neighboring block networks, p.14¶2 contemplating differences with respect to a “best” block; hence, a prediction of a future decoder weights based on prior blocks, the performance of prior blocks, combinations of prior blocks, spatially or temporally at the encoder side in order to generate weight updates and a prediction function, e.g., neighbor, best, average, space, time, baseline, etc., , hence, the decoder receives a weight update prediction error with respect to a predicted weight);
and predict a weight-update based on one or more reference weight updates, and a prediction function or algorithm (ibid: weight update is predicted based on best, average, prior frame, etc.);
reconstruct a weight update by combining the predicted weight-update and a prediction error (fig.7, p.17¶5).
For claim 114, Cricri discloses: an apparatus comprising:
at least one processor (fig.1, p.5¶1-2); and
at least one non-transitory memory including computer program code (ibid);
wherein the at least one non-transitory memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to (ibid):
perform a prediction process, on an encoder-side, to generate a predicted weight-update based on one or more reference weight updates and a prediction function or algorithm (p.10-11¶2 gives overview of the algorithm, in particular, an encoder network and a decoder network, such as over a transmission channel (see p.11¶2) is progressively updated or overfitted to local data, such as image block data, this being achieved by the encoder-decoder being adjusted or overfitted at the transmitter end and the decoder weight differences being transmitted; see p.15¶2-p.17¶2 providing details of decoder overfitting or weight adjustments, in particular, p.16:5-15 disclosing the transmitting of decoder weight residuals, p.16¶4-5 disclosing transmission of residuals with respect to baseline network, neighboring frame networks, or any baseline applicable to the encoder overfitting (“first embodiment”); in particular, see p.13¶4 contemplating transmitting weight differences with respect to a combination (e.g., average) of neighboring block networks, p.14¶2 contemplating differences with respect to a “best” block; hence, a prediction of a future decoder weights based on prior blocks, the performance of prior blocks, combinations of prior blocks, spatially or temporally at the encoder side in order to generate weight updates and a prediction function, e.g., neighbor, best, average, space, time, baseline, etc.));
generate a weight-update prediction error based on a weight-update and on the predicted weight-update (ibid, in particular p.16:5-15: a weight update difference or error from prediction is generated);
encode the weight-update prediction error (ibid: the weigh-update prediction error is encoded for transmission to the decoder side);
provide the encoded weight-update prediction error to a decoder-side (ibid); and wherein the decoders-side decodes the encoded weight-update prediction error, predicts the weight-update based on the one or more reference weight updates and the prediction function or algorithm, and reconstructs a weight update by combining the predicted weight-update and the decoded weight-update prediction error (ibid, particularly p.16:5-15, ¶4-5: the decoder, in order to reconstruct its weights, would need to access the correct prediction function, apply the prediction, apply the prediction error to the prediction function, as claimed, see also fig.7, p.17¶5).
For claim 115, Cricri discloses: a method comprising:
training or finetuning at least one neural network (NN) based at least on a temporal persistence scope ((p.10-11¶2 gives overview of the algorithm, in particular, an encoder network and a decoder network, such as over a transmission channel (see p.11¶2) is progressively updated or overfitted to local data, such as image block data, this being achieved by the encoder-decoder being adjusted or overfitted at the transmitter end and the decoder weight differences being transmitted; see p.15¶2-p.17¶2 providing details of decoder overfitting or weight adjustments, in particular, p.16:5-15 disclosing the transmitting of decoder weight residuals, p.16¶4-5 disclosing transmission of residuals with respect to baseline network, neighboring frame networks, or any baseline applicable to the encoder overfitting (“first embodiment”); in particular, see p.13¶4 contemplating transmitting weight differences with respect to a combination (e.g., average) of neighboring block networks, p.14¶2 contemplating differences with respect to a “best” block; hence, a prediction of a future decoder weights based on prior blocks, the performance of prior blocks, combinations of prior blocks, spatially or temporally at the encoder side in order to generate weight updates and a prediction function, e.g., neighbor, best, average, space, time, baseline, etc., , hence, encoder and decoder neural networks are tuned based on temporal persistence scope including baseline, prior frame, prior block, etc.); and
encoding or decoding one or more media elements based at least on the trained or finetuned at least one neural network (figs.3-4 show encoding for transmission of media image data based on finetuned neural networks, see also p.9¶2).
For claim 116 Cricri discloses the apparatus of claim 115, as described above. Cricri further discloses: wherein the temporal persistence scope comprises one or more of following:
any test video, and wherein the at least one NN is used to encode or decode the any test video;
a first set of videos, and wherein the at least one NN is used to encode or decode a video in the first set of videos (p.10¶2: baseline network trained on a large corpus of data, with p.4:25-35 contemplating video data);
a first video, and wherein the at least one NN is used to encode or decode any frame or any patch of the first video;
one or more sets of consecutive video frames from a second video, and wherein the at least one NN is used to encode or decode any frame or any patch in the one or more sets of consecutive video frames from the second video;
one or more video frames from a third video, and wherein, the at least one NN is used to encode or decode any patch in the one or more video frames from the third video; or
one or more patches from one or more video frames, and wherein the at least one NN is used to encode or decode the one or more patches from a video frame of the one or more video frames from a fourth video (p.14¶2-3).
For claim 119 Cricri discloses the apparatus of claim 115, as described above. Cricri further discloses: signaling a unique identifier for each NN (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc).
For claim 120 Cricri discloses the apparatus of claim 119, as described above. Cricri further discloses: signaling a flag to indicate whether a NN comprises a base NN (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, signaling via the unique identifier identifying a base NN to update, the baseline potentially being a baseline NN (p.16:27), hence, the passing of the baseline neural network being a flag indicating a reference NN comprises a base NN).
For claim 121 Cricri discloses the apparatus of claim 115, as described above. Cricri further discloses: associating the each of the one or more media elements as an identifier of an associated NN (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, associating current media element block with a reference NN identifier).
For claim 122 Cricri discloses the apparatus of claim 121, as described above. Cricri further discloses: wherein the identifier comprises ref_nn_id, and wherein the ref_nn_id comprises one of predetermined values of an nn_id (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, selected reference decoder constitutes a ref_nn_id identifier, the identifier being one of the various stored and predetermined nn_id’s).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 107, 117 are rejected under 35 U.S.C. 103 as being unpatentable over Cricri-wo (WO 2020008104 A1) in view of rfc3986 ("Uniform Resource Identifier (URI): Generic Syntax", published 2005).
For claim 107, Cricri discloses the apparatus of claim 104, as described above. Cricri further discloses: wherein the apparatus is further caused to:
encode at least one of a topology, weights, or weight-update of at least one NN (p.13¶2) specifying resource indicator from which at least one of the topology or weights of the at least one NN are obtained (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc.).
Cricri does not disclose: wherein the resource indicator is a universal resource indicator (URI).
Rfc3986 discloses: wherein the resource indicator is a universal resource indicator (URI) (p.1:Abstract discloses using a URI to identify resources).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the apparatus of Cricri by incorporating the URI technique of Rfc3986. Both concern the art of computer resource management, and the incorporation would have, according to Rfc3986, allow uniformity, ease of interpretation across different contexts (§1.1 ¶1).
For claim 117 Cricri discloses the apparatus of claim 115, as described above. Cricri further discloses: further comprising encoding at least one of a topology, weights, or weight-update of the at least one NN (p.13¶2) specifying a resource indicator from which at least one of the topology or weights of the at least one NN are obtained (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc.).
Cricri does not disclose: wherein the resource indicator is a universal resource indicator (URI).
Rfc3986 discloses: wherein the resource indicator is a universal resource indicator (URI) (p.1:Abstract discloses using a URI to identify resources).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the apparatus of Cricri by incorporating the URI technique of Rfc3986. Both concern the art of computer resource management, and the incorporation would have, according to Rfc3986, allow uniformity, ease of interpretation across different contexts (§1.1 ¶1).
Claim(s) 108, 118 are rejected under 35 U.S.C. 103 as being unpatentable over Cricri-wo (WO 2020008104 A1) in view of rfc3986 ("Uniform Resource Identifier (URI): Generic Syntax", published 2005) in view of Sjoberg ("Overview of HEVC high-level syntax and reference picture management", published 2012).
For claim 108, Cricri discloses the apparatus of claim 107, as described above. Cricri further discloses: wherein the apparatus is further caused to signal an indication of which base NN to update (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, signaling via the unique identifier identifying a base NN to update).
Cricri does not disclose: wherein the indication comprises a first high-level syntax element.
Sjoberg discloses: wherein the indication comprises a first high-level syntax element (§I¶2: use of high level syntax for interfacing with systems in upcoming standards, §VIII use of high level syntax in supplemental enhancement information).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the apparatus of Cricri by incorporating the high level syntax encoding of Sjoberg . Both concern the art of video encoding and the incorporation would have, according to Sjoberg , allow common handling of messaging (§VIII ¶1), allow conformity with various standards (§I ¶2).
For claim 118 Cricri discloses the apparatus of claim 117, as described above. Cricri further discloses: signaling an indication of which base NN to update (p.16:34: unique ID, such as applied to the various reference decoders contemplated above in p.14¶2, p.13¶4, etc., hence, signaling via the unique identifier identifying a base NN to update).
Cricri does not disclose: wherein the indication comprises a first high-level syntax element.
Sjoberg discloses: wherein the indication comprises a first high-level syntax element (§I¶2: use of high level syntax for interfacing with systems in upcoming standards, §VIII use of high level syntax in supplemental enhancement information).
It would have been obvious before the effective filing date to a person of ordinary skill in the art to modify the apparatus of Cricri by incorporating the high level syntax encoding of Sjoberg . Both concern the art of video encoding and the incorporation would have, according to Sjoberg , allow common handling of messaging (§VIII ¶1), allow conformity with various standards (§I ¶2).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Lee (US 20210125380 A1) discloses adaptive neural video encoding.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LIANG LI whose telephone number is (303)297-4263. The examiner can normally be reached Mon-Fri 9-12p, 3-11p MT (11-2p, 5-1a ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. The examiner is available for interviews Mon-Fri 6-11a, 2-7p MT (8-1p, 4-9p ET).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/LIANG LI/
Primary examiner AU 2143
Prosecution Insights