DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) were submitted July 24, 2024, February 12, 2025, and October 23, 2025. The submissions comply with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claims 2, 11, and 12 objected to because of the following informalities: The claims include “9ii) a proxy.” For compact prosecution, the “9ii) a proxy” is treated as reciting “ii) a proxy.” Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1- 6, 11, 13 and 14 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Said US 11825090 B1 (hereinafter referred to as “Said”).
Regarding claim 1 (Original), Said discloses a hybrid codec for training a neural coding system, the hybrid codec (“FIG. 6A is a block diagram 600 illustrating an example implementation of a video codec neural boosting system.” Said, Col. 3:25-27.) comprising:
a real codec for encoding data pre-processed by a first neural network (“Differentiable codec proxy 652.” “ML pre-processing engine 656.” “ML post-processing engine 668.”. Said, Fig. 6B.), and for decoding the encoded data for post-processing by a second neural network (“ML post-processing engine 668”. Said, Fig. 6B.); and
a proxy codec that is a differentiable representation of the real codec (“Differentiable codec proxy 652.” Said, fig. 6B), and is for back-propagating a loss function (“The loss measurement engine 668 may compare the output from the ML post-processing engine 668 and estimated bit rate 670 to a ground truth reference to compute a loss function and gradient 672. This gradient 672 may be back propagated to the ML pre-processing engine 656, the differentiable codec proxy 652, and ML post-processing engine 668 for training.” Said, Col. 25:4-30.), representative of a coding bit-rate of the real codec and a coding distortion between the post-processed data and the original data, for training the weights of at least the first neural network (“[T]he differentiable codec proxy 652 depends on the accuracy of the proxy estimations, which are defined by two factors…bit rate R [and] distortion D” Said, Col. 25:31-52; Equation 1. “[T]he output gradient of Equation 12 may be determined and used for back propagation as a part of a loss function, for example by a loss measurement system, such as loss measurement engine 668 of FIG. 6, for training a pre-processor, such as ML pre-processing engine 656” Said, Col. 29:40-47.).
Regarding claim 2 (Previously Presented), Said discloses a neural coding system for streaming (“FIG. 6A is a block diagram 600 illustrating an example implementation of a video codec neural boosting system.” Said, Col. 3:25-27; Fig. 3.), the neural coding system comprising:
a pre-processing unit configured to pre-process streaming data to obtain pre-processed data using a first neural network trained to pre-process streaming data (“ML pre-processing engine 656”. Said, Fig. 6B.);
a hybrid codec for training the neural coding system, the hybrid codec (Said, Fig. 6B) comprising: (i) a real codec for encoding data pre-processed by a first neural network (“ML pre-processing engine 656”. Said, Fig. 6B.), and for decoding the encoded data for post-processing by a second neural network (“ML post-processing engine 668”. Said, Fig. 6B.); and
9ii) a proxy codec that is a differentiable representation of the real codec (“Differentiable codec proxy 652.” Said, fig. 6B), and is for back-propagating a loss function (“The loss measurement engine 668 may compare the output from the ML post-processing engine 668 and estimated bit rate 670 to a ground truth reference to compute a loss function and gradient 672. This gradient 672 may be back propagated to the ML pre-processing engine 656, the differentiable codec proxy 652, and ML post-processing engine 668 for training.” Said, Col. 25:4-30.), and is for back-propagating a loss function, representative of a coding bit-rate of the real codec and a coding distortion between the post-processed data and the original data, for training the weights of at least the first neural network (“[T]he differentiable codec proxy 652 depends on the accuracy of the proxy estimations, which are defined by two factors…bit rate R [and] distortion D” Said, Col. 25:31-52; Equation 1. “[T]he output gradient of Equation 12 may be determined and used for back propagation as a part of a loss function, for example by a loss measurement system, such as loss measurement engine 668 of FIG. 6, for training a pre-processor, such as ML pre-processing engine 656” Said, Col. 29:40-47.);
an encoding unit configured to encode, using the real codec, the pre-processed data (Said, Figs. 3 and 6A and 6B);
a decoding unit configured to decode, using the real codec, the encoded pre-processed data (Said, Figs. 3 and 6A and 6B); and
a post-processing unit configured to post-process the decoded pre-processed data to obtain a reconstruction of the streaming data using a second neural network trained to post-process the decoded pre-processed data (“ML pre-processing engine 656.” Said, Figs. 6A and 6B).
Regarding claim 3 (Original), Said discloses the neural coding system according to claim 2, comprising: a loss calculation unit configured to calculate a loss function representative of a coding bit- rate for the streaming data and a coding distortion between the streaming data and the reconstruction of the streaming data (“The loss measurement engine 668 may compare the output from the ML post-processing engine 668 and estimated bit rate 670 to a ground truth reference to compute a loss function and gradient 672. This gradient 672 may be back propagated to the ML pre-processing engine 656, the differentiable codec proxy 652, and ML post-processing engine 668 for training.” Said, Col. 25:4-30); and a backpropagation unit configured to further train the weights of the first neural network by back-propagating the loss function using the proxy codec that is a differentiable representation of the real codec (“To adjust the weights, a learning algorithm may compute a gradient vector for the weights. The weights may then be adjusted to reduce the error. This manner of adjusting the weights may be referred to as “back propagation” as it involves a “backward pass” through the neural network.” Said, Col. 21:63- 22:8. “[T]he output gradient of Equation 12 may be determined and used for back propagation as a part of a loss function, for example by a loss measurement system, such as loss measurement engine 668 of FIG. 6, for training a pre-processor, such as ML pre-processing engine 656” Said, Col. 29:40-47).
Regarding claim 4 (Original), Said discloses the neural coding system according to claim 3, wherein the backpropagation unit is configured to further train the weights of the second neural network by back-propagating the loss function via the second neural network prior to the backpropagation of the loss function using the proxy codec (“To adjust the weights, a learning algorithm may compute a gradient vector for the weights. The weights may then be adjusted to reduce the error. This manner of adjusting the weights may be referred to as “back propagation” as it involves a “backward pass” through the neural network.” Said, Col. 21:63- 22:8. “In some cases, the ML post-processing engine 668 may be a ML post-processing engine 618 that is being trained. Output from the ML post-processing engine 668 may be passed to the loss measurement engine 668.” Said, Col. 25:20-30).
Regarding claim 5 (Original), Said discloses the neural coding system according to claim 2, comprising: a training repository unit configured to store the streaming data for further training the first neural network and the second neural network ( [T]he encoding device 204 may store encoded video bitstream data in storage 208…the decoding device 212 can access stored video data from the storage device [216].” Said, Col. 14:10, 14:41-44. “[T]he ML pre-processing engine 656 may be a ML pre-processing engine 606 that is being trained….the ML post-processing engine 668 may be a ML post-processing engine 618 that is being trained.” Said, 25:10-30.).
Regarding claim 6 (Original), Said discloses the neural coding system according to claim 5, wherein the training repository unit is configured to store the reconstruction of the streaming data in association with the streaming data for further training the first neural network and the second neural network ( [T]he encoding device 204 may store encoded video bitstream data in storage 208…the decoding device 212 can access stored video data from the storage device [216].” Said, Col. 14:10, 14:41-44. “[T]he ML pre-processing engine 656 may be a ML pre-processing engine 606 that is being trained…. the ML post-processing engine 668 may be a ML post-processing engine 618 that is being trained.” Said, 25:10-30.).
Regarding claim 11 (Previously Presented), claim 2 is substantially similar to claim 11. Therefore, claim 11 is rejected for the same reasons as claim 2. (“Processes and methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer-readable media.” Said, Col. 37:16-19.).
Regarding claim 13 (Previously Presented), claim 1 is substantially similar to claim 13. Therefore, claim 13 is rejected for the same reasons as claim 1. (“Processes and methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer-readable media.” Said, Col. 37:16-19. “A computer-readable medium may include a non-transitory medium.” Said, Col. 37:32-35).
Regarding claim 14 (Currently Amended), claim 2 is substantially similar to claim 14. Therefore, claim 14 is rejected for the same reasons as claim 2. (“Processes and methods according to the above-described examples can be implemented using computer-executable instructions that are stored or otherwise available from computer-readable media.” Said, Col. 37:16-19. “A computer-readable medium may include a non-transitory medium.” Said, Col. 37:32-35).
15. (Cancelled)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
1. Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Said in further view of Guleryuz et al., US 20250045968 A1 (hereinafter referred to as “Guleryuz”).
Regarding claim 10 (Original), Said does not explicitly disclose the neural coding system according to claim 2, wherein a portion of the streaming data is in a format not supported by the real codec; the first neural network is trained to convert, during the pre-processing, the portion of the streaming data into latent data that is in a format that is supported by the real codec; and the second neural network is trained to convert, during the post-processing, the decoded latent data, comprised by the decoded pre-processed data, into a reconstruction of the portion of the streaming data that is in the format not supported by the real codec.
In the same field of endeavor, related to codecs and methods for neural coding (Guleryuz, Fig. 7), Guleryuz discloses the claimed feature.
Guleryuz discloses wherein a portion of the streaming data is in a format not supported by the real codec; the first neural network is trained to convert, during the pre-processing, the portion of the streaming data into latent data that is in a format that is supported by the real codec (“[A]s prior to preprocessing, the input image data may express color values using a non-linear primary color space, such as R′G′B′, and preprocessing the input image may include image format conversion of the input image data to the operative image format of the codec.” Guleryuz, ¶ [0103]); and the second neural network is trained to convert, during the post-processing, the decoded latent data, comprised by the decoded pre-processed data, into a reconstruction of the portion of the streaming data that is in the format not supported by the real codec (“Postprocessing image format conversion from the operative image format of the codec to the output, or display, image format.” ¶ [0135]).
It would have been obvious to one with ordinary skill in the art at the time of the invention was filed to modify Said with converting the format of streaming data, as taught by Guleryuz, in order to improve coding efficiency.
Claims 9 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Said in further view of Doshi et al., US 20160330460 A1 (hereinafter referred to as “Doshi”).
Regarding claim 9 (Original), Said does not disclosed the neural coding system according to claim 2, comprising: a splitting unit configured to split the streaming data, prior to the streaming data being pre- processed by the pre-processing unit, into a plurality of streaming data content components in dependence upon the content in the streaming data, wherein a respective streaming data content component is streamed by the neural coding system independently of the other streaming data content components to obtain a respective reconstruction of the respective streaming data content component; and a composing unit configured to combine the plurality of reconstructions of respective streaming data content components for output by the neural coding system.
However, Doshi discloses a splitting unit configured to split the streaming data, prior to the streaming data being pre- processed by the pre-processing unit, into a plurality of streaming data content components in dependence upon the content in the streaming data, wherein a respective streaming data content component is streamed by the neural coding system independently of the other streaming data content components to obtain a respective reconstruction of the respective streaming data content component; and a composing unit configured to combine the plurality of reconstructions of respective streaming data content components for output by the neural coding system (“The picture parallel decoder 110 may analyze VCL NAL and NVCL data separately in order to efficiently parallelize the linearly encoded bitstream.” Doshi, ¶ [0017]).
It would have been obvious to one with ordinary skill in the art at the time of the invention was filed to modify Said with by a splitting unit configured to split the streaming data, prior to the streaming data being pre- processed by the pre-processing unit from Doshi in order to improve video encoding efficiency by providing different video components to separate modules.
Regarding claim 12 (Previously Presented), Said discloses a method, comprising: streaming using neural coding via a hybrid codec for training a neural coding system (“FIG. 6A is a block diagram 600 illustrating an example implementation of a video codec neural boosting system.” Said, Col. 3:25-27.), the hybrid codec comprising:
(i) a real codec for encoding data pre-processed by a first neural network, and for decoding the encoded data for post-processing by a second neural network (“Differentiable codec proxy 652.” “ML pre-processing engine 656.” “ML post-processing engine 668.”. Said, Fig. 6B.); and
9ii) a proxy codec that is a differentiable representation of the real codec (“Differentiable codec proxy 652.” Said, Fig. 6B.) and is for back-propagating a loss function (“The loss measurement engine 668 may compare the output from the ML post-processing engine 668 and estimated bit rate 670 to a ground truth reference to compute a loss function and gradient 672. This gradient 672 may be back propagated to the ML pre-processing engine 656, the differentiable codec proxy 652, and ML post-processing engine 668 for training.” Said, Col. 25:4-30.), representative of a coding bit-rate of the real codec and a coding distortion between the post-processed data and the original data, for training the weights of at least the first neural network (“[T]he differentiable codec proxy 652 depends on the accuracy of the proxy estimations, which are defined by two factors…bit rate R [and] distortion D” Said, Col. 25:31-52; Equation 1. “[T]he output gradient of Equation 12 may be determined and used for back propagation as a part of a loss function, for example by a loss measurement system, such as loss measurement engine 668 of FIG. 6, for training a pre-processor, such as ML pre-processing engine 656” Said, Col. 29:40-47.);
pre-processing streaming data to obtain pre-processed data using a first neural network trained to pre-process the streaming data (“ML pre-processing engine 656.” Said, Figs. 3, 6A and 6B.);
encoding, using the real codec, the pre-processed data (“ML pre-processing engine 656.” Said, Figs. 3, 6A and 6B);
decoding, using the real codec, the encoded pre-processed data (“ML post-processing engine 668.”. Said, Figs. 3, 6A and 6B.);
post-processing the decoded pre-processed data to obtain a reconstruction of the streaming data using a second neural network trained to post-process the decoded pre-processed data (“ML post-processing engine 668.”. Said, Figs. 3, 6A and 6B.);
training the first neural network by:
(i) calculating a loss function representative of a coding bit-rate for streaming data coded, and a coding distortion between the streaming data and the reconstruction of the streaming data obtained (“Loss Measurement Engine 668.” Fig. 6B. “[T]he differentiable codec proxy 652 depends on the accuracy of the proxy estimations, which are defined by two factors…bit rate R [and] distortion D” Said, Col. 25:31-52; Equation 1. “[T]he output gradient of Equation 12 may be determined and used for back propagation as a part of a loss function, for example by a loss measurement system, such as loss measurement engine 668 of FIG. 6, for training a pre-processor, such as ML pre-processing engine 656” Said, Col. 29:40-47.)
further training the weights of the first neural network by back-propagating the loss function using the proxy codec (“Loss Measurement Engine 668.” Fig. 6B. “[T]he differentiable codec proxy 652 depends on the accuracy of the proxy estimations, which are defined by two factors…bit rate R [and] distortion D” Said, Col. 25:31-52; Equation 1. “[T]he output gradient of Equation 12 may be determined and used for back propagation as a part of a loss function, for example by a loss measurement system, such as loss measurement engine 668 of FIG. 6, for training a pre-processor, such as ML pre-processing engine 656” Said, Col. 29:40-47.).
Said does not disclose (i) a splitting unit configured to split the streaming data, prior to the streaming data being pre-processed by the pre-processing unit, into a plurality of streaming data content components in dependence upon the content in the streaming data, wherein a respective streaming data content component is streamed by the neural coding system independently of the other streaming data content components to obtain a respective reconstruction of the respective streaming data content component; and (ii) a composing unit configured to combine the plurality of reconstructions of respective streaming data content components for output by the neural coding system.
However, Doshi discloses, (i) a splitting unit configured to split the streaming data, prior to the streaming data being pre-processed by the pre-processing unit, into a plurality of streaming data content components in dependence upon the content in the streaming data, wherein a respective streaming data content component is streamed by the neural coding system independently of the other streaming data content components to obtain a respective reconstruction of the respective streaming data content component; and (ii) a composing unit configured to combine the plurality of reconstructions of respective streaming data content components for output by the neural coding system (“The picture parallel decoder 110 may analyze VCL NAL and NVCL data separately in order to efficiently parallelize the linearly encoded bitstream.” Doshi, ¶ [0017]).
It would have been obvious to one with ordinary skill in the art at the time of the invention was filed to modify Said with by a splitting unit configured to split the streaming data, prior to the streaming data being pre- processed by the pre-processing unit from Doshi in order to improve video encoding efficiency by providing different video components to separate modules.
Allowable Subject Matter
Claims 7 and 8 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEIRDRE L BEASLEY whose telephone number is (571)270-0452. The examiner can normally be reached Monday-Friday 8 a.m. -5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at (571) 272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DLB/Patent Examiner, Art Unit 2482
/CHRISTOPHER S KELLEY/Supervisory Patent Examiner, Art Unit 2482
Prosecution Insights