DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application is a continuation application of PCT RU2021/000496. Priority to RU2021/000496 with a priority date of 11/11/2021 is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 09/16/2024, 09/19/2024, and 02/09/2025 have been considered and placed in the application file.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: "Conditional coding of brightness and color components for image compression".
Claim Objections
Claim 17 is objected to because of the following informalities: Claim 17 currently reads "is one of larger than, smaller than and equal to the size of the second latent tensor in the channel dimension." in reference to size of the first latent tensor. It is unclear as to if the Claim is supposed to read "is one of larger than, smaller than or equal to the size of the second latent tensor in the channel dimension." As it is impossible for the size to be larger than, smaller than and equal to as these would all denote different sizes of the tensor. Appropriate correction is required.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the
plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claim 7 recite “or ” then listing “wherein the size of the first latent tensor in the channel dimension is larger than, smaller than or equal to the size of the second latent tensor in the channel dimension, or wherein the first tensor is transformed into the first latent tensor through a first neural network, and the concatenated tensor is transformed into the second latent tensor through a second neural network different from the first neural network, or wherein the first bitstream is generated based on a first entropy model, and the second bitstream is generated based on a second entropy model different from the first entropy model.” Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim 20 recite “or” then listing “wherein the primary component of the image is a luma component and the at least one secondary component of the image is a chroma component, or wherein the primary component of the image is a chroma component and the at least one secondary component of the image is a luma component.” Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-6, 8, and 24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claims 1, 8, and 24 and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1-6, 8, and 24 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101. The Examiner will analyze Claim 1. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a process since the claim is directed to method.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of to encode (convert information) different parts of an image based on brightness and color is directed to an abstract idea. The claim is related to mental process, the claims recite mental processes that include separating parts of images based on brightness and color. Under Berkheimer v. HP, The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing images, because the steps were recited at a high level of generality and merely used computers as a tool to perform the processes. Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018). If the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The Examiner interprets that Claim 1 limitation does not provide additional elements or combination of additional elements to a practical application since the claims are not adding insignificant extra-solution activity to the judicial exception. The abstract idea is not integrated into a practical application because the additional elements fail to provide a technical improvement. Limiting the encoding to "primary component" or "secondary component" is a non-qualifying field-of-use limitation. Data gathering (encoding an image) and separating the primary and secondary component do not add a practical application. Reciting an “encoding a primary component of the image independently from at least one secondary component of the image” functionally does not reflect a specific technical improvement to the method. Specifically, the analysis method does not integrate a judicial exception into practical application. See Genetic Techs. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016) (eligibility "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself."). For a claim reciting a judicial exception to be eligible, the additional elements (if any) in the claim must "transform the nature of the claim" into a patent-eligible application of the judicial exception, Alice Corp., 573 U.S. at 217, 110 USPQ2d at 1981, either at Prong Two or in Step 2B. If there are no additional elements in the claim, then it cannot be eligible. In such a case, after making the appropriate rejection, it is a best practice for the examiner to recommend an amendment, if possible, that would resolve eligibility of the claim.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claims state analyzing images for well-known characteristics with a high level of generality. The claims lack an inventive concept because the elements, considered individually and as an order combination are an abstract idea, adding the words "apply it" (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, e.g., a limitation indicating that a particular function such as creating and maintaining electronic records is performed by a computer, as discussed in Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984 (see MPEP § 2106.05(f)).
Furthermore, the generic computer components of the method applied to an electronic encoding device, recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
Claims 2-6 depend on the independent claim/s include all the limitation of the independent claim. The Examiner finds that Claim 2 does not state significantly more since the claim adds limitation that encoding is performing concurrently, which is an additional element and under Step 2A prong 2 to be a mere recitation of mental process. It is a recitation of observation and evaluation and can be done as well in a human mind, such as a human can look at an image and interpret the color and brightness at the same time in the image which is further mental process, see 2106.05(g). The Examiner finds that Claim 3 does not state significantly more since the claim adds definition of components of brightness and color which the human mind can easily see, which is an additional element and under Step 2A prong 2, to be mere recitation of a mental process. It is a recitation of observation and evaluation and can be done as well in a human mind, such as a human can tell the difference between color and brightness in the image which is further mental process, see 2106.05(g). The Examiner finds that Claim 4 does not state significantly more since the claim adds definition of components of brightness and color which the human mind can easily see, which is an additional element and under Step 2A prong 2, to be mere recitation of a mental process. It is a recitation of observation and evaluation and can be done as well in a human mind, such as a human can tell the difference between color and brightness in the image which is further mental process, see 2106.05(g). The Examiner finds that Claim 5 does not state significantly more since the claim recites adding a tensor for interpretation of the data, which is an additional element and under Step 2A prong 2, to be mere data gathering. It is a recitation of data gathering and mathematical concept, see 2106.05(g). The Examiner finds that Claim 6 does not state significantly more since the claim recites manipulating data for the surgical plan based on safety margins, which is an additional element and under Step 2A prong 2, to be mere data manipulation see 2106.05 (g). It is also a recitation of observation and evaluation and is well known in field as doctors frequently have weight safety margins before performing surgery, see 2106.05(d).
Thus, Claims 2-6 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more. Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
For the analogous independent claim 8 to the independent claim 1, the analogous limitations can be analyzed in the same way as above for the claim 1 hence rejected under 101. Moreover, the claim of 8 further recites at the same statutory category of “method” which is a limitation that the examiner interprets that the claims is related to a process since the claim is directed to a method to encode (convert information) different parts of an image based on brightness and color, and is consistent with the abstract ideas, the claims recite mental processes that include separating parts of images based on brightness and color, the examiner interprets the addition of the “residual component” as mental processes as it is just an additional component that could be another chroma element such as color, and a human could differentiate between two different colors. Under Berkheimer v. HP, The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing images, because the steps were recited at a high level of generality and merely used computers as a tool to perform the processes. Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018). This limitation of a residual component just further implements the abstract ideas to be performed by generic computer or software/hardware components of additional elements of the different types of analyzation methods. Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
For the analogous independent claim 24 to the independent claim 1 and the analogous limitations can be analyzed in the same way as above for the claim 1 hence rejected under 101. Moreover, the claim of 24 further recites a different statutory category of “apparatus” which is a limitation that the examiner interprets that the claims is related to a device and is consistent with the abstract ideas, the claims recite mental processes that include separating parts of images based on brightness and color. Under Berkheimer v. HP, The Federal Circuit determined that these claims were directed to mental processes of parsing and comparing images, because the steps were recited at a high level of generality and merely used computers as a tool to perform the processes. Berkheimer v. HP, Inc., 881 F.3d 1360, 125 USPQ2d 1649 (Fed. Cir. 2018). This limitation of a residual component just further implements the abstract ideas to be performed by generic computer or software/hardware components of additional elements of the different types of analyzation methods. Therefore, the Examiner interprets that the claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-7, 15 and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. The Examiner strongly suggested that appropriate corrections be made to clarify the claim scope.
With respect to Claim 6, the claim recites the following, each of which renders the claim indefinite:
“ the size” on line 10 (unclear antecedent basis as there is no definition or reference to “size” in claim 1 or in earlier parts of claim 6, however if applicant is intending for the height and width dimension of the tensor to be interpreted as size this needs to be explicitly stated).
With respect to Claim 7, it depends on Claim 6, therefore it is also rejected. Appropriate correction is required.
With respect to Claim 15, the claim recites the following, each of which renders the claim indefinite:
“ the size” on line 10 (unclear antecedent basis as there is no definition or reference to “size” in claim 9 or in earlier parts of claim 15, however if applicant is intending for the height and width dimension of the tensor to be interpreted as size this needs to be explicitly stated).
With respect to Claim 17, the claim recites the following, each of which renders the claim indefinite:
“ the size” on line 3 (unclear antecedent basis as there is no definition or reference to “size” in claim 9 or in earlier parts of claim 17, however if applicant is intending for the dimension of the tensor to be interpreted as size this needs to be explicitly stated).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 8, and 24 are rejected under 35 U.S.C. 102(a)(1) as being clearly anticipated by Zhu (Zhu, Linwei, et al. "Deep learning-based chroma prediction for intra versatile video coding." IEEE Transactions on Circuits and Systems for Video Technology 31.8 (2020): 3168-3181.).
Regarding Claim 1, Zhu discloses a method of encoding (Zhu Pg 2 Col 2 ¶02 and Pg 5, Col 1, ¶05 and Pg 6 Col 1 ¶04 discloses employing an encoder-decoder structure and an encoder side of the neural network) at least a portion of an image (Zhu Pg 1 Col 1 ¶03, and Pg 2 Col 1 ¶01 discloses partitioning the image into blocks), the method applied to an electronic encoding device (Zhu Pg 8 Col1 ¶03 discloses using a Windows 10 Enterprise 64-bit operating system, to run the experiment) and comprising
encoding a primary component of the image (Zhu Fig 7a discloses the chroma encoding) independently from at least one secondary component of the image (Zhu Pg 2 Col 2 ¶01 and Pg 3 Col 2 ¶04 discloses two separate networks one to process luma and one to process chroma, meaning they are processed separately); and
encoding the at least one secondary component of the image (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) using information from the primary component (Zhu Pg 3 Col 2 ¶04 discloses the model that performs the encoding using both luma and chroma pixels).
Regarding Claim 2, Zhu discloses the method according to claim 1, wherein the primary component (Zhu Fig 7a discloses the chroma encoding) and the at least one secondary component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) are encoded concurrently (Zhu Pg 7 Col 1 ¶02, Pg 4 Col 1 ¶04 discloses the networks operating simultaneously).
Regarding Claim 3, Zhu discloses the method according to claim 1, wherein the primary component of the image is a luma component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) and the at least one secondary component of the image is a chroma component (Zhu Fig 7a discloses the chroma encoding).
Regarding Claim 4, Zhu discloses the method according to claim 1, wherein the primary component of the image is a chroma component (Zhu Fig 7a discloses the chroma encoding) and the at least one secondary component of the image is a luma component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding).
Regarding Claim 8, Zhu discloses a method of encoding (Zhu Pg 2 Col 2 ¶02 and Pg 5, Col 1, ¶05 and Pg 6 Col 1 ¶04 discloses employing an encoder-decoder structure and an encoder side of the neural network) at least a portion of an image (Zhu Pg 1 Col 1 ¶03, and Pg 2 Col 1 ¶01 discloses partitioning the image into blocks), the method applied to an electronic encoding device (Zhu Pg 8 Col1 ¶03 discloses using a Windows 10 Enterprise 64-bit operating system, to run the experiment) and comprising:
providing a residual (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal and luma residual signal) comprising a primary residual component for a primary component of the image (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal) and at least one secondary residual component for at least one secondary component (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal and luma residual signal) of the image that is different from the primary component (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal and luma residual signal which are different);
encoding (Zhu Fig 7a discloses the chroma encoding) the primary residual component (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal and luma residual signal) independently from the at least one secondary residual component (Zhu Pg 2 Col 2 ¶01 and Pg 3 Col 2 ¶04 discloses two separate networks one to process luma and one to process chroma, meaning they are processed separately); and
encoding (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) the at least one secondary residual component (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal and luma residual signal) using information from the primary residual component(Zhu Pg 3 Col 2 ¶04 discloses the model that performs the encoding using both luma and chroma pixels).
Regarding Claim 24, Zhu discloses a processing apparatus (Zhu Pg 8 Col1 ¶03 discloses using a Windows 10 Enterprise 64-bit operating system, to run the experiment) for encoding at least a portion of an image (Zhu Pg 2 Col 2 ¶02 and Pg 5, Col 1, ¶05 and Pg 6 Col 1 ¶04 discloses employing an encoder-decoder structure and an encoder side of the neural network Pg 1 Col 1 ¶03, and Pg 2 Col 1 ¶01 discloses partitioning the image into blocks), the processing apparatus comprising a processing circuitry (Zhu Pg 8 Col1 ¶03 discloses using a Windows 10 Enterprise 64-bit operating system including a CPU, to run the experiment) configured for:
encoding a primary component of the image (Zhu Fig 7a discloses the chroma encoding) independently from at least one secondary component of the image (Zhu Pg 2 Col 2 ¶01 and Pg 3 Col 2 ¶04 discloses two separate networks one to process luma and one to process chroma, meaning they are processed separately); and
encoding the at least one secondary component of the image (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) using information from the primary component (Zhu Pg 3 Col 2 ¶04 discloses the model that performs the encoding using both luma and chroma pixels).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 5-7, 9-23, and 25 are rejected under 35 U.S.C. 103 as unpatentable over Zhu (Zhu, Linwei, et al. "Deep learning-based chroma prediction for intra versatile video coding." IEEE Transactions on Circuits and Systems for Video Technology 31.8 (2020): 3168-3181.) in view of Besenbruch et al (WO Patent Publication WO2021220008 A1, hereafter referred to as Besenbruch).
Regarding Claim 5, Zhu teaches the method according to claim 1, wherein the encoding the primary component (Zhu Fig 7a discloses the chroma encoding) further comprises:
representing the primary component by a first tensor (Zhu Fig 5 discloses the chroma predication network arranged into n x n sizes and Pg 7 Col 2 ¶02 discloses TensorFlow being used for network training which implies that the fundamental data structure of the networks are tensors);
wherein the encoding the at least one secondary component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) further comprises:
representing the at least one secondary component by a second tensor different from the first tensor (Zhu Fig 5 discloses the luma network arranged into n x n sizes and Pg 7 Col 2 ¶02 discloses TensorFlow being used for network training which implies that the fundamental data structure of the networks are tensors) ;
concatenating the second tensor and the first tensor to obtain a concatenated tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors).
Zhu does not explicitly disclose transforming the first tensor into a first latent tensor; and
processing the first latent tensor to generate a first bitstream; and
transforming the concatenated tensor into a second latent tensor; and
processing the second latent tensor to generate a second bitstream.
Besenbruch is in the same field of image compression while preserving image quality. Further, Besenbruch teaches transforming the first tensor into a first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) ; and
processing the first latent tensor to generate a first bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream); and
transforming the concatenated tensor into a second latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent); and
processing the second latent tensor to generate a second bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhu by incorporating the transformation of the tensors into latent tensors and bitstreams in both actions of reducing dimensionality and increasing dimensionality as taught by Besenbruch; to make an invention that utilizes the reduced dimensionality to ease the computational burden; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to increase the compression, while preserving displayed image quality, or to increase the displayed image quality, while not increasing the amount of data that is actually transmitted across the communications networks. This would help to reduce the demands on communications networks, compared to the demands that otherwise would be made as disclosed by Besenbruch in ¶0004.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 6, Zhu teaches the method according to claim 1, wherein the encoding the primary component (Zhu Fig 7a discloses the chroma encoding) further comprises:
representing the primary component by a first tensor (Zhu Fig 5 discloses the chroma predication network arranged into n x n sizes and Pg 7 Col 2 ¶02 discloses TensorFlow being used for network training which implies that the fundamental data structure of the networks are tensors) having a height dimension and a width dimension (Zhu Pg 4 Col 2 ¶04 discloses the component being the size of 4N x 4N which the examiner is interpreting as the height and width of the component);
wherein the encoding the at least one secondary component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) further comprises:
representing the at least one secondary component by a second tensor different from the first tensor (Zhu Fig 5 discloses the luma network arranged into n x n sizes and Pg 7 Col 2 ¶02 discloses TensorFlow being used for network training which implies that the fundamental data structure of the networks are tensors) and having a height dimension and a width dimension (Zhu Pg 4 Col 2 ¶04 discloses the component being the size of 4N x 4N which the examiner is interpreting as the height and width of the component);
determining whether the size or a sub-pixel offset of samples of the second tensor in at least one of the height and width dimensions differs from the size or sub-pixel offset of samples in at least one of the height and width dimensions of the first tensor (Zhu Pg 7 Col 2 ¶03 discloses the chroma and luma having different partition and block sizes), and based on a determination that the size or sub-pixel offset of samples of the second tensor differs from the size or sub-pixel offset of samples of the first tensor, adjusting the sample locations of the first tensor to match the sample locations of the second tensor to obtain an adjusted first tensor (Zhu Pg 7 Col 2 ¶03- Pg 8 Col1 ¶01-¶02 discloses motion compensation methods that handle the differences in images of rotation, zooming and shearing when processing both components including shape adaptive transform);
concatenating the second tensor and the adjusted first tensor to obtain a concatenated tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors) only based on a determination that the size or sub-pixel offset of samples of the second tensor differs from the size or sub-pixel offset of samples of the first tensor (Zhu Pg 7 Col 2 ¶03 discloses the chroma and luma having different partition and block sizes), and else concatenating the second tensor and the first tensor to obtain a concatenated tensor(Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors).
Zhu does not explicitly disclose transforming the first tensor into a first latent tensor; and
processing the first latent tensor to generate a first bitstream; and
transforming the concatenated tensor into a second latent tensor; and
processing the second latent tensor to generate a second bitstream.
Besenbruch is in the same field of image compression while preserving image quality. Further, Besenbruch teaches transforming the first tensor into a first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) ; and
processing the first latent tensor to generate a first bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream); and
transforming the concatenated tensor into a second latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent); and
processing the second latent tensor to generate a second bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhu by incorporating the transformation of the tensors into latent tensors and bitstreams in both actions of reducing dimensionality and increasing dimensionality as taught by Besenbruch; to make an invention that utilizes the reduced dimensionality to ease the computational burden; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to increase the compression, while preserving displayed image quality, or to increase the displayed image quality, while not increasing the amount of data that is actually transmitted across the communications networks. This would help to reduce the demands on communications networks, compared to the demands that otherwise would be made as disclosed by Besenbruch in ¶0004.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 7, Zhu in view of Besenbruch teaches the method according to claim 6, wherein the first latent tensor comprises a channel dimension (Besenbruch Pg 103 ¶02 discloses where the tensor to be modelled is split into blocks (in spatial and or channel dimensions) and the second latent tensor comprises a channel dimension (Besenbruch Pg 103 ¶02 discloses where the tensor to be modelled is split into blocks (in spatial and or channel dimensions) and
wherein the size of the first latent tensor in the channel dimension is larger than, smaller than or equal to the size of the second latent tensor in the channel dimension (Besenbruch Pg 103 ¶02 discloses that the channel dimension size can differ and can be adjusted to match the input), or
wherein the first tensor is transformed into the first latent tensor through a first neural network (Besenbruch ¶0104 discloses using a first neural network to produce a latent representation), and the concatenated tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors) is transformed into the second latent tensor through a second neural network different from the first neural network (Besenbruch ¶0105-¶0106 discloses using a second neural network to quantize the latent tensor), or
wherein the first bitstream is generated based on a first entropy model (Besenbruch ¶0119 discloses entropy encoding the quantized latent into a bitstream); and the second bitstream is generated based on a second entropy model different from the first entropy model (Besenbruch ¶0120 discloses entropy decoding the bitstream, wherein decoding is different from encoding indicating a separate model). See Claim 6 for rationale, its parent claim.
Regarding Claim 9, Zhu teaches a method of reconstructing (Zhu Pg 3 Col 1 ¶04 and Pg 4 Col 2 ¶01-¶02 discloses reconstructing the luma and chroma pixels of the image) at least a portion of an image (Zhu Pg 1 Col 1 ¶03, and Pg 2 Col 1 ¶01 discloses partitioning the image into blocks), representing a primary component of the image (Zhu Fig 7a discloses the chroma encoding);
representing at least one secondary component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) of the image using information from the first latent tensor (Zhu Pg 3 Col 2 ¶04 discloses the model that performs the encoding using both luma and chroma pixels).
Zhu does not explicitly disclose processing a first bitstream based on a first entropy model to obtain a first latent tensor; processing the first latent tensor to obtain a first tensor,
processing a second bitstream different from the first bitstream based on a second entropy model different from the first entropy model to obtain a second latent tensor different from the first latent tensor; and
processing the second latent tensor to obtain a second tensor.
Besenbruch is in the same field of image compression while preserving image quality. Further, Besenbruch teaches processing a first bitstream based on a first entropy model to obtain a first latent tensor (Besenbruch ¶0119 discloses entropy encoding the quantized latent into a bitstream); processing the first latent tensor to obtain a first tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent)
processing a second bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream) different from the first bitstream based on a second entropy model different from the first entropy model (Besenbruch ¶0120 discloses entropy decoding the bitstream, wherein decoding is different from encoding indicating a separate model) to obtain a second latent tensor different from the first latent tensor (Besenbruch ¶0105-¶0106 discloses using a second neural network to quantize the latent tensor); and
processing the second latent tensor to obtain a second tensor (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhu by incorporating the transformation of the tensors into latent tensors and bitstreams in both actions of reducing dimensionality and increasing dimensionality as taught by Besenbruch; to make an invention that utilizes the reduced dimensionality to ease the computational burden; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to increase the compression, while preserving displayed image quality, or to increase the displayed image quality, while not increasing the amount of data that is actually transmitted across the communications networks. This would help to reduce the demands on communications networks, compared to the demands that otherwise would be made as disclosed by Besenbruch in ¶0004.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 10, Zhu in view of Besenbruch teaches the method according to claim 9, wherein the first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) is processed independently (Zhu Pg 2 Col 2 ¶01 and Pg 3 Col 2 ¶04 discloses two separate networks one to process luma and one to process chroma, meaning they are processed separately) from the processing of the second latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent). See Claim 9 for rationale, its parent claim.
Regarding Claim 11, Zhu in view of Besenbruch teaches the method according to claim 9, wherein the primary component of the image is a luma component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) and the at least one secondary component of the image is a chroma component (Zhu Fig 7a discloses the chroma encoding). See Claim 9 for rationale, its parent claim.
Regarding Claim 12, Zhu in view of Besenbruch teaches the method according to claim 9, wherein the primary component of the image is a chroma component (Zhu Fig 7a discloses the chroma encoding) and the at least one secondary component of the image is a luma component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding). See Claim 9 for rationale, its parent claim.
Regarding Claim 13, Zhu in view of Besenbruch teaches the method according to claim 11, wherein the second tensor(Zhu Pg 2 Col 2 ¶01 and Pg 3 Col 2 ¶04 discloses two separate networks one to process luma and one to process chroma, meaning they are processed separately) represents two secondary components one of which being a chroma component and the other one being another chroma component (Zhu Pg 7 Col 1 ¶2 and Pg 7 Col 2 ¶03 discloses two chroma components being predicted). See Claim 9 for rationale, its parent claim.
Regarding Claim 14, Zhu in view of Besenbruch teaches the method according to claim 9, wherein
the processing of the first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) comprises transforming the first latent tensor into the first tensor (Besenbruch Pg 135 Section 12.2.5 discloses using decomposition which involves breaking down the tensor into lower dimensions); and
the processing of the second latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) comprises concatenating the second latent tensor and the first latent tensor to obtain a concatenated tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors) and transforming the concatenated tensor into the second tensor ( Besenbrucnh Pg 135 Section 12.2.4 discloses concatenating the blocks in a separate dimension to reduce the number of channels by half, creating a new tensor). See Claim 9 for rationale, its parent claim.
Regarding Claim 15, Zhu in view of Besenbruch teaches the method according to claim 9, wherein each of the first and second latent tensors has a height and a width dimension (Zhu Pg 4 Col 2 ¶04 discloses the component being the size of 4N x 4N which the examiner is interpreting as the height and width of the component) and
the processing of the first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) comprises transforming the first latent tensor into the first tensor (Besenbruch Pg 135 Section 12.2.5 discloses using decomposition which involves breaking down the tensor into lower dimensions); and
the processing of the second latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) comprises determining whether the size or a sub-pixel offset of samples of the second latent tensor in at least one of the height and width dimensions differs from the size or sub-pixel offset of samples in at least one of the height and width dimensions of the first latent tensor (Zhu Pg 7 Col 2 ¶03 discloses the chroma and luma having different partition and block sizes), and when it is determined that the size or sub-pixel offset of samples of the second latent tensor differs from the size or sub-pixel offset of samples of the first latent tensor, adjusting the sample locations of the first latent tensor to match the sample locations of the second latent tensor thereby obtaining an adjusted first latent tensor (Zhu Pg 7 Col 2 ¶03- Pg 8 Col1 ¶01-¶02 discloses motion compensation methods that handle the differences in images of rotation, zooming and shearing when processing both components including shape adaptive transform);
concatenating the second latent tensor and the adjusted first latent tensor to obtain a concatenated latent tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors) only when it is determined that the size or sub-pixel offset of samples of the second latent tensor differs from the size or sub-pixel offset of samples of the first latent tensor (Zhu Pg 7 Col 2 ¶03 discloses the chroma and luma having different partition and block sizes) and else concatenating the second latent tensor and the first latent tensor to obtain a concatenated latent tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors); and
transforming the concatenated latent tensor into the second tensor ( Besenbrucnh Pg 135 Section 12.2.4 discloses concatenating the blocks in a separate dimension to reduce the number of channels by half, creating a new tensor). See Claim 9 for rationale, its parent claim.
Regarding Claim 16, Zhu in view of Besenbruch teaches the method according to claim 9, wherein the first bitstream is processed by a first neural network (Besenbrucnh ¶0185-¶0189 discloses a computer that processes the second bitstream, the computer having two different neural networks) and the second bitstream is processed by a second neural network different from the first neural network (Besenbrucnh ¶0177-¶0183 discloses a computer that processes the second bitstream, the computer having two different neural networks).See Claim 9 for rationale, its parent claim.
Regarding Claim 17, Zhu in view of Besenbruch teaches the method according to claim 9, wherein the first latent tensor comprises a channel dimension (Besenbruch Pg 103 ¶02 discloses where the tensor to be modelled is split into blocks (in spatial and or channel dimensions) and the second latent tensor comprises a channel dimension (Besenbruch Pg 103 ¶02 discloses where the tensor to be modelled is split into blocks (in spatial and or channel dimensions) and wherein the size of the first latent tensor in the channel dimension is one of larger than, smaller than and equal to the size of the second latent tensor in the channel dimension (Besenbruch Pg 103 ¶02 discloses that the channel dimension size can differ and can be adjusted to match the input). See Claim 9 for rationale, its parent claim.
Regarding Claim 18, Zhu teaches a method of reconstructing (Zhu Pg 3 Col 1 ¶04 and Pg 4 Col 2 ¶01-¶02 discloses reconstructing the luma and chroma pixels of the image) at least a portion of an image (Zhu Pg 1 Col 1 ¶03, and Pg 2 Col 1 ¶01 discloses partitioning the image into blocks),
representing a primary residual component of a residual for a primary component of the image (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal),
representing at least one secondary residual component of the residual for at least one secondary component (Zhu Pg 3 Col 2 ¶02 discloses a residual including chroma residual signal and luma residual signal) of the image using information from the first latent tensor (Zhu Pg 3 Col 2 ¶04 discloses the model that performs the encoding using both luma and chroma pixels).
Zhu does not explicitly disclose processing a first bitstream based on a first entropy model to obtain a first latent tensor;
processing the first latent tensor to obtain a first tensor,
processing a second bitstream different from the first bitstream based on a second entropy model different from the first entropy model to obtain a second latent tensor different from the first latent tensor; and
processing the second latent tensor to obtain a second tensor.
Besenbruch is in the same field of image compression while preserving image quality. Further, Besenbruch teaches processing a first bitstream based on a first entropy model to obtain a first latent tensor (Besenbruch ¶0119 discloses entropy encoding the quantized latent into a bitstream); processing the first latent tensor to obtain a first tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent)
processing a second bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream) different from the first bitstream based on a second entropy model different from the first entropy model (Besenbruch ¶0120 discloses entropy decoding the bitstream, wherein decoding is different from encoding indicating a separate model) to obtain a second latent tensor different from the first latent tensor (Besenbruch ¶0105-¶0106 discloses using a second neural network to quantize the latent tensor); and
processing the second latent tensor to obtain a second tensor (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhu by incorporating the transformation of the tensors into latent tensors and bitstreams in both actions of reducing dimensionality and increasing dimensionality as taught by Besenbruch; to make an invention that utilizes the reduced dimensionality to ease the computational burden; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to increase the compression, while preserving displayed image quality, or to increase the displayed image quality, while not increasing the amount of data that is actually transmitted across the communications networks. This would help to reduce the demands on communications networks, compared to the demands that otherwise would be made as disclosed by Besenbruch in ¶0004.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 19, Zhu in view of Besenbruch teaches the method according to claim 18, wherein the first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) is processed independently (Zhu Pg 2 Col 2 ¶01 and Pg 3 Col 2 ¶04 discloses two separate networks one to process luma and one to process chroma, meaning they are processed separately) from the processing of the second latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent). See Claim 18 for rationale, its parent claim.
Regarding Claim 20, Zhu in view of Besenbruch teaches the method according to claim 18, wherein the primary component of the image is a luma component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) and the at least one secondary component of the image is a chroma component (Zhu Fig 7a discloses the chroma encoding), or
wherein the primary component of the image is a chroma component (Zhu Fig 7a discloses the chroma encoding) and the at least one secondary component of the image is a luma component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding). See Claim 18 for rationale, its parent claim.
Regarding Claim 21, Zhu in view of Besenbruch teaches the method according to claim 18, wherein
the processing of the first latent tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent) comprises transforming the first latent tensor into the first tensor (Besenbruch Pg 135 Section 12.2.5 discloses using decomposition which involves breaking down the tensor into lower dimensions); and
the processing of the second latent tensor comprises concatenating the second latent tensor (Zhu Fig 5 discloses concatenating the tensors to output a chroma predication network that utilizes tensors) and transforming the concatenated latent tensor into the second tensor ( Besenbrucnh Pg 135 Section 12.2.4 discloses concatenating the blocks in a separate dimension to reduce the number of channels by half, creating a new tensor). See Claim 18 for rationale, its parent claim.
Regarding Claim 22, Zhu in view of Besenbruch teaches the method according to claim 18, wherein the first bitstream is processed by a first neural network (Besenbrucnh ¶0185-¶0189 discloses a computer that processes the second bitstream, the computer having two different neural networks) and the second bitstream is processed by a second neural network different from the first neural network (Besenbrucnh ¶0177-¶0183 discloses a computer that processes the second bitstream, the computer having two different neural networks). See Claim 18 for rationale, its parent claim.
Regarding Claim 23, Zhu in view of Besenbruch teaches a non-transitory computer-readable medium comprising a code which when executed by one or more processors performs the method according to claim 18 (Besenbruch ¶0165, ¶0176, ¶0291, ¶0308 discloses a computer program product that is executed by processors). See Claim 18 for rationale, its parent claim.
Regarding Claim 25, Zhu teaches a processing apparatus (Zhu Pg 8 Col1 ¶03 discloses using a Windows 10 Enterprise 64-bit operating system, to run the experiment) for reconstructing (Zhu Pg 3 Col 1 ¶04 and Pg 4 Col 2 ¶01-¶02 discloses reconstructing the luma and chroma pixels of the image) at least a portion of an image, (Zhu Pg 2 Col 2 ¶02 and Pg 5, Col 1, ¶05 and Pg 6 Col 1 ¶04 discloses employing an encoder-decoder structure and an encoder side of the neural network Pg 1 Col 1 ¶03, and Pg 2 Col 1 ¶01 discloses partitioning the image into blocks), the processing apparatus comprising a processing circuitry (Zhu Pg 8 Col1 ¶03 discloses using a Windows 10 Enterprise 64-bit operating system including a CPU, to run the experiment);
representing a primary component of the image (Zhu Fig 7a discloses the chroma encoding);
representing at least one secondary component (Zhu Fig 5, Pg 7 Col 1 ¶1 discloses the luma down sampling network and encoding) of the image using information from the first latent tensor (Zhu Pg 3 Col 2 ¶04 discloses the model that performs the encoding using both luma and chroma pixels).
Zhu does not explicitly disclose processing a first bitstream based on a first entropy model to obtain a first latent tensor;
processing the first latent tensor to obtain a first tensor;
processing a second bitstream different from the first bitstream based on a second entropy model different from the first entropy model to obtain a second latent tensor different from the first latent tensor; and
processing the second latent tensor to obtain a second tensor.
Besenbruch is in the same field of image compression while preserving image quality. Further, Besenbruch teaches processing a first bitstream based on a first entropy model to obtain a first latent tensor (Besenbruch ¶0119 discloses entropy encoding the quantized latent into a bitstream);
processing the first latent tensor to obtain a first tensor (Besenbruch ¶0093 discloses the first computer system is configured to quantize the latent representation to produce a quantized latent)
processing a second bitstream (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream) different from the first bitstream based on a second entropy model different from the first entropy model (Besenbruch ¶0120 discloses entropy decoding the bitstream, wherein decoding is different from encoding indicating a separate model) to obtain a second latent tensor different from the first latent tensor (Besenbruch ¶0105-¶0106 discloses using a second neural network to quantize the latent tensor); and
processing the second latent tensor to obtain a second tensor (Besenbruch ¶0094 discloses the first computer system is configured to entropy encode the quantized latent into a bitstream).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Zhu by incorporating the transformation of the tensors into latent tensors and bitstreams in both actions of reducing dimensionality and increasing dimensionality as taught by Besenbruch; to make an invention that utilizes the reduced dimensionality to ease the computational burden; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need to increase the compression, while preserving displayed image quality, or to increase the displayed image quality, while not increasing the amount of data that is actually transmitted across the communications networks. This would help to reduce the demands on communications networks, compared to the demands that otherwise would be made as disclosed by Besenbruch in ¶0004.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent US-20220191553-A1 to Auyeung et al. discloses method and apparatus for video coding that incorporates neural networks into the video coding process.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL ROBERTS whose telephone number is (571) 272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights