DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant claims the benefit of US Provisional Application No. 63/594,772 filed 10/31/2023. Claims 1-20 have been afforded the benefit of this filing date.
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the
plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification.
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claim 5 recite “or” then listing “includes a color value or a transparency value”. Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim 7 recite “or” then listing “color or transparency channel sample values”. Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim 8 and Claim 17 recite “or” then listing “real or virtual camera”. Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Objections
Claim 7 and 16 is objected to because of the following informalities:
Line 3 of Claim 7 and 16 currently read “that are similar” when referring to the single channel sample function and the interpolated sample functions. The Examiner finds that it is unclear from the claims and specification how “similar” affects the scope of the claim when referring to the single and double sample functions. There is no definition or metric that defines how similar the values have to be, or how they are being compared, therefore rendering the scope of the claim indefinite.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as unpatentable over Ye et al (Ye, Yanming, et al. "Improved Upsampling Based Depth Image Super-Resolution Reconstruction." IEEE Access 11 (2023): 46782-46792, hereafter referred to as Ye) in view of Duch et al (Duch, Marc Maceira, Josep-Ramon Morros, and Javier Ruiz-Hidalgo. "Depth map compression via 3D region-based representation." Multimedia Tools and Applications 76.12 (2017): 13761-13784, hereafter referred to as Duch) in further view of Smirnov et al (US Patent Publication US 2017/0316602 Al, hereafter referred to as Smirnov).
Regarding Claim 1, Ye teaches for resampling deep images (Ye Introduction ¶01, ¶08, Fig 1 discloses restructuring the depth images to obtain a higher resolution output image)
receiving a deep image (Ye Fig 1, discloses the input of a low resolution depth image), where the deep image includes one or more channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels, the examiner is interpreting that the edge and planar regions are channels contain pixels), each channel sample value including one or more associated depth values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values);
wherein each 3D region is defined by one or more planar surfaces each including an associated depth value (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image);
generating an upsampled image having a greater resolution than the deep image (Ye Fig 1, Abstract, and Introduction disclose the output being an image with a higher resolution than the input deep image, Section II discloses the up sampling used to create the higher resolution image),
interpolating, based on a specified interpolation technique (Ye Introduction ¶012 and Fig 1 discloses using two different interpolation techniques one for planar and one for edge), one or more new channel sample values associated with pixels included in the upsampled image (Ye Introduction ¶09 discloses correlative depths of seeds are chosen as interpolation values, wherein the seeds are correlation coefficients of interpolated pixels and Section II C 1 discloses using interpolation to compute a pseudo depth image with interpolated pixels from the coefficient seeds); and
generating an output deep image (Ye Fig 1 and Section II ¶03 discloses that outputting a high resolution depth image) based on the upsampled image and the one or more new channel sample values (Ye Fig 1 and Abstract and Section II discloses that the output image Is based on the pseudo depth images with the interpolated values and the upsampled image in the structure and edge informative side of the figure).
Ye does not explicitly disclose partitioning the deep image into one or more three-dimensional (3D) regions.
Duch is in the same field of depth preservation in image analysis. Further, Duch teaches partitioning the deep image into one or more three-dimensional (3D) regions (Duch Abstract, Section 3 and Fig 2 disclose segmenting images where regions are represented using a planar model in the 3D world scene).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Ye by incorporating the 3D image segmentation to produce 3D regions with a starting and ending depth value as taught by Duch; to make an invention that can atomically segment and determine the depth values for a 3D image for use in the interpolation of further depth points to more accurately represent the scene; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need prevent coding artifacts along sharp depth discontinuities while efficiently encoding homogeneous areas. The compression scheme considered relies on the choice of an adequate image segmentation method. as disclosed by Duch in the Introduction.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Ye and Duch in combination do not explicitly disclose a computer-implemented method, the computer-implemented method.
Smirnov is in the same field of depth preservation for high resolution images in image analysis. Further, Smirnov teaches a computer-implemented method (Smirnov ¶0025, ¶0163 discloses a computer program product that executes a method) the computer-implemented method (Smirnov ¶0025, ¶0163 discloses a computer program product that executes a method).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Ye in view of Duch by incorporating the use of computer implemented methods and apparatuses in conjunction with cameras and specific sightlines as taught by Smirnov; to make an invention that can automate the process of creating higher resolution images while preserving depth information of an image captured by a camera; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need address the problem that for many two-sensor 3D cameras it may not be possible to filter depth map using available color image until it is aligned with a color
view as disclosed by Smirnov in ¶0005.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are provided by a user (Ye Section II C discloses that the assignment process of their algorithm can be interpreted as a kind of segmentation process, therefore the examiner views the quantized depth value as a label for each seed, provided by the executer of the algorithm). See Claim 1 for rationale, its parent claim.
Regarding Claim 3, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are determined dynamically (Ye Introduction references adaptive techniques to enhance the depth images) based on one or more depth values associated with channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values) included in the upsampled image (Ye Fig 1, Abstract, and Introduction disclose the output being an image with a higher resolution than the input deep image, Section II discloses the up sampling used to create the higher resolution image). See Claim 1 for rationale, its parent claim.
Regarding Claim 4, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are automatically calculated to generate a predetermined number (Duch Section 3.1 discloses the number of regions in the color partition N fixes the minimum rate for the depth coding method as is the minimum number of planes encoded) of 3D regions within the deep image (Duch Abstract, Section 3 and Fig 2 discloses segmenting images where regions are represented using a planar model in the 3D world scene). See Claim 1 for rationale, its parent claim.
Regarding Claim 5, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein each of the one or more channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels, the examiner is interpreting that the edge and planar regions are channels contain pixels) includes a color value or a transparency value (Ye Fig 1 and Section II B discloses that the color image information is used in conjunction with the depth values). See Claim 1 for rationale, its parent claim.
Regarding Claim 6, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, further comprising replacing two interpolated channel sample value functions with a single channel sample value function (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image) having an associated range of depth values corresponding to the ranges of depth values (Ye Section II, C, 1, discloses quantify the depth range of LR depth image and obtain a pseudo LR depth mage by mapping disjoint depth interval to its median value and the pseudo LR depth image is mapped into pseudo depth image and forms a set of seeds) associated with the two interpolated channel sample values (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image). See Claim 1 for rationale, its parent claim.
Regarding Claim 7, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 6, wherein the single channel sample value function generates depth-dependent color or transparency channel sample values (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image) that are similar to the depth-dependent color or transparency channel sample values (Ye Section II, B, discloses comparing the depth value of a pixel to its neighboring pixel) generated by the two interpolated channel sample value functions (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image) to within one or more predefined thresholds (Ye Section II, B, discloses comparing the depth value of a pixel to its neighboring pixel and comparing it to a threshold value). See Claim 1 for rationale, its parent claim.
Regarding Claim 8, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein each of the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) is parallel to a plane (Smirnov ¶0260 discloses the camera sightline needing to be parallel for Z depth) associated with a real or virtual camera used to capture the deep image (Ye Introduction ¶08 and Section IV B discloses using a camera to capture the image in a real world application). See Claim 1 for rationale, its parent claim.
Regarding Claim 9, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein one of the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) divides a channel sample into two depth-adjacent channel samples at a calculated depth value (Ye Section II, C, 3, discloses dividing the pseudo depth image values into different sets to produce a higher accuracy coefficient for the calculated final depth value). See Claim 1 for rationale, its parent claim.
Regarding Claim 10, Ye in view of Duch in further view of Smirnov teaches the computer-implemented method of claim 1, wherein at least one of the one or more channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels, the examiner is interpreting that the edge and planar regions are channels contain pixels) includes an associated starting depth value and an associated ending depth value (Duch Section 3.2-3.4 discloses the max and minimum depth values of the image used for 3D representation), the ending depth value being greater than the starting depth value (Duch Section 3.4-3.2 discloses the max and minimum depth values of the image used for 3D representation specifically low and high values being used to determine the depth range of the image). See Claim 1 for rationale, its parent claim.
Regarding Claim 11, Ye teaches receiving a deep image (Ye Fig 1, discloses the input of a low resolution depth image), where the deep image includes one or more channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels, the examiner is interpreting that the edge and planar regions are channels contain pixels), each channel sample value including one or more associated depth values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values);
wherein each 3D region is defined by one or more planar surfaces each including an associated depth value (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image);
generating an upsampled image having a greater resolution than the deep image (Ye Fig 1, Abstract, and Introduction disclose the output being an image with a higher resolution than the input deep image, Section II discloses the up sampling used to create the higher resolution image),
interpolating, based on a specified interpolation technique (Ye Introduction ¶012 and Fig 1 discloses using two different interpolation techniques one for planar and one for edge), one or more new channel sample values associated with pixels included in the upsampled image (Ye Introduction ¶09 discloses correlative depths of seeds are chosen as interpolation values, wherein the seeds are correlation coefficients of interpolated pixels and Section II C 1 discloses using interpolation to compute a pseudo depth image with interpolated pixels from the coefficient seeds) ; and
generating an output deep image (Ye Fig 1 and Section II ¶03 discloses that outputting a high resolution depth image) based on the upsampled image and the one or more new channel sample values (Ye Fig 1 and Abstract and Section II discloses that the output image Is based on the pseudo depth images with the interpolated values and the upsampled image in the structure and edge informative side of the figure).
Ye does not explicitly disclose partitioning the deep image into one or more three-dimensional (3D) regions.
Duch is in the same field of depth preservation in image analysis. Further, Duch teaches partitioning the deep image into one or more three-dimensional (3D) regions (Duch Abstract, Section 3 and Fig 2 disclose segmenting images where regions are represented using a planar model in the 3D world scene).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Ye by incorporating the 3D image segmentation to produce 3D regions with a starting and ending depth value as taught by Duch; to make an invention that can atomically segment and determine the depth values for a 3D image for use in the interpolation of further depth points to more accurately represent the scene; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need prevent coding artifacts along sharp depth discontinuities while efficiently encoding homogeneous areas. The compression scheme considered relies on the choice of an adequate image segmentation method. as disclosed by Duch in the Introduction.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Ye and Duch in combination do not explicitly disclose one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps.
Smirnov is in the same field of depth preservation for high resolution images in image analysis. Further, Smirnov teaches one or more non-transitory computer-readable media storing instructions (Smirnov ¶0025 discloses A nontransitory computer-readable medium, the instructions when executed by a processor) that, when executed by one or more processors, cause the one or more processors to perform the steps (Smirnov ¶0050 discloses nontransitory computer-readable medium, the instructions when executed by a processor).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Ye in view of Duch by incorporating the use of computer implemented methods and apparatuses in conjunction with cameras and specific sightlines as taught by Smirnov; to make an invention that can automate the process of creating higher resolution images while preserving depth information of an image captured by a camera; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need address the problem that for many two-sensor 3D cameras it may not be possible to filter depth map using available color image until it is aligned with a color
view as disclosed by Smirnov in ¶0005.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 12, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 11, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are provided by a user (Ye Section II C discloses that the assignment process of their algorithm can be interpreted as a kind of segmentation process, therefore the examiner views the quantized depth value as a label for each seed, provided by the executer of the algorithm). See Claim 11 for rationale, its parent claim.
Regarding Claim 13, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 11, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are determined dynamically (Ye Introduction references adaptive techniques to enhance the depth images) based on one or more depth values associated with channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values)included in the upsampled image (Ye Fig 1, Abstract, and Introduction disclose the output being an image with a higher resolution than the input deep image, Section II discloses the up sampling used to create the higher resolution image). See Claim 11 for rationale, its parent claim.
Regarding Claim 14, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 11, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are automatically calculated to generate a predetermined number (Duch Section 3.1 discloses the number of regions in the color partition N fixes the minimum rate for the depth coding method as is the minimum number of planes encoded) of 3D regions within the deep image (Duch Abstract, Section 3 and Fig 2 discloses segmenting images where regions are represented using a planar model in the 3D world scene). See Claim 11 for rationale, its parent claim.
Regarding Claim 15, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 11, further comprising replacing two interpolated channel sample value functions with a single channel sample value function (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image) having an associated range of depth values corresponding to the ranges of depth values (Ye Section II, C, 1, discloses quantify the depth range of LR depth image and obtain a pseudo LR depth mage by mapping disjoint depth interval to its median value and the pseudo LR depth image is mapped into pseudo depth image and forms a set of seeds) associated with the two interpolated channel sample values (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image). See Claim 11 for rationale, its parent claim.
Regarding Claim 16, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 15, wherein the single channel sample value function generates depth-dependent color or transparency channel sample values (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image) that are similar to the depth-dependent color or transparency channel sample values (Ye Section II, B, discloses comparing the depth value of a pixel to its neighboring pixel) generated by the two interpolated channel sample value functions (Ye Introduction ¶08 and Fig 1 and Section II, C, 2 discloses taking the two depth values, one from color and one from depth into a single value to produce the higher resolution output image) to within one or more predefined thresholds (Ye Section II, B, discloses comparing the depth value of a pixel to its neighboring pixel and comparing it to a threshold value). See Claim 11 for rationale, its parent claim.
Regarding Claim 17, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 11, wherein each of the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) is parallel to a plane (Smirnov ¶0260 discloses the camera sightline needing to be parallel for Z depth) associated with a real or virtual camera used to capture the deep image (Ye Introduction ¶08 and Section IV B discloses using a camera to capture the image in a real world application). See Claim 11 for rationale, its parent claim.
Regarding Claim 18, Ye in view of Duch in further view of Smirnov teaches the one or more non-transitory computer-readable media of claim 11, wherein one of the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) divides a channel sample into two depth-adjacent channel samples at a calculated depth value (Ye Section II, C, 3, discloses dividing the pseudo depth image values into different sets to produce a higher accuracy coefficient for the calculated final depth value). See Claim 11 for rationale, its parent claim.
Regarding Claim 19, Ye teaches receive a deep image (Ye Fig 1, discloses the input of a low resolution depth image), where the deep image includes one or more channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels, the examiner is interpreting that the edge and planar regions are channels contain pixels), each channel sample value including one or more associated depth values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values);
wherein each 3D region is defined by one or more planar surfaces each including an associated depth value (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image);
generating an upsampled image having a greater resolution than the deep image (Ye Fig 1, Abstract, and Introduction disclose the output being an image with a higher resolution than the input deep image, Section II discloses the up sampling used to create the higher resolution image),
interpolate, based on a specified interpolation technique (Ye Introduction ¶012 and Fig 1 discloses using two different interpolation techniques one for planar and one for edge), one or more new channel sample values associated with pixels included in the upsampled image (Ye Introduction ¶09 discloses correlative depths of seeds are chosen as interpolation values, wherein the seeds are correlation coefficients of interpolated pixels and Section II C 1 discloses using interpolation to compute a pseudo depth image with interpolated pixels from the coefficient seeds) and
generate an output deep image (Ye Fig 1 and Section II ¶03 discloses that outputting a high resolution depth image) based on the upsampled image and the one or more new channel sample values (Ye Fig 1 and Abstract and Section II discloses that the output image Is based on the pseudo depth images with the interpolated values and the upsampled image in the structure and edge informative side of the figure).
Ye does not explicitly disclose partition the deep image into one or more three-dimensional (3D) regions.
Duch is in the same field of depth preservation in image analysis. Further, Duch teaches partition the deep image into one or more three-dimensional (3D) regions (Duch Abstract, Section 3 and Fig 2 disclose segmenting images where regions are represented using a planar model in the 3D world scene).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Ye by incorporating the 3D image segmentation to produce 3D regions with a starting and ending depth value as taught by Duch; to make an invention that can atomically segment and determine the depth values for a 3D image for use in the interpolation of further depth points to more accurately represent the scene; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need prevent coding artifacts along sharp depth discontinuities while efficiently encoding homogeneous areas. The compression scheme considered relies on the choice of an adequate image segmentation method. as disclosed by Duch in the Introduction.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Ye and Duch in combination do not explicitly disclose a system comprising: one or more memories storing instructions; and one or more processors for executing the instructions.
Smirnov is in the same field of depth preservation for high resolution images in image analysis. Further, Smirnov teaches a system comprising: one or more memories storing instructions (Smirnov ¶0163 discloses computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor) ; and
one or more processors for executing the instructions (Smirnov ¶0161 discloses A processor is embodied as an executor of software instructions, the instructions may specifically configure the processor to perform the algorithms and/or operations).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Ye in view of Duch by incorporating the use of computer implemented methods and apparatuses in conjunction with cameras and specific sightlines as taught by Smirnov; to make an invention that can automate the process of creating higher resolution images while preserving depth information of an image captured by a camera; thus one of ordinary skilled in the art would be motivated to combine the references since there is a need address the problem that for many two-sensor 3D cameras it may not be possible to filter depth map using available color image until it is aligned with a color
view as disclosed by Smirnov in ¶0005.
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 20, Ye in view of Duch in further view of Smirnov teaches the system of claim 19, wherein the depth values associated with the one or more planar surfaces (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values, Section II discloses the up sampling used to create the higher resolution image) are determined dynamically (Ye Introduction references adaptive techniques to enhance the depth images) based on one or more depth values associated with channel sample values (Ye Section II B, discloses that the depth image has determined planar areas and edge regions made up of pixels that have associated depth values)included in the upsampled image (Ye Fig 1, Abstract, and Introduction disclose the output being an image with a higher resolution than the input deep image, Section II discloses the up sampling used to create the higher resolution image). See Claim 19 for rationale, its parent claim.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
WO Patent Pub WO-2022025769-A1 to Hillman et al. discloses a method and apparatus for managing pixel interpolation of deep pixels in a deep image while retaining structure that allows for easy editing of the deep image
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on (313) 446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights