Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 8 and 16 are objected to because of the following informalities: In each of these claims, the set of formulas for Pos_x and Pos_y contains typographical errors, where the formulas recite “DethRateX” and “DethRateY”, which should be amended to read --DepthRateX-- and --DepthRateY--. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 4, 9, 12, 17, and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Zhao et al. (U.S. Patent Application Publication No. 2023/0145498), referred herein as Zhao.
Regarding claim 1, Zhao teaches a 3D image generating method, comprising: separating a target 2D image to obtain an initial color image and an initial depth image (paragraph 48, lines 1-3; paragraph 49, lines 1-7; paragraph 73, lines 1-8; a 2D image is separated into its corresponding color image and depth map image);
performing associative processing on the initial color image and the initial depth image to obtain a first color image and a target depth image, wherein the target depth image comprises a plurality of reference pixels (paragraph 50, lines 1-11; paragraph 51, lines 1-7; paragraph 78, lines 1-13; a target depth image comprising reference pixels is obtained by performing associative processing on the color and depth images);
determining target pixels according to depth values of the reference pixels, wherein the target pixels are pixels that are holes in the first color image, and determining hole filling values of the target pixels according to the reference pixels, filling the target pixels, and generating a target color image (paragraph 53; paragraph 54, lines 1-8; paragraphs 101 and 102; using the depth image and reference pixels, target pixels are determined that are holes in the first image, and values are determined that are used to fill the holes in a target color image), and
interleaving the initial color image and the target color image to generate a target 3D image (paragraph 55, the last 5 lines; paragraph 84; paragraph 91, lines 1-8; the initial and target color images are interleaved and a target 3D image is generated).
Regarding claim 4, Zhao teaches the method according to claim 1, wherein the determining the target pixels according to the depth values of the reference pixels comprises: in a condition that I d2 (x, y) ≤ 0 is met, determining that a pixel with coordinate values (x, y) in the first color image is a target pixel which is a hole, wherein I d2 is the target depth image, I d2 (x, y) is a depth value of a reference pixel with coordinate values (x, y) in the target depth image (paragraph 91; paragraphs 99 and 101; pixels are identified in the first color image that comprise target pixels that are holes, by referencing the corresponding depth map, which has holes at depth values less than or equal to zero).
Regarding claim 9, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the device, memory, program, and processor, which are taught by Zhao, fig 13, device 1300, memory 1304, and processor 1302; paragraph 166); thus they are rejected on similar grounds.
Regarding claim 12, the limitations of this claim substantially correspond to the limitations of claim 4; thus they are rejected on similar grounds.
Regarding claim 17, the limitations of this claim substantially correspond to the limitations of claim 1 (except for the medium, program, and processor, which are taught by Zhao, fig 13, memory 1304 and processor 1302; paragraph 166); thus they are rejected on similar grounds.
Regarding claim 20, the limitations of this claim substantially correspond to the limitations of claim 4; thus they are rejected on similar grounds.
Allowable Subject Matter
Claims 2, 3, 5-8, 10, 11, 13-16, 18, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, the prior art teaches the method of claim 1, as discussed above, and teaches utilizing reference pixels to determine hole filling values (see, for example, the citations to Zhao provided above). In the context of claims 1 and 2 as a whole, however, the prior art does not appear to teach the method of claim 1, wherein the determining the hole filling values of the target pixels according to the reference pixels comprises setting preset paths of traversing the reference pixels, wherein the preset paths are set in the target depth image, traversing the preset paths based on coordinate values of the reference pixels, and determining a first target pixel and a second target pixel that meet a preset condition, and determining the hole filling values according to coordinate values of the first target pixel and coordinate values of the second target pixel. Thus claim 2 comprises allowable subject matter.
Regarding claim 3, this claim comprises allowable subject matter insomuch as it depends from claim 2, which comprises allowable subject matter.
Regarding claim 5, the prior art teaches the method of claim 1, as discussed above, and teaches utilizing point clouds and the depth images to reproject and resize the images (see, for example, Zhao, paragraphs 84 and 85, among others). In the context of claims 1 and 5 as a whole, however, the prior art does not appear to teach the method of claim 1, wherein the performing associative processing on the initial color image and the initial depth image to obtain the first color image and the target depth image comprises: determining an initial point cloud corresponding to a first depth image, wherein the first depth image is obtained by remapping of the initial depth image; adjusting coordinate values of each point in the initial point cloud according to a preset relative displacement to obtain a target point cloud; processing coordinate values of each point in the target point cloud to obtain a reference image, wherein the reference image is a depth image that matches a size of the initial depth image; processing pixels in the initial color image and pixels in the initial depth image according to depth values of pixels in the reference image to obtain the first color image and the target depth image. Thus claim 5 comprises allowable subject matter.
Regarding claims 6 and 7, these claims comprise allowable subject matter insomuch as they depend directly or indirectly from claim 5, which comprises allowable subject matter.
Regarding claim 8, the prior art teaches the method of claim 1, as discussed above. In the context of claims 1 and 8 as a whole, however, the prior art does not appear to teach the method of claim 1, wherein the performing associative processing on the initial color image and the initial depth image to obtain a first color image and a target depth image, wherein the target depth image comprises a plurality of reference pixels comprises determining a preset depth of field of the target 3D image, the preset depth of field of the target 3D image on an x-axis being nx0 to nx1, and the preset depth of field on the y-axis being ny0 to ny1, determining a ratio of the preset depth of field to a depth range of the initial depth image according to formulas:
D
e
p
t
h
R
a
t
e
X
=
(
n
x
1
-
n
x
0
)
/
(
D
m
a
x
-
D
m
i
n
)
, and
D
e
p
t
h
R
a
t
e
Y
=
(
n
y
1
-
n
y
0
)
/
(
D
m
a
x
-
D
m
i
n
)
,
wherein
D
e
p
t
h
R
a
t
e
X
is a ratio of the preset depth of field to the depth range of the initial depth image on the x-axis,
D
e
p
t
h
R
a
t
e
Y
is a ratio of the preset depth of field to the depth range of the initial depth image on the y-axis, and
D
m
a
x
is a maximum depth value and
D
m
i
n
is a minimum depth value of the initial depth image, determining Pos_x and Pos_y according to formulas:
Pos_x = x + (Id (x, y ) – Dmin) * DepthRateX + nx0,
Pos_y = y + (Id (x, y ) – Dmin) * DepthRateY + ny0;
and determining the first color image and the target depth image according to formulas:
Ic1 (Pos_x, Pos_y ) = Ic (x, y ), and
Id2 (Pos_x, Pos_y ) = Id (x, y ),
wherein Ic is the initial color image, Ic1 is the first color image, Ic (x, y ) is a pixel value of a pixel with coordinate values (x, y ) in the initial color image, Ic1 (Pos_x, Pos_y ) is a pixel value of a pixel with coordinate values (Pos_x, Pos_y ) in the first color image, Id is the initial depth image, Id2 is the target depth image, and Id2 (Pos_x, Pos_y ) is a depth value of a pixel with coordinate values (Pos_x, Pos_y ) in the target depth image. Thus claim 8 comprises allowable subject matter.
Regarding claims 10, 11, and 13-16, the limitations of these claims substantially correspond to the limitations of claims 2, 3, and 5-8, respectively; thus they comprise allowable subject matter for similar reasons to those discussed above.
Regarding claims 18 and 19, the limitations of these claims substantially correspond to the limitations of claims 2 and 3, respectively; thus they comprise allowable subject matter for similar reasons to those discussed above.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Yao (U.S. Patent Application Publication No. 2010/0208994); Filling holes in depth maps.
Nguyen (U.S. Patent Application Publication No. 2011/0115886); System for executing 3D propagation for depth image-based rendering.
Evertt (U.S. Patent Application Publication No. 2013/0100119); Object refinement using many data sets.
Wagner (U.S. Patent Application Publication No. 2016/0073080); Method and apparatus for efficient depth image transformation.
Graziosi (U.S. Patent Application Publication No. 2016/0360177); Methods for full parallax compressed light field synthesis utilizing depth information.
Martin (U.S. Patent No. 12,361,519); Neural blending for novel view synthesis.
Bigos (U.S. Patent Application Publication No. 2023/0281912); Method and system for generating a target image from plural multi-plane images.
McDaniel (U.S. Patent No. 12,541,827); Depth image interpolation for multiview item recognition.
Kontkanen (U.S. Patent Application Publication No. 2025/0218109); Rendering videos with novel views from near-duplicate photos.
Cho et al. ("Hole-Filling of RealSense Depth Images Using a Color Edge Map"); IEEE; 2020.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID T WELCH whose telephone number is (571)270-5364. The examiner can normally be reached on Monday-Thursday, 8:30-5:30 EST, and alternate Fridays, 9:00-2:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID T. WELCH
Primary Examiner
Art Unit 2613
/DAVID T WELCH/Primary Examiner, Art Unit 2613
Prosecution Insights