Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
CONTINUED EXAMINATION UNDER 37 CFR. 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/2/26 has been entered.
RESPONSE TO ARGUMENTS
Double Patenting Rejection
Applicant arguments filed 4/2/26 in view of double patenting are acknowledged. Applicants position that applicant will consider submitting a terminal disclaimer in future is noted. As claims remain sufficient for rejection under double patenting guidelines rejection is maintained.
PRIOR ART REJECTION
The examiner acknowledges the amendment of claims 1-7 & 10-12 filed 4/2/26. Applicants’ arguments filed on (4/2/26) have been fully considered but are deemed moot in view of new grounds of rejection. Due to the variation in claim scope via amendments a new ground of rejection is proper.
DOUBLE PATENTING
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP §§ 706.02(l)(1) - 706.02(l)(3) for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1-12 are rejected on the ground of nonstatutory double patenting as being unpatentable over (U.S. Patent 10,706,568) in view of LI et al. (U.S. Publication 2002/0092741).
As to claims 1 & 10-12, instant application discloses
one or more processors that execute a program stored in a memory and thereby cause the information processing apparatus to: obtain first 3D shape data representing a shape of a 3D object (U.S. Patent 10,706,568 - Claim 1), obtain data of a shot image of a field of view including the 3D object (U.S. Patent 10,706,568 - Claims 3 & 8), generate a depth map corresponding to the field of view of the shot image by projecting the first 3D shape data onto an image plane corresponding to the shot image, increase a resolution of an area of the depth map corresponding to an area of interest that is set for the shot image (U.S. Patent 10,706,568 - Claims 1 & 3),
(U.S. Patent 10,706,568) is silent to increase a resolution of an area of the depth map corresponding to an area of interest set for the image, and generate second 3D shape data based on the depth map; wherein the second 3d shape data includes a first portion corresponding to the area of interest and a second portion being outside the area of interest, and wherein a first resolution of the first portion is higher than a second resolution of the second portion.
However, Li discloses to increase a resolution of an area of the depth map corresponding to an area of interest set for the image, and generate second 3D shape data based on the depth map; wherein the second 3d shape data includes a first portion corresponding to the area of interest and a second portion being outside the area of interest, and wherein a first resolution of the first portion is higher than a second resolution of the second portion. ([0186] discloses region detection circuitry/logic 2306 determines the region, depth super resolution circuitry/logic determines the regions, depth super resolution circuitry 2307 determines the region(s) then upscales the resolution of ROIs and the point cloud generator then generates a point cloud using the upscaled ROI. [0185] discloses a depth camera 2301 captures a depth map of the environment which is used by point cloud generator 2302 to formulate a corresponding point cloud and point cloud registration circuitry/logic 2304 registers all point clouds to generate the final 3D point cloud model for the scene. [0189] discloses the trained model 2414 is used to sample the depth map returned by the sensor in a mixed reality system and generate a super resolution map of the region of interest (ROI). [0192-0193] further discloses one or more regions of interest are identified within the raw image and a trained model generated by machine learning techniques is used to generate a super resolution map of the one or more regions of interest. )
It would have been obvious to one of ordinary skill in the art at the time of filing to modify (U.S. Patent 10,706,568) to include the above limitations in order to generate final 3D shape data for the field of view from a depth map in which only the area of interest has been super resolved, thereby preserving broad field of view reconstruction while concentrating higher 3D detail in the area of interest.
As to claims 2-9, these claims are rejected due to their dependence on claims 1 & 10-12 and are rejected for the same reasons.
CLAIM REJECTIONS - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12 are rejected under 35 U.S.C. 103 as being unpatentable over
HIGAKI et al. (U.S. Publication 2019/0259169) in view of Wang et al. (U.S. Publication 2020/0202495) & LI et al. (U.S. Publication 2022/0092741)
As to claims 1 & 10-12, HIGAKI discloses obtain 3D shape data representing a shape of a 3D object ([0020] discloses point cloud data represents three dimensional coordinates of a plurality of points on a target object that has three dimensional shape.), obtain data of a shot image of a field of view including the 3D object ([0004] discloses a depth map that represents distances from a predetermined viewpoint in a field of view. [0021] discloses camera parameters such as a focal length, an orientation and a position of a predetermined viewpoint for observing a target object. ), generate a depth map corresponding to the field of view of the shot image based on the first 3D shape data onto an image plane corresponding to the shot image ([0018] discloses projects each of a plurality of three dimensional coordinates representing the shape of a target object onto a two dimensional image and generates a range image (a depth map). [0022] discloses a point cloud projection unit 103 specifies projected positions for projecting three dimensional positions of each point represented by the point cloud data, onto a range image (a depth map) and the depth map is a projection plane).
HIGAKI is silent to increase a resolution of an area of the depth map corresponding to an area of interest that is set for the shot image; and generate second 3D shape data includes a first portion corresponding to the area of interest and a second portion being outside the area of interest, and wherein a first resolution of the first portion is higher than a second resolution of the second portion.
However, Wang discloses increase a resolution of an area of the depth map corresponding to an area of interest that is set for the shot image; ([0007] discloses sets a three dimensional (3D) region of interest according to a predefined feature of a salient object, the high resolution image and the first depth map; and computes a second depth map whose depth resolution is greater than the depth resolution of the first depth map in the 3D region of interest. )([0017] discloses a high resolution depth measurement is performed inside a pre-defined region of interest (ROI), and a low resolution depth measurement is performed outside the region. [0024] discloses in Step S13, a 3D region of interest (ROI) is set. In Step S15, a second depth map is obtained, wherein in the 3D region of interest (ROI), the depth resolution of the second depth map is greater than the depth resolution of the first depth map. [0030] compute the second depth map to enhance the depth resolution along the Z-axis.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI’s disclosure to include the above limitations in order to improve depth detail in the area of interest while avoiding the computational cost of uniformly increasing depth resolution across the entire field of view.
HIGAKI in view of Wang is silent to generate second 3D shape data for the field of view based on the depth map;
Wherein the second 3D shape data includes a first portion corresponding to the area of interest and a second portion being outside the area of interest, wherein the second 3D shape data includes a first portion corresponding to the area of interest and a second portion being outside the area of interest;
However, LI discloses generate second 3D shape data for the field of view based on the depth map. ([0185] discloses components 2301-2305 represents a typical process for 3D model generation. A depth camera 2301 captures a depth map of the environment which is used by point cloud generator 2302 to formulate a corresponding point cloud. A point cloud registration circuitry/logic registers all point clouds to generate the final 3D point cloud model for the scene.)
wherein the second 3D shape data includes a first portion corresponding to the area of interest and a second portion being outside the area of interest; ([0186] discloses region detection circuitry/logic 2306 determines the region. Depth super resolution circuitry/logic 2307 then upscales the resolution of ROIs. The point cloud generator then generates a point cloud using the upscaled ROIs. The input to the region detection circuitry/logic 2306 is a low resolution depth map, and the output of the depth super resolution circuitry module 2307 is high resolution depth map. [0192] discloses one or more regions of interest are identified within the raw image. [0193] discloses a trained model generated by machine learning techniques is used to generate a super resolution map of the one or more regions of interest.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang’s disclosure to include the above limitations in order to generate final 3D shape data for the field of view from a depth map in which only the area of interest has been super resolved thereby preserving broad field of view reconstruction while concentrating higher 3D detail in the area of interest.
As to claim 2, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 1 but is silent to wherein the information processing apparatus generates the second 3D shape data based on data of an area of the depth map with the first resolution, and the one or more processors further cause the information processing apparatus to combine the first 3D shape data and the second 3D shape data.
However, Wang discloses the second 3D shape data is generated based on data of an area of the depth map with increased resolution because Wang recomputes the depth map in the ROI at higher resolution. ([0007] discloses computing a second depth map whose depth resolution is greater than the depth resolution of the first depth map in the 3d region of interest.)
LI discloses using the upscaled ROIs to generate a point cloud and then registering point clouds to generate the final 3D point cloud model for the scene, which corresponds to combining the first 3D shape data and the second 3D shape data. [0185] discloses a depth camera 2301 captures a depth map of the environment which is used by point cloud generator 2302 to formulate a corresponding point cloud. Point cloud registration circuitry/logic 2304 registers all point clouds to generate the final 3D point cloud model for the scene. [0186] discloses depth super resolution circuitry/logic 2307 then upscales the resolution of ROIs and the plot cloud generator then generates a point cloud using the upscaled ROIs.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to combine pre-existing 3D shape data with ROI-enhanced 3D shape data and thereby improve local detail while preserving a coherent overall 3D model for the field of view.
As to claim 3, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 1 but is silent to wherein the information processing apparatus uses data of the area of interest when increasing the resolution of the depth map.
However, Wang discloses wherein the information processing apparatus uses data of the area of interest when increasing the resolution of the depth map. ([0007] discloses sets a three-dimensional (3D) region of interest according to a pre-defined feature of a salient object, the high-resolution image and the first depth map; and computes a second depth map whose depth resolution is greater than the depth resolution of the first depth map in the 3D region of interest. [0024] discloses In Step S13, a 3D region of interest (ROI) is set according to a predefined feature of a salient object OB, the high-resolution image MG2 and the first depth map. In step S14, a disparity map in sub-pixel values in the 3D region of interest (ROI) is recomputed. In step S15 a second depth map is obtained.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to focus the resolution increase on the user relevant region instead of expending resources across the entire field of view.
As to claim 4, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 3 but is silent to wherein a resolution of the area of interest used when increasing the resolution of the depth map is higher than a resolution of the area of interest in the shot image.
However, Wang discloses wherein a resolution of the area of interest used when increasing the resolution of the depth map is higher than a resolution of the area of interest in the shot image. ([0007] discloses the image capture module obtains a high-resolution image whose resolution is higher than the resolution of depth capture module. [0022] discloses in MG2 mode, the resolution of the second camera 112 is set to be higher than that of the first camera 112, being the image capture module 120 to capture the high-resolution image MG2. [0024] discloses in Step S13, a 3D region of interest (ROI) is set according to the high-resolution image MG2 and first depth map.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to improve the precision of the depth resolution increase in the area of interest.
As to claim 5, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 4 but is silent to wherein the information processing apparatus obtains data of the area of interest used when increasing the resolution of the depth map by making an image capture apparatus shoot an image containing the area of interest at a shooting magnification higher than a shooting magnification of the shot image.
However, Wang discloses wherein the information processing apparatus obtains data of the area of interest used when increasing the resolution of the depth map by making an image capture apparatus shoot an image containing the area of interest at a shooting magnification higher than a shooting magnification of the shot image. ([0007] discloses the image capture module obtains a high-resolution image whose resolution is higher than the resolution of the depth capture module. [0022] discloses in MG2 mode, the resolution of the second camera is set to be higher than that of the first camera 112, being the image capture module 120 to capture the high-resolution image MG2. [0024] discloses In Step S11, a set of images MG1 for disparity computation and a high-resolution image MG2 are synchronously obtained.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to provide more detailed area of interest information for the subsequent depth resolution increase.
As to claim 6, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 4 but is silent to wherein the information processing apparatus obtains data of the area of interest used when increasing the resolution of the depth map by increasing a resolution of the area of interest of the shot image using image processing.
However, Wang discloses reinforces that the ROI is the region in which the high resolution depth maps is computed. ([0028] discloses the computing unit 130 also can compute a high-resolution depth map in the 3D region of interest (ROI))
LI discloses obtaining area-of-interest data by increasing the resolution of the ROI using image processing/super-resolution processing.
([0189] discloses the trained model 2414 is used to sample the depth map returned by the sensor in a mixed reality system and generate a super resolution map of the region of interest. (ROI). [0192] discloses one or more regions of interest are identified within the raw image. [0193] discloses a trained model generated by machine learning techniques is used to generate a super resolution map of the one or more regions of interest. [0186] discloses a depth super resolution circuitry/logic 2307 then upscales the resolution of ROIs and the output of the depth super resolution circuitry/module 2307 is high resolution depth map (at
least for some ROIs))
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to enhance the detail available for the depth refinement operation in the area of interest.
As to claim 7, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 1 but is silent to wherein the one or more processors further cause the information processing apparatus to set the area of interest for the shot image, and wherein an area of a specific object included in the shot image is set as the area of interest.
However, Wang discloses wherein the one or more processors further cause the information processing apparatus to set the area of interest for the shot image, and wherein an area of a specific object included in the shot image is set as the area of interest.
([0019] discloses Fig. 1B illustrates a 3D region of interest (ROI) containing a salient object OB and the 3D region of interest (ROI) is set by the computing unit 130 according to a predefined feature of a salient object OB, the high resolution image MG2 and first depth map. [0017] discloses the three dimensional (3D) region of interest may be a human face, a unique shape, an object with closed boundary, or an object feature. [0025] discloses when the feature of an object in the 3D region of interest is highly similar with the feature of a human face, the unique shape of an object or a pre-defined feature of an object, this object can be specified as a salient object OB)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to focus the increased depth resolution on the object of interest.
As to claim 8, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 7 but is silent to wherein in a case where an area of the specific object is not detected in the shot image, an in-focus area of the shot image or an area surround by a contour is set as the area of interest.
However, Wang discloses wherein in a case where an area of the specific object is not detected in the shot image, an in-focus area of the shot image or an area surround by a contour is set as the area of interest. ([0017] discloses the three dimensional (3D) region of interest may be object. [0020] discloses the computing unit 130 can automatically detect the position of the salient object OB according to the high-resolution image MG2, the features between adjacent pixels, and the distribution of the corresponding first depth map.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to preserve ROI selection using image defined object boundaries.
As to claim 9, HIGAKI in view of Wang & LI discloses everything as disclosed in claim 1 but is silent to wherein the first 3D shape data and the second 3D shape data are point group data.
However, Higaki discloses supplying the first 3D shape data as a point cloud data.
([0020] discloses point cloud data represents three dimensional coordinates of a plurality of points on a target object.)
However, Li discloses supplying the second 3D shape data/ final 3D output as point cloud data.
([0185] discloses a depth camera 2301 captures a depth map of the environment which is used by point cloud generator 2302 to formulate a corresponding point. Point cloud registration circuitry/logic 2304 registers all point clouds to generate the final 3D point cloud model for the scene.)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify HIGAKI in view of Wang & LI’s disclosure to include the above limitations in order to preserve compatibility between the 3D data representations used for projection, ROI refinement, and final 3D model generation.
CONCLUSION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stephen P Coleman whose telephone number is (571)270-5931. The examiner can normally be reached Monday-Thursday 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Stephen P. Coleman
Primary Examiner
Art Unit 2675
/STEPHEN P COLEMAN/Primary Examiner, Art Unit 2675
Prosecution Insights