Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Oath/Declaration
The receipt of the Oath/Declaration is acknowledged.
Drawings
The drawing(s) filed on March 13, 2024 are accepted by the Examiner.
Status of Claims
Claims 1-20 are now pending in this application.
Priority
Receipt is acknowledged of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on March 13, 2024 is in compliance with the provisions on 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Amendments
Acknowledgment of receiving amendments to the claims, which were received by the Office on 03/06/2026.
Response to Arguments
Applicant's arguments filed 03/02/2026 have been fully considered but they are not persuasive.
In the remarks, applicant argues in substance:
Applicant argues:
The combination of Kouperman and Varekamp does not disclose determining a completeness of the first plurality of depth components, as specifically claimed in independent claims 1 and 11, specifically that the completeness refers to how much of the scene is captured in the depth components.
Examiner’s Response: The Examiner respectfully disagrees. Applicant’s assertion that “the amount of overlap between images provides no information with regard to the completeness of the depth information” is not persuasive and is based on a seemingly narrow characterization of the cited references.
Kouperman expressly teaches that overlapping fields of view between frames and cameras are used to register image data, align camera sequences, and determine a shared coordinate system for 3D reconstruction (Kouperman, [0044]-[0048], [0055]-[0068], [0070]-[0072]). In Kouperman, overlap is the mechanism by which the system determines whether the image data can be aligned and whether the scene is sufficiently covered to support 3D space generation and virtual viewpoint interpolation. Thus, overlap necessarily informs whether the captured views are adequate for reconstruction of scene geometry, which is directly relevant to depth completeness.
Varekamp further teaches evaluating depth data for completeness by determining whether the depth components are complete, undercomplete, or overcomplete, including by considering proximity of objects, visibility of camera-facing surfaces, occlusion, artifacts, and the sufficiency of depth coverage across multiple views (Varekamp, [0016]-[0019], [0026]-[0044], [0045]-[0054], [0077]-[0104]). Varekamp also teaches dynamically selecting fewer or additional camera pairs based on whether the previously generated depth data is overcomplete or undercomplete.
Applicant further argues that “the completeness refers to how much of the scene is captured in the depth components” does not overcome the rejection because Varekamp expressly states that completeness of depth components is determined by whether the depth information captures enough of the scene to render the frame without missing depth information, and further explains that completeness may be evaluated by assessing continuity, gaps, occlusions, missing depth information, and artifacts (Varekamp, [0016]-[0019], [0051]-[0054], [0077]-[0104]). Thus, the applicant’s statement is not a distinction over the prior art. It seems the teaching of Varekamp reads on the limitation.
Kouperman also teaches that overlapping fields of view and aligned camera sequences are used to obtain a sufficient 3D scene representation, which necessarily depends on how much of the scene is captured by the available views (Kouperman, [0044]-[0048], [0055]-[0068], [0070]-[0072]).
Accordingly, the cited references collectively teach that completeness is a measure of scene coverage in the depth components, and teach the claimed limitations. The rejection of independent claims 1 and 11 remains proper.
The combination of Kouperman and Varekamp does not disclose that if the first plurality of depth components is overcomplete, then the number of depth components in the second plurality of depth components is arranged to be smaller than the number of depth components in the first plurality of depth components, as claimed in independent claims 1 and 11.
Examiner’s Response: The Examiner respectfully disagrees. Applicant’s argument is not persuasive. Varekamp expressly teaches that if a first plurality of depth components is overcomplete, then a subsequent plurality of depth components is selected to be smaller than the first plurality (Varekamp, [0015]-[0018], [0051]-[0054], [0077]-[0104]). Varekamp further explains that an overcomplete depth set contains redundant or duplicated depth information and that, after analyzing completeness, the system selects a second set of images/camera pairs having fewer depth components when the first set is overcomplete. Thus, Varekamp directly discloses the very limitation to which applicant objects.
Kouperman supplements this teaching by disclosing the use of multiple cameras, overlapping views, and dynamic selection/alignment of camera sequences for 3D reconstruction, which supports the broader premise that not all available views need be used at all times (Kouperman, [0044]-[0048], [0055]-[0068], [0070]-[0072]).
Accordingly, the combination of Kouperman and Varekamp does disclose, and at a minimum render obvious, selecting a second plurality of depth components with fewer depth components than the first plurality when the first plurality is overcomplete.
The applicant has therefore not identified a deficiency in the rejection. The amended claims and the new claims are fully addressed in the rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kouperman et al. (US 2017/0094259 A1) (herein after referred to as Kouperman) in view of Varekamp et al. (US 2022/0148207 A1) (herein after referred to as Varekamp).
Regarding claim 1, Kouperman teaches a method comprising: obtaining a first plurality of images of a scene (Kouperman Fig 1, paragraph [0023], …Referring to Fig 1, one example of scene 100 with object 102 may be captured by an array of cameras…; [0032], [0074], [0087]-[0090]);
generating a first plurality of depth components based on the first plurality of images (Kouperman, paragraph [0032]-[0034], where the depth data and/or camera depth maps correspond to the recited depth components; [0050]-[0058], [0073-[0078]);
obtaining a second plurality of images of the scene (Kouperman, paragraph [0032]-[0034], [0056]-[0060], wherein the scenario where 1 out of 3 frames is considered to reduce computation load. [0070]-[0072]); and
generating a second plurality of depth components based on the second plurality
of images (Kouperman, paragraph [0058], where the load reducing strategy is used in case the minimum overlap is maintained with a reduced number of frames (1 out of 3 frames for instance; [0032]-[0034], [0056]-[0060], [0074]-[0082]). In this context, an overlap between consecutive frames above the minimum overlap corresponds to the recited overcomplete analysis.)
the first plurality of depth components is overcomplete (Kouperman, paragraph [0058], where the load reducing strategy is used in case the minimum overlap is maintained with a reduced number of frames (1 out of 3 frames for instance; [0032]-[0034], [0056]-[0060], [0074]-[0082]). In this context, an overlap between consecutive frames above the minimum overlap corresponds to the recited overcomplete analysis.)
While, Kouperman teaches overlap or coverage as a prerequisite for 3D reconstruction and alignment (Kouperman, [0032]-[0034], it does not explicitly teach determining a completeness of the first plurality of depth components, wherein the completeness refers to how much of the scene is captured in the depth components; nor
wherein if the first plurality of depth components is overcomplete then, the number of depth components in the second plurality of depth components is arranged to be smaller than the number of depth components in the first plurality of depth components.
In reference Varekamp, Varekamp discloses a method for processing depth maps that includes determining a completeness of the first plurality of depth components, wherein the completeness refers to how much of the scene is captured in the depth components (Varekamp, [0016]-[0019], [0034]-[0044], [0051]-[0054], [0101]-[0104], teaches evaluating depth maps for completeness, undercompleteness, overcompleteness, occlusion, artifacts, and confidence) and wherein if the first plurality of depth components is overcomplete then, the number of depth components in the second plurality of depth components is arranged to be smaller than the number of depth components in the first plurality of depth components (Varekamp, [0015]-[0018], [0051]-[0054], [0077]-[0104]).
These arts are analogous since they are both related to imaging devices that process depth maps. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Kouperman with the processing depth maps that includes wherein if the first plurality of depth components is overcomplete then, the number of depth components in the second plurality of depth components is arranged to be smaller than the number of depth components in the first plurality of depth components as seen in Varekamp to reduce redundancy of components for efficient storage and processing as seen in (Varekamp [0013], [0154], among other citations in the specification of Varekamp).
Regarding claim 2, the combination of Kouperman and Varekamp teaches the method of claim 1, wherein the first plurality of images are obtained with a first plurality of cameras (Kouperman Fig 2, paragraph [0024] static cameras 202, Fig 5, plurality of camera 502, paragraphs [0031-0032]),
wherein the determining comprised determining an indication of how close at least one of the objects in the scene are to the first plurality of cameras (Kouperman, paragraph [0023-0025, 0029, 0032, 0043], …synchronized cameras that are pre-positioned to a static position of the overlapping fields of view…).
Regarding claim 3, the combination of Kouperman and Varekamp teaches the method of claim 1, wherein the first plurality of depth components comprises a first depth map (Varekamp, paragraphs [0015, 0022-0023]),
wherein the first depth map is generated by performing depth estimation on at least two images of the first plurality of images (Varekamp, paragraphs [0015, 0022-0023]),
wherein the second plurality of depth components comprises a second depth map (Varekamp, paragraphs [0015, 0023, 0052]),
wherein the second depth map is generated by performing depth estimation on at least two images of the second plurality of images (Varekamp, paragraphs [0104, 0164]).
Regarding claim 4, the combination of Kouperman and Varekamp teaches the method of claim 1, further comprising determining if a camera-facing surface of any objects in the scene is not visible in the field of view of at least two cameras of the first plurality of cameras, wherein generating a second plurality of depth components is based on the determination of the visibility (Kouperman, paragraph [0023, 0033] and Varekamp, paragraph [0078, 0092]).
Regarding claim 5, the combination of Kouperman and Varekamp teaches the method of claim 1, wherein determining the completeness of the first plurality of depth components comprises determining whether any object in the scene is at least partly occluded in the first plurality of depth components (Varekamp, paragraph [0017, 0179]).
Regarding claim 6, the combination of Kouperman and Varekamp teaches the method of claim 1, wherein determining the completeness of the first plurality set of depth components comprises determining if the first plurality of depth components has any visual artifacts and/or depth artifacts (Kouperman, paragraphs [0050], Varekamp, paragraph [0078, 0092]).
Regarding claim 7, the combination of Kouperman and Varekamp teaches the method of claim 1, further comprising arranging a plurality of cameras so as to obtain the second plurality of images based on the completeness of the first plurality of depth components (Kouperman, paragraph [0056], where the overlap of at least 40% between consecutive frames corresponds to the recited completeness).
Regarding claim 8, the combination of Kouperman and Varekamp teaches the method of claim 7, wherein arranging the plurality of cameras comprises selecting a first plurality of cameras from a group of pluralities of cameras,
wherein each plurality of cameras of the group of pluralities of cameras is associated with a minimum distance at which an object in the scene is guaranteed to be within the field of view of at least two cameras in the first plurality group of cameras (Kouperman, paragraph [0023-0025, 0029, 0032, 0043], …synchronized cameras that are pre-positioned to a static position of the overlapping fields of view…).
Regarding claim 9, the combination of Kouperman and Varekamp teaches the method of claim 1, wherein, the first plurality of depth components is undercomplete then, the second plurality of depth components has a larger number of depth components than the first plurality of depth components (Varekamp, paragraph, [0052, 0104]).
Regarding claim 19, Kouperman teaches a method comprising: obtaining a first plurality of images of a scene from a plurality of cameras (Kouperman, [0032], [0074], [0087]-[0090], obtaining image data/sequences from multiple cameras capturing a common scene) wherein the plurality of cameras comprises a first set of pairs of cameras (Kouperman, [0044]-[0048], [0055]-[0068], teaches multiple cameras and overlapping fields of view).
However, Kouperman does not explicitly teach wherein the cameras of at least one pair of cameras in the first set are separated by a first minimum distance among the first set of pairs of cameras;
generating depth information based on images of at least one pair of cameras;
determining a minimum depth of a camera-facing surface in the images; selecting a second set of pairs of cameras based on the minimum depth,
wherein the second set of pairs of cameras include cameras of at least one pair of cameras in the second set that are separated by a second minimum distance among the second set of cameras,
wherein the second minimum distance is greater than the first minimum distance.
In reference Varekamp, Varekamp discloses a method for processing depth maps that includes wherein the cameras of at least one pair of cameras in the first set are separated by a first minimum distance among the first set of pairs of cameras (Varekamp, [0051]-[0054], [0077]-[0104], [123] Varekamp uses multiple camera pairs/views for depth generation and discusses choosing camera pairs based on depth conditions and view relationships);
generating depth information based on images of at least one pair of cameras (Varekamp, [0015]-[0019], [0051]-[0054], [0077]-[0104] teaches generating, updating, and refining depth maps/depth components based on images from different viewpoints);
determining a minimum depth of a camera-facing surface in the images; selecting a second set of pairs of cameras based on the minimum depth (Varekamp, [0016]-[0019], [0034]-[0044], [0051]-[0054], [0077]-[0104], teaches evaluating depth maps for completeness, visibility, occlusion, and scene coverage, which can include identifying the nearest/camera-facing depth content in the depth representation)
wherein the second set of pairs of cameras include cameras of at least one pair of cameras in the second set that are separated by a second minimum distance among the second set of cameras (Varekamp, [0016]-[0019], [0034]-[0044], [0051]-[0054], [0077]-[0104], supports changing the selection of camera pairs/views)
wherein the second minimum distance is greater than the first minimum distance (Varekamp, [0016]-[0019], [0034]-[0044], [0051]-[0054], [0077]-[0104], supports dynamic adjustment of which views/camera pairs are used based on depth completeness. This makes the “greater minimum distance” an obvious choice.
These arts are analogous since they are both related to imaging devices that process depth maps. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Kouperman with the wherein the cameras of at least one pair of cameras in the first set are separated by a first minimum distance among the first set of pairs of cameras; generating depth information based on images of at least one pair of cameras; determining a minimum depth of a camera-facing surface in the images; selecting a second set of pairs of cameras based on the minimum depth, wherein the second set of pairs of cameras include cameras of at least one pair of cameras in the second set that are separated by a second minimum distance among the second set of cameras, wherein the second minimum distance is greater than the first minimum distance as seen in Varekamp to reduce redundancy of components for efficient storage and processing as seen in (Varekamp [0013], [0154], among other citations in the specification of Varekamp).
Regarding claim 20, the combination of Kouperman and Varekamp teaches the method of claim 19, wherein, the second set of pairs of cameras is selected to include fewer cameras than the first set of pairs of cameras based on the minimum depth (Varekamp, 0015]-[0019], [0051]-[0054], [0077]-[0104, Varekamp teaches selecting a subsequent set with fewer depth components when the first set is overcomplete. Although Varekamp describes depth components rather than cameras, the teaching of reducing the amount of data used in a second pass based on depth conditions renders obvious selecting fewer camera pairs for a second set based on minimum depth).
Claim 10 is rejected for the same reasons as claim 1.
Claim 11 is rejected for the same reasons as claim 1.
Claim 12 is rejected for the same reasons as claim 2.
Claim 13 is rejected for the same reasons as claim 4.
Claim 14 is rejected for the same reasons as claim 7.
Claim 15 is rejected for the same reasons as claim 8.
Claim 16 is rejected for the same reasons as claim 3.
Claim 17 is rejected for the same reasons as claim 5.
Claim 18 is rejected for the same reasons as claim 9.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the Supervisory Patent Examiner should be directed to TWYLER L. HASKINS whose telephone number is (571)272-7406. The Supervisory Patent Examiner can normally be reached Monday-Thursday 6:00 am - 4:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the Supervisory Patent Examiner by telephone are unsuccessful, the Supervisory Patent Examiner’s Group Director, James Kramer can be reached at 571-272-6783. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TWYLER L HASKINS/ Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights