DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/17/2026 has been entered.
Response to Arguments
As an initial matter, the 35 USC 112 rejections set forth in the previous office action have been withdrawn in view of Applicant's amendments and arguments.
Applicant's arguments regarding the 35 USC 103 rejections with respect to the amended claims 1-6, 8-16, 18-20, 29 have been fully considered but they are not persuasive.
Applicant argues in page 8 against Kim that “Identifying bins of points larger than a predefined size does not suggest identifying flat surfaces having a size greater than a size threshold.”. The Examiner contests that this appears to be Applicant’s own conclusion, rather than evidence. In contrast Kim clearly teaches the feature of “wherein the first set of the points is identified based on identifying a flat surface having a size greater than a size threshold” in Figs 4-6, abstract, page 289 col 2-page 290 col 1, wherein portions of the input point cloud satisfying size threshold and/or angular similarity criteria substituted as planar mesh, wherein each of the bin represents the points within a cluster that are selected based on the angle and size of the bin clusters contributing to the planar surface with better visual quality, eliminating small object (page 289 col 2) for the mesh representation. The newly cited reference Zolanvari describes typical/known methods of segmentation and feature extraction from point clouds that includes applying predefined thresholds/constraints/criteria on characteristics including size, shape, etc to remove noise, small objects etc and ensure high quality surface/object/feature representations (see page 335 cols 1-2).
Applicant's remaining arguments regarding the 35 USC 103 rejections with respect to the amended limitations of “and shape that satisfies a shape criterion and determining that each point of the first set of the points represents a portion of the flat surface based on determining a respective distance between each point of the first set of points and the flat surface;” in the claims 1-6, 8-16, 18-20, 29 have been considered but are moot in view of the new ground(s) of rejection to teach the amended limitations necessitated by the amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4-6, 9-12, 14-16, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al (Kim, Han-Ul, Jin-Hwan Kim, and Chang-Su Kim. "Hybrid representation and rendering of indoor environments using meshes and point clouds." 2014 11th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI). IEEE, 2014. Applicant cited), in view of Zolanvari et al (Zolanvari, SM Iman, and Debra F. Laefer. "Slicing Method for curved façade and window extraction from point clouds." ISPRS Journal of Photogrammetry and Remote Sensing 119 (2016): 334-346.), and further in view of Bell et al (US 20190051050 A1).
RE claim 1, Kim teaches A method comprising: at a processor of a first device:
generating a three-dimensional (3D) representation of a physical environment, the 3D representation comprising points each having a 3D location and representing a portion of the physical environment (Abstract, Figs 1-5, page 290 cols 1-2);
identifying a first set of the points of the 3D representation satisfying a replacement criterion; and replacing the first set of the points in the 3D representation with a planar element depicting a surface of the physical environment wherein the first set of the points is identified based on identifying a flat surface having a size greater than a size threshold (Figs 4-6, abstract, page 289 col 2-page 290 col 1, wherein portions of the input PC satisfying size threshold and/or angular similarity criteria substituted as planar mesh).
Kim is silent RE and shape that satisfies a shape criterion and determining that each point of the first set of the points represents a portion of the flat surface based on determining a respective distance between each point of the first set of points and the flat surface;
However Zolanvari teaches in page 335 cols 1-2 as well known methods to extract features and planes from a facade point cloud. This is readily available or can equally applied in Kim to effectively extract the planar surface and surface features from the point cloud.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Kim a system and method of shape that satisfies a shape criterion and determining that each point of the first set of the points represents a portion of the flat surface based on determining a respective distance between each point of the first set of points and the flat surface, as suggested by Zolanvari, in order to effectively extract the planar surface and features from the point cloud and thereby ensuring system effectiveness and user experience.
Kim as modified by Zolanvari is silent RE: the planar element having an appearance provided by one or more images.
However Bell teaches in Figs 7-8, abstract, [0115], [0127]-[0131] to facilitate transitioning between 2D models or images and 3D models allowing multiple viewing options. This is readily available or can equally applied in Kim as modified by Zolanvari to effectively utilize the texture images in fig 4 for color/texture mapping the selected planar areas.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Kim as modified by Zolanvari a system and method of the planar element having an appearance provided by one or more images, as suggested by Bell, in order to effectively color/texture mapping the selected planar areas and thereby increasing system effectiveness and user experience.
RE claim 2, Kim as modified by Zolanvari and Bell teaches wherein an appearance of the planar element in the view is based on the one or more images of the surface (Bell teaches in Figs 7-8, abstract, [0039], [0051]-[0052], [0115], [0127]-[0131] etc).
Kim as modified by Zolanvari and Bell is silent RE further comprising providing the 3D representation to a second device different than the first device, wherein the second device provides a view based on the points and the planar element.
However Bell teaches in Figs 4, 14, [0041], [0172] etc.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Kim as modified by Zolanvari and Bell a system and method for providing the 3D representation to a second device different than the first device, wherein the second device provides a view based on the points and the planar element, as suggested by Bell, in order to allow communicate and sharing the view within multiple devices and thereby increasing system effectiveness and user experience.
RE claim 4, Kim as modified by Zolanvari and Bell teaches where the surface is a flat surface, wherein the planar element comprises a mesh defining the flat surface, wherein an appearance of the mesh is defined based on the one or more images (Kim Figs 4-6, abstract, page 289 col 2-page 290 col 1, Bell abstract, Figs 7-8, abstract, [0039], [0051]-[0052], [0115], [0127]-[0131] etc).
RE claim 5, Kim as modified by Zolanvari and Bell teaches wherein the planar element is defined using less data than the first set of the points (Kim page 289 col 2-page 290 col 1).
RE claim 6, Kim as modified by Zolanvari and Bell teaches wherein the first set of the points is identified by: determining a confidence value that each point of the first set of the points represents a portion of a flat surface; and determining that the confidence value of each point of the first set of the points exceeds a threshold (Kim Fig 2, page 290 col 1, in addition Bell [0073]-[0075]).
RE claim 9, Kim as modified by Zolanvari and Bell teaches further comprising updating an appearance of the planar element as new sensor data is obtained (Bell [0052], [0054], [0108], [0139], [0146]).
RE claim 10, Kim as modified by Zolanvari and Bell teaches wherein updating the appearance of the planar element is based on determining that the new sensor data satisfies an image criterion (Bell [0053]).
Claims 11-12, 14-16, 19 recite limitations similar in scope with limitations of claim 1-2, 4-6, 10 and therefore rejected under the same rationale. In addition Kim as modified by Zolanvari and Bell teaches A first device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium (Kim abstract, method performed by typical computing device as Bell Fig 1, [0044]).
Claims 20 recite limitations similar in scope with limitations of claim 1 and therefore rejected under the same rationale. In addition Kim as modified by Zolanvari and Bell teaches A non-transitory computer-readable storage medium storing program instructions executable via one or more processors (Kim abstract, method instructions stored in typical CRM as Bell Fig 1, [0044]).
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kim as modified by Zolanvari and Bell, and further in view of Cui et al (US 20220036080 A1).
RE claim 3, Kim as modified by Zolanvari and Bell is silent RE wherein the first device and the second device concurrently provide views of the physical environment during a communication session.
However Cui teaches in Figs 3-4, [0007], [0044]-[0047].
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Kim as modified by Zolanvari and Bell a system and method wherein the first device and the second device concurrently provide views of the physical environment during a communication session, as suggested by Cui, in order to allow viewing concurrently with the second device and thereby increasing system effectiveness and user experience.
Claim 13 recites limitations similar in scope with limitations of claim 3 and therefore rejected under the same rationale.
Claims 8 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kim as modified by Zolanvari and Bell, and further in view of Kaiser et al (KAISER ET AL: "Geometric Proxies for Live RGB-D Stream Enhancement and Consolidation", ARXIV.ORG, 21 January 2020, XP081582810, Applicant cited).
RE claim 8, Kim as modified by Zolanvari and Bell is silent RE wherein the replacing occurs in real time during a scanning process in which: sensor data is obtained over a time period; active depth sensing and imaging are used to generate the points for inclusion in the 3D representation as sensor data is received during the time period; and points are selectively replaced with one or more planar elements based on the replacement criterion during the time period.
However Kaiser teaches generating and updating through time a single set of compact local statistics parameterized over detected geometric proxies, which are fed from raw RGB-D data in abstract, Figs 1, 4, 8, page 8 col 2- page 9 col 1 etc providing an enhanced 3D model in real time. This is readily available or can be equally applied by applying the metho steps of claim 1 over a time period, as Bell readily teaches capturing/updating 3D imagery including video, streaming and live navigation interface ([0041], [0045]-[0046], [0052]), as readily recognized by one of ordinary skill in the art.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Kim as modified by Zolanvari and Bell a system and method wherein the replacing occurs in real time during a scanning process in which: sensor data is obtained over a time period; active depth sensing and imaging are used to generate the points for inclusion in the 3D representation as sensor data is received during the time period; and points are selectively replaced with one or more planar elements based on the replacement criterion during the time period, as set forth above applying Kaiser, in order to provide real time processing and viewing and thereby increasing system effectiveness and user experience.
Claim 18 recites limitations similar in scope with limitations of claim 8 and therefore rejected under the same rationale.
Claim 29 is rejected under 35 U.S.C. 103 as being unpatentable over Kim as modified by Zolanvari and Bell, and further in view of Mishaev et al (US 20220036080 A1).
RE claim 29, Kim as modified by Zolanvari and Bell teaches wherein the first set of the points is identified based on identifying the flat surface based further on determining that the flat surface has sufficient flatness, lack of irregularities (Kim page 290 col 1, Zolanvari page 335 col 2-page 336 col 1-2 and Bell [0076], [0147] wherein sufficient flatness and lack of irregularities are defined quality metrics to ensure complete dataset and removing small/unwanted/holes features).
Kim as modified by Zolanvari and Bell is silent RE and transparency level.
However Mishaev teaches determine transparency level in abstract, [0177]- [0179] to improve surface layer detection.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include in Kim as modified by Zolanvari and Bell a transparency level, as suggested by Mishaev, in order to include the transparency level for improve the surface detection and thereby increasing system effectiveness and user experience.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure (See attached 892).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SULTANA MARCIA ZALALEE whose telephone number is (571)270-1411. The examiner can normally be reached Monday- Friday 8:00am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Sultana M Zalalee/ Primary Examiner, Art Unit 2614
Prosecution Insights