DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This communication is in response to the Application Filed on 03/06/2026
Claims 1–10, 12 and 21–34 are pending in this application.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/22/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Amendment
Applicant’s Amendments filed on 03/06/2026 has been entered and made of record.
Currently pending Claim(s):
Independent Claim(s):
Amended Claim(s):
Cancelled Claim(s):
New Claim(s):
1–10, 12 and 21–34
1, 21 and 28
1
11 and 13–20
21–34
Response to Applicant’s Arguments
This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment received on 03/06/2026.
In view of the amendments filed on 03/06/2026 to the drawing, the drawing objection is withdrawn.
In view of applicant Arguments/Remarks and amendment filed on 03/06/2026 with respect to independent claims 1, 21 and 28 under 35 U.S.C 103, claim rejection has been fully considered and the arguments are found to be persuasive (See Page 11–15), therefore the claim rejection with respect to 35 U.S.C. 103 still applies.
Applicant’s Reply (March 6th, 2026) includes amendments to the claims. This Office action has been updated with a new grounds of rejection addressing those amendments that have broadened the scope of the claims by incorporating only part of limitations for claim 11 into 1 and not incorporating any of the limitations of claim 10. Further Applicant’s Arguments/Remarks with respect to independent claims 1, 21 and 28 has been considered but are moot because the arguments do not apply to any of the references being used in the current rejection and the arguments are now rejected by newly cited art Totty et al. (US 20220020210 A1) as explained in the body of the rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claim(s) 1–6, 10, 21–25, 27–32 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (See NPL attached, "Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image", hereafter, "Yang") in view of Phalak et al. (US 20210279950 A1, hereafter, "Phalak") further in view of Totty et al. (US 20220020210 A1, hereafter, “Totty”).
Regarding claim 1, Yang teaches a method executed by one or more computing devices for layout extraction (See Yang, [Abstract], Single-image room layout reconstruction aims to reconstruct the enclosed 3D structure of a room from a single image), the method comprising:
[storing a plurality of scene priors corresponding to an image of a scene, the plurality of scene priors comprising a semantic map indicating semantic labels associated with a plurality of pixels in the image], geometry information corresponding to the plurality of pixels in the image (See Yang, [3.1. Plane and Line Detection]. To reconstruct the 3D room layout, we further estimate the 3D parameters for each plane. The 3D plane parameters include its surface normal
n
∈
S
2
and offset d. Note: Examiner is interpreting the 3D parameters as geometry information), and one or more line segments corresponding to the scene (See Yang, [3.1. Plane and Line Detection], By introducing the virtual plane passing through the occlusion line, all adjacent walls in the image space are physically connected in the 3D space. Thus, we directly calculate the boundary of adjacent walls with their 3D parameters. Note: Examiner is interpreting the vertical line between walls as the border);
generating one or more borders based on the one or more line segments, each border representing a separation between two layout planes in a plurality of layout planes of the scene (See Yang, [3.1. Plane and Line Detection], By introducing the virtual plane passing through the occlusion line, all adjacent walls in the image space are physically connected in the 3D space. Thus, we directly calculate the boundary of adjacent walls with their 3D parameters. Note: Examiner is interpreting the vertical line between walls as the border); and
[generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene based at least in part on superimposing the semantic map on the one or more borders].
However, Yang fail(s) to teach storing a plurality of scene priors corresponding to an image of a scene, the plurality of scene priors comprising a semantic map indicating semantic labels associated with a plurality of pixels in the image; generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene based at least in part on superimposing the semantic map on the one or more borders.
Phalak, working in the same field of endeavor, teaches: storing a plurality of scene priors corresponding to an image of a scene, the plurality of scene priors comprising a semantic map indicating semantic labels associated with a plurality of pixels in the image (See Phalak, ¶ [0270], One or more local features capturing geometric structures may be extracted at 1504D at least by recursively performing semantic feature extraction on nested partitioning of the set of points. A PointNet-based module may be employed to extract local features or points).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference to storing a plurality of scene priors corresponding to an image of a scene, the plurality of scene priors comprising a semantic map indicating semantic labels associated with a plurality of pixels in the image based on the method of Phalak’s reference. The suggestion/motivation would have been to efficiently and accurately generate the layout of an indoor scene (See Phalak, ¶ [0146]).
However, Yang and Phalak fail(s) to teach generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene based at least in part on superimposing the semantic map on the one or more borders.
Totty, working in the same field of endeavor, teaches: generating a plurality of plane masks (See Totty, ¶ [0117], See Totty, ¶ [0117], Refining segmentation masks S530, which can function to determine refined segmentation labels per pixel (or a subset thereof) in one or more models) corresponding to the plurality of layout planes that estimate the geometry of the scene based at least in part on superimposing the semantic map on the one or more borders (See Totty, ¶ [0117], In a specific example, S530 can be performed based on refined planes from S510 and/or S560, one or more prior-enhanced geometric representations from S520, high level information from S400 and/or any other suitable information. ¶ [0096], S560 can then include determining the wall distance away from the virtual camera (e.g., to scale the plane equations), …, In a second example, the lines associated with a wall-floor intersection (e.g., determined by combining semantic segmentation from S530 with lines from S410) can be used to determine the wall distance by calculating the wall distance based on the wall-floor intersection and the (scaled) floor plane equation (e.g., from S510). Note: Examiner is interpreting the combining the semantic segmentation with lines as superimposing the semantic map with the borders).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference to generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene based at least in part on superimposing the semantic map on the one or more borders based on the method of Totty’s reference. The suggestion/motivation would have been to accurately reconstruct the indoor scenery to combine 3D geometry with photorealism (See Totty, ¶ [0002–0005]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Phalak and Totty with Yang to obtain the invention as specified in claim 1.
Regarding claim 2, Yang teaches the method of claim 1, wherein the plurality of layout planes comprise at least one of a planar plane or a curved plane (See Yang, [3.1. Plane and Line Detection], Then, we use different CNN-based heads to detect the planes, vertical lines between adjacent walls and regress 3D parameters of planes, respectively).
Regarding claim 3, Yang teaches the method of claim 1, wherein the semantic labels comprise at least one of a wall, a ceiling, or a floor (See Yang, [3.1. Plane and Line Detection], Each channel of the center likelihood map C represents semantic different categories, i.e., wall, floor, and ceiling).
Regarding claim 4, Yang teaches the method of claim 1, wherein the image is a red-green-blue (RGB) image (See Yang, [3. Our Method], Our goal is to reconstruct the 3D room layout from a single RGB image).
Regarding claim 5, Yang teaches the method of claim 1, wherein the plurality of scene priors further comprises one or more of: a gravity vector corresponding to the scene;
an edge map corresponding to a plurality of edges in the scene;
a normal map corresponding to a plurality of normals in the scene;
camera parameters of a camera configured to capture the image (See Yang, [3.2. 3D Layout Reconstruction], Next, we calculate the intersection line of two adjacent walls by their 3D parameters and project it into image space with the known camera intrinsic matrix. Note: only one needs to be taught);
or an orientation map corresponding to a plurality of orientation values in the scene.
Regarding claim 6, Yang in view of Phalak further in view of Totty teaches the method of claim 1, wherein the geometry information comprises one or more of: [a depth map corresponding to the plurality of pixels];
photogrammetry points corresponding to a plurality of three-dimensional point values in the plurality of pixels;
a sparse depth map corresponding to the plurality of pixels;
a plurality of depth pixels storing both color information and depth information. a mesh representation corresponding to the plurality of pixels;
a voxel representation corresponding to the plurality of pixels;
or depth information associated with one or more polygons corresponding to the plurality of pixels.
However, Yang fail(s) to teach a depth map corresponding to the plurality of pixels.
Phalak, working in the same field of endeavor, teaches: a depth map corresponding to the plurality of pixels (See Phalak, ¶ [0367], A depth map and a wall segmentation mask may be generated at 1504E by using, for example, a multi-view depth estimation network and a PSPNet-based and/or a Resnet-based segmentation module. In some embodiments, a per-frame dense depth map may be generated at 1502E with, for example, a Multiview depth estimation network. Note: only one needs to be taught).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference a depth map corresponding to the plurality of pixels based on the method of Phalak’s reference. The suggestion/motivation would have been to efficiently and accurately generate the layout of an indoor scene (See Phalak, ¶ [0146]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Phalak with Yang and Totty to obtain the invention as specified in claim 6.
Regarding claim 10, Yang in view of Phalak further in view of Totty teaches the method of claim 1, [wherein the generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene comprises: generating a plurality of initial plane masks, the plurality of initial plane masks corresponding to the plurality of layout planes];
generating a plurality of plane connectivity values based at least in part on the one or more borders, each plane connectivity value indicating connectivity between two layout planes in the plurality of layout planes (See Yang, [3.2. 3D Layout Reconstruction], Next, we calculate the intersection line of two adjacent walls by their 3D parameters and project it into image space with the known camera intrinsic matrix. [3.2. 3D Layout Reconstruction], Each potential intersection line region has at most one detected line, and we choose the one with the highest confidence when there are multiple detected lines. Note: The examiner is interpreting the intersection line confidence as the connectivity value as the intersection line connects two adjacent walls (e.g., planes) and the confidence is how likely there is a line that connects two planes); and
[refining the plurality of initial plane masks based at least in part on an estimated geometry of the plurality of layout planes, the estimated geometry based at least in part on at least one of at least one of the plurality of scene priors, the plurality of initial plane masks, and the plurality of connectivity values].
However, Yang fail(s) to teach wherein the generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene comprises: generating a plurality of initial plane masks, the plurality of initial plane masks corresponding to the plurality of layout planes; refining the plurality of initial plane masks based at least in part on an estimated geometry of the plurality of layout planes, the estimated geometry based at least in part on at least one of at least one of the plurality of scene priors, the plurality of initial plane masks, and the plurality of connectivity values.
Totty, working in the same field of endeavor, teaches: wherein the generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene (See Totty, ¶ [0097], Segmenting the scene information S420 functions to determine segmentation mask(s) for the scene. For example, S420 can determine a class label (e.g., “floor”, “wall”, “couch”, etc.)) comprises: generating a plurality of initial plane masks, the plurality of initial plane masks corresponding to the plurality of layout planes (See Totty, ¶ [0097], Segmenting the scene information S420 functions to determine segmentation mask(s) for the scene. For example, S420 can determine a class label (e.g., “floor”, “wall”, “couch”, etc.));
refining the plurality of initial plane masks based at least in part on an estimated geometry of the plurality of layout planes, the estimated geometry based at least in part on at least one of at least one of the plurality of scene priors, the plurality of initial plane masks, and the plurality of connectivity values (See Totty, ¶ [0117], Refining segmentation masks S530, which can function to determine refined segmentation labels per pixel (or a subset thereof) in one or more models, …, In a specific example, S530 can be performed based on refined planes from S510 and/or S560, one or more prior-enhanced geometric representations from S520, high level information from S400 and/or any other suitable information. ¶ [0112], S520 can be performed based on one or more scene priors. Note: Examiner is teaching refining the intial planes based on the geometry of the layout planes and the estimated geometry based scene priors).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference wherein the generating a plurality of plane masks corresponding to the plurality of layout planes that estimate the geometry of the scene comprises: generating a plurality of initial plane masks, the plurality of initial plane masks corresponding to the plurality of layout planes; refining the plurality of initial plane masks based at least in part on an estimated geometry of the plurality of layout planes, the estimated geometry based at least in part on at least one of at least one of the plurality of scene priors, the plurality of initial plane masks, and the plurality of connectivity values based on the method of Totty’s reference. The suggestion/motivation would have been to accurately reconstruct the indoor scenery to combine 3D geometry with photorealism (See Totty, ¶ [0002–0005]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Totty with Yang and Phalak to obtain the invention as specified in claim 10.
Regarding claim 21, claim 21 is rejected the same as claim 1 and the arguments similar to that presented above for claim 1 are equally applicable to the claim 21, and all of the other limitations similar to claim 1 are not repeated herein, but incorporated by reference. Furthermore, Yang teaches [An apparatus for layout extraction, the apparatus comprising: one or more processors; and one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors]
However, Yang fail(s) to teach an apparatus for layout extraction, the apparatus comprising: one or more processors; and one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors.
Phalak, working in the same field of endeavor, teaches: An apparatus for layout extraction, the apparatus comprising: one or more processors; and one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors (See Phalak, ¶ [0021], According to some embodiments, a system having a processor and memory is provided).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference An apparatus for layout extraction, the apparatus comprising: one or more processors; and one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors based on the method of Phalak’s reference. The suggestion/motivation would have been to efficiently and accurately generate the layout of an indoor scene (See Phalak, ¶ [0146]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Phalak with Yang and Totty to obtain the invention as specified in claim 21.
Regarding claim 22, claim 22 is rejected the same as claim 2 and the arguments similar to that presented above for claim 2 are equally applicable to the claim 22, and all of the other limitations similar to claim 2 are not repeated herein, but incorporated by reference.
Regarding claim 23, claim 23 is rejected the same as claim 3 and the arguments similar to that presented above for claim 3 are equally applicable to the claim 23, and all of the other limitations similar to claim 3 are not repeated herein, but incorporated by reference.
Regarding claim 24, claim 24 is rejected the same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to the claim 24, and all of the other limitations similar to claim 5 are not repeated herein, but incorporated by reference.
Regarding claim 25, claim 25 is rejected the same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to the claim 25, and all of the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Regarding claim 27, claim 27 is rejected the same as claim 10 and the arguments similar to that presented above for claim 10 are equally applicable to the claim 27, and all of the other limitations similar to claim 10 are not repeated herein, but incorporated by reference.
Regarding claim 28, claim 28 is rejected the same as claim 1 and the arguments similar to that presented above for claim 1 are equally applicable to the claim 28, and all of the other limitations similar to claim 1 are not repeated herein, but incorporated by reference. Furthermore, Yang teaches [at least one non-transitory computer-readable medium storing computer- readable instructions for layout extraction that, when executed by one or more computing devices, cause at least one of the one or more computing devices to]
However, Yang fail(s) to teach at least one non-transitory computer-readable medium storing computer- readable instructions for layout extraction that, when executed by one or more computing devices, cause at least one of the one or more computing devices to.
Phalak, working in the same field of endeavor, teaches: at least one non-transitory computer-readable medium storing computer- readable instructions for layout extraction that, when executed by one or more computing devices, cause at least one of the one or more computing devices to (See Phalak, ¶ [0021], According to some embodiments, a system having a processor and memory is provided).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference at least one non-transitory computer-readable medium storing computer- readable instructions for layout extraction that, when executed by one or more computing devices, cause at least one of the one or more computing devices to based on the method of Phalak’s reference. The suggestion/motivation would have been to efficiently and accurately generate the layout of an indoor scene (See Phalak, ¶ [0146]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Phalak with Yang and Totty to obtain the invention as specified in claim 28.
Regarding claim 29, claim 29 is rejected the same as claim 2 and the arguments similar to that presented above for claim 2 are equally applicable to the claim 29, and all of the other limitations similar to claim 2 are not repeated herein, but incorporated by reference.
Regarding claim 30, claim 30 is rejected the same as claim 3 and the arguments similar to that presented above for claim 3 are equally applicable to the claim 30, and all of the other limitations similar to claim 3 are not repeated herein, but incorporated by reference.
Regarding claim 31, claim 31 is rejected the same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to the claim 31, and all of the other limitations similar to claim 5 are not repeated herein, but incorporated by reference.
Regarding claim 32, claim 32 is rejected the same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to the claim 32, and all of the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Regarding claim 34, claim 34 is rejected the same as claim 10 and the arguments similar to that presented above for claim 10 are equally applicable to the claim 34, and all of the other limitations similar to claim 10 are not repeated herein, but incorporated by reference.
Claim(s) 7, 26 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (See NPL attached, "Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image", hereafter, "Yang") in view of Phalak et al. (US 20210279950 A1, hereafter, "Phalak") further in view of Totty et al. (US 20220020210 A1, hereafter, “Totty”) and further in view of Gallagher (US 7,583,858 B2, hereafter, "Gallagher").
Regarding claim 7, Yang in view of Phalak further in view of Totty teaches the method of claim 1, wherein generating one or more borders based on the one or more line segments (See Yang, [3.1. Plane and Line Detection], By introducing the virtual plane passing through the occlusion line, all adjacent walls in the image space are physically connected in the 3D space. Thus, we directly calculate the boundary of adjacent walls with their 3D parameters. Note: Examiner is interpreting the vertical line between walls as the border) comprises computing an [orientation map by: detecting a horizontal line from a pixel of the plurality of pixels in the image;
calculating a vanishing point based on the horizontal line and a gravity vector associated with the pixel; and
combining the vanishing point with the gravity vector to determine a plurality of normal estimates].
However, Yang, Phalak and Totty fail(s) to teach orientation map by: detecting a horizontal line from a pixel of the plurality of pixels in the image; calculating a vanishing point based on the horizontal line and a gravity vector associated with the pixel; and combining the vanishing point with the gravity vector to determine a plurality of normal estimates.
Gallagher, working in the same field of endeavor, teaches: orientation map (See Gallagher, [Col. 7, ln. 17–19], In general, the transform 60 is created by determining preferred positions for the gravity vanishing point (and possibly additional vanishing points). Note: Examiner is interpreting the transforms as the orientation map) by: detecting a horizontal line from a pixel of the plurality of pixels in the image (See Gallagher, [Col. 6, ln. 40–43], For example, in a brick wall, the lines along rows of bricks define a horizontal vanishing point while the lines along columns of bricks are vertical scene lines defining a vertical vanishing point (coincident to the gravity vector));
calculating a vanishing point based on the horizontal line and a gravity vector associated with the pixel (See Gallagher, [Col. 6, ln. 43–50], A set of two vanishing points related to two orthogonal sets of lines (i.e. the vertical lines parallel to gravity and the horizontal lines parallel to the scene ground plane are orthogonal) define a vanishing line for planes parallel to both sets of lines. The data processor 20 then generates the transform 60 based on the gravity vector and possibly additional vanishing points 50 found with image analysis); and
combining the vanishing point with the gravity vector to determine a plurality of normal estimates (See Gallagher, [Col. 7, ln. 66–67 – Col. 8, ln. 1–8], The transform HIR is used to remove the tilt that is apparent in images when the camera is unintentionally rotated with respect to the scene (i.e. when the gravity vector is not orthogonal to the x-axis or y-axis of the imaging system). The angle a represents the negative of the angle of rotation of the camera from a vertical orientation, and the transform HIR is applied by the image processor 36 to produce an enhanced digital image 120 rotated by angle a relative to the original digital image 102, thereby removing the effect of undesirable rotation of the camera from the image).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference orientation map by: detecting a horizontal line from a pixel of the plurality of pixels in the image; calculating a vanishing point based on the horizontal line and a gravity vector associated with the pixel; and combining the vanishing point with the gravity vector to determine a plurality of normal estimates based on the method of Gallagher’s reference. The suggestion/motivation would have been to accurately avoid the effect of perspective distortion in the image (See Gallagher, [Col. 1, ln. 25–45 and Col. 6, ln. 50–62]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Gallagher with Yang, Phalak and Totty to obtain the invention as specified in claim 7.
Regarding claim 26, claim 26 is rejected the same as claim 7 and the arguments similar to that presented above for claim 7 are equally applicable to the claim 26, and all of the other limitations similar to claim 7 are not repeated herein, but incorporated by reference.
Regarding claim 33, claim 33 is rejected the same as claim 7 and the arguments similar to that presented above for claim 7 are equally applicable to the claim 33, and all of the other limitations similar to claim 7 are not repeated herein, but incorporated by reference.
Claim(s) 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (See NPL attached, "Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image", hereafter, "Yang") in view of Phalak et al. (US 20210279950 A1, hereafter, "Phalak") further in view of Totty et al. (US 20220020210 A1, hereafter, “Totty”) and further in view of Elmekies (US 20170069142 A1, hereafter, "Elmekies").
Regarding claim 8, Yang in view of Phalak further in view of Totty teaches the method of claim 1, wherein generating one or more borders based on the one or more line segments (See Yang, [3.1. Plane and Line Detection], By introducing the virtual plane passing through the occlusion line, all adjacent walls in the image space are physically connected in the 3D space. Thus, we directly calculate the boundary of adjacent walls with their 3D parameters. Note: Examiner is interpreting the vertical line between walls as the border) comprises: [detecting a first set of borders comprising lines that form seams between two walls];
detecting a second set of borders comprising lines that separate walls in the scene (See Yang, [3.2. 3D Layout Reconstruction], If two walls are physically connected in 3D space, we directly calculate the boundary with their 3D parameters. Furthermore, we expect to construct a geometrically consistent 3D room layout between detected planes and lines); and
[detecting a third set of borders comprising lines that separate walls from floors or ceilings in the scene].
However, Yang, Phalak and Totty fail(s) to teach detecting a first set of borders comprising lines that form seams between two walls; detecting a third set of borders comprising lines that separate walls from floors or ceilings in the scene.
Elmekies, working in the same field of endeavor, teaches: detecting a first set of borders comprising lines that form seams between two walls (See Elmekies, ¶ [0024], 3. Edge Categorization: In this stage, the merged edges are now each ranked as likely candidates for a seam between two surfaces in the target virtual room. The “target virtual space” is our assumed arrangement of walls and ceiling, e.g., in our current implementation, we start with a simple virtual box, the interiors of which represent the walls, floor and ceiling of an idealized room);
detecting a third set of borders comprising lines that separate walls from floors or ceilings in the scene (See Elmekies, ¶ [0021], So, in the ranking stage described here we determine the likelihood, e.g., that a given edge is the seam between the ceiling and the back wall. We do this by giving each edge a “score” that represents the likelihood that it is a given seam (i.e., back-wall/ceiling, back-wall/left-wall, etc.). Every edge is given a score based on a series of rules. For instance, one rule might say that edges that are recognized as being lower down in the room are more likely to be floor/wall seams than ceiling/wall seams).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference to detecting a first set of borders comprising lines that form seams between two walls; detecting a third set of borders comprising lines that separate walls from floors or ceilings in the scene based on the method of Elmekies’s reference. The suggestion/motivation would have been to accurately replicate an entire room or portion of room based on video or images (See Elmekies, ¶ [0009]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Elmekies with Yang, Phalak and Totty to obtain the invention as specified in claim 8.
Regarding claim 9, Yang in view of Phalak further in view of Totty teaches the method of claim 8, [wherein the detecting the first set of borders comprising lines that form seams between two walls comprises, for each line segment in the one or more line segments:
determining a first end and a second end of the line segment; and
determining whether the line segment forms a seam between two walls based at least in part on the first end and the second end and a normal map of the scene].
However, Yang, Phalak and Totty fail(s) to teach wherein the detecting the first set of borders comprising lines that form seams between two walls comprises, for each line segment in the one or more line segments: determining a first end and a second end of the line segment; and determining whether the line segment forms a seam between two walls based at least in part on the first end and the second end and a normal map of the scene.
Elmekies, working in the same field of endeavor, teaches: wherein the detecting the first set of borders comprising lines that form seams between two walls (See Elmekies, ¶ [0024], 3. Edge Categorization: In this stage, the merged edges are now each ranked as likely candidates for a seam between two surfaces in the target virtual room. The “target virtual space” is our assumed arrangement of walls and ceiling, e.g., in our current implementation, we start with a simple virtual box, the interiors of which represent the walls, floor and ceiling of an idealized room) comprises, for each line segment in the one or more line segments:
determining a first end and a second end of the line segment (See Elmekies, ¶ [0023], 2. Edge Merging: In the Edge Merging stage, line segments (Edges) that have been extracted from the image are compared and, if their end points and slopes are suitably close and/or have commonality, are merged to form longer segments); and
determining whether the line segment forms a seam between two walls based at least in part on the first end and the second end and a normal map of the scene (See Elmekies, ¶ [0024], 3. Edge Categorization: In this stage, the merged edges are now each ranked as likely candidates for a seam between two surfaces in the target virtual room. The “target virtual space” is our assumed arrangement of walls and ceiling, e.g., in our current implementation, we start with a simple virtual box, the interiors of which represent the walls, floor and ceiling of an idealized room).
Thus, it would have been obvious to one of ordinary skills in the art before the effective filing date of the claimed invention to modify Yang’s reference wherein the detecting the first set of borders comprising lines that form seams between two walls comprises, for each line segment in the one or more line segments: determining a first end and a second end of the line segment; and determining whether the line segment forms a seam between two walls based at least in part on the first end and the second end and a normal map of the scene based on the method of Elmekies’s reference. The suggestion/motivation would have been to accurately replicate an entire room or portion of room based on video or images (See Elmekies, ¶ [0009]).
Further, one skilled in the art could have combined the elements as described above by known method with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Elmekies with Yang, Phalak and Totty to obtain the invention as specified in claim 9.
Allowable Subject Matter
Claim(s) 12 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim(s) 12 contain subject matter that is not disclosed or made obvious in the cited art.
In regard to claim 12, when considering claim 12 as a whole, prior art of record fails to disclose or render obvious, alone or in combination:
“The method of claim 10, wherein the refining the plurality of initial plane masks based at least in part on an estimated geometry of the plurality of layout planes comprises: applying a non-linear optimization function based at least in part on the plurality of initial plane masks, the plurality of connectivity values, and at least one of the one or more scene priors to generate an initial estimated geometry of the plurality of layout planes, the initial estimated geometry comprising confidence values associated with the plurality of layout planes;
detecting and refining one or more low confidence layout planes in the plurality of layout planes in the initial estimated geometry having confidence values below a predetermined threshold to generate a refined estimated geometry; and
refining the plurality of initial plane masks based at least in part on the refined estimated geometry to generate the plurality of plane masks”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Horak (US 20150304558 A1) teaches The invention relates to a method of 3D reconstruction of a scene by means of 2D panoramic images of the scene, which comprises a step of processing these images. These images arise from a panoramic system moving in displacement along a determined trajectory, such that the image of at least one point of the scene is in at least 3 successive images obtained according to various directions; the step of processing these 2D successive images comprises the sub-steps: a) determining reconstruction planes in the scene to be reconstructed, b) determining, on the basis of pairs of panoramic images and for each pair, rectification planes corresponding to the reconstruction planes and projecting onto each of them a sector of each image of the pair, in a direct manner, so as to obtain two 2D rectified images, c) matching the two 2D rectified images so as to obtain an intermediate 3D reconstruction, d) transforming each intermediate 3D reconstruction into a 3D frame including the reconstruction planes so as to obtain a transformed intermediate 3D reconstruction, e) repeating steps b) to d) on the basis of a new pair of 2D panoramic images and of at least one other rectification plane, so as to obtain at least one other transformed intermediate 3D reconstruction, f) temporally fusing at least two transformed intermediate 3D reconstructions so as to obtain a 3D reconstruction of the scene.
Molyneaux et al. (US 20200372718 A1) teaches A method to efficiently update and manage outputs of real time or offline 3D reconstruction and scanning in a mobile device having limited resource and connection to the Internet is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data, in either single user applications or multi-user applications sharing and updating the same 3D reconstruction data.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DION J SATCHER whose telephone number is (703)756-5849. The examiner can normally be reached Monday - Thursday 5:30 am - 2:30 pm, Friday 5:30 am - 9:30 am PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DION J SATCHER/Patent Examiner, Art Unit 2676
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights