DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This is in response to applicant’s amendment/response filed on 01/13/2026, which has been entered and made of record. Claims 1, 8, and 15 have been amended. No Claim has been cancelled. No Claim has been added. Claims 1-20 are pending in the application.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 8, and 15, and the dependent claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant’s arguments directed to amended limitation have been addressed in the detail rejection below with new reference by Guo et al.
The arguments regarding dependent claims for the virtue of their dependency are moot because the independent claims are not allowable.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8-13, and 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shechtman et al. (US 20130287318 A1), and in view of Sadalgi et al. (US 20220084296 A1), and further in view of Guo et al. (US 20230245448 A1).
Regarding Claim 15, Shechtman discloses A system (¶4 reciting “an automatic system”. Fig. 24) comprising:
one or more memories storing instructions; (Fig. 24 showing memory 1020. ¶193 reciting “System memory 1020 may be configured to store program instructions”) and
one or more processors for executing the instructions to: (¶186 reciting “The computer-readable storage medium may store program instructions executable by the one or more processors to cause the computing apparatus to perform the camera calibration technique and/or the reprojection technique, as described herein”. Further, ¶193 reciting “System memory 1020 may be configured to store program instructions and/or data accessible by processor 1010.”)
identify, based on a two-dimensional (2D) input scene, one or more line segments included in the input scene; (¶53 reciting “FIG. 3 is a flowchart of a reprojection technique for correcting an image of a scene such as a digital or digitized photograph”. Further, ¶58 reciting “As indicated at 300, a set of line segments from an input image may be obtained. A low-level line detection technique may be applied to an input image (or to each image in an input set of images, such as the frames of a video sequence) to detect lines and edges in the image.”)
generate one or more vanishing points associated with the input scene based on the one or more line segments; (¶58 reciting “As indicated at 320, an energy function may be iteratively optimized to simultaneously estimate camera intrinsic parameter matrix "K," orientation matrix "R" (e.g., a rotation matrix that may be used to describe a rotation in two dimensional or three dimensional space), and vanishing points for the input image.”)
estimate one or more scene parameters associated with the scene; (¶58 reciting “As indicated at 320, an energy function may be iteratively optimized to simultaneously estimate camera intrinsic parameter matrix "K," orientation matrix "R" (e.g., a rotation matrix that may be used to describe a rotation in two dimensional or three dimensional space), and vanishing points for the input image.”)
However, Shechtman does not explicitly disclose to estimate scene parameters based on the one or more vanishing points; and insert a world object into the input scene based on the one or more scene parameters.
Sadalgi teaches “a method for generating a two-dimensional (2D) image of one or more products within a physical scene is provided.” (ABST). ¶206 teaches obtaining one or more camera setting values, and recites “The system may be configured to estimate the optical center and the focal length of the camera using one or more vanishing points in the image of the physical scene.” Further, ¶219 teaches to configure the virtual camera such that it renders the 2D image to replicate capture of an image of a physical scene by a camera of the device based on obtained camera setting values. Furthermore, ¶224 teaches inserting a 2D image of a world product into the scene, and recites “the system may be configured to render the 2D image by applying a ray-tracing technique . . . By applying raytracing using the virtual camera (e.g., configured as described at block 312) to the 3D scene generated at blocks 302-310”. Thus Sadalgi teaches obtaining camera parameters based on one or more vanishing points, and inserting a world object (a product) into the scene based on the obtained camera parameters.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the System (taught by Shechtman) to obtain camera parameters based on one or more vanishing points, and insert a world object (a product) into the scene based on the obtained camera parameters (taught by Sadalgi). The suggestions/motivations would have been to “provide more accurate, photorealistic visualizations while reducing the overall computational resources needed by the user's device” (¶102), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Shechtman discloses to identify, based on a two-dimensional (2D) input scene, one or more line segments included in the input scene.
However, Shechtman in view of Sadalgi does not explicitly disclose transforming a two-dimensional (2D) input scene from an image space representation into a latent feature space representation; performing a search of the latent feature space representation based at least on a latent feature vector representation of a straight line; identifying the line segments based at least on one or more results obtained from the search.
Guo teaches “The systems and methods described herein may be configured to encode the input image into a feature vector in an embedding space. The closest template feature vector in the processor determines the current step previously saved feature bank determines the current step.” (¶29). Further, ¶85 recites “At 504, the method 500 receives the captured image. At 506, the method 500 received image may be converted into an oriented edge map via holistically-nested edge detection. That is followed by edge-based non-maximum suppression and edge orientation computation. At 508, the method 500 queries the resulting edge map against all the available templates in the template storage 110. . . The template that most closely matches the captured image is determined; the matched template is designated as the initial prediction”. Therefore, Guo teaches transforming an input scene into a latent feature space representation (i.e. feature vector); performing a search of the feature space representation of previously saved template; and identifying the closest match.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to adapt the method taught by Guo for identifying line segments included in the input scene taught by Shechtman. The suggestions/motivations would have been “to provide robust and accurate results” (¶37), and to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Regarding Claim 16, Shechtman in view of Sadalgi and Guo discloses The system of claim 15, wherein the input scene is captured by a camera and the one or more scene parameters include intrinsic camera parameters. (Shechtman, ABST disclosing input digital image adjustment according to a camera calibration technique. ¶58 reciting “As indicated at 320, an energy function may be iteratively optimized to simultaneously estimate camera intrinsic parameter matrix "K,"”)
Regarding Claim 17, Shechtman in view of Sadalgi and Guo discloses The system of claim 16, wherein the intrinsic camera parameters include a relative focal length associated with the camera and a principal point associated with the camera. (Shechtman, ¶58 reciting “The camera intrinsic parameter matrix "K" may describe values such as a focal length, pixel size in one or more directions (e.g., "x" and "y" directions), and a principal point (e.g., which may be defined using "x" and "y" coordinates).”)
Regarding Claim 18, Shechtman in view of Sadalgi and Guo discloses The system of claim 15, wherein the input scene is captured by a camera and the one or more scene parameters include extrinsic camera parameters. (Shechtman, ¶145 reciting “In some embodiments, the server 104 may be configured to obtain camera information for an obtained image of a physical scene. The camera information may include camera settings. For example, the camera settings may include a camera exposure offset (EV) and/or a camera field of view. In another example, the camera information may include a rotation angle of a panorama relative to a camera position (e.g., determined from a 3D scene).” The suggestions/motivations would have been the same as that of Claim 15 rejections.)
Regarding Claim 19, Shechtman in view of Sadalgi and Guo discloses The system of claim 18, wherein the extrinsic camera parameters include a camera position and a camera orientation. (Shechtman, ¶145 reciting “In some embodiments, the server 104 may be configured to obtain camera information for an obtained image of a physical scene. The camera information may include camera settings. For example, the camera settings may include a camera exposure offset (EV) and/or a camera field of view. In another example, the camera information may include a rotation angle of a panorama relative to a camera position (e.g., determined from a 3D scene).” The suggestions/motivations would have been the same as that of Claim 15 rejections.)
Regarding Claim 20, Shechtman in view of Sadalgi and Guo discloses The system of claim 15, wherein the world object includes a 2D representation of a three-dimensional (3D) object, one or more real-world size dimensions associated with the world object, and a desired insertion point expressed as a 2D location within the input scene. (Sadalgi, ABST reciting “rendering a 2D image of a second product in the physical scene using the image of the physical scene, the position information, and a 3D model of the second product;” In addition, ¶199 reciting “the system may be configured to determine a location in the 3D scene at which to position a 3D product model by using dimensions of a boundary (e.g., a box) enclosing the 3D product model.”)
Claim 1, has similar limitations as of Claim(s) 15, therefore it is rejected under the same rationale as Claim(s) 15.
Claim 2, has similar limitations as of Claim(s) 16, therefore it is rejected under the same rationale as Claim(s) 16.
Claim 3, has similar limitations as of Claim(s) 17, therefore it is rejected under the same rationale as Claim(s) 17.
Claim 4, has similar limitations as of Claim(s) 18, therefore it is rejected under the same rationale as Claim(s) 18.
Claim 5, has similar limitations as of Claim(s) 19, therefore it is rejected under the same rationale as Claim(s) 19.
Claim 6, has similar limitations as of Claim(s) 20, therefore it is rejected under the same rationale as Claim(s) 20.
Claim 8, has similar limitations as of Claim(s) 15, therefore it is rejected under the same rationale as Claim(s) 15.
Claim 9, has similar limitations as of Claim(s) 16, therefore it is rejected under the same rationale as Claim(s) 16.
Claim 10, has similar limitations as of Claim(s) 17, therefore it is rejected under the same rationale as Claim(s) 17.
Claim 11, has similar limitations as of Claim(s) 18, therefore it is rejected under the same rationale as Claim(s) 18.
Claim 12, has similar limitations as of Claim(s) 19, therefore it is rejected under the same rationale as Claim(s) 19.
Claim 13, has similar limitations as of Claim(s) 20, therefore it is rejected under the same rationale as Claim(s) 20.
Claim(s) 7 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shechtman et al. (US 20130287318 A1), and in view of Sadalgi et al. (US 20220084296 A1) and Guo, and further in view of Zagaynov et al. (US 20190197693 A1).
Regarding Claim 14, Shechtman in view of Sadalgi and Guo discloses The one or more non-transitory computer-readable media of claim 8.
However, Shechtman in view of Sadalgi and Guo does not explicitly disclose wherein generating the one or more vanishing points further comprises classifying, via a machine learning model, each of the one or more line segments based on a horizontal or vertical orientation associated with the line segment.
Zagaynov teaches “receiving an image comprising one or more documents and detecting a set of lines in the image; identifying a plurality of intersection points corresponding to the set of lines; determining a vanishing point based on the plurality of intersection point” (ABST). More specifically, ¶62 recites “a first vanishing point may be determined based on a set of horizontal lines detected in the image and a second vanishing point may be determined based on a set of vertical lines detected in the image, as discussed in more detail below in regards to FIG. 13.” Further, ¶91 recites “there are two different line sets (e.g., horizontal line set and vertical line set) and each line set may used to determine a respective set of intersection points and vanishing points.” Thus. Zagaynov teaches grouping horizontal lines and vertical lines into two set for identifying vanishing points. It is well known in the art to classify features via machine learning. In addition, ¶62 teaches determining vanishing point via machine learning.
It would have been obvious to one with ordinary skill, before the effective filing date of the claimed invention, to modify the system (taught by Shechtman in view of Sadalgi and Guo) to classify line segments based on horizontal or vertical orientation via machine learning, for vanishing point identification (taught by Zagaynov). The suggestions/motivations would have been to apply a known technique to a known device (method, or product) ready for improvement to yield predictable results.
Claim 7, has similar limitations as of Claim(s) 14, therefore it is rejected under the same rationale as Claim(s) 14.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YI WANG whose telephone number is (571)272-6022. The examiner can normally be reached 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YI WANG/Primary Examiner, Art Unit 2619