CTNF 18/876,910 CTNF 101726 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. Information Disclosure Statement The information disclosure statement (IDS) submitted on 12 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Specification 07-29-04 The disclosure is objected to because it contains an embedded hyperlink and/or other form of browser-executable code (see para. [0027], [0030]]. Applicant is required to delete the embedded hyperlink and/or other form of browser-executable code; references to websites should be limited to the top-level domain name without any prefix such as http:// or other browser-executable code. See MPEP § 608.01. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-7, 9, 12-13, and 15-21 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. (US 20210056751 A1; IDS REF) (herein after Zhao) in view of He et al. (US 20220091486 A1; IDS REF) (hereinafter He) . Regarding Claim 1, Zhao discloses a method of determining a scale factor, the method comprising: (Abstract, "The present disclosure discloses a photography-based 3D modeling system and method , and an automatic 3D modeling apparatus and method"; para. [0083], “para. [0083], " Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") capturing a real-world environment using a camera resting on a plane, thereby generating an image, wherein the camera comprises a supporting base including a first reference point ; (para. [0068], " the mobile device can be attached to a stand, such as a tripod . independently or jointly with another electronic terminal device such as a camera , to cooperate with applications running in the Android system to implement the implementation method in the present disclosure, or to cooperate with applications running in other operating systems such as the iOS system, the Windows system, and HarmonyOS to implement the implementation method in the present disclosure.”; para. [0069], "FIG. 2 is a schematic structural diagram illustrating an implementation of a photography-based 3D modeling system according to the present disclosure. As shown in FIG. 2, the photography-based 3D modeling system in the present implementation includes: a photo capture unit 201, configured to capture a first image of each of multiple space s. Herein, the first image may be, for example, an image used for 3D modeling, including an ordinary photo, a panoramic photo, and a processed (for example, undistorted) panoramic photo. The photo capture unit 201 can be implemented by a photo capture module in the mobile device.") based on the image, identifying a first three-dimensional (3D) point of a virtual 3D environment that is a reconstruction of the real-world environment, wherein the first 3D point is mapped to the first reference point of the supporting base ; and (para. [0072], "A 3D model generation unit 202 is configured to generate a 3D model of each space based on the image used for 3D modeling that is captured by the photo capture unit 201 for each space."; para. [0074], "Certainly, in one or more implementations, for example, the 3D model generation unit 202 can further predict a depth of each pixel or depths of some pixels in the image used for 3D modeling by using a deep learning method, and calculate a normal direction of each pixel or normal directions of some pixels or predict the normal direction of each pixel or the normal directions of some pixels by directly using the deep learning method, so as to generate a 3D model of each space."; para. [0081], "the 3D model assembling unit 204 can further convert local coordinates of the 3D model of a single room into global coordinates, for example, by using a transformation matrix based on the position and capture direction information obtained by the capture position acquisition unit 203 when each room is captured, so as to obtain the overall 3D model of all photo capture points ."; para. [0082], "the method for converting local coordinates of the 3D model of a single room into global coordinates includes : enabling the photo capture unit 201 to move a predetermined distance, and obtaining, by the capture position acquisition unit 203, coordinates of two endpoints of the predetermined distance (for example, one meter), where a ratio of a difference between the coordinates of the two endpoints to the predetermined distance is the scale of the local coordinates to the global coordinates; or estimating, by using a feature point identified by the capture position acquisition unit 203, a ratio of a height of a plane on which a floor or a ceiling of the space is located to a height of the photo capture unit 201, to obtain the scale of the local coordinates to the global coordinates. Before performing photo capture at a first photo capture point or during movement of subsequent photo capture, the photo capture unit 201 moves a predetermined distance to obtain a predetermined quantity of the feature points.") determining the scale factor based on a coordinate of the first 3D point. (para. [0083], " Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") Zhao does not explicitly disclose capturing a real-world environment using a camera resting on a plane, thereby generating an image, wherein the camera comprises a supporting base including a first reference point; and based on the image, identifying a first three-dimensional (3D) point of a virtual 3D environment that is a reconstruction of the real-world environment, wherein the first 3D point is mapped to the first reference point of the supporting base; However, He more explicitly teaches capturing a real-world environment using a camera resting on a plane, thereby generating an image, wherein the camera comprises a supporting base including a first reference point; and based on the image, identifying a first three-dimensional (3D) point of a virtual 3D environment that is a reconstruction of the real-world environment, wherein the first 3D point is mapped to the first reference point of the supporting base; ([0024] According to the tripod for 3D modeling and the method for determining camera capture parameters provided in the present disclosure, contrasting colors are used at the bottom of the tripod where it touches the floor, e.g., the connecting rod 6 and the foot cover 8, to help recognize the points where the tripod touches the floor, and the height of the tripod can be accurately calculated by measuring the distance between these points in the image, e.g., by measuring a distance between a foot cover 8 and the bottom of the telescopic rod 1 or by measuring a distance between two foot covers 8; the mobile phone holder is attached to the tripod, and a direction sensor of the mobile phone is used to record the direction during capture, so that the accurate capture direction can be obtained even when the camera does not have a direction sensor; the tripod features a simple structure, easy operation, low costs, and a compact storage which is easy to carry.”; para. [0025] “The present disclosure further provides a method for determining a camera height by using the above tripod.”; para. [0026], “In operation (S1), a view directly below a camera is obtained by rendering the panoramic image captured by a panoramic camera as texture inside a 3D globe. In some implementations, the panoramic camera includes two lenses, each capturing a portion of the panoramic image. In some implementations, the image portion captured by a lens of the two lenses are checked to see whether the image portion includes two foot covers 8 . In a typical scenario, at least one of the two lenses will capture two foot covers 8. If none of the two lenses captures two foot covers 8, e.g., a foot cover 8 is adjacent to a border line between the two image portions captured by the two lenses, calibration or adjustment of the shooting angle of the panoramic camera may be conducted. For example, the shooting angle of the panoramic camera may be adjusted to ensure that at least one of the two lenses captures two foot covers 8.”; para. [0003], "The present disclosure provides a tripod for 3D modeling and a method for determining camera capture parameters, including the height and the direction of the camera lens (also referred below as the height and the direction of the camera), which are difficult to obtain with existing techniques.” As both Zhao and HE are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhao to include a supporting base including a first reference point and wherein the first 3D point is mapped to the first reference point of the supporting base in the context of determining a scale factor according to the teaching of He in order to measure the height of camera when using a tripod (para. [0002] of He). Regarding claim 2 , Zhao fails to teach the method of claim 1, wherein the supporting base is a tripod, and the first reference point is a tip of the tripod . However, He more explicitly teaches the method of claim 1, wherein the supporting base is a tripod, and the first reference point is a tip of the tripod .(para. [0024], “According to the tripod for 3D modeling and the method for determining camera capture parameters provided in the present disclosure, contrasting colors are used at the bottom of the tripod where it touches the floor, e.g., the connecting rod 6 and the foot cover 8, to help recognize the points where the tripod touches the floor, and the height of the tripod can be accurately calculated by measuring the distance between these points in the image, e.g., by measuring a distance between a foot cover 8 and the bottom of the telescopic rod 1 or by measuring a distance between two foot covers 8 ; the mobile phone holder is attached to the tripod, and a direction sensor of the mobile phone is used to record the direction during capture, so that the accurate capture direction can be obtained even when the camera does not have a direction sensor; the tripod features a simple structure, easy operation, low costs, and a compact storage which is easy to carry.”; para. [0025], “[0025] The present disclosure further provides a method for determining a camera height by using the above tripod.”; para. [0026], “In operation (S1), a view directly below a camera is obtained by rendering the panoramic image captured by a panoramic camera as texture inside a 3D globe. In some implementations, the panoramic camera includes two lenses, each capturing a portion of the panoramic image. In some implementations, the image portion captured by a lens of the two lenses are checked to see whether the image portion includes two foot covers 8 . In a typical scenario, at least one of the two lenses will capture two foot covers 8. If none of the two lenses captures two foot covers 8, e.g., a foot cover 8 is adjacent to a border line between the two image portions captured by the two lenses, calibration or adjustment of the shooting angle of the panoramic camera may be conducted. For example, the shooting angle of the panoramic camera may be adjusted to ensure that at least one of the two lenses captures two foot covers 8.”) Zhao and He are combined for the reason set forth above with respect to claim 1. Regarding claim 3 , Zhao discloses the method of claim 1, wherein the method further comprises: based on the coordinate of the first 3D point, calculating a first distance value indicating a real-world distance between the camera and the plane; and (para. [0082], "estimating, by using a feature point identified by the capture position acquisition unit 203, a ratio of a height of a plane on which a floor or a ceiling of the space is located to a height of the photo capture unit 201 "; para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera ." calculating a second distance value indicating a virtual distance between the camera and the plane, wherein the scale factor is determined based on the first and second distance values. (para. [0083], “" Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") Regarding claim 4 , Zhao discloses the method of claim 3, wherein the method further comprises: determining a first directional vector between the first 3D point and the camera; and (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera.") determining an angle value indicating an angle formed by the first directional vector with respect to a reference axis, wherein (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known , e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera." Examiner’s note: using the angle between L1 and L2, and a trigonometric function, angle value indicating an angle formed by the first directional vector with respect to a reference axis can be calculated.) the first distance value is calculated using the angle value, and (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera ." Examiner’s note: L3 corresponds to the first distance value.) the reference axis is perpendicular to the plane. (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera." Examiner’s note: L3 is perpendicular to the plane (floor or ceiling)). Regarding claim 5 , Zhao discloses the method of claim 4, wherein the first distance value is calculated using the angle value and a third distance value, and the third distance value indicates a real-world distance between the first reference point of the supporting base and an intersection point of the reference axis and the plane. (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") Regarding claim 6 , Zhao discloses The method of claim 5, wherein PNG media_image1.png 42 101 media_image1.png Greyscale where H is the first distance value, Ls is the third distance value, and β is the angle value. (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera." Examiner’s note: When using well-known triangulation to calculate a distance or length, as illustrated in FIG 6 of applicant’s drawings, Ls= Lv * Sin(β), H=Lv * Cos(β) therefore, Ls/H = tan(β) which is equal to H = Ls/tan(β)). He also teaches determining height of the camera and captured image parameters using an algebraic/trigonometric function (see para. [0012]). Although Zhao and He fail to explicitly recite the claimed equation, it would have been obvious to one of ordinary skill in the art before the effective filing date to combine the disclosure of Zhao with the He disclosure to obtain the invention as specified in claim 9 to determine a scale based on the image, reference points and triangulation based algebraic/trigonometric functions because doing so merely combines known methods to yield predictable results of determining a distance. Regarding claim 7 , Zhao discloses the method of claim 3, wherein the method further comprises: based on the image, identifying a 3D floor point on the plane; and (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor , so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera.") determining a coordinate of an intersection point of a reference axis and the plane based on a coordinate of the 3D floor point and a coordinate of a basis point of the camera, wherein the second distance value is calculated based on the coordinate of the intersection point and the coordinate of the basis point of the camera. (para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor , so that these three points form a triangle. Assume that the projection line is L1 , the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera.") Regarding claim 9 , Zhao discloses the method of of claim 1, wherein the supporting base comprises a plurality of reference points including the first reference point, and the method further comprises: based on the image, identifying a plurality of 3D points of the virtual 3D environment, wherein each of the plurality of 3D points is mapped to each of the plurality of reference points; and determining the scale factor based on coordinates of the plurality of 3D points. (para. [0082]. "Herein, the method for converting local coordinates of the 3D model of a single room into global coordinates includes: enabling the photo capture unit 201 to move a predetermined distance, and obtaining, by the capture position acquisition unit 203, coordinates of two endpoints of the predetermined distance (for example, one meter), where a ratio of a difference between the coordinates of the two endpoints to the predetermined distance is the scale of the local coordinates to the global coordinates; or estimating, by using a feature point identified by the capture position acquisition unit 203, a ratio of a height of a plane on which a floor or a ceiling of the space is located to a height of the photo capture unit 201, to obtain the scale of the local coordinates to the global coordinates . Before performing photo capture at a first photo capture point or during movement of subsequent photo capture, the photo capture unit 201 moves a predetermined distance to obtain a predetermined quantity of the feature points .”; para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor , so that these three points form a triangle. Assume that the projection line is L1 , the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") Zhao does not explicitly disclose the method of of claim 1, wherein the supporting base comprises a plurality of reference points including the first reference point, and the method further comprises: based on the image, identifying a plurality of 3D points of the virtual 3D environment, wherein each of the plurality of 3D points is mapped to each of the plurality of reference points; and determining the scale factor based on coordinates of the plurality of 3D points. However, He more explicitly discloses the method of of claim 1, wherein the supporting base comprises a plurality of reference points including the first reference point, and (FIG1; PNG media_image2.png 693 316 media_image2.png Greyscale para. 0024, "the connecting rod 6 and the foot cover 8, to help recognize the points where the tripod touches the floor, and the height of the tripod can be accurately calculated by measuring the distance between these points in the image, e.g., by measuring a distance between a foot cover 8 and the bottom of the telescopic rod 1 or by measuring a distance between two foot covers 8;") the method further comprises: based on the image, identifying a plurality of 3D points of the virtual 3D environment, wherein each of the plurality of 3D points is mapped to each of the plurality of reference points; and (para. [0026], "In operation (S1), a view directly below a camera is obtained by rendering the panoramic image captured by a panoramic camera as texture inside a 3D globe. In some implementations, the panoramic camera includes two lenses, each capturing a portion of the panoramic image. In some implementations, the image portion captured by a lens of the two lenses are checked to see whether the image portion includes two foot covers 8. In a typical scenario, at least one of the two lenses will capture two foot covers 8 . If none of the two lenses captures two foot covers 8, e.g., a foot cover 8 is adjacent to a border line between the two image portions captured by the two lenses, calibration or adjustment of the shooting angle of the panoramic camera may be conducted. For example, the shooting angle of the panoramic camera may be adjusted to ensure that at least one of the two lenses captures two foot covers 8.” para. [0003], "The present disclosure provides a tripod for 3D modeling and a method for determining camera capture parameters, including the height and the direction of the camera lens (also referred below as the height and the direction of the camera), which are difficult to obtain with existing techniques.”) determining the scale factor based on coordinates of the plurality of 3D points. As both Zhao and HE are from the same field of endeavor, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhao to include wherein the supporting base comprises a plurality of reference points including the first reference point, and the method further comprises: based on the image, identifying a plurality of 3D points of the virtual 3D environment, wherein each of the plurality of 3D points is mapped to each of the plurality of reference points, in the context of determining a scale factor, by Zhao according to the teaching of He, in order to measure the height of camera when using a tripod that has a plurality of reference points (para. [0002] of He). Additionally, it would have been obvious to combine the disclosure of Zhao with the He disclosure to obtain the invention as specified in claim 9 because doing so merely combines known methods to yield predictable results. Regarding claim 12 , Zhao discloses the method of of claim 1, wherein the scale factor is for determining a real-world dimension of an item included in the real-world environment. (para. [0083], " Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") Regarding claim 13 , Zhao discloses a non-transitory computer readable storage medium storing a computer program comprising instructions for configuring an apparatus to perform the method of claim 1. (para. [0268], “In particular, according to the implementations of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the implementations of the present disclosure include a computer program product that includes a computer program that is carried on a non-transient computer readable medium . The computer program includes program code for performing the method shown in the flowchart . In such an implementation, the computer program can be downloaded and installed from a network by using the communications apparatus, or installed from the storage apparatus. When the computer program is executed by the processing apparatus, the above functions defined in the method in the implementations of the present disclosure are executed.”) Regarding claim 15 , claim 15 is the apparatus claim that has similar limitations of claim 1, except claim 15 is the apparatus further comprising: a processing circuitry; and a memory storing instructions executable by the processing circuitry, wherein the apparatus is configured to perform a method (see Zhao, Abstract, “The present disclosure discloses a photography-based 3D modeling system and method, and an automatic 3D modeling apparatus and method,”, para. [0259], ”The processing apparatus can include one or more processors for executing instructions to perform all or some of the steps of the above method.”; para. [0260], “More specific examples of the computer-readable storage media can include but are not limited to an electrical connection with one or more conducting wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.”); therefore, the claim is rejected under same rationale as claim 1. Regarding claim 16, 17, 18, 19, 20, and 21 , the claims are apparatus claims of the method claims 2, 3, 4, 5, 6, and 7 respectively. The claims are similar in scope to claims 2, 3, 4, 5, 6, and 7 respectively and they are rejected under the same rationale as claims 2, 3, 4, 5, 6, and 7 respectively . 07-21-aia AIA Claim s 8 is rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. (US 20210056751 A1; IDS REF) (herein after Zhao) in view of He et al. (US 20220091486 A1; IDS REF) (hereinafter He) and further in view of Mr.H (How to project a point onto a plane in 3D?, StackOverflow, https://web.archive.org/web/20210927024753/https://stackoverflow.com/questions/9605556/how-to-project-a-point-onto-a-plane-i, 2021) (hereinafter H) . Regarding claim 8 , the combination of Zhao and He disclose the method of claim 7, wherein P*=P−(n·(P−O)) n , where P* is the coordinate of the intersection point, P is the coordinate of the basis point of the camera, O is the coordinate of the 3D floor point, and n is a normal vector of the plane. (Zhao, para. [0074], "Certainly, in one or more implementations, for example, the 3D model generation unit 202 can further predict a depth of each pixel or depths of some pixels in the image used for 3D modeling by using a deep learning method, and calculate a normal direction of each pixel or normal directions of some pixels or predict the normal direction of each pixel or the normal directions of some pixels by directly using the deep learning method, so as to generate a 3D model of each space."; para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor , so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera."; [0182], "Areas such as a floor, a ceiling, a wall, and a roof in the image are identified by using deep learning method. For a plane on which one of these areas is located, either its normal direction (as in the case of the floor and ceiling) is known or its normal is on a horizontal plane (as in the case of a wall)." Examiner’s note: In order to have P* (point), two points and a normal vector are used. It’s general mathematical concept in geometry math.) The combination of Zhao and He does not explicitly disclose the method of claim 7, wherein P*=P−(n·(P−O)) n, where P* is the coordinate of the intersection point, P is the coordinate of the basis point of the camera, O is the coordinate of the 3D floor point, and n is a normal vector of the plane. H more explicitly teaches the method of claim 7, wherein P*=P−(n·(P−O)) n, where P* is the coordinate of the intersection point, P is the coordinate of the basis point of the camera, O is the coordinate of the 3D floor point, and n is a normal vector of the plane. (page 6, PNG media_image3.png 188 685 media_image3.png Greyscale ) As Zhao, He, and H are from the same field of endeavor, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to include wherein P*=P−(n·(P−O)) n, in the context of calculating a scale factor, by the combination of Zhao and He according to the teaching of H in order to measure the height of camera when using a tripod . 07-21-aia AIA Claim s 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al. (US 20210056751 A1; IDS REF) (herein after Zhao) in view of He et al. (US 20220091486 A1; IDS REF) (hereinafter He) and further in view of Muramatsu et al. (JPH05188276A) (hereinafter Muramatsu) . Regarding claim 10 , the combination of Zhao and He disclose the method of claim 9, wherein the method further comprises: determining a directional vector between each of the plurality of 3D points and a virtual basis point of the camera; (Zhao, para. para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera."; He, para. [0026], "In operation (S1), a view directly below a camera is obtained by rendering the panoramic image captured by a panoramic camera as texture inside a 3D globe. In some implementations, the panoramic camera includes two lenses, each capturing a portion of the panoramic image. In some implementations, the image portion captured by a lens of the two lenses are checked to see whether the image portion includes two foot covers 8. In a typical scenario, at least one of the two lenses will capture two foot covers 8 . If none of the two lenses captures two foot covers 8, e.g., a foot cover 8 is adjacent to a border line between the two image portions captured by the two lenses, calibration or adjustment of the shooting angle of the panoramic camera may be conducted. For example, the shooting angle of the panoramic camera may be adjusted to ensure that at least one of the two lenses captures two foot covers 8.) determining an angle value indicating an angle formed by each of the determined directional vectors with respect to a reference axis; and (Zhao, para. para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera.") calculating a plurality of distance values using the determined angle values and one or more reference distance values, wherein (Zhao, para. para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera.") said one or more reference distance values indicate a reference distance between each of the plurality of reference points of the supporting base and an intersection point of the reference axis and the plane within the real-environment, and ( Zhao, para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera."; He, para. [0026], "In operation (S1), a view directly below a camera is obtained by rendering the panoramic image captured by a panoramic camera as texture inside a 3D globe. In some implementations, the panoramic camera includes two lenses, each capturing a portion of the panoramic image. In some implementations, the image portion captured by a lens of the two lenses are checked to see whether the image portion includes two foot covers 8. In a typical scenario, at least one of the two lenses will capture two foot covers 8 . If none of the two lenses captures two foot covers 8, e.g., a foot cover 8 is adjacent to a border line between the two image portions captured by the two lenses, calibration or adjustment of the shooting angle of the panoramic camera may be conducted. For example, the shooting angle of the panoramic camera may be adjusted to ensure that at least one of the two lenses captures two foot covers 8.) the scale factor is determined based on an average of the plurality of distance values. (Zhao, para. [0083], "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera .") The combination of Zhao and He does not explicitly disclose the scale factor is determined based on an average of the plurality of distance values . However, Muramatsu more explicitly teaches an average of the plurality of distance values. (para. [0004], “for calculating the second distance measurement value which is the average value of the plurality of distance measurement values are provide") As Zhao, He, and Muramatsu are from the same field of endeavor, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to include an average of the plurality of distance values, in the context of calculating a scale factor, by the combination of Zhao and He according to the teaching of Muramatsu in order to minimize errors when measuring the height of camera when using a tripod that has a plurality of reference points. Regarding claim 11, the combination of Zhao and He disclose the method of claim 10, wherein PNG media_image4.png 46 150 media_image4.png Greyscale where S is the scale factor, Hin is the second distance value, and Havg is the average of the plurality of distance values. (Zhao, "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2, and the line from the projection point to the feature point is L3 . The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera ." "Herein, for example, the method for estimating the ratio of the height of the plane on which the floor or the ceiling of the space is located to the height of the photo capture unit 201 is projecting the photo capture point vertically onto the floor plane, and then connecting the feature points, e.g., on the floor, so that these three points form a triangle. Assume that the projection line is L1, the line from the photo capture point to the feature point is L2 , and the line from the projection point to the feature point is L3. The angle between L1 and L2 is known, e.g., based on the characteristics of the panoramic image, L1 can be calculated by using a trigonometric function based on a length of L3 and the above angle, and a scale is calculated based on an actual height of the camera." Examiner’s note: Calculating a scale factor is a well-known mathematical equation. In the equation y = Cx, C is the scale factor for x. C is also the coefficient of x, and may be called the constant of proportionality of y to x. The equation in the claim corresponds to calculating a scale factor.) The combination of Zhao and He does not explicitly disclose where S is the scale factor, Hin is the second distance value, and Havg is the average of the plurality of distance values . However, Muramatsu more explicitly teaches an average of the plurality of distance values. (para. [0004], “for calculating the second distance measurement value which is the average value of the plurality of distance measurement values are provide") As Zhao, He, and Muramatsu are from the same field of endeavor, it would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to include an average of the plurality of distance values, in the context of calculating a scale factor, by the combination of Zhao and He according to the teaching of Muramatsu in order to minimize errors when measuring the height of camera when using a tripod that has a plurality of reference points. Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hyorim Park whose telephone number is (571)272-3859. The examiner can normally be reached Monday - Friday. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at (571) 272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /Hyorim Park/Examiner, Art Unit 2615 /ALICIA M HARRINGTON/Supervisory Patent Examiner, Art Unit 2615 Application/Control Number: 18/876,910 Page 2 Art Unit: 2615 Application/Control Number: 18/876,910 Page 3 Art Unit: 2615 Application/Control Number: 18/876,910 Page 4 Art Unit: 2615 Application/Control Number: 18/876,910 Page 5 Art Unit: 2615 Application/Control Number: 18/876,910 Page 6 Art Unit: 2615 Application/Control Number: 18/876,910 Page 7 Art Unit: 2615 Application/Control Number: 18/876,910 Page 8 Art Unit: 2615 Application/Control Number: 18/876,910 Page 9 Art Unit: 2615 Application/Control Number: 18/876,910 Page 10 Art Unit: 2615 Application/Control Number: 18/876,910 Page 11 Art Unit: 2615 Application/Control Number: 18/876,910 Page 12 Art Unit: 2615 Application/Control Number: 18/876,910 Page 13 Art Unit: 2615 Application/Control Number: 18/876,910 Page 14 Art Unit: 2615 Application/Control Number: 18/876,910 Page 15 Art Unit: 2615 Application/Control Number: 18/876,910 Page 16 Art Unit: 2615 Application/Control Number: 18/876,910 Page 17 Art Unit: 2615 Application/Control Number: 18/876,910 Page 18 Art Unit: 2615 Application/Control Number: 18/876,910 Page 19 Art Unit: 2615 Application/Control Number: 18/876,910 Page 20 Art Unit: 2615 Application/Control Number: 18/876,910 Page 21 Art Unit: 2615 Application/Control Number: 18/876,910 Page 22 Art Unit: 2615 Application/Control Number: 18/876,910 Page 23 Art Unit: 2615 Application/Control Number: 18/876,910 Page 24 Art Unit: 2615 Application/Control Number: 18/876,910 Page 25 Art Unit: 2615 Application/Control Number: 18/876,910 Page 26 Art Unit: 2615 Application/Control Number: 18/876,910 Page 27 Art Unit: 2615 Application/Control Number: 18/876,910 Page 28 Art Unit: 2615 Application/Control Number: 18/876,910 Page 29 Art Unit: 2615 Application/Control Number: 18/876,910 Page 30 Art Unit: 2615
Prosecution Insights