DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 5-8 are canceled. Claims 1-4 and 9-19 are pending.
Response to Arguments
Applicant’s response to the last Office Action dated 04/02/2026, as well as amendments to claims and the specification, filed on 07/28/2026, have been entered and made of record.
In light of Applicant's amendment of the Specification, Drawings, and Claims to change the minor informalities, the objection of record with respect to the Specification, Drawings, and Claims has been withdrawn.
In light of Applicant's amendment of the specification to change the title of the invention to a more descriptive title, the objection of record with respect to the specification has been withdrawn.
Applicant’s arguments with respect to amended claim(s) 1, 3, 9, and 11-13 have been considered but are moot because of these arguments being merely directed to the amended portion of the claims. Applicant’s amendment of claims has altered the scope of the claims, and therefore, necessitated new grounds of rejection relying on new references as well as the cited prior art of record (Park, S. Jung, Iwakoshi, Lee, Tanaka), as presented in the following sections. THIS ACTION IS MADE FINAL.
Claim Rejections - 35 USC § 103
Claims 1 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (KR 102158316 B1) in view of S. Jung et al. (KR 20170100718 A), Park et al. (KR 20110084028 A), Felipe Barajas (US 20120297253 A1), Curtis et al. (US 2022/0121852), Tanaka et al. (JP 2023029181 A), Iwakoshi et al. (JP 2010025836 A) and Irie et al. (US 20100040289 A).
Regarding Claim 1, Lee discloses “A method performed by a computing device, the method comprising” (Lee, Paragraph [0013], discloses: “A point cloud information processing method according to an embodiment includes the steps of: obtaining three-dimensional point cloud information for a three-dimensional space; obtaining one or more additional images photographing at least a part of the three dimensional space and direction information indicating a gravity direction on a coordinate system in which the one or more additional images are photographed; converting one axis of the coordinate system of the three-dimensional point cloud information to match the gravity direction based on the one or more additional images and the direction information; and indicating the three-dimensional point cloud information using the converted coordinate system of the three-dimensional point cloud information.”); “determining transformation relationship information between a sensing coordinate system and a reference surface coordinate system for a sensing direction of a 3D sensing unit, the determining including:” (Lee, Paragraph [0053], discloses “In an embodiment, the processing unit 106 may rotate the coordinate system of the 3d point cloud information based on the rotation relationship to match one axis of the coordinate system of the 3d point cloud information with the gravitational direction of the coordinate system in which the additional image is captured.”; Paragraph [0054] discloses “After the transformation is performed, the processing unit 106 may indicate the 3d point cloud information using the coordinate system of the transformed 3d point cloud information to clarify which position the coordinates of each point in the 3d point cloud information indicate in the real space”; Paragraph [0066] discloses “the processor 106 may calculate transformation relationship information between a coordinate system in which the one or more additional images are captured and a coordinate system of 3d point cloud information from the mapping result.”); “acquiring, by the 3D sensing unit, a calibration depth image for a reference surface based on the sensing direction” (Lee, Paragraph [0014], discloses “The acquiring of the additional image and the direction information may include acquiring sensing information in which the gravity direction is detected, and acquiring the direction information by reflecting the sensing information in a coordinate system in which the one or more additional images are captured.”); “acquiring point cloud data corresponding to the reference surface based on the calibration depth image” (Lee, Paragraph [0013], discloses “A point cloud information processing method according to an embodiment includes the steps of: obtaining three-dimensional point cloud information for a three-dimensional space; obtaining one or more additional images photographing at least a part of the three dimensional space and direction information indicating a gravity direction on a coordinate system in which the one or more additional images are photographed; converting one axis of the coordinate system of the three-dimensional point cloud information to match the gravity direction based on the one or more additional images and the direction information; and indicating the three-dimensional point cloud information using the converted coordinate system of the three-dimensional point cloud information.”); and “determining the transformation relationship information” (Lee, Paragraphs [0053], [0054], and [0066], please refer to Page 4); “acquiring, by the 3D sensing unit, a target depth image for a target object positioned on the reference surface” (Lee, Abstract, discloses: “the processing unit which converts based on at least one addition image taking a picture of the point group (point cloud) information about three dimensional space); “wherein “the sensing direction” (Lee, Abstract, discloses: “an additional information acquisition unit for acquiring one or more additional images obtained by photographing at least a part of the three-dimensional space and direction information indicating a direction of gravity on a coordinate system in which the one or more additional images are photographed”; Paragraph [0010] also discloses: “The additional information acquisition unit may acquire sensing information in which the gravity direction is detected, and acquire the direction information by reflecting the sensing information in a coordinate system in which the one or more additional images are captured.”)(Lee, Paragraph [0009], discloses, “a point cloud information acquisition unit for acquiring three-dimensional point cloud information for a three-dimensional space”); and “acquiring point cloud data for a reference surface coordinate system by applying the transformation relationship information to the point cloud data for the sensing coordinate system” (Lee, Paragraph [0011], discloses, “The processor may extract a plurality of feature points from the one or more additional images, map each of the plurality of feature points and each point in the three-dimensional point cloud information, calculate conversion relationship information between a coordinate system in which the one or more additional images are captured and a coordinate system of the three-dimensional point cloud information from the mapping result, and convert one axis of the coordinate system of the three-dimensional point cloud information to coincide with the gravity direction based on the conversion relationship information.”) Lee Paragraph [0010]).
Lee does not explicitly disclose “determining an equation of a plane for the reference surface by applying a singular value decomposition (SVD) to the point cloud data corresponding to the reference surface to calculate a unit vector matrix and a centroid matrix”, “based on the equation of the plane”, “at an acute angle with respect to the reference surface”to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for determining an equation of a plane for the reference surface and based on the equation of the plane, Park discloses “a plane equation calculation unit calculating a plane equation of a portion corresponding to a floor surface among image data acquired from an image acquisition unit” (Park, Paragraph [0005]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method seen in Lee with the technique for finding the equation of the plane described in Park to achieve the same claim limitations described in Claim 1.
The combination of Lee and Park does not explicitly disclose by applying a singular value decomposition (SVD) to the point cloud data corresponding to the reference surface to calculate a unit vector matrix and a centroid matrix”, “at an acute angle with respect to the reference surfaceto generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for applying a singular value decomposition to the point cloud data, S. Jung discloses “The mathematical model generating unit which especially processes the singular value decomposition (singular value decomposition the pair of the point of the lower set and produces the transform matrix And a point group matching unit for converting points of the remaining point groups into a coordinate system of the reference point group by applying the conversion matrix to the point group” (S. Jung, Paragraph [0017]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method seen in the combination of Lee and Park with the S. Jung mathematical model generating unit that uses singular value decomposition to yield predictable results for determining the plane equation for a reference surface to have an improved volume measurement method. By using singular value decomposition, one of ordinary skill in the art could transform the point cloud data from the reference surface coordinate system to develop a plane equation as described in the claimed invention to perform an accurate calibration before assessing the volume of an object. Therefore, it would have been obvious for one of ordinary skill in the art to combine the Lee, Park, and S. Jung references to achieve the above-described limitation seen in Claim 1.
The combination of Lee, Park, and S. Jung does not explicitly disclose “to calculate a unit vector matrix and a centroid matrix”, “at an acute angle with respect to the reference surfaceto generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for calculating a unit vector matrix, Barajas discloses “...generating a unit vector matrix comprising the unit vector for each vector for each data entry in the data repository...” (Barajas, Paragraph [0107]). Therefore, it would have been obvious for one of ordinary skill in the art to combine the method seen in the combination of Lee, Park, and S. Jung with the Barajas technique of calculating the unit vector matrix to achieve the above-described limitation described in Claim 1.
The combination of Lee, Park, S.Jung, and Barajas does not explicitly disclose “a centroid matrix”, “at an acute angle with respect to the reference surfaceto generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for a centroid matrix, Curtis teaches of a system and method for three-dimensional counting that has a third neural network with a centroid matrix, where they describe the following for the given equations in Paragraphs [0088], [0089], and [0090]:
PNG
media_image1.png
679
412
media_image1.png
Greyscale
Curtis describes earlier about O, which is explained in Paragraph [0078]:
PNG
media_image2.png
106
412
media_image2.png
Greyscale
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method seen in the combination of Lee, Park, S. Jung, and Barajas with the centroid matrix seen in Curtis to achieve the same limitation described in Claim 1 (the combination will hereinafter be described as “LPSBC”).
The combination of LPSBC does not explicitly disclose “which is an acute angle with respect to the reference surface” or “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for the acute angle with respect to the reference surface, Tanaka discloses “the three dimensional camera 20 is disposed downward (specifically, so as to face a direction slightly inclined with respect to the vertical direction in the YZ plane (see FIG. 4)) at a position spaced further upward from the upper surface of the measurement object 80” (Tanaka, Paragraph [0041]). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the point cloud data acquisition techniques found in the combination of LPSBC with the inclined 3D camera seen in Tanaka to achieve the above-described limitation seen in Claim 1. By using the simple substitution of the gravity direction described in Lee with the sensing direction, one of ordinary skill in the art could enhance the function of the 3D sensing unit to acquire point cloud data of objects placed on the reference surface. Additionally, the sensing direction can be enhanced through an incline (or acute angle) with respect to the reference surface as seen in Tanaka with the 3D sensing unit to obtain an accurate depth image that corresponds to the shape of the object. Therefore, it would have been obvious to combine the LPSBC and Tanaka references to achieve the limitation described in Claim 1 (The combination hereinafter will be described as “LPSBCT”).
The combination of LPSBCT does not explicitly disclose “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for generating a bird eye view image, Iwakoshi discloses “a method of inspecting a three-dimensional structure by nondestructive inspection will be described as a third embodiment. In the third embodiment, as shown in FIG. 12, a bird' s-eye view is obtained by observing the surface of the object to be inspected from an oblique direction” (Iwakoshi, Paragraph [0062] and Figure 12 (see below)).
PNG
media_image3.png
643
694
media_image3.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have used the technique for obtaining a bird’s eye view image seen in Iwakoshi with the method described in Claim 1 from the combination of LPSBCT to achieve an easily inspectable form of point cloud data to observe what has been captured by the 3D sensing unit. Thus, it would have been obvious to have used the technique for acquiring a bird’s eye view image seen in Iwakoshi with the method described in the combination of LPSBCT to achieve the same limitation seen in Claim 1 (the combination will hereinafter be described as “LPSBCTI”).
Finally, the combination of LPSBCTI does not explicitly disclose “wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor, Irie discloses “the image generation section 121 generates a top-view image 910 (plan view of the surface of the earth as viewed from sky) corresponding to the frame image 900 as viewed from the vertical direction” (Irie, Paragraph [0032]). It is important to note that top-view images are synonymous to bird’s eye view images. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method seen in the combination of LPSBCTI with the Irie technique of having the bird eye view image representing a vertical viewing direction to the reference surface to achieve a complete volume measurement method. By combining the Irie technique of the bird’s eye view image with a vertical viewing direction with the method described in LPSBCTI, one of ordinary skill in the art allows for the 3D sensing unit to have a clearer representation of the volume of the package or other shaped object that it measures. Therefore, it would have been obvious to combine the Lee, Park, S. Jung, Barajas, Curtis, Tanaka, Iwakoshi, and Irie references to achieve the same volume measurement method described in Claim 1 (the combination hereinafter will be described as “LPSBCTII”).
Claim 12 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the LPSBCTII references, presented in rejection of Claim 1, apply to this claim. Finally, the combination of LPSBCTII discloses a computer readable storage medium (for example, see Lee, Paragraph [0073], where it discloses: “The computer-readable storage medium 16 is configured to store computer-executable instructions or program code, program data, and/or other suitable form of information. The program 20 stored in the computer-readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, the computer-readable storage medium 16 may be a memory (volatile memory, such as random access memory, nonvolatile memory, or any suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media accessible by the computing device 12 and capable of storing the desired information, or any suitable combination thereof.”).
Claims 2, 3, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Park, S. Jung, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Zhang (CN 114820736 A).
Regarding Claim 2, the combination of LPSBCTII discloses “the method of claim 1” (Please refer to the above-described analysis for Claim 1), wherein “the transformation relationship information between the sensing coordinate system and the reference surface coordinate system includes a rotation transformation matrix” (Lee, Paragraph [0012] and [0016], disclose: “The conversion relationship information may include a rotation relationship between a coordinate system in which the one or more additional images are captured and a coordinate system of the 3d point cloud information. The rotation relationship may be expressed as a three-dimensional rotation matrix, a quaternion, an euler angle, an axis angle, or other methods.”) . The combination of LPSBCTII does not explicitly disclose “a translation transformation matrix”. However, in an analogous field of endeavor, Zhang discloses “calculating the registration translation transformation matrix according to the coordinate of the centre center of mass the two center of mass point sets; S5, using the registration rotation transformation matrix and the registration translation transformation matrix to transform the coordinate of the second three-dimensional volume data to finish the registration” (Zhang, Abstract). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the conversion relationship with the rotation transformation matrix described in the combination of LPSBCTII with the translation transformation matrix seen in Zhang to improve the acquisition of point cloud data using the 3D sensing unit in the same way. By using the conversion relationship with the rotation transformation matrix found in the combination of LPSBCTII with the translation transformation matrix seen in Zhang, one of ordinary skill in the art could effectively create both a rotation transformation matrix and translation transformation matrix within the 3D sensing unit to effectively stitch the images acquired from the 3D sensing unit into a unified 3D model to measure the volume of the object. Thus, it would have been obvious to have used the LPSBCTII and Zhang references to achieve the same method described in Claim 2 (The combination will hereinafter be described as “LPSBCTIIZ”).
Regarding Claim 3, the combination of LPSBCTIIZ discloses “the method of claim 2” (Please refer to the above-described analysis for Claim 1), “wherein the rotation transformation matrix is” (Lee, Paragraph [0012] and [0016], disclose: “The conversion relationship information may include a rotation relationship between a coordinate system in which the one or more additional images are captured and a coordinate system of the 3d point cloud information. The rotation relationship may be expressed as a three-dimensional rotation matrix, a quaternion, an euler angle, an axis angle, or other methods.”); “a unit vector matrix of the reference surface according to the sensing coordinate system” (Lee, Paragraph [0037], discloses: “the sensor may form a coordinate system such that the detected gravitational direction coincides with the direction of one axis, or may form an arbitrary coordinate system including a unit vector in the detected gravitational direction in the coordinate space.”; Lee, Paragraph [0039], discloses: “The sensing information generated through the method (1) or (2) is a coordinate system in which one axis coincides with the direction of gravity or an arbitrary coordinate system including a unit vector in the direction of gravity in the coordinate space. Let this coordinate system be referred to as a 'sensed coordinate system'”); and “the translation transformation matrix is” (Zhang, Abstract, discloses “calculating the registration translation transformation matrix according to the coordinate of the centre center of mass the two center of mass point sets; S5, using the registration rotation transformation matrix and the registration translation transformation matrix to transform the coordinate of the second three-dimensional volume data to finish the registration.”); “a centroid matrix of the point cloud data” (Curtis discloses the following in Paragraphs [0088], [0089], and [0090]:
PNG
media_image1.png
679
412
media_image1.png
Greyscale
Curtis describes earlier about O, which is explained in Paragraph [0078]:
PNG
media_image2.png
106
412
media_image2.png
Greyscale
As mentioned in the analysis for Claim 1, centroid matrix is clearly defined in [0088]-[0090]); “corresponding to the reference surface according to the sensing coordinate system” (Lee, Paragraph [0039], discloses “The sensing information generated through the method (1) or (2) is a coordinate system in which one axis coincides with the direction of gravity or an arbitrary coordinate system including a unit vector in the direction of gravity in the coordinate space. Let this coordinate system be referred to as a 'sensed coordinate system'”).
Claim 14 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 2. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the LPSBCTIIZ references, presented in rejection of Claim 2, apply to this claim.
Claim 15 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 3. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the LPSBCTIIZ references, presented in rejection of Claim 3, apply to this claim.
Claims 4 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Park, S. Jung, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Zhang and Smith et al. (US 9,536,322).
Regarding Claim 4, the combination of LPSBCTIIZ discloses “the method of claim 3, wherein the transformation relationship information between the sensing coordinate system and the reference surface coordinate system is based on” (Please refer to the above-described analysis for Claim 3) . The combination of LPSBCTIIZ does not explicitly disclose the equation of the transformation relationship that “is based on pp = RTpc – RTt, where, pc represents the sensing coordinate system, pp represents the reference surface coordinate system, RT represents a transposed matrix of the rotation transformation matrix, and t represents the translation transformation matrix”. However, in an analogous field of endeavor, Smith discloses “a “primed” coordinate system that has been translated and rotated relative to an “unprimed” coordinate system. A point represented in the unprimed coordinate system has coordinate values R(x, y, z). The same point has coordinate values R′(x′, y′, z′) relative to the unprimed coordinate system.” (Smith, Col. 44, lines 39-46). Smith recites afterward in Col 44, lines 46-53, the following equation and explanation for the primed coordinate system:
PNG
media_image4.png
166
458
media_image4.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have substituted the primed coordinate system for the reference surface coordinate system, unprimed coordinate system for the sensing coordinate system, and position vector for the translation transformation matrix in the linear transformation equation for Smith in combination with the LPSBCTIIZ references to obtain the same equation described in the current claim. By using this substitution, one of ordinary skill in the art can implement an effective transformation relationship between the two coordinate systems that can enable the point cloud data from object to be utilized to measure their volume. Thus, it would have been obvious to use the linear transformation equation in Smith in combination with the LPSBCTIIZ references to yield predictable results.
Claim 16 recites a computer-readable storage medium storing a program with instructions corresponding to the steps recited in Claim 4. Therefore, the recited programming instructions of this claim are mapped to the proposed combination in the same manner as the corresponding steps in its corresponding method claim. Additionally, the rationale and motivation to combine the LPSBCTIIZS references, presented in rejection of Claim 4, apply to this claim.
Claim 9 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Park, S. Jung, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Deok Lee (KR 20220095169 A, which will from now on be described as “Lee 2”).
Regarding Claim 9, the combination of LPSBCTII teaches “The method of claim 1, further comprising:” (Please refer to the above-described analysis for Claim 1); bird-eye view image”. However, in an analogous field of endeavor, Lee 2 teaches “...The model according to the embodiment can estimate a 3D boundary volume by using a LiDAR-based bird's eye view image and apply it to reality....” (Lee 2, Paragraph [0035]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the volume measurement method seen in Lee 2 with the point cloud acquisition method described in the combination of LPSBCTII to achieve an appropriate method for calculating the volume of object placed on the volume measurement apparatus, as seen in Claim 9 (the combination will hereinafter be described as “LPSBCTIIL”).
Claim 10 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Park, S. Jung, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Lee 2 and Kono et al. (JP 2013196355 A).
Regarding Claim 10, the combination of LPSBCTIIL teaches “the method of claim 9, wherein the determining of the volume of the target object based on the point cloud data for the reference surface coordinate system includes:” (Please refer to the above-described analysis for Claim 9); information of mesh data may be used to extract a cluster of point cloud data. In addition, a clustering method such as a k-means method may be used. The measurement unit 114 extracts a minimum value and a maximum value for each coordinate axis of the three-dimensional space from the extracted point cloud data. The measurement unit 114 detects a surface including each of the extracted minimum and maximum values as a surface of the circumscribed rectangular parallelepiped” (Kono, Paragraph [0036]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method seen in LPSBCTIIL with the Kono technique of identifying the top surface of a rectangular parallelepiped to improve the function of the volume measurement method, as shown in Claim 10.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Park, S. Jung, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Lee 2, and Y. Jung (KR 20030062774 A).
Regarding Claim 11, the combination of LPSBCTIIL discloses “The method of claim 9, wherein the determining of the volume of the target object based on the point cloud data for the reference surface coordinate system includes:” (Please refer to the above-described analysis for Claim 9); cloud data for the reference surface coordinate system when the target object has an atypical shape, and determining approximate values of a width, a length, and a height of the target object buy using the cube type bounding box of the target object”. However, in an analogous field of endeavor, Y. Jung teaches “a method for determining a hexahedral (cube) box of a minimum size surrounding an object. That is, according to the present invention, the minimum circumscribed rectangle is obtained with respect to the object area on the image plane to measure the dimension of the object, thereby effectively measuring the dimension even in the case of an object having an arbitrary shape” (Y. Jung, Abstract). Y. Jung also teaches that “the minimum circumscribed rectangle is acquired with respect to the object area on the image plane to measure the dimension of the object, thereby effectively measuring the dimension even in the case of an object having an arbitrary shape, calculating the information on the length and width of the object using the spatial geometric characteristics of the four corner points acquired by the minimum circumscribed rectangle, thereby calculating the dimension at high speed, and using the height sensor and the ccd camera” (Y. Jung, Paragraph [0060], Drawing 1 (See below)), whereas each of the dimensions found can be utilized to measure the volume of an object. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method for determining a cube box surrounding an object described in Y. Jung with the volume measurement method seen in the combination of LPSBCTIIL in order to obtain the same limitation found in Claim 11.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Tanaka, Kono, Park, S. Jung, Barajas, Curtis, Iwakoshi, and Irie.
Regarding Claim 13, Lee teaches ; determine transformation relationship information between a sensing coordinate system and a reference surface coordinate system for a sensing direction of a 3D sensing unit, the determining including:” (Lee, Paragraph [0053], discloses “In an embodiment, the processing unit 106 may rotate the coordinate system of the 3d point cloud information based on the rotation relationship to match one axis of the coordinate system of the 3d point cloud information with the gravitational direction of the coordinate system in which the additional image is captured.”; Paragraph [0054] discloses “After the transformation is performed, the processing unit 106 may indicate the 3d point cloud information using the coordinate system of the transformed 3d point cloud information to clarify which position the coordinates of each point in the 3d point cloud information indicate in the real space”; Paragraph [0066] discloses “the processor 106 may calculate transformation relationship information between a coordinate system in which the one or more additional images are captured and a coordinate system of 3d point cloud information from the mapping result.”); “acquiring, by the 3D sensing unit, a calibration depth image for a reference surface based on the sensing direction” (Lee, Paragraph [0014], discloses “The acquiring of the additional image and the direction information may include acquiring sensing information in which the gravity direction is detected, and acquiring the direction information by reflecting the sensing information in a coordinate system in which the one or more additional images are captured.”); “acquiring point cloud data corresponding to the reference surface based on the calibration depth image” (Lee, Paragraph [0013], discloses “A point cloud information processing method according to an embodiment includes the steps of: obtaining three-dimensional point cloud information for a three-dimensional space; obtaining one or more additional images photographing at least a part of the three dimensional space and direction information indicating a gravity direction on a coordinate system in which the one or more additional images are photographed; converting one axis of the coordinate system of the three-dimensional point cloud information to match the gravity direction based on the one or more additional images and the direction information; and indicating the three-dimensional point cloud information using the converted coordinate system of the three-dimensional point cloud information.”); and “determining the transformation relationship information” (Lee, Paragraphs [0053], [0054], and [0066], please refer to Page 23 or Page 4); the sensing coordinate system based on the target depth image” (Lee, Paragraph [0009], discloses: “a point cloud information acquisition unit for acquiring three-dimensional point cloud information for a three-dimensional space”); and “acquires point cloud data for the reference surface coordinate system by applying the transformation relationship information to the point cloud data for the sensing coordinate system” (Lee, Paragraph [0011], discloses, “The processor may extract a plurality of feature points from the one or more additional images, map each of the plurality of feature points and each point in the three-dimensional point cloud information, calculate conversion relationship information between a coordinate system in which the one or more additional images are captured and a coordinate system of the three-dimensional point cloud information from the mapping result, and convert one axis of the coordinate system of the three-dimensional point cloud information to coincide with the gravity direction based on the conversion relationship information.”) Lee Paragraph [0010]).
Lee does not disclose “A volume measurement apparatus, comprising: including a stand having an upper plane on which configured to support a target objecta 3D sensing unit arranged such that a sensing direction of a 3D sensing unit is oblique relative to the upper plane; and controller configured to” “determining an equation of a plane for the reference surface by applying a singular value decomposition (SVD) to the point cloud data corresponding to the reference surface to calculate a unit vector matrix and a centroid matrix” “based on the equation of the plane; acquirethe upper plane a using the 3D sensing unit”, and “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor, Tanaka teaches for the volume measurement apparatus of a package reception system that “includes a measurement unit configured to measure a size that is a total value of lengths in three different directions of a package” (Tanaka, [0021]), which indicates finding the length, width, and height of a package placed on the system (or apparatus) which can determine the size (i.e. volume). Later on, Tanaka also teaches for the body structure, 3D sensing unit being oblique relative to the upper plane, and object positioned on an upper plane how within this system, “The three-dimensional camera 20 can capture a captured image of the object to be measured 80 as viewed from above in a state where the object to be measured 80 is held by the holding unit 50 (a state where the object to be measured 80 is placed on the 51a (placement surface) of the transparent member 51 of the holding unit 50). Specifically, the three-dimensional camera 20 is disposed downward (specifically, so as to face a direction slightly inclined with respect to the vertical direction in the YZ plane (see FIG. 4)) at a position spaced further upward from the upper surface of the measurement object 80. The three-dimensional camera 20 is fixed by a camera fixing member 27,28. The camera fixing member 27 is a column member that is fixed to the floor surface 90 and extends in the vertical direction, and the camera fixing member 28 is a beam member that extends in the horizontal direction from the vicinity of the uppermost portion of the column member 27 toward a position above the measurement object 80 (see FIG. 2). The three-dimensional camera 20 is fixed downward (obliquely downward) in the vicinity of the distal end portion of the camera fixing member 28 (and above the measurement object 80)” (Tanaka, Paragraph [0041], Figure 2 (see first picture below), and Figure 4 (see second picture below)).
PNG
media_image5.png
655
446
media_image5.png
Greyscale
Figure 2 (Tanaka). Apparatus for measuring size of object
PNG
media_image6.png
550
392
media_image6.png
Greyscale
Figure 4 (Tanaka). Apparatus for measuring size of object with three-dimensional camera at an incline.
Here, although the amendment clarifies the 3D sensing unit to be oblique relative to the upper plane, Tanaka thoroughly describes this through Paragraph [0041] and Figures 2 and 4, as the 3D sensing unit (in the form of the 3D camera) is facing obliquely downward in the vicinity of the object placed on the upper plane, which is being interpreted as 51a in Figures 2 and 4. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to use the measurement unit and apparatus found in the package reception system seen in Tanaka in combination with the point cloud acquisition and transformation methods seen in Lee to achieve the above-described limitations of Claim 13.
The combination of Lee and Tanaka does not explicitly disclose, “controller configured to” “determining an equation of a plane for the reference surface by applying a singular value decomposition (SVD) to the point cloud data corresponding to the reference surface to calculate a unit vector matrix and a centroid matrix” “based on the equation of the plane”; acquirethe upper plane a using the 3D sensing unit”, and “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor, Kono teaches that its “object measurement apparatus 100 includes a storage device for storing the measurement result. The control unit 110 stores the measurement result measured by the measurement unit 114 in the storage device. In addition, the control unit 110 may be configured to store the image of the object in the storage device together with the measurement result” (Kono, Paragraph [0106]), whereas “The image acquisition unit 120 acquires a distance image of a target object. The image obtaining unit 120 inputs the obtained range image to the control unit 110” (Kono, Paragraph [0112]). Kono further teaches that its “object measuring apparatus 100 can calculate the length, area, or volume of the measurement target on the target object based on the arranged shape data” (Kono, Paragraph [0042]), whereas “the shape data is point cloud data representing a three dimensional shape of an object by a set of points having coordinate values in a three dimensional space corresponding to a surface of the object.” (Kono, Paragraph [0016]). Therefore, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to combine the volume measurement apparatus seen in the combination of Lee and Tanaka with the control unit and volume measurement technique performed by acquiring distance (depth) images seen in Kono to allow the apparatus to effectively measure the volume of objects via the 3D sensing unit and achieve the above-described limitations seen in Claim 13.
The combination of Lee, Tanaka and Kono does not explicitly disclose “determining an equation of a plane for the reference surface by applying a singular value decomposition (SVD) to the point cloud data corresponding to the reference surface to calculate a unit vector matrix and a centroid matrix”, “based on the equation of the plane” or “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for determining an equation of a plane for the reference surface and based on the equation of the plane, Park discloses “a plane equation calculation unit calculating a plane equation of a portion corresponding to a floor surface among image data acquired from an image acquisition unit” (Park, Paragraph [0005]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of Lee, Tanaka, and Kono with the technique for finding the equation of the plane described in Park to achieve the same claim limitations described in Claim 13.
The combination of Lee, Tanaka, Kono and Park does not explicitly disclose “by applying a singular value decomposition (SVD) to the point cloud data corresponding to the reference surface to calculate a unit vector matrix and a centroid matrix”, or “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for applying a singular value decomposition to the point cloud data, S. Jung discloses “The mathematical model generating unit which especially processes the singular value decomposition (singular value decomposition the pair of the point of the lower set and produces the transform matrix And a point group matching unit for converting points of the remaining point groups into a coordinate system of the reference point group by applying the conversion matrix to the point group” (S. Jung, Paragraph [0017]). Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of Lee, Tanaka, Kono and Park with the S. Jung mathematical model generating unit that uses singular value decomposition to yield predictable results for determining the plane equation for a reference surface to have an improved volume measurement method. By using singular value decomposition, one of ordinary skill in the art could transform the point cloud data from the reference surface coordinate system to develop a plane equation as described in the claimed invention to perform an accurate calibration before assessing the volume of an object. Therefore, it would have been obvious for one of ordinary skill in the art to combine the Lee, Tanaka, Kono, Park, and S. Jung references to achieve the above-described limitation seen in Claim 13.
The combination of Lee, Tanaka, Kono, Park, and S. Jung does not explicitly disclose “to calculate a unit vector matrix and a centroid matrix”, or “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for calculating a unit vector matrix, Barajas discloses “...generating a unit vector matrix comprising the unit vector for each vector for each data entry in the data repository...” (Barajas, Paragraph [0107]). Therefore, it would have been obvious for one of ordinary skill in the art to combine the apparatus seen in the combination of Lee, Tanaka, Kono, Park, and S. Jung with the Barajas technique of calculating the unit vector matrix to achieve the above-described limitation described in Claim 13.
The combination of Lee, Tanaka, Kono, Park, S.Jung, and Barajas does not explicitly disclose “a centroid matrix”, or “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for a centroid matrix, Curtis teaches of a system and method for three-dimensional counting that has a third neural network with a centroid matrix, where they describe the following for the given equations in Paragraphs [0088], [0089], and [0090]:
PNG
media_image1.png
679
412
media_image1.png
Greyscale
Curtis describes earlier about O, which is explained in Paragraph [0078]:
PNG
media_image2.png
106
412
media_image2.png
Greyscale
Accordingly, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of Lee, Tanaka, Kono, Park, S. Jung, and Barajas with the centroid matrix seen in Curtis to achieve the same limitation described in Claim 1 (the combination will hereinafter be described as “LTKPSBC”).
The combination of LTKPSBC does not explicitly disclose “to generate a bird-eye view image corresponding to the target depth image, wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor for generating a bird eye view image, Iwakoshi discloses “a method of inspecting a three-dimensional structure by nondestructive inspection will be described as a third embodiment. In the third embodiment, as shown in FIG. 12, a bird' s-eye view is obtained by observing the surface of the object to be inspected from an oblique direction” (Iwakoshi, Paragraph [0062] and Figure 12 (see below)).
PNG
media_image3.png
643
694
media_image3.png
Greyscale
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have used the technique for obtaining a bird’s eye view image seen in Iwakoshi with the apparatus described in Claim 13 from the combination of LTKPSBC to achieve an easily inspectable form of point cloud data to observe what has been captured by the 3D sensing unit. Thus, it would have been obvious to have used the technique for acquiring a bird’s eye view image seen in Iwakoshi with the apparatus described in the combination of LTKPSBC to achieve the same limitation seen in Claim 13 (the combination will hereinafter be described as “LTKPSBCI”).
Finally, the combination of LTKPSBCI does not explicitly disclose “wherein the bird-eye view image represents a viewing direction vertical to the reference surface”. However, in an analogous field of endeavor, Irie discloses “the image generation section 121 generates a top-view image 910 (plan view of the surface of the earth as viewed from sky) corresponding to the frame image 900 as viewed from the vertical direction” (Irie, Paragraph [0032]). It is important to note that top-view images are synonymous to bird’s eye view images. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the apparatus seen in the combination of LTKPSBCI with the Irie technique of having the bird eye view image representing a vertical viewing direction to the reference surface to achieve a complete volume measurement apparatus. By combining the Irie technique of the bird’s eye view image with a vertical viewing direction with the apparatus described in LTKPSBCI, one of ordinary skill in the art allows for the 3D sensing unit to have a clearer representation of the volume of the package or other shaped object that it measures. Therefore, it would have been obvious to combine the Lee, Tanaka, Kono, Park, S. Jung, Barajas, Curtis, Iwakoshi, and Irie references to achieve the same volume measurement method described in Claim 13.
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view S. Jung, Park, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Kono.
Regarding Claim 17, the combination of LPSBCTII teaches “The non-transitory computer readable storage medium of claim 12, further comprising:” (Please refer to the above-described analysis for Claim 12) The combination of LPSBCTII does not explicitly disclose “determining a volume of the target object based on the point cloud data for the reference surface coordinate system”. However, in an analogous field of endeavor, Kono teaches “the object measuring apparatus 100 can calculate the length, area, or volume of the measurement target on the target object based on the arranged shape data” (Kono, Paragraph [0042]), where “the shape data is point cloud data representing a three dimensional shape of an object by a set of points having coordinate values in a three dimensional space corresponding to a surface of the object” (Kono, Paragraph [0016]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the volume measurement method seen in Kono with the computer-readable medium described in the combination of LPSBCTII to achieve an appropriate method for calculating the volume of object, as seen in the computer-readable medium of Claim 17 (the combination will hereinafter be described as “LPSBCTIIK”).
Regarding Claim 18, the combination of LPSBCTIIK teaches “the method of claim 17, wherein the determining of the volume of the target object based on the point cloud data for the reference surface coordinate system includes:” (Please refer to the above-described analysis for Claim 17); and “identifying a top surface of the target object in order to determine a height of the target object when the target object has a rectangular parallelepiped shape” (Kono, Paragraph [0036], discloses: “the measurement unit 114 detects the floor surface on which the target object is placed as the maximum plane based on the shape data arranged with the position angle adjusted. The measurement unit 114 removes the detected floor surface, and extracts the object from the background as a mass of point cloud data close to the photographed center. Note that adjacency information of mesh data may be used to extract a cluster of point cloud data. In addition, a clustering method such as a k-means method may be used. The measurement unit 114 extracts a minimum value and a maximum value for each coordinate axis of the three-dimensional space from the extracted point cloud data. The measurement unit 114 detects a surface including each of the extracted minimum and maximum values as a surface of the circumscribed rectangular parallelepiped.”).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view S. Jung, Park, Barajas, Curtis, Tanaka, Iwakoshi, and Irie, and further in view of Kono and Y.Jung.
Regarding Claim 19, the combination of LPSBCTIIK discloses “The non-transitory computer readable storage medium of Claim 17, wherein the determining of the volume of the target object based on the point cloud data for the reference surface coordinate system includes:” (Please refer to the above-described analysis for Claim 17); (Y. Jung, Abstract). Y. Jung also teaches that “the minimum circumscribed rectangle is acquired with respect to the object area on the image plane to measure the dimension of the object, thereby effectively measuring the dimension even in the case of an object having an arbitrary shape, calculating the information on the length and width of the object using the spatial geometric characteristics of the four corner points acquired by the minimum circumscribed rectangle, thereby calculating the dimension at high speed, and using the height sensor and the ccd camera” (Y. Jung, Paragraph [0060], Drawing 1 (See below)
PNG
media_image7.png
224
438
media_image7.png
Greyscale
), whereas each of the dimensions found can be utilized to measure the volume of an object. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to combine the method for determining a cube box surrounding an object described in Y. Jung with the volume measurement computer-readable medium seen in the combination of LPSBCTIIK in order to obtain the same limitation found in Claim 19.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SORIE I KOROMA JR whose telephone number is (571)272-9259. The examiner can normally be reached Monday - Friday 8AM-6:00PM; Alternate Fridays Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SORIE I KOROMA JR/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
