Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN-202210757474.5, filed on 06/29/2022.
Information Disclosure Statement
The information disclosure statements (IDSs) submitted on 12/18/2024 and 08/29/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claims 1-4, 7, 9-14, and 17-20 are objected to because of the following informalities:
In Claims 1, 9, and 10, “rendering scanning image” is unclear, for the sake of further prosecution, Examiner will interpret this as “rendering a scan”.
In Claims 1, 9, and 10, “a current frame scanning image” is unclear, for the sake of further prosecution, Examiner will interpret this as “a current frame of a scan”.
In Claims 1, 9, and 10, "scanning image, to obtain a target fusion field" should read "scanning image to obtain a target fusion field ".
In Claims 1, 9, and 10, "rendering color, to obtain a rendering result" should read "rendering color to obtain a rendering result".
In Claims 2, 12, and 18, all instances of “performing calculation” should read “performing a calculation”.
In Claims 2, 12, and 18, "scanning image, to obtain the target fusion field" should read "scanning image to obtain the target fusion field ".
In Claims 2, 12, and 18, "parameter information, to obtain a score value" should read "parameter information to obtain a score value ".
In Claims 3, 13, and 19, "parameter information, to obtain the score value" should read "parameter information to obtain the score value ".
In Claims 3, 13, and 19, all instances of “performing calculation” should read “performing a calculation”.
In Claims 3, 13, and 19, “on weight of the target point cloud set” should read “on a weight of the target point cloud set”.
In Claims 3, 13, and 19, "preset radius, to obtain a target point cloud set" should read "preset radius to obtain a target point cloud set ".
In Claims 3, 13, and 19, "target point cloud set, to obtain a weight value" should read "target point cloud set to obtain a weight value".
In Claims 3, 13, and 19, "target point cloud, to obtain a position" should read "target point cloud to obtain a position".
In Claims 4, 14, and 20, "target point cloud set, to obtain the weight value" should read "target point cloud set to obtain the weight value ".
In Claims 4, 14, and 20, all instances of “performing calculation” should read “performing a calculation”.
In Claims 6 and 16, “and controlling the total weight remain unchanged,” is grammatically incorrect. Examiner will interpret this to convey “and keeping the total weight unchanged,”.
In Claims 7 and 17, "images, to obtain a total fusion field" should read "images to obtain a total fusion field".
In Claims 7 and 17, "total rendering color, to obtain a total rendering result" should read "total rendering color to obtain a total rendering result ".
In Claim 11, “each control point controls scores within a certain range” should read “each control point scores within a certain range”.
Appropriate correction is required.
The numbering of claims is not in accordance with 37 CFR 1.126 which requires the original numbering of the claims to be preserved throughout the prosecution. When claims are canceled, the remaining claims must not be renumbered. When new claims are presented, they must be numbered consecutively beginning with the number next following the highest numbered claims previously presented (whether entered or not). Claims 11-20 are numbered incorrectly. Claim 11 depends on Claim 1 and is improper because it is separated by Claims 9 and 10, claims that do not depend on Claim 1. Claims 12-20 are improper to for the same reason. See MPEP § 608.01(n).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Regarding Claim 1, it recites “and based on the weight of each display point in the display point cloud, determining each display point and a rendering color of a region”, this is indefinite because a display point must already be determined if something is being based on the weight of a display point. For the sake of further prosecution, Examiner will interpret this as “and based on the weight of each display point in the display point cloud, determining a rendering color of each display point in a region”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 7, 9, 10, 12, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 2019/0272671 A1), hereinafter referenced as Zhang in view of Landrieu et al. (“A structured regulation framework for spatially smoothing semantic labeling of 3D point clouds”, 2017), hereinafter referenced as Landrieu.
Regarding Claim 1, Zhang discloses a method for rendering scanning image (Zhang: [Abs], discloses a method for constructing a 3D scene), comprising:
acquiring a current frame scanning image (Zhang: [0024], discloses obtaining scanned data <a frame scanning image>);
updating information of all control points in a fusion field according to a point cloud corresponding to the current frame scanning image, to obtain a target fusion field (Zhang: [0029], discloses converting a point cloud from the current frame of scanned data <the 3D coordinate system this point cloud exists in is interpreted as the fusion field>; [0155], discloses the point cloud for the first frame may serve as a reference frame point cloud <interpreted as the target fusion field>);
acquiring a display point cloud (Zhang: [0047], discloses a fused point cloud <display point cloud> from the reference frame point cloud <target fusion field>);
Zhang does not disclose
acquiring a display point cloud and weight of each display point in the display point cloud from the target fusion field,
and based on the weight of each display point in the display point cloud, determining each display point and a rendering color of a region in which each display point is located;
and rendering the display point cloud based on the rendering color, to obtain a rendering result and display the rendering result.
However, Landrieu discloses
acquiring a display point cloud and weight of each display point in the display point cloud (Landrieu: [1. Introduction], discloses acquiring a 3D point cloud <interpreted as the display point cloud>; [4. Regularizing soft labelings on a weighted graph], discloses calculating a confidence value <weight> of each point <display point> in the point cloud <display point cloud>),
and based on the weight of each display point in the display point cloud, determining each display point and a rendering color of a region in which each display point is located (Landrieu: [Fig. 5(d)], illustrates a rendered point cloud <display point cloud> with different colors; [Fig. 5. description] discloses based on the determined weight <confidence> of each point in the point cloud <each display point in the display point cloud>, a rendering color of a region <entirety/parts of buildings/objects> in which each point <display point> is located is represented <determined> in a range from green to red);
PNG
media_image1.png
432
464
media_image1.png
Greyscale
PNG
media_image2.png
78
1696
media_image2.png
Greyscale
and rendering the display point cloud based on the rendering color, to obtain a rendering result and display the rendering result (Landrieu: [Fig. 5(d)], illustrates a rendered point cloud based on the rendering color, obtaining a rendering result and displaying the rendering result).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the point cloud registration method disclosed by Zhang by implementing the confidence-based pseudo coloring as taught by Landrieu. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to visualize the discrepancy between high and low confidence areas in the registered point cloud.
Regarding Claim 9, it recites limitations similar to the method of Claim 1, but as an electronic device. As shown in the rejection the combination of Zhang and Landrieu disclose the method of Claim 1, the combination of Zhang and Landrieu further disclose
an electronic device (Zhang: [0103], discloses an electronic device), comprising:
a processor (Zhang: [0103], discloses a processor);
and a memory configured to store an executable instruction executed by the processor, wherein the processor is configured to read the executable instruction from the memory, and execute the executable instruction (Zhang: [0103], discloses a memory used to store executable program codes and the processor reads the executable program codes stored on the memory and executes a program corresponding to the executable program codes to carry out the method) to: …
Regarding Claim 10, it discloses limitations similar to Claims 1 and 9 but as a non-transitory computer-readable storage medium. As shown in the rejection, the combination of Zhang and Landrieu disclose the method and electronic device of Claims 1 and 9 respectively. The combination of Zhang and Landrieu further disclose
a non-transitory computer-readable storage medium, wherein the non- transitory computer-readable storage medium stores a computer program (Zhang: [Claim 24], discloses a non-transitory storage medium, wherein the storage medium is configured for storing executable program codes that, when executed, carry out a method), and the computer program is used to: …
Regarding Claims 2, 12, and 18, the combination of Zhang and Landrieu disclose the method, electronic device, and non-transitory computer-readable storage medium of Claims 1, 9, and 10 respectively. The combination of Zhang and Landrieu further disclose wherein updating the information of all control points in the fusion field according to the point cloud corresponding to the current frame scanning image, to obtain the target fusion field comprises:
acquiring parameter information of a point cloud corresponding to the current frame scanning image (Zhang: [0130-0131], discloses a first point cloud corresponding to a current frame <interpreted as the current frame scanning image> having position coordinates <acquired parameter information>; [0137], further teaches each point in a point cloud has both a position and a color);
performing calculation based on the parameter information, to obtain a score value of each control point (Zhang: [0043], discloses calculating distance between the first point and each second point in the reference frame <based on position, which is interpreted as parameter information to obtain a distance <score value of each control point>);
and obtaining the target fusion field based on the score value of each control point (Zhang: [0047], discloses obtaining the fused point cloud <derived from the reference frame point cloud and interpreted as the target fusion field> based on fusing points determined by their calculated distance value <score value of each control point>).
Regarding Claims 7 and 17, the combination of Zhang and Landrieu disclose the method and electronic device of Claims 1 and respectively. The combination of Zhang and Landrieu further disclose
acquiring all frame scanning images (Zhang: [0003], discloses acquiring each frame <frame scanning images>);
acquiring information of all control points in the fusion field according to point clouds corresponding to all frame scanning images, to obtain a total fusion field (Zhang: [0169], discloses acquiring all fused points in the coordinate system according to point clouds corresponding to a offset frame point cloud that has been transformed with the reference frame point cloud; [0035] discloses fusing each offset point cloud to the reference point cloud <all frame scanning images> <therefore obtaining a total fusion field>);
acquiring a total display point cloud from the total fusion field, (Zhang: [0172], discloses acquiring a fused point cloud <total display point cloud> from the fused points <total fusion field>);
and determining a total rendering color based on a weight set of the total display point cloud (Landrieu: [6.4 Experimental results], discloses color <total rendering color> is based on the confidence scores <weight set> of the point cloud <total display point cloud>)
and rendering the total display point cloud based on the total rendering color, to obtain a total rendering result and display the total rendering result (Landrieu: [Fig. 5], illustrates a rendered 3D point cloud <total display point cloud> based on color <total rendering color> <therefore obtaining a total rendering result and display the total rendering result>).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method disclosed by the combination of Zhang and Landrieu by rendering the point cloud based of color determined by confidence as further taught by Landrieu. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to visualize the discrepancy between high and low confidence areas in the registered point cloud.
Claims 5, 15, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Zhang and Landrieu in view of Ochiai et al. (US 2022/0397320 A1), hereinafter referenced as Ochiai.
Regarding Claims 5, 15, and 21, the combination of Zhang and Landrieu disclose the method, electronic device, and non-transitory computer-readable storage medium of Claims 1, 9, and 10 respectively. The combination of Zhang and Landrieu further disclose wherein based on the weight of each display point in the display point cloud, determining each display point and the rendering color of the region in which each display point is located comprises:
in a case that the weight of the display point in the display point cloud is (Landrieu: [Fig. 5(d)], illustrates in a case that the confidence <weight> of the point <display point> in the point cloud <display point cloud> is uncertain <a preset first threshold, lower confidence corresponds with uncertainty>, determining the rendering color of the region in which the point <display> point is located as red <a first color>);
PNG
media_image3.png
20
232
media_image3.png
Greyscale
in a case that the weight of the display point in the display point cloud is greater than the first threshold and less than a preset second threshold, determining the rendering color of the region in which the display point is located as a second color, wherein the first threshold is less than the second threshold (Landrieu: [Fig. 5(d)], illustrates in a case that the weight of the point <display point> in the point cloud <display point cloud> is greater than the first threshold and less than a confident <preset second, higher confidence corresponds with more confidence> threshold, determining the rendering color of the region in which the display point cloud is located as a color between green and red <a second color>, wherein the first threshold is less than the second threshold <first threshold is uncertain, second is confident>);
and in a case that the weight of the display point in the display point cloud is (Landrieu: [Fig. 5(d)], illustrates in a case that the weight of the point <display point> in the point cloud <display point cloud> is confident <preset second threshold>, determining the rendering color of the region in which the point is located as green <a third color>).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method or electronic device disclosed by the combination of Zhang and Landrieu by using confidence thresholds to determining the color of a point in the point cloud as further taught by Landrieu. One of ordinary skill in the art would be motivated to make this modification to visualize the discrepancy between high, medium, and low confidence areas in the registered point cloud.
The combination of Zhang and Landrieu fail to disclose
less than or equal to a preset first threshold
greater than or equal to a preset second threshold
However, Ochiai discloses
less than or equal to a preset first threshold (Ochiai: [0091], discloses less than or equal to a preset first threshold)
greater than or equal to a preset second threshold (Ochiai: [0091], discloses greater than or equal to a preset second threshold)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method disclosed by the combination of Zhang and Landrieu by defining metrics as less/greater than or equal to a respective first and second threshold as disclosed by Ochiai. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to enable clearer trend visibility with a tiered response system.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Zhang, Landrieu, and Ochiai in view of “Thresholding” (Github, 2022), hereinafter referenced as bioImageBook in further view of Hellman et al. (US 2013/0184777 A1), hereinafter referenced as Hellman.
Regarding Claims 6 and 16, the combination of Zhang, Landrieu, and Ochiai disclose the method and electronic device of Claims 5 and 15 respectively. The combination of Zhang, Landrieu, and Ochiai fail to disclose the limitations of Claims 6 and 16, however, bioImageBook discloses
acquiring an image frame sample set, wherein each image frame has corresponding weight (bioImageBook: [Fig. 64(A)], illustrates loading an image <interpreted as an image frame sample set, where each pixel is a 1x1 image frame and the set is the image>, wherein each pixel <image frame> has a corresponding value <weight>, see below);
PNG
media_image4.png
38
232
media_image4.png
Greyscale
PNG
media_image5.png
812
866
media_image5.png
Greyscale
determining total weight of the image frame sample set based on the weight of each image frame (bioImageBook: [Fig. 64(B)], illustrates graphing <determining> the total value <total weight> of the image frame sample set on a histogram based on the value of each pixel <weight of each frame>);
setting an initial threshold, and generating a rendering test result based on the total weight and the initial threshold (bioImageBook: [Fig. 64], illustrates setting thresh1 to 140 <an initial threshold> and rendering [Fig. 64(C)] based on the initial threshold which was determined based on the distribution of the total value <total weight>, see below);
PNG
media_image6.png
54
228
media_image6.png
Greyscale
and controlling the total weight remain unchanged, (bioImageBook: [Fig. 64], illustrates keeping the total value <weight> unchanged, , and setting <determining> a second threshold, see below).
PNG
media_image7.png
20
116
media_image7.png
Greyscale
the rendering test result (bioImageBook: illustrated in [Fig. 64(C-D)])
the initial threshold (bioImageBook: [Fig. 64(C)], illustrates an initial threshold set to 140)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method disclosed by the combination of Zhang, Landrieu, and Ochiai by thresholding as disclosed by bioImageBook. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to automate the segmentation of weight values, therefore saving time while obtaining reproducible results.
The combination of Zhang, Landrieu, Ochiai, and bioImageBook fail to disclose
adjusting the initial threshold, and until the rendering test result changes, determining the second threshold
However, Hellman discloses
adjusting the initial threshold, and until the ER changes, determining the pulse magnitude (Hellman: [0087], discloses the known threshold increases <adjusts> and until a change in ER is detected, determining the pulse magnitude)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method disclosed by the combination of Zhang, Landrieu, Ochiai, and bioImageBook by using changepoint detection as taught by Hellman. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification to precisely pinpoint a shift in data.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over the combination of Zhang and Landrieu in view of Yoshida (US 2004/0246496 A1), hereinafter referenced as Yoshida.
Regarding Claim 11, the combination of Zhang and Landrieu disclose the method Claim 1. The combination of Zhang and Landrieu fail to disclose the limitations of Claim 11, however, Yoshida discloses wherein
the fusion field is composed of control points uniformly distributed (Yoshida: [Fig. 20], illustrates a grating pattern where measurement points <control points> are uniformly distributed, therefore forming a grating pattern<therefore composing a fusion field>),
PNG
media_image8.png
376
452
media_image8.png
Greyscale
the control points are distributed on a scanning surface (Yoshida: [Fig. 20], illustrates measurement points <control points> distributed on a surface; [Abs], discloses a grating pattern composed of measurement points <fusion field composed of control points> projected on a surface),
and each control point controls scores within a certain range (Yoshida: [0025], discloses the measurement points are calculated <scored>; [Fig. 3], illustrates the measurement points can score within a certain range, between the light source of the light being projected and the furthest point of the wall or surface behind the object).
PNG
media_image9.png
530
814
media_image9.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply and/or modify the method disclosed by the combination of Zhang and Landrieu by using a structured pattern of light projected onto a surface and measurement points that score within a certain range as taught by Yoshida. One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to make this modification for fast data acquisition while maintaining high precision and detail.
Allowable Subject Matter
Claims 3-4, 13-14, and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lapa et al. (US 2009/0059241 A1) discloses projecting structured light.
Sharapov et al. (US 2020/0318955 A1), discloses a 3D scanner.
Xu et al. (CN 110009732 B) discloses point attributes including color, position, normal vector, and weight.
Whelan et al. (“ElasticFusion: Dense SLAM Without A Pose Graph”, 2016) discloses attributes of position, normal, color, and weight.
Erze (US 12051147 B1) discloses points of a point cloud having a color, texture, normal direction, position, or other values.
Kamal et al. (US 2020/0273147 A1) discloses assigning confidence values to control points.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISABELLA OCHSNER whose telephone number is (571)272-9322. The examiner can normally be reached 9:30 - 6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona Faulk can be reached at (571) 272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.O./Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights