DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Mai et al. (Method to perform 3D localization of Text in Shipboard Point Cloud Data Using Corresponding 2D Image, Feb 2021, hereinafter “Mai”)
Kang et al. (US 20220180610 A1, hereinafter “Kang”)
Li et al. (CN 115063459 A, hereinafter “Li”)
Rizk et al. (US 20230306740 A1, hereinafter “Rizk”)
Response to Arguments
Applicant's arguments filed on 01/21/2026 with regard to the 35 U.S.C. 112 rejection have been fully considered but they are persuasive. Therefore, the rejection under 35 U.S.C. 112 has been withdrawn.
Applicant's arguments filed on 01/21/2026 with regard to the 35 U.S.C. 103 rejection have been fully considered but they are moot in view of the new ground of rejection set forth below.
Applicant’s Arguments/Remarks:
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 7-9, 11-12, 14, 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Mai et al. (Method to perform 3D localization of Text in Shipboard Point Cloud Data Using Corresponding 2D Image, Feb 2021, hereinafter “Mai”) in view of Kang et al. (US 20220180610 A1, hereinafter “Kang”) and in view of Rizk et al. (US 20230306740 A1, hereinafter “Rizk”).
Regarding claims 1 and 11:
Lai teaches a method of object recognition with a virtual camera, comprising:
providing a three-dimensional (3D) point cloud and an associated two- dimensional (2D) panoramic image, each comprising at least one object of interest (Mail page 134 right column last paragraph, where Mai teaches Consider a colorized 3D point cloud of single scan, corresponding 2D RGB panorama image from the same location, and a one-to-one mapping describing that correspondence between them which we term the “grid" Note that the grid allows construction of the 2D image from the 3D point cloud and vice versa if needed. Further, if starting with just a 3D point cloud, one can construct a 2D panorama and define the grid mapping) ;
constructing a one-to-one grid map, wherein each point in the 3D point cloud correlates with a pixel in the 2D panoramic image (Mail page 134 right column last paragraph and page 435 right column first paragraph (Lai [0043]-[0044], where Lai teaches Such projection may include projecting or mapping each 3D image point in the 3D image to a pixel in the 2D image. In some embodiments, the system may project the complete 3D image on to the reference 2D image. Alternatively, the system may project an area of the 3D image that includes at least the object of interest on the reference 2D image. As such, rather than projecting all 3D image points to every available pixel in a 2D reference image, the system may first filter out pixels that are not pixels showing an image of the object of interest);
performing detection and localization on the object of interest (bullseye) (Mai page 436 left column last paragraph, where Mai teaches In this section, we will discuss the experiments and present the results from 3D text detection and localization of bullseyes on three ships, in order to validate the performance of the algorithm, wherein the texts in the bullseye are in a standard text format).
constructing a 3D bounding box around the object of interest (Mai page 435 fig. 4, where Mai teaches constructing 3D bounding box around the object).
obtaining an object recognition prediction based on the best synthetic image (Mai page 437, page 438 section C).
Lai fails to teach forming a virtual camera system around the bounding box oriented towards the object (bullseye) of interest; rotating the virtual camera around the bounding box, at a discrete angular intervals around a non-major axis of the bounding box wherein the virtual camera generates a plurality of synthetic images at discrete rotation angles by orthogonal projection using transformation matrix;
calculating a recognition score for each of the plurality of synthetic images using an optical character recognition algorithm; determining a best angle based on the recognition score; generating a best synthetic image based on the best angle;
However, Mai teaches Contending with grid mapping issues likely can be managed by a combination of averaging over nearby points (either in 2D or 3D) or extracting the 2D image directly from the 3D point cloud via projection. Virtually moving to a different perspective to construct a better 2D image (of the bullseye), especially in denser data combining several scan positions, may prove beneficial (Mai page 438 section C, where finding better image is to score every image and finding better image by performing rotation to improve the quality of the recognition task).
Kang further teaches the best view spot generating system 10 may dispose a bounding hemi-sphere capable of fully including the three-dimensional observation target and may then dispose the sample spots on the surface of the bounding hemi-sphere. The sample spots may be distributed and disposed at regular intervals. The best view spot generating system 10 may dispose virtual cameras at all the sample spots and may count the number of feature markers that are fully viewed from the virtual camera disposed at each sample spot. A way to count feature markers, which are described in operation S113, will be described in detail with reference to FIG. 5 (Kang [0004], [0044]-[0045], [0049-[0052], [0061]-[0062], a best view spot among the plurality of the view spots candidate is selected).
Therefore, taking the teachings of Lai and Kang as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to generate a plurality of image based on a plurality of virtual cameras places around an object of interest, in order to obtain and provide to a user an image of the object that makes it easy to observe and/or to recognize the object (Mai page 438 section C; Kang [0061]).
Mai in view of Kang fails to explicitly teach wherein automatically calculating a recognition score for each of the plurality of synthetic images using an optical character recognition algorithm.
However, Rizk teaches system and method for image verification wherein In embodiments, a fidelity score can be calculated based on the above comparisons to indicate the match between the stream segments before and after transmission over the network 150. In embodiments, a visual cue can be provided to the presenter/participants to indicate the results of the comparison. For example, a green light/red light or thumbs up/thumbs down can be displayed on the presenter's screen (e.g., within the web-based conference software), indicating whether the video feed quality is acceptable or not (e.g., based on a comparison between the fidelity score and a threshold). As another example, the fidelity score and the corresponding data used to calculate the fidelity score (e.g., OCR match percentage, STT match percentage, pixel match percentage, etc.) can be displayed to the presenter (Rizk [0019], [0028], [0048], [0056]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to calculate or determine the OCR score of all the images taken around the bounding box such that they can be compared and the image with the better score can be selected, in order to increase fidelity of the OCR and/or to chose the best image among the images taken.
Regarding claim 2:
Mai in view of Kang and in in view of Rizk teaches wherein detection and localization is performed on the 3D point cloud (Mai page 437, page 438 section C).
Regarding claim 3:
Mai in view of Kang and in in view of Rizk teaches wherein detection and localization is performed on the 2D panoramic image, and utilizes the grid map to extrapolate the 2D localization to a 3D localization (Mai page 438).
Regarding claim 4:
Mai in view of Kang and in in view of Rizk teaches wherein calculating a bounding box around the object of interest further comprises:
determining a 2D midpoint of the object of interest (Mai page 435 right column; page 436 right column);
mapping the 2D midpoint to the 3D point cloud via the grid map; calculating a 3D centroid of the object of interest; and cropping the 3D point cloud to a fixed distance of the 3D centroid, wherein the fixed distance enables efficient processing (Mai 434 right column).
Regarding claims 5 and 15:
Mai in view of Kang and in in view of Rizk teaches further comprising: calculating an ideal normal vector of the object of interest; and wherein the virtual camera system’s orientation is opposite a normal vector of the bounding box (Mai page 435 right column, page 437).
Regarding claims 7 -9, and 17-19:
Mai in view of Kang and in in view of Rizk fails to recite explicitly the limitations of Claims 7-9. However, Mai in view of Kang teaches determining the best position and angle of the virtual camera (rotate the virtual camera around the object maintaining a desired radius) such that the best image can be taking and used for the object localization and recognition It would have been obvious to one having ordinary skill in the art at the time the invention was made to obtain the position and perform the rotation similar to applicant’s claims, since it has been held that where the general conditions of a claim are disclosed in the prior art, discovering the optimum or workable ranges involves only routine skill in the art. In re A11er, 105 USPQ 233.
Regarding claim 12:
Mai in view of Knag teaches wherein detecting and locating the bullseye further comprises:
utilizing a hierarchical clustering distance method to eliminate text that does not
meet the standardized format (Mai page 436 last paragraph page 437 right column, where when feature to our text (standard text) or interest is considered as source of error and when the error is large it is become false positive and therefore would be good to be removed).
Regarding claim 14:
Mai in view of Knag teaches further comprising:
extrapolating a 3D bounding box from the 2D bounding box; determining a centroid of a 3D bounding box; cropping the point cloud area to within about 0.5 meters of the centroid; performing plane estimation on the 3D bounding box; and determining a normal vector of the 3D bounding box based on the plane estimation (Mai 434 right column, page 435 right column page 436 right column).
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Mai et al. (Method to perform 3D localization of Text in Shipboard Point Cloud Data Using Corresponding 2D Image, Feb 2021, hereinafter “Mai”) in view of Kang et al. (US 20220180610 A1, hereinafter “Kang”) in view of Rizk et al. (US 20230306740 A1, hereinafter “Rizk”) and in view of Li et al. (CN 115063459 A, hereinafter “Li”).
Regarding claim 6:
Mai in view of Kang and in view of Rizk fails to teach further comprising: down-sampling the 3D point cloud; removing a plurality of points in a low-density region of the 3D point cloud to reduce noise.
However, Li teaches pre-processing the first sample point cloud and the second sample point cloud, obtaining the first low density sample point cloud and the second low density sample point cloud, wherein the pre-processing comprises down-sampling processing and noise reduction processing. the point of the first low density sample point cloud is reduced relative to the point of the first sample point cloud, the point number of the second low density sample point cloud is reduced relative to the point number of the second sample point cloud. wherein the downsampling process can reduce the density of the point cloud The original point cloud may have hundreds of thousands of points, and the point cloud after the down-sampling process is about 200 thousand points. The denoising process can delete the outlier. Because the point around the group point is few, the local feature can be extracted is less, so the group point for subsequent registration of the significance is not large (Li [0058]-61]).
Therefore, taking the teaching of Mai, Knag, Rizk and Li as a whole, it would have been obvious to one with ordinary skill in the art before the effective filing date of the application to down-sample the 3D point cloud; removing a plurality of points in a low-density region of the 3D point cloud to reduce noise, in order for the group point for subsequent registration are not significantly large, for example.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 March 24, 2026
Prosecution Insights