DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: LIDAR object-detection system that fuses deep-learning detection results with signal processing detection results.
The disclosure is objected to because of the following informalities: Para [00129] (see below) recites two different outcomes for the same condition, “overlap ratios r^S_i,j is equal to or smaller than the predetermined reference value t_s,h,”. One of theses instances should instead be recites as, “… greater than…”. According to corresponding Fig. 6: S403, S405 (see below), Examiner believed that the first recitation should be changed. Appropriate correction is required.
PNG
media_image1.png
612
1563
media_image1.png
Greyscale
PNG
media_image2.png
1067
944
media_image2.png
Greyscale
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 and 16 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by machine translation of KR 102597652 B1 (Kim).
As per claim 1, Kim teaches an object detection apparatus including:
a processor configured to detect an object based on deep learning using data obtained from Light Detection and Ranging (LiDAR) and to detect the object based on signal processing (Kim:
para 2: “3D point cloud clustering and deep learning fusion object detection enhanced recognition method, device, and computer program”;
para 6: “deep learning-based object recognition method is a method of recognizing objects by analyzing sensor data (e.g., point cloud data) using a deep learning-based artificial intelligence model previously learned using learning data.”;
paras 8: “recognizing objects by fusion of deep learning-based object recognition method and rule-based object recognition method, more accurate object recognition”;
paras 14, 49, 94, 95, 97-99, 102-104: LIDAR;
para 10: “The 3D point cloud clustering and deep learning fusion object detection enhanced recognition method according to an embodiment of the present invention to solve the above-described problem is a method performed by a computing device, based on deep learning. Deriving a first object recognition result by analyzing point cloud data corresponding to the region, deriving a second object recognition result by analyzing the point cloud data based on a rule base, and deriving the result.”;
para 11: “the step of deriving the first object recognition result includes identifying one or more first objects by analyzing the point cloud data through a pre-learned object recognition model, and identifying the first object as a result of first object recognition. and deriving information about one or more first objects, wherein the pre-learned object recognition model uses a plurality of point cloud data labeled with information about the object as training data, and uses a pre-learned deep It could be a running model.”;
para 12: “deriving the second object recognition result may include extracting one or more regions of interest from the point cloud data, and selecting at least one point among a plurality of points included in the extracted one or more regions of interest. It may include clustering, identifying one or more second objects based on attributes of the at least one clustered point, and deriving information about the identified one or more second objects as a result of second object recognition”;
paras 14-15: clustering;
para 20: “analyzing the point cloud data based on a rule base”;
para 71: “Referring to FIG. 3, in step S110, the computing device 100 may derive a first object recognition result by analyzing point cloud data corresponding to a predetermined area based on deep learning. For example, the computing device 100 may recognize objects included in the point cloud data by analyzing the point cloud data through a deep learning-based learning model. Hereinafter, it will be described in more detail with reference to FIG. 4.”;
“72. Figure 4 is a flowchart for explaining a deep learning-based object recognition method in various embodiments.”;
Para 76: deep neural network;
Fig. 3 (shown below): mainly s110-s120;
Fig. 4
PNG
media_image3.png
594
848
media_image3.png
Greyscale
Fig. 5
PNG
media_image4.png
732
806
media_image4.png
Greyscale
); and
a storage operatively connected to the processor and configured to store algorithms and data driven by the processor (Kim:
Fig. 1 (shown below): mainly 100, 200;
Fig. 2 (shown below): mainly 120, 150, 151;
Fig. 2
PNG
media_image5.png
775
941
media_image5.png
Greyscale
),
wherein the processor is configured to output a final object detection result by fusing a result of the detecting based on the deep learning and a result of the detecting based on the signal processing (Kim:
para 2: “3D point cloud clustering and deep learning fusion object detection enhanced recognition method, device, and computer program”;
paras 8: “recognizing objects by fusion of deep learning-based object recognition method and rule-based object recognition method, more accurate object recognition”;
para 23: “recognizing objects by fusing a deep learning-based object recognition method and a rule-based object recognition method, not only more accurate object recognition is possible, but also a relatively further distance compared to existing object recognition methods. It has the advantage of being able to more accurately recognize objects located in the distance”;
para 47: “perform 3D point cloud clustering and deep learning fusion object detection enhanced recognition processes”;
para 51: “the computing device 100 may be connected to the user terminal 200 through the network 400, and may perform 3D point cloud clustering and deep learning fusion in response to an object recognition request obtained from the user terminal 200. The object detection enhanced recognition process can be performed, and the results (e.g., object recognition results) derived by performing the 3D point cloud clustering and deep learning fusion object detection enhanced recognition process can be provided to the user terminal 200”;
para 55: “Hereinafter, with reference to FIG. 2, the hardware configuration of the computing device 100 that performs the 3D point cloud clustering and deep learning fusion object detection enhanced recognition method will be described”;
“82. Referring again to FIG. 3, in step S120, the computing device 100 may derive a second object recognition result by analyzing point cloud data corresponding to a predetermined area based on a rule base. For example, the computing device 100 may recognize objects included in the point cloud data by analyzing the point cloud data based on predefined rules. Hereinafter, it will be described in more detail with reference to FIG. 5.”;
“84. Referring to FIG. 5 , in step S310, the computing device 100 may extract one or more regions of interest (ROI) from point cloud data corresponding to a predetermined region.”;
Fig. 3 (shown below): mainly S130;
Fig. 3
PNG
media_image6.png
878
932
media_image6.png
Greyscale
Fig. 1 (shown below): mainly 200;
Fig. 1
PNG
media_image7.png
869
1231
media_image7.png
Greyscale
).
As per claim(s) 16, arguments made in rejecting claim(s) 1 are analogous.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Kim as applied to claim 1 above, and further in view of Official Notice.
As per claim 2, Kim teaches the object detection apparatus of claim 1, wherein the processor is further configured:
to perform
to determine an overlap ratio between a
to determine association between the object detected based on the deep learning and the object detected based on the signal processing (Kim:
“19. In various embodiments, the step of determining whether the object is the same includes comparing the sum of the area of the first area corresponding to the first object and the area of the second area corresponding to the second object. If the ratio of overlapping areas between regions is greater than or equal to a preset reference value, the method may include determining that the first object and the second object are the same object.
“113. In various embodiments, the computing device 100 may perform an Intersection over Union (IoU) operation between a first object recognition result and a second object recognition result. For example, the computing device 100 compares the area of the first area corresponding to the first object with the area of the second area corresponding to the second object (the union of the area of the first area and the area of the second area). The ratio of the overlapped area between the first area and the second area (intersection of the area of the first area and the area of the second area) can be calculated.”;
“114. Thereafter, the computing device 100 selects the information included in the first object recognition result based on the comparison result between the information about the first object included in the first object recognition result and the information about the second object included in the second object. It may be determined whether the first object and the second object included in the second object recognition result are the same object. For example, the computing device 100 may determine that the first object and the second object are the same object when the result derived from performing the IoU operation is greater than or equal to a preset reference value (e.g., 0.5), and the first object One object ID corresponding to the and second objects can be created and assigned to the first object and the second object.”;
“115. Meanwhile, if the result derived from performing the IoU operation is less than a preset reference value, the computing device 100 may determine that the first object and the second object are different objects, and the first object and the second object may be Object IDs corresponding to each can be individually created and assigned to the first object and the second object, respectively.”).
Kim does not teach bounding box. Examiner provides Official Notice that these limitations were well known prior to filing.
One of ordinary skill in the art, prior to filing, would have recognized the advantage of being fundamental tools in computer vision used to localize and identify objects within images or videos with advantages being their simplicity, computational efficiency, and ease of annotation, making them the industry standard for large-scale machine learning tasks bounding box processing. The teachings of the prior art could have been incorporated into Kim in that the object region is designated with a bounding box.
Allowable Subject Matter
Claims 3-15 and 17-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: Considering claims 3 and 17, from which all other claims objected to as containing allowing subject matter depend, limitations pertaining to “deep learning-based overlap ratio” and “signal processing-based overlap ratio”, in conjunction with other limitations present in the instant claims, corresponding intervening claims, and corresponding independent claim(s), distinguish over the prior art.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Atiba Fitzpatrick whose telephone number is (571) 270-5255. The examiner can normally be reached on M-F 10:00am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on (571) 270-5183. The fax phone number for Atiba Fitzpatrick is (571) 270-6255.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Atiba Fitzpatrick
/ATIBA O FITZPATRICK/
Primary Examiner, Art Unit 2677
Prosecution Insights