DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 3/29/26 have been fully considered but they are not persuasive.
Regarding applicant’s argument for claim 11, on page 10, that Taghavi does not disclose a shared backbone network and a plurality of head networks, examiner disagrees. Taghavi discloses multiple object classifiers (e.g. column 13, lines 20-25) that are treated in various independent ways (e.g. figure 5, items 504 and 506) for classification/labeling which examiner has interpreted as “head networks” (examiner notes that this interpretation appears to be congruent with applicant’s own use of the term “head network” as shown in figure 3). The “shared backbone network” has essentially been interpreted as the learned model 224 itself as the model creates the cross class confusion matrices (figure 5) that diffuse the information (i.e. loss) across the head networks during retraining of the model. Therefore the argument is overcome and the previous rejection remains.
Regarding applicant’s argument for claim 11, on page 10, that Taghavi does not disclose a loss for each class, examiner disagrees. Figure 4, item 410 specifically discloses a step of determining deficient object classes using accuracy thresholds (column 15, line 32 – column 16, line 8) which is interpreted as determining a “loss”. Therefore the argument is overcome and the previous rejection remains.
Regarding applicant’s argument for claim 1, on page 11, that Taghavi does not disclose object associated position information that defines a placement region, examiner disagrees. Column 17, lines 16-23 and 35-50 details specific spatial rules for where a particular object class can be placed including only corresponding point cloud frames, point cloud frames being “placement regions”. Therefore the argument is ovcercome and the previous rejection remains.
Claim Objections
Claim 21 is objected to because of the following informalities: word “wights” in the 4th line should be “weights”. Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
1) Claim(s) 1-8, 10-16, 18 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. patent 11,410,388 by Taghavi et al.
2) Regarding claim 1, Taghavi teaches a method comprising: obtaining, by a processor (figure 2, item 202; a processor), at least one virtual object dataset from a point cloud database (figure 3, item 214; column 11, lines 20-33; point cloud objects can be obtained from dataset); obtaining learning datasets by adding, based on object-associated position information that defines for each object class, a corresponding placement region in a scene (column 17, lines 16-23 and 35-50; details specific spatial rules for where a particular object class can be placed including only corresponding point cloud frames, point cloud frames being “placement regions”), the at least one virtual object dataset to a first dataset of at least one object (column 21, lines 23-26; object is added to a scene based upon context dependent policy detailed in column 17, lines 7-23 where a pedestrian, for example, can only be added to an appropriate position in a scene); training a LiDAR-based deep learning object perception apparatus based on the learning datasets (column 21, lines 42-54; model is trained based on LIDAR point clouds [column 9, lines 32-53]); and identifying, via the trained LiDAR-based deep learning object perception apparatus, the at least one object (figure 4; column 13, line 65 – column 14, line 4; objects are labeled [i.e. identified] through each iteration).
3) Regarding claim 2, Taghavi teaches the method of claim 1, wherein the at least one virtual object dataset comprises at least one dynamic object dataset obtained from at least one point cloud dataset of the point cloud database (column 12, table 1; augment objects can be any of the dynamic objects listed, e.g. a pedestrian).
4) Regarding claim 3, Taghavi teaches the method according to claim 2, wherein the at least one dynamic object dataset comprises at least one of: a pedestrian, a bicycle, a motorcycle, a passenger vehicle, a truck, a bus, a trailer, or construction equipment (column 12, table 1; augment objects can be any of the dynamic objects listed, e.g. a pedestrian).
5) Regarding claim 4, Taghavi teaches the method according to claim 1, wherein the object-associated position information for the at least one object is determined according to point label information of the first dataset (column 12, lines 36 – 65; scene dictionary provides point cloud for labeled objects).
6) Regarding claim 5, Taghavi teaches the method according to claim 4, wherein the point label information comprises at least one of: a sidewalk, a drivable area, or a road (column 17, lines 7-23; sidewalk and drivable area are among the point cloud label information of scene types).
7) Regarding claim 6, Taghavi teaches the method according to claim 5, wherein the object-associated position information comprises at least one of: an association between: at least one of: a motorcycle, a passenger vehicle, a truck, a bus, a trailer, or construction equipment, and the road, an association between a pedestrian and the sidewalk, or an association between a bicycle and one of the sidewalk or the road (column 17, lines 7-23; pedestrian and sidewalk are associated).
8) Regarding claim 7, Taghavi teaches the method of claim 1, wherein the LiDAR-based deep learning object perception apparatus comprises: non-transitory memory storing object perception software based on a deep learning model (figure 2, item 208 a memory); and at least one processor (figure 2, item 202; a processor) executing the object perception software in the non-transitory memory, and wherein the deep learning model comprises a shared backbone network and a plurality of head networks that each outputs a loss for each class (column 13, line 58 – column 14, line 30 and column 18, line 29-61; confusion analysis [i.e. a loss function] applies confusion analysis to each individual object to find deficient objects [i.e. “head” networks] and then combines the analysis [column 18, line 55] for object combination predictions [a “backbone” network]).
9) Regarding claim 8, Taghavi teaches the method of claim 7, further comprising outputting, via the deep learning model, a final loss, wherein the final loss comprises a weighted sum of losses output from the plurality of head networks (column 18, line 55; object accuracies represented as mean intersection over union [weighted] are summed).
10) Regarding claim 10, Taghavi teaches the method of claim 1, wherein the point cloud database comprises at least one of: a nuScenes dataset or a Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset. (column 2, lines 16-38; nuScenes and KITTI are disclosed).
11) Claims 11 is taught in the same manner as described in the rejections of claims 1 and 7 above.
12) Claims 12-16, 18 and 20 are taught in the same manner as described in the rejections of claims 2-6, 8 and 10 above, respectively.
13) Claim 22 is taught in the same manner as described in the rejection of claim 1 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13) Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. patent 11,410,388 by Taghavi et al. as applied to claims 1 and 11 above, and further in view of U.S. patent application publication 2022/0245343 by Brun et al.
Taghavi does not specifically teach the method of claim 18, further comprising determining the final loss based on a dynamic weight average (DWA) scheme.
Brun teaches the method of claim 18, further comprising determining the final loss based on a dynamic weight average (DWA) scheme (paragraphs 83 and 84; DWA is utilized to balance loses corresponding to multiple outputs).
Taghavi and Brun are combinable because they are both from the neural network loss function field of endeavor.
It would have been obvious to a person of ordinary skill in the art at the time the invention was effectively filed to combine Taghavi with Brun to add a DWA scheme. The motivation for doing so would have been to balance multiple loss outputs (paragraph 83). Therefore it would have been obvious to combine Taghavi and Brun to obtain the invention of claim 19.
Allowable Subject Matter
Claim 21 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN O DULANEY whose telephone number is (571)272-2874. The examiner can normally be reached Mon-Fri 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abderrahim Merouan can be reached at (571)270-5254. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
BENJAMIN O. DULANEY
Primary Examiner
Art Unit 2676
/BENJAMIN O DULANEY/ Primary Examiner, Art Unit 2683
Prosecution Insights