DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
In response to applicant’s amendment received on 5/8/26, all requested changes to the specification and claims have been entered. Claims 1-21 were previously and are currently pending. The amendments have resolved the pending objection of claim 1 and the 112(b) rejection of claim 10. The amendment has also resolved one of the 112(b) rejections associated with claim 18, however “storing the processed plurality of predicted paths” is still indefinite as indicated below.
Response to Arguments
Applicant’s arguments with respect to claims 1, 10 and 18 have been considered but are moot because the new ground of rejection, necessitated by the amendment, does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 18 recites the limitation "storing the processed plurality of predicted paths" in line 8. There is insufficient antecedent basis for this limitation in the claim. Specifically, it’s unclear what is meant by “the processed” portion of the limitation. There is not a step of processing the paths prior to this step like there is in other independent claims, however there is the step of “generating a plurality of predicted paths”, is the limitation in question here supposed to be a reference to those generated paths?
Claims 19-21 are rejected by the virtue of their dependency upon rejected claim 18.
Claim Rejections - 35 USC § 112(d)
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 15 and 16 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 15 recites “processing the plurality of predicted paths of travel based on at least one additional ground truth label” which is already disclosed in parent claim 10, lines 12-13. Therefore claim 15 fails to further limit the subject matter of claim 10 upon which it depends.
Claim 16 recites “wherein further processing the plurality of predicted paths of travel based on at least one additional ground truth label includes identifying at least one static object that may interfere with a predicted path of travel” which is already disclosed in claim 10, lines 12-16. Therefore claim 16 fails to further limit the subject matter of claim 10 upon which it depends.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-21 are rejected under 35 U.S.C. 103 as being unpatentable over US 2020/0174481 to Van Heukelom et al. (“Van Heukelom”) (from the IDS filed 4/15/24 Citation No. “A438”) in view of US 2019/0367021 to Zhao et al. (“Zhao”).
Regarding claim 1, Van Heukelom discloses a system for managing vision systems in vehicles, the system comprising:
one or more computing systems including processing devices and memory, that execute computer-executable instructions (Fig. 7; paragraphs 75, 113 and 114), for implementing a vision system processing component (Fig. 8, paragraph 118) operative to:
obtain first ground truth label data associated with collected vision data from one or more vision systems, wherein the obtained first ground truth label data corresponds to attributes of travel surfaces including at least one of road edges ground truth labels, lane line ground truth labels, or road markings (Fig. 8, element 806; paragraphs 12, 31-32 and 121, wherein lane lines, lane dividers (i.e. road edges) and other road markings (e.g. crosswalks) are detected/labeled in captured images (i.e. vision data));
obtain second ground truth label data associated with collected vision data from one or more vision systems, wherein the obtained second ground truth label data corresponds to attributes of one or more detected dynamic objects (Fig. 8, element 806; paragraphs 12, 30, 32 and 121, wherein attributes associated with vehicles and pedestrians (i.e. dynamic objects) are detected/labeled in captured images (i.e. vision data));
process the obtained first and second ground truth label data associated with the collected vision data to form a plurality of predicted paths of travel, wherein each individual predicted path of travel is associated with a confidence value (Fig. 1; Fig. 8, element 806; paragraphs 13-16, 31-35 and 121, wherein the labeled/detected data is used to predict a probabilities (i.e. confidence value) associated with possible paths of travel);
process the plurality of predicted paths of travel based on at least one additional ground truth label (Fig. 1; Fig. 8, element 808; paragraphs 38 and 122, wherein an overlapping region (130) and region probability are determined based a future region of a vehicle (112*) (i.e. additional ground truth label data) and prediction probabilities (124) (i.e. predicted paths of travel), which corresponds to processing the paths based on at least one additional label),
store the process plurality of predicted paths of travel and associated confidence value (Fig. 8, elements 812-820; paragraphs 38, 39, 124-129, wherein the overlapping regions and probabilities, comprising predicted paths and confidence values, are stored and further processed to, for example, determine collision risk).
As indicated by the double strikethroughs above, the prior art of Van Heukelom does not disclose expressly wherein the at least one additional ground truth label identifies a static object on the travel surface, and wherein processing the plurality of predicted paths of travel comprises modifying or removing at least one predicted path of travel generated for a detected dynamic object based on the static object.
Zhao discloses a similar process of managing vision systems in vehicles that includes processing a plurality of predicted paths of travel based on additional ground truth label that identifies a static object on the travel surface, and modifying or removing at least one predicted path of travel generated for a detected dynamic object based on the static object (Fig. 8 and paragraphs 158-172, wherein the drivable area and trajectory (i.e. predicted paths) of vehicle is adjusted/modified based identified static objects and their effect on other dynamic objects).
Van Heukelom & Zhao are combinable because they are from the same art of image processing, specifically managing vision systems in vehicles.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of processing a plurality of predicted paths of travel based on additional ground truth label that identifies a static object on the travel surface, and modifying or removing at least one predicted path of travel generated for a detected dynamic object based on the static object, as taught by Zhao, into the system for managing vision systems in vehicles disclosed by Van Heukelom.
The suggestion/motivation for doing so would have been to provide/identify safe and not safe drivable areas based on static and dynamic objects (Zhao, paragraphs 03 and 158).
Therefore, it would have been obvious to combine Zhao with Van Heukelom to obtain the invention as specified in claim 1.
Regarding claim 2, the combination of Van Heukelom and Zhao discloses the system as recited in Claim 1,wherein the vision system processing component processes the obtained first and second ground truth label data associated with the collected vision data to form a plurality of predicted paths of travel based on selecting potential paths of travel exceeding a minimal confidence value threshold (Van Heukelom, Fig. 8, elements 812-818; paragraphs 124-129, wherein paths associated with overlapping regions exceeding a minimal confidence/probability threshold value are selected for further processing).
Regarding claim 3, the combination of Van Heukelom and Zhao discloses the system as recited in Claim1, wherein the first and second ground truth label data corresponds to one or more objects detected within a horizon of the captured video data (Van Heukelom, Fig. 1, element 104; paragraph 27, wherein an image/video captured by the vehicle captures the first and second label/detected data within a horizon perspective).
Regarding claim 4, the combination of Van Heukelom and Zhao discloses the system as recited in Claim 3, wherein the first and second ground truth label data corresponds to one or more objects detected beyond a current defined location of the vehicle (Van Heukelom, Fig. 1, element 104; paragraph 27, wherein an image/video captured by the vehicle captures the first and second label/detected data surrounding/beyond the vehicle).
Regarding claim 5, the combination of Van Heukelom and Zhao discloses the system as recited in Claim 1, wherein the attributes of one or more detected dynamic objects corresponds to at least one of yaw, velocity or acceleration of the dynamic object (Van Heukelom, paragraphs 12, 13, 77 and 82, wherein yaw, velocity and acceleration of dynamic objects (e.g. pedestrian, animal, other vehicles, etc.) are detected).
Regarding claim 6, the combination of Van Heukelom and Zhao discloses the system as recited in Claim 1,wherein the vision system processing component processes the plurality of predicted paths of travel based on at least one additional ground truth label by identifying at least one static object that may interfere with a predicted path of travel (Van Heukelom, Fig. 1; Fig. 8, element 808; paragraphs 12, 38 and 122, wherein static objects (e.g. lane lines, lane dividers, crosswalks) correspond to first or additional labels that are identified and may interfere with the predicted path associated with the overlapping regions).
Regarding claim 7, the combination of Van Heukelom and Zhao discloses the system as recited in Claim 1,wherein a sum of confidence values associated with two or more of the plurality of predicted paths of travel exceeds 100% (Van Heukelom, paragraph 35, wherein the path prediction probability distribution adds up to 1 corresponding to 100%).
Regarding claim 8, the combination of Van Heukelom and Zhao discloses the system as recited in Claim 1, wherein a sum of confidence values associated with the plurality of predicted paths of travel does not exceed 100% (Van Heukelom, paragraph 35, wherein the path prediction probability distribution adds up to 1 corresponding to 100%).
Regarding claim 9, the combination of Van Heukelom and Zhao discloses the system as recited in Claim1,wherein the vision system processing component processes the plurality of predicted paths of travel based on modeled feasibility cone for a detected dynamic object (Van Heukelom, Fig. 3 and paragraphs 51-52, wherein the heat map of prediction probabilities corresponds to a “modeled feasibility cone” for the dynamic object/vehicle).
Regarding claim 10, Van Heukelom discloses a method for managing vision systems in vehicles, the method comprising:
obtaining first ground truth label data associated with collected vision data from one or more vision systems, wherein the obtained first ground truth label data corresponds to attributes of travel surfaces (Fig. 8, element 806; paragraphs 12, 31-32 and 121, wherein attributes of the travel surface like lane lines, lane dividers (i.e. road edges) and other road markings (e.g. crosswalks) are detected/labeled in captured images (i.e. vision data));
obtaining second ground truth label data associated with collected vision data from one or more vision systems, wherein the obtained second ground truth label data corresponds to attributes of one or more detected dynamic objects (Fig. 8, element 806; paragraphs 12, 30, 32 and 121, wherein attributes associated with vehicles and pedestrians (i.e. dynamic objects) are detected/labeled in captured images (i.e. vision data));
processing the obtained first and second ground truth label data associated with the collected vision data to form a plurality of predicted paths of travel, wherein each individual predicted path of travel is associated with a confidence value (Fig. 1; Fig. 8, element 806; paragraphs 13-16, 31-35 and 121, wherein the labeled/detected data is used to predict a probabilities (i.e. confidence value) associated with possible paths of travel);
processing the plurality of predicted paths of travel based on at least one additional ground truth label (Fig. 1; Fig. 8, element 808; paragraphs 38 and 122, wherein an overlapping region (130) and region probability are determined based a future region of a vehicle (112*) (i.e. additional ground truth label data) and prediction probabilities (124) (i.e. predicted paths of travel), which corresponds to processing the paths based on at least one additional label),
storing the process plurality of predicted paths of travel and associated confidence value (Fig. 8, elements 812-820; paragraphs 38, 39, 124-129, wherein the overlapping regions and probabilities, comprising predicted paths and confidence values, are stored and further processed to, for example, determine collision risk).
As indicated by the double strikethroughs above, the prior art of Van Heukelom does not disclose expressly wherein the at least one additional ground truth label identifies a static object on a travel surface, and wherein processing the plurality of predicted paths of travel comprises modifying or removing at least one predicted path of travel generated for a detected dynamic object based on the static object.
Zhao discloses a similar process of managing vision systems in vehicles that includes processing a plurality of predicted paths of travel based on additional ground truth label that identifies a static object on the travel surface, and modifying or removing at least one predicted path of travel generated for a detected dynamic object based on the static object (Fig. 8 and paragraphs 158-172, wherein the drivable area and trajectory (i.e. predicted paths) of vehicle is adjusted/modified based identified static objects and their effect on other dynamic objects).
Van Heukelom & Zhao are combinable because they are from the same art of image processing, specifically managing vision systems in vehicles.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of processing a plurality of predicted paths of travel based on additional ground truth label that identifies a static object on the travel surface, and modifying or removing at least one predicted path of travel generated for a detected dynamic object based on the static object, as taught by Zhao, into the system for managing vision systems in vehicles disclosed by Van Heukelom.
The suggestion/motivation for doing so would have been to provide/identify safe and not safe drivable areas based on static and dynamic objects (Zhao, paragraphs 03 and 158).
Therefore, it would have been obvious to combine Zhao with Van Heukelom to obtain the invention as specified in claim 10.
Regarding claims 11, 12, 14, 16 and 17, please refer to the rejections of claims 2, 3, 5, 6 and 9, respectively, above.
Regarding claim 13, the combination of Van Heukelom and Zhao discloses the method as recited in Claim 10,wherein the obtained first ground truth label data corresponds to attributes of travel surfaces including at least one of road edges ground truth labels, lane line ground truth labels, or road markings (Van Heukelom, Fig. 8, element 806; paragraphs 12, 31-32 and 121, wherein attributes of the travel surface like lane lines, lane dividers (i.e. road edges) and other road markings (e.g. crosswalks) are detected/labeled in captured images (i.e. vision data)).
Regarding claim 15, the combination of Van Heukelom and Zhao discloses the method as recited in Claim 10 further comprising processing the plurality of predicted paths of travel based on at least one additional ground truth label (Van Heukelom, Fig. 1; Fig. 8, element 808; paragraphs 38 and 122, wherein an overlapping region (130) and region probability are determined based a future region of a vehicle (112*) (i.e. additional ground truth label data) and prediction probabilities (124) (i.e. predicted paths of travel), which corresponds to processing the paths based on at least one additional label).
Regarding claim 18, Van Heukelom discloses a method for managing vision systems in vehicles, the method comprising:
obtaining ground truth label data associated with collected vision data from one or more vision systems, wherein the obtained first ground truth label data corresponds to attributes of travel surfaces and one or more detected dynamic objects (Fig. 8, element 806; paragraphs 12, 30-32 and 121, wherein attributes of the travel surface like lane lines, lane dividers (i.e. road edges) and other road markings (e.g. crosswalks) as well as vehicles and pedestrians (i.e. dynamic objects) are detected/labeled in captured images (i.e. vision data));
generating a plurality of predicted paths of travel based on the obtained ground truth label data associated with the collected vision data, wherein each individual predicted path of travel is associated with a confidence value (Fig. 1; Fig. 8, element 806; paragraphs 13-16, 31-35 and 121, wherein the labeled/detected data is used to predict a probabilities (i.e. confidence value) associated with possible paths of travel),
storing the process plurality of predicted paths of travel and associated confidence value (Fig. 8, elements 812-820; paragraphs 38, 39, 124-129, wherein the overlapping regions and probabilities, comprising predicted paths and confidence values, are stored and further processed to, for example, determine collision risk).
As indicated by the double strikethroughs above, the prior art of Van Heukelom does not disclose expressly wherein the plurality of predicted paths of travel correspond to future paths of travel of the one or more detected dynamic objects predicted based on an existence of one or more static objects.
Zhao discloses a similar process of managing vision systems in vehicles including predicting paths of travel that take into account, and therefore correspond to/with, future paths of travel of the one or more detected dynamic objects predicted based on an existence of one or more static objects (Fig. 8 and paragraphs 158-172, wherein the drivable area and trajectory (i.e. predicted paths) of a vehicle is adjusted/modified based identified static objects and their effect on future paths of travel associated with other dynamic objects).
Van Heukelom & Zhao are combinable because they are from the same art of image processing, specifically managing vision systems in vehicles.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention to incorporate the technique of predicting paths of travel that take into account, and therefore correspond to/with, future paths of travel of the one or more detected dynamic objects predicted based on an existence of one or more static objects, as taught by Zhao, into the system for managing vision systems in vehicles disclosed by Van Heukelom.
The suggestion/motivation for doing so would have been to provide/identify safe and not safe drivable areas based on static and dynamic objects (Zhao, paragraphs 03 and 158).
Therefore, it would have been obvious to combine Zhao with Van Heukelom to obtain the invention as specified in claim 18.
Regarding claims 19-21, please refer to the rejections of claims 2, 13 and 5, respectively, above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See attached PTO-892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON W CARTER whose telephone number is (571)272-7445. The examiner can normally be reached 8am - 5pm (Mon - Fri).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON W CARTER/Primary Examiner, Art Unit 2661
Prosecution Insights