DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 03/10/2026 and 04/27/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Response to Amendment
The Amendment filed on 02/10/2026 has been entered. Claims 1 and 3-6 are pending in the application. In response to Applicant's amendments, Examiner withdraws the previous objections to claims 1-3; withdraws the previous rejections; and maintains the previous objection to claim 4. Examiner notes that all previous objections and rejections pertaining to canceled claim 2 are withdrawn.
Response to Arguments
Applicant’s arguments with respect to claims 1 and 3-6 regarding 35 U.S.C. 102 have been fully considered. Applicant’s argument that “Mori fails to disclose the use of AI technology to recognize surrounding targets” (page 8) regarding amended independent claim 1 is persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Bill et al. (US 20220101723 A1), which discloses object recognition using spatial features extracted by a machine learning models from sensor data [0099].
Claim Objections
Claims 4 is objected to because of the following informalities:
Claim 4 recites the limitation “notify one of the recognized surrounding targets”. However, the specification suggests, particularly in paragraph [0025], that notifying the recognition target means notifying about the recognition target, not sending a notification to the recognition target. As such, “notify one of the recognized surrounding targets” should read “notify about one of the recognized surrounding targets” in claim 4.
Appropriate correction is required.
Claim Interpretation
The claimed electronic control unit is interpreted as a specially programmed computer in view of [0016] and therefore does not invoke 112(f).
Claim 3 recites “wherein the electronic control unit is configured to display, in the recognized surrounding targets, a part recognized by the electronic control unit and a part not recognized by the electronic control unit in different modes when notifying about the recognized surrounding targets whose confidence level is equal to or lower than the reference value.” In the context of [0025], “a part not recognized by the recognition unit” has been interpreted as including “a part of the recognition target whose confidence level is equal to or lower than a reference value” and a part not recognized without a known calculated confidence level. Furthermore, a recognized surrounding target may comprise more than one object, for example, left and right lane lines, which are set as the target serving as the starting point of the control in [0020-0021].
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6 are rejected under 35 U.S.C. 103 as being unpatentable over Mori et al. (US 20180065552 A1; hereafter “Mori”) in view of Bill et al. (US 20220101723 A1; hereafter “Bill”).
Regarding claim 1, Mori discloses
A recognition state notification device comprising an electronic control unit (See electronic control unit (ECU) 10 of autonomous driving system 100 comprising external situation recognition unit 11, vehicle position recognition unit 12, travel plan generation unit 14, and display control unit 16 [0036-0037]. See also Fig. 1.) configured to:
acquire detection results from external sensors (See “the external situation recognition unit 11 [in ECU 10] acquires the detection result of the external sensor 2 as the surroundings information of the vehicle V” [0038] and “recognizes the external situation of the vehicle V based on the acquired information” [0039]. See also [0020-0023].);
calculate a confidence level of the… recognition of surrounding targets (See “When recognizing the external situation of the vehicle V, the external situation recognition unit 11 may calculates a certainty level [confidence level] of the recognition in association with the recognition result” of a surrounding target [0040]. Various examples of recognition targets in the surrounding environment are given in [0039-0040], including road lines and a vehicle. See “When recognizing the travel lane of the vehicle V or the lateral position of the vehicle V, the vehicle position recognition unit 12 may calculate the certainty level of the recognition in association with the recognition result” [0044]. See also [0042].);
notify a supervisor of a vehicle about the detection results relating to the recognized surrounding targets whose confidence level is equal to or lower than a reference value, and of the recognized surrounding targets (See “The display control unit 16 [of ECU 10] display[s] the camera video image H1 and the images H1, H2, and H3… in a superimposed manner [as shown in Fig. 4A]. Accordingly, it is possible to present the facts that the autonomous driving system 100 recognizes the other vehicles V2 and V3 [and] that the certainty level of recognizing the other vehicle V3 is low… to the driver [supervisor] in correspondence with the actual scene” [0069]. Notification of low confidence level: see “since the certainty level of the recognizing the other vehicle V3 [a surrounding target] is lower than the predetermined [reference] value, the rectangular object T3 of a dashed line is adopted as the recognition result information of the other vehicle V3” [0067]. The display control unit displays (notifies the driver of) the generated recognition objects on display unit 8b [0066]. A confidence level equal to or lower than a reference value is equivalent to a confidence level lower than the closest value higher than the reference value; for example, x ≤ 3 is equivalent to x < 4 when only considering integer values. Therefore, a “certainty level of the recognizing the other vehicle V3 [which] is lower than the predetermined value” [0067] is the same as a “confidence level [which] is equal to or lower than a reference value.” See also [0053], [0062-0074], and Figs. 4A-5D.);
execute autonomous driving of the vehicle using the detection results (Autonomous driving system 100 controls the driving of vehicle V [0018] via travel control unit 15 [0050] of ECU 10 [0037] and actuator 9 [0034] according to a plan generated by travel plan generation unit 14 [0050] based on the recognition results from external situation recognition unit 11 and vehicle position recognition unit 12 [0046-0047], which in turn are based on the detection results of the external sensor 2 [0038-0039], [0042]. See also [0035-0036], [0051-0052].).
However, Mori does not explicitly teach “input the acquired detection results into an Al technology to recognize surrounding targets; [and] calculate a confidence level of the Al technology's recognition of surrounding targets.”
Bill, in the same field of endeavor (object recognition for autonomous vehicles), teaches
input the acquired detection results into an Al technology to recognize surrounding targets (See “At 510, a spatial feature extractor is used to extract spatial features from the sensor data of the data stream 508. The spatial feature extractor may be implemented at 510 by inputting the data stream 508 into a trained machine-learning model, such as a convolution neural network, whose inferred output comprises the spatial features. The spatial features may comprise groups of pixels with respective values in a multi-dimensional coordinate system (e.g., two- or three-dimensional) that may represent lane lines, stop lines, pedestrians, other vehicles, traffic flow control devices, etc.” [0099]. Detected objects are classified into types (e.g., “a type of a traffic flow control device such as a traffic light, a stop sign, a yield sign, a single flashing light, etc.”) using the extracted spatial features [0104]. See also [0101-0102], [0106], [0141], and Fig. 5.);
calculate a confidence level of the Al technology's recognition of surrounding targets (See “The spatial features extracted at 510 are used as input to an object detector at 512. The object detector parses the spatial features into likely objects… The object detector output from the processing at 512 may also include… a level of confidence that the bounding box is associated with an object instead of noise, a level of confidence that the bounding box encompasses the entirety of an object, …or some combination thereof” [0100]. See “a determination is made at 604 as to whether the detected type of the traffic flow control device is relatively accurate, such that the detected type is or may be acceptable as the existing type. If the detected type of the traffic flow control device is determined with less than the defined level of detection confidence (e.g., it has a level of detection confidence less than the defined level), or the traffic flow control device is undetected such that no detected type exists, processing advances to 606” [0114]. See also [0108-0111], [0113], [0116-0117], and Fig. 5.).
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the autonomous driving system of Mori with the machine learning model of Bill. One of ordinary skill in the art would have been motivated to make this modification for the benefit of addressing “low-confidence object detection conditions for the vehicle to successfully operate within a vehicle transportation network. Accurately identifying and subsequently annotating and mapping objects, such as traffic flow control devices, reduce reliance on the remotely-generated (e.g., high definition) maps” (Bill, [0019]).
Regarding claim 3, Mori/Bill discloses the limitations of claim 1 as addressed above, and Mori additionally discloses
wherein the electronic control unit is configured to display, in the recognized surrounding targets, a part recognized by the electronic control unit and a part not recognized by the electronic control unit in different modes when notifying about the recognized surrounding targets whose confidence level is equal to or lower than the reference value (As stated in the Claim Interpretation section, a recognized surrounding target may comprise more than one object, for example, left and right lane lines. See “In the example in FIG. 5C, since the certainty [confidence] level of the recognizing the boundary line L1 [a part of the recognition target recognized by the recognition unit] is equal to or higher than the predetermined [reference] value, the [display] object T3 of a solid line [first mode] is adopted as the recognition result information of the boundary line L1. On the other hand, since the certainty level of recognizing the boundary lines L2 and L3 is lower than the predetermined value, the [display] objects T4 and T5 of dashed lines [second mode] are adopted as the recognition result information of the boundary lines L2 and L3” (a part of the recognition target that is not recognized by the recognition unit) [0072]. This displaying is performed by display control unit 16 of ECU 10 [0071], [0074]. The different modes may include shape, type of line (e.g., solid vs dashed), color, color density, and transparency of the display object “according to the certainty level” and whether or not a display object is generated and displayed [0067]. See also [0068], [0070-0074], and Figs. 5A-5D.).
Additionally, even if interpreted such that a surrounding target is only one object—for example, boundary line L3 in Fig. 5A, Mori still discloses displaying “a part recognized by the electronic control unit and a part not recognized by the electronic control unit in different modes.” See the annotations below of Fig. 5A: the display object T5, which marks the recognized part of boundary line L3 [0072], does not include the entirety of boundary line L3. Thus, there is a part of boundary line L3 that is displayed in one mode (object T5 with certainty level lower than the predetermined value) and a part that is displayed in a different mode (no display object generated/shown).
PNG
media_image1.png
327
737
media_image1.png
Greyscale
Figure A: annotated Fig. 5A of Mori
Regarding claim 4, Mori/Bill discloses the limitations of claim 1 as addressed above, and Mori additionally discloses
wherein the electronic control unit is further configured to set a target serving as a starting point of control for the autonomous driving based on a course plan for the autonomous driving in the vehicle (The autonomous driving is implemented by travel control unit 15 according to a target speed, target vehicle-to-vehicle distance, and target lateral position, which are set in a travel plan (course plan) by travel plan generation unit 14 [0050-0052]. By setting a target vehicle-to-vehicle distance, the ECU 10 sets a vehicle in front of the autonomous vehicle V as a target serving as a starting point of control; by setting a target lateral position, the ECU 10 sets the lane boundary lines as a target serving as a starting point of control. The external situation recognition unit 11 then recognizes and stores these targets [0038-0039], which feed into the travel plan [0046-0048]. See also [0018], [0037], and [0042].); and
notify one of the recognized surrounding targets that is set as the target serving as the starting point of the control and has the confidence level equal to or lower than the reference value (When a vehicle is the control target: see “the display control unit 16 generates the object according to the certainty level of the recognition… In the example in FIG. 4C, …since the certainty level of the recognizing the other vehicle V3 [recognition target] is lower than the predetermined [reference] value, the rectangular object T3 of a dashed line is adopted as the recognition result information of the other vehicle V3” [0067]. When boundary lane lines are the control target: see “In the example in FIG. 5C, …since the certainty level of recognizing the boundary lines L2 and L3 [recognition target] is lower than the predetermined [reference] value, the [display] objects T4 and T5 of dashed lines are adopted as the recognition result information of the boundary lines L2 and L3” [0072]. The display control unit 16 of ECU 10 displays (notifies the driver of) the generated recognition objects on display unit 8b [0066], [0071]. See also [0053], [0062-0074], and Figs. 4A-5D.).
Regarding claim 5, Mori/Bill discloses the limitations of claim 1 as addressed above, and Mori additionally discloses
wherein the electronic control unit communicates with a camera or a lidar, and the detection results comprise an image from the camera or a point cloud image from the lidar (See “The camera is an imaging device that images the external situation of the vehicle V… The image information relating to the external situation of the vehicle V is output to the ECU 10” [0021]. See “The LIDAR is a detection device that detects an object outside of the vehicle V using light… The LIDAR outputs the detected object information to the ECU 10” [0023]. See also [0020], [0038], [0063], and [0067].).
Regarding claim 6, Mori/Bill discloses the limitations of claim 1 as addressed above, and Mori additionally discloses
wherein the [ECU] recognizes a position, a shape, and a type of the surrounding targets (See “The external situation recognition unit 11 [of ECU 10] recognizes the external situation of the vehicle V based on the acquired information. The external situation of the vehicle V may include… a position of the center of the lane, a width of the road, and a type of the road. The type of the road may be, for example, a curvature of the travelling lane, a gradient change of the road effective for estimating the prospects of the external sensor 2, and an undulation” [0039]. The lane and road are surrounding targets. A curvature or undulation is a shape of a lane or road. See also [0038] and [0040].).
Bill additionally teaches
wherein the AI technology recognizes a position, a shape, and a type of the surrounding targets (Position: “The object detector output from the processing at 512 may also include locations within the multi-dimensional coordinate system” [0100]. Type: “The object classification at 522 may use pattern recognition software, or some other means, of classifying the object as a traffic flow control device or another object within the intersection. For example, the object classifier may classify a matching object at 522 as a type of a traffic flow control device such as a traffic light, a stop sign, a yield sign, a single flashing light, etc.” [0104]. See also [0101-0103] and [0106].).
Therefore, the combination as a whole teaches the claim.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Moya Ly whose telephone number is (571)272-5832. The examiner can normally be reached Monday-Friday 10:00 am-6:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached at (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MOYA LY/Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658
