DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed 09/02/2026 has been entered. Claims 1-20 remain pending in the application.
Response to Arguments
Applicant’s arguments filed 09/02/2026 have been fully considered.
Applicant’s argument (REMARKS pages 6-8) about amended Claims 1, 8, and 15 is moot based on the new ground rejections.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1,3, 5-8, 10, 12-15, 17, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Buerkle et al . (US 20200225622, hereafter Buerkle) in view of Mahjourian et al.( US 2022/0301182, hereafter Mahjourian).
Regarding claim 1, Buerkle (‘622) discloses that An apparatus {Fig.1; Fig.2; [0042] lines 1 (Fig.1), 3-7 (vehicle 100 may include one or more processors 102 , one or more image acquisition devices 104 , one or more position sensors 106 , one or more speed sensors 108 , one or more radar sensors 110 , and / or one or more LIDAR sensors 112 .)} comprising:
at least one radar sensor { Fig.1 item 110 (radar); Fig.2 item 110 (radar); [0042] line 6 (one or more radar sensors 110)};
at least one camera { Fig.1 item 104 (image acquisition devices); Fig.2 item 104 (image acquisition devices); [0044] lines 5-6 (one or more image acquisition devices 104 ( e.g. , one or more cameras)};
at least one memory { Fig.2 item 202 (memory); [0044] lines 11-12 (safety system 200 may further include one or more memories 202 )}; and
at least one processor communicatively coupled to the at least one memory, the at least one radar sensor, and the at least one camera { Fig.1 item 102 (processor); Fig.2 items 102 (processor), 104 (image acquisition devices), 110 (radar), 202 (memory), 214, 216, 218 (processor); [0044] lines 4-6 (the safety system 200 may include one or more processors 102, one or more image acquisition devices 104 ( e.g. , one or more cameras)), 9 (one or more radar sensors 110), 11-12 (safety system 200 may further include one or more memories 202 ); [0045] lines 1-3 (the one or more processors 102 may include an application processor 214 , an image processor 216 , a communication processor 218)}, and configured to:
obtain radar measurement data from the at least one radar sensor { Fig.2 items 110, 102, 214, 216, 218; [0052] lines 11-17 (The radar sensors 110 and / or the LIDAR sensors 112 may be configured to provide pre - processed sensor data , The third data interface may couple, the one or more radar sensors 110 and the one or more LIDAR sensors 112 to at least one of the one or more processors 102)};
obtain camera-derived data based on at least one image obtained by the at least one camera { Fig.2 items 104 (image acquisition devices), 102, 214, 216, 218; [0046] lines 6-8 (to transmit image data acquired by the one or more image acquisition devices 104 to the one or more processors 102 ( e.g. , to the image processor 216 ).)}, ; and
determine a dynamic occupancy grid based on the radar measurement data and the camera-derived data {Fig.2 items 104 (image acquisition devices), 110 (radar); Fig.5 item 502a-n (sensor 1, sensor N), 506 (grid 1 creation), 508 (Grid N creation); Fig.10; Fig.12 item 1206 (Determining from sensor data a probability of whether a region relative to the ego vehicle corresponding to the cell and the selected cell size is occupied 1206); [0016] (Fig.10, modification of the occupancy grid); [0038] lines 5-6 (An input set may include sensor data , such as image data , radar data , LIDAR data); [0041] lines 1-3 (groupings of occupancy grid ( static or dynamic ) cells are referred to as sections or regions of cells .); [0044] lines 5-6 (one or more image acquisition devices 104 ( e.g. , one or more cameras), 9 (one or more radar sensors 110); [0058] lines 10-11 (An occupancy grid may be employed to represent a dynamic environment around the vehicle); [0064] line 6 (dynamically reconfigure the non - uniform grid)}, the dynamic occupancy grid comprising a plurality of cells {[0083] line 8 (the dynamic cells in the grid)},
However, Buerkle (‘622) does not explicitly disclose that (see words with underline) “the camera-derived data comprising occupancy flow data” and “the occupancy flow data indicating an occupancy probability and a motion prediction for each cell of the plurality of cells”. In the same field of endeavor, Mahjourian (‘182) discloses that
the camera-derived data comprising occupancy flow data {Fig.1 items 130 (sensor system), 152 (occupancy flow predictions); Fig.2; [0027] lines 2 (sensor subsystems 130 include), 6 (camera systems); [0029] lines 1 (The sensor subsystems 130), 2-3 from bottom (Each group of camera sensor measurements can be represented as an image patch)};
the occupancy flow data indicating an occupancy probability and a motion prediction for each cell of the plurality of cells {abstract lines 4-13 (the future movement is predicted through occupancy flow fields that specify, for each future time point in a sequence of future time points and for each agent type in a set of one or more agent types: an occupancy prediction for the future time step that specifies, for each grid cell, an occupancy likelihood that any agent of the agent type will occupy the grid cell at the future time point, and a motion flow prediction that specifies, for each grid cell, a motion vector that represents predicted motion of agents of the agent type within the grid cell at the future time point.); Examiner’s note: likelihood for “probability”}.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Buerkle (‘622) with the teachings of Mahjourian (‘182) {generate occupancy flow data (e.g. occupancy likelihood, predicted motion, motion vector) for each grid cell based on sensor measurements (e.g. camera)} to generate occupancy flow data (e.g. occupancy likelihood, predicted motion, motion vector) for each grid cell based on sensor measurements (e.g. camera). Doing so would predict the motion of objects in an environment using sensors (e.g. radar, lidar, camera, etc.) so as to plan motion for an autonomous vehicle accordingly, as recognized by Mahjourian (‘182) {[0003] lines 3-5 (Predicting the motion of objects is a task required for motion planning, e.g., by an autonomous vehicle.)}.
Regarding claim 3, which depends on claim 1, the combination of Buerkle (‘622) and Mahjourian (‘182) discloses that in the apparatus, the apparatus further comprises
at least one lidar sensor communicatively coupled to the at least one processor { see Buerkle (‘622) Fig.2 items 214, 216, 218 (processor) in 102(processor), 112 (lidar); [0042] lines 4 (one or more processors 102), 6-7 (one or more LIDAR sensors 112); [0045] lines 1-3 (the one or more processors 102 may include an application processor 214 , an image processor 216 , a communication processor 218)}, and
wherein the at least one processor is configured to determine the dynamic occupancy grid based on lidar data obtained from the at least one lidar sensor { see Buerkle (‘622) Fig.12 item 1206 (Determining from sensor data a probability of whether a region relative to the ego vehicle corresponding to the cell and the selected cell size is occupied 1206); [0038] lines 5-6 (An input set may include sensor data , such as image data , radar data , LIDAR data); [0058] lines 10-11 (An occupancy grid may be employed to represent a dynamic environment around the vehicle); [0064] line 6 (dynamically reconfigure the non - uniform grid)}.
Regarding claim 5, which depends on claim 1, the combination of Buerkle (‘622) and Mahjourian (‘182) discloses that in the apparatus, the at least one processor is configured to:
determine a radar measurement grid based on the radar measurement data { see Buerkle (‘622) Fig.2 item 110 (radar); [0081] lines 4 (FIG . 5 as sensor one 502a through sensor n 502n), 11-12 (image sensor data ( e.g. one or more cameras , one or more lidar sensors , one or more radar sensors ,); [0082] lines 1-3 (Once the sensor data has been received , the occupancy grid manager may convert the sensor measurements at time t into an input grid 506 and 506.)};
determine at least one camera measurement grid based on the camera-derived data { see Buerkle (‘622) Fig.2 item 104 ((image acquisition devices); [0044] lines 4-5 (the safety system 200 may include one or more processors 102, one or more image acquisition devices 104 ( e.g. , one or more cameras)); [0081] lines 4 (FIG . 5 as sensor one 502a through sensor n 502n), 11-12 (image sensor data ( e.g. one or more cameras , one or more lidar sensors , one or more radar sensors ,); [0082] lines 1-3 (Once the sensor data has been received , the occupancy grid manager may convert the sensor measurements at time t into an input grid 506 and 506.)};
determine a fused measurement grid based on the radar measurement grid and the at least one camera measurement grid { see Buerkle (‘622) Fig.2 items 104 ((image acquisition devices), 110 (radar); Fig.5 item 510 (grid fusion); [0081] lines 4 (FIG . 5 as sensor one 502a through sensor n 502n), 11-12 (image sensor data ( e.g. one or more cameras , one or more lidar sensors , one or more radar sensors ,); [0083] lines 1-2 (one or more processors may combine the sensor grids into one common grid 510)}; and
determine the dynamic occupancy grid based on the fused measurement grid { see Buerkle (‘622) [0083] lines 1-2 (one or more processors may combine the sensor grids into one common grid 510), 7-9 (The particle filter may track the dynamic cells in the grid . The particle filter may generate an updated dynamic grid representation)}.
Regarding claim 6, which depends on claims 1 and 5, the combination of Buerkle (‘622) and Mahjourian (‘182) discloses that in the apparatus, the at least one processor is configured to
determine a predicted grid, and to determine the fused measurement grid based further on the predicted grid { see Buerkle (‘622) Fig.5 item 510 (grid fusion); [0083] lines 3-4 (The occupancy grid manager may combine the scan grid 510 with a predicted grid of t - 1) , 12 (carry out any of the actions of the grid fusion 510)}.
Regarding claim 7, which depends on claims 1 and 5-6, the combination of Buerkle (‘622) and Mahjourian (‘182) discloses that in the apparatus, the at least one processor is configured to
determine the dynamic occupancy grid based further on a localization input including localization information of the apparatus { see Buerkle (‘622) Fig.1 item 106 (position sensors); Fig.2 item 106 (position sensors); Fig.3 (ego vehicle, occupied cells); Fig.12 item 1204 (Selecting based on the one or more context factors a cell size for a cell of an occupancy grid comprising a plurality of cells , each cell representing a region relative to a vehicle), 1206 (Determining from sensor data a probability of whether a region relative to the ego vehicle corresponding to the cell and the selected cell size is occupied); [0009] lines 1-2 (Fig.3, a uniform occupancy grid of a non - uniform occupancy grid); [0018] lines 1-2 (Fig.12, method of occupancy grid management); [0042] line 5 (one or more position sensors 106); [0044] lines 6-8 (one or more position sensors 106 ( e.g. , a Global Navigation Satellite System ( GNSS ) , a Global Positioning System ( GPS ))}.
Regarding claim 8, Buerkle (‘622) discloses that A method, for determining a dynamic occupancy grid {[0016] (Fig.10, modification of the occupancy grid); [0018] lines 1-2 (Fig.12, method of occupancy grid management); [0058] lines 10-11 (An occupancy grid may be employed to represent a dynamic environment around the vehicle); [0064] line 6 (dynamically reconfigure the non - uniform grid)}, comprising:
obtaining radar measurement data from at least one radar sensor of an apparatus;
obtaining camera-derived data based on at least one image obtained by at least one camera of the apparatus, the camera-derived data comprising occupancy flow data; and
determining the dynamic occupancy grid based on the radar measurement data and the camera-derived data, the dynamic occupancy grid comprising a plurality of cells, the occupancy flow data indicating an occupancy probability and a motion prediction for each cell of the plurality of cells.
{The claim limitations above are the same or substantially the same scope as the corresponding claim limitations in claim 1. Therefore the claim limitations above are rejected in the same or substantially the same manner as in claim 1. See the rejections of claim 1}.
Regarding claims 10, 12-14, Applicant recites claim limitations of the same or substantially the same scope as that of claims 3, 5-7, respectively. Accordingly, claims 10, 12-14 are rejected in the same or substantially the same manner as claims 3, 5-7, respectively, shown above.
Regarding claim 15, Buerkle (‘622) discloses that An apparatus {Fig.1; Fig.2; [0042] lines 1 (Fig.1), 3-7 (vehicle 100 may include one or more processors 102 , one or more image acquisition devices 104 , one or more position sensors 106 , one or more speed sensors 108 , one or more radar sensors 110 , and / or one or more LIDAR sensors 112 .)} comprising:
means for obtaining radar measurement data from at least one radar sensor;
means for obtaining camera-derived data based on at least one image obtained by at least one camera of the apparatus, the camera-derived data comprising occupancy flow data; and
means for determining a dynamic occupancy grid based on the radar measurement data and the camera-derived data, the dynamic occupancy grid comprising a plurality of cells, the occupancy flow data indicating an occupancy probability and a motion prediction for each cell of the plurality of cells.
{The claim limitations above are the same or substantially the same scope as the corresponding claim limitations in claim 1. Therefore the claim limitations above are rejected in the same or substantially the same manner as in claim 1. See the rejections of claim 1}.
Regarding claims 17, 19-20, Applicant recites claim limitations of the same or substantially the same scope as that of claims 3, 5-6, respectively. Accordingly, claims 17, 19-20 are rejected in the same or substantially the same manner as claims 3, 5-6, respectively, shown above.
Claims 2, 9, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Buerkle (‘622) and Mahjourian (‘182) as applied to claims 1, 8, 15, respectively, above, and further in view of Hieida et al. (US 20220169245, hereafter Hieida).
Regarding claim 2, which depends on claim 1, t Buerkle (‘622) and Mahjourian (‘182) do not explicitly disclose “the camera-derived data further comprise semantic segmentation data comprising a semantic classification label for each pixel of the at least one image”. In the same field of endeavor, Hieida (‘245) discloses that in the apparatus, the camera-derived data further comprise
semantic segmentation data comprising a semantic classification label for each pixel of the at least one image {[0163] lines 1-6 (The image region estimation unit 802 estimates respective regions in the image input via the image input unit 801 by using the semantic segmentation technology , and outputs , for each pixel , information to which a label for identifying a category has been added . Objects are extracted identifying a category has been added .)}.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the combination of Buerkle (‘622) and Mahjourian (‘182) with the teachings of Hieida (‘245) { semantic segmentation technology in image processing } to semantic segmentation technology in image processing. Doing so would identify which category a pixel belongs for each pixel of an image so as to obtain region information, estimate object on the basis of an image, and provide warning to a driver or an autonomous operation of a brake to avoid collision, as recognized by Hieida (‘245) {[0002] lines 7-9 (warning is given to a driver , or an auxiliary operation or an autonomous operation of a brake is performed); [0011] lines 1-2 (The region estimation unit estimates the object on the basis of the image by using semantic segmentation.); [0020] line 6 (to avoid the collision); [0021] lines 4-6 (predicting a collision between a mobile body and an object on the basis of region information obtained by semantic segmentation); [0109] lines 3-5 (Semantic segmentation is a technology for identifying which category a pixel belongs for each pixel of an image .) }.
Regarding claims 9 and 16, Applicant recites claim limitations of the same or substantially the same scope as that of claim 2. Accordingly, claims 9 and 16 are rejected in the same or substantially the same manner as claim 2, shown above.
Claims 4, 11, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Buerkle (‘622) and Mahjourian (‘182) as applied to claims 1, 8, 15, respectively, above, and further in view of Moawad et al. (US 12174641, hereafter Moawad) and Wang et al . (US 2022/0065657, hereafter Wang).
Regarding claim 4, which depends on claim 1, Buerkle (‘622) and Mahjourian (‘182) do not explicitly disclose that “the at least one processor is configured to determine the dynamic occupancy grid based on high-definition map data stored in the at least one memory”. In the same field of endeavor, Moawad (‘641) discloses that in the apparatus,
the at least one processor is configured to determine the dynamic occupancy grid based on high-definition map data stored in the at least one memory {Fig.14 item 1002 (HD map), 308 (Radar Occupancy Grid); col.3 lines 8-9 (Fig.14, determining a radar occupancy grid using an HD map); col.15 lines 30-32 (The adjusted polylines 1414 are used to mark corresponding grid cells of the radar occupancy grid 308 as occupied.); Examiner’s note: “adjusted” in Fig.14 item 1414 for “dynamic”}.
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the combination of Buerkle (‘622) and Mahjourian (‘182) with the teachings of Moawad (‘641) { determining a radar occupancy grid using an HD map } to determining a radar occupancy grid using an HD map. Doing so would obtain object attributes from HD map (e.g. street signs, overpasses, guard rails, traffic control devices, posts, buildings, k-rails) for polylines determination in determining occupancy grid so that the radar occupancy grid 308 may be generated without necessitating a vehicle driving through the corresponding area, as recognized by Moawad (‘641) { col.12 lines 16-21 (The HD map 1002 contains object attributes 1004 that are determined at 1006 for HD map objects 1008 within the HD map 1002. The HD map objects 1008 may comprise street signs, overpasses, guard rails, traffic control devices, posts, buildings, k-rails, or other semi-permanent objects.); col.15 lines 35-37 (the radar occupancy grid 308 may be generated without necessitating a vehicle driving through the corresponding area)}.
However, Moawad (‘641) does not explicitly disclose that (see word with underline) “high-definition map data stored in the at least one memory”. In the same field of endeavor, Wang (‘657) discloses that
high-definition map data stored in the at least one memory { [0091] lines 26-27 (the HD map may be stored to memory ( See FIG . 6 ) .)}.
A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that applying a known technique (e.g. the HD map may be stored to memory) to a known device (e.g. vehicle) ready for improvement to yield predictable results (e.g. use HD map for localization and mapping) and result in an improved system (e.g. collect data from surrounding environment and process it with aid of HD map so as to generate Simultaneous localization and mapping for a surrounding environment (e.g. occupancy grid) based on information determined from the processed collected data, as recognized by Wang (‘657) {[0003] lines 11-15 (Simultaneous localization and mapping ( SLAM ), a high - definition ( HD ) map, simultaneously tracking the vehicle's autonomous vehicle location within the environment ) ; [0091] lines 5-6 (collect a second set of signals 106 , from said the surrounding environment 101), 12-16 (the second set of signals may be processed by the SAR - based system 100 with aid of the controller 130 to generate the HD map of the surrounding environment 101. Processing the second set of signals may involve using), 18-19 (Occupancy Grid SLAM), 22-25 (generate a map of a surrounding environment based on, may be determined from the second set of signals received by the SAR - based system)}).
Regarding claims 11 and 18, Applicant recites claim limitations of the same or substantially the same scope as that of claim 4. Accordingly, claims 11 and 18 are rejected in the same or substantially the same manner as claim 4, shown above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200272834 discloses that “the camera-derived data further comprise semantic segmentation data comprising a semantic classification label for each pixel of the at least one image” {Fig.9; [0164] lines (on the basis of the unpolarized image and the normal vector map , the semantic segmentation section 306 labels through semantic segmentation each pixel in the unpolarized image as to which subject in the image the pixel belongs ,)}, which also support the rejections of claims 2, 9, and 16.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YONGHONG LI whose telephone number is (571)272-5946. The examiner can normally be reached 8:30am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vladimir Magloire can be reached at (571)270-5144. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YONGHONG LI/ Primary Examiner, Art Unit 3648
