Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to communications filed on 1/14/2024. Claims 1 & 11-20 have been amended. Accordingly, claims 1- 20 are pending.
Response to Arguments
Applicant's arguments filed 1/14/2026 have been fully considered but they are not persuasive. Applicant's representative argues, in substance, that Shoeb/’573 fails to teach and/or disclose: (A)comparing a current optical flow map to reference optical flow maps and then determining a position of a UAV based on the comparing.
In response to (A), the examiner respectfully disagrees. Initially the examiner would like to point out that applicant's representative employs broad language in the instant claims; and as such the examiner reserves the right to interpret the claims broadly. Furthermore, examiner would like to point out that applicant's representative admits Shoeb discloses optical flow maps, however argues that they are used to detect close encounters and not determining a position. The examiner contends that Shoeb's disclosure of utilizing optical flow maps to avoid encounters/collisions with objects/obstacle along a route (see fig. 2) and the use of objection detection along with GPS location to help more accurately detect/avoid obstacles would require knowing/determining the position of both the UAV and the object/obstacle in order to avoid collision (see par. 20-25). As such the examiner contends that Shoeb still reads on the instant claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
Claims 1- 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shoeb et al. (US 2024/016573 A1).
Shoeb discloses:
1: A computer implemented method for localization of an unmanned aerial vehicle (UAV), the computer-implemented method comprising:
acquiring aerial images of a terrain below the UAV with an onboard camera system of the UAV while the UAV is flying a mission along a preplanned route over the terrain (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22fig. 1-3B, 5-5D; acquire video stream of ground area below);
generating a current optical flow map based upon image pixel motion between consecutive images in a sequence of the aerial images (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22fig. 1-3B, 5-5D; acquire video stream of ground area below);
comparing the current optical flow map to reference optical flow maps stored on board the UAV, wherein the reference optical flow maps are precomputed from a model of the terrain along the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-3B, 5-5D; acquire video stream of ground area below, video stream saved onboard memory of UAV, UAV’s navigation system with specific routes taken between two locations); and
determining a position of the UAV in at least two lateral dimensions based on the comparing (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-3B, 5-5D; acquire video stream of ground area below, video stream saved onboard memory of UAV, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
2: wherein the model comprises a geo-registered point cloud of the terrain along the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-3B, 5-5D; acquire video stream of ground area below, video stream saved onboard memory of UAV, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
3: wherein the geo-registered point cloud is derived from a lidar scan flown over the terrain (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 42-46 fig. 1-3B, 5-5D; acquire video stream of ground area below, video stream saved onboard memory of UAV, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
4: wherein the reference optical flow maps comprise collections of the reference optical flow maps where the collections are associated with candidate paths that each align with the preplanned route or a corresponding one of a plurality of lateral offsets of the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; acquire video stream of ground area below, video stream saved onboard memory of UAV, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
5: wherein comparing the current optical flow map to the reference optical flow maps comprises: dividing the terrain along the preplanned route into tiles of a predetermined size; and searching the reference optical flow maps corresponding to one of the tiles over which the UAV is currently flying to identify one of the candidate paths that matches a current flight path of the UAV (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; map fusing, combination depth map, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
6: wherein some of the candidate paths correspond to vertically offsets of the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; map fusing, combination depth map, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
7: further comprising: scaling either the reference or current optical flow maps to identify a candidate path that is vertically offset from the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; map fusing, combination depth map, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
8: further comprising: determining when the UAV is flying straight and level; and limiting localization of the UAV using the reference optical flow maps when the UAV is flying straight and level (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; map fusing, combination depth map, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
9: further comprising: semantically segmenting the aerial images to identify pixels within the aerial images associated with either moving objects or transitory objects; and masking any portion of the current optical flow map that aligns with an instance of either the moving objects or the transitory objects (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; map fusing, combination depth map, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment, masking encounter flags etc.).
10: wherein determining the position based on the comparing comprises a backup localization for the UAV when a global navigation satellite system (GNSS)-based localization is insufficiently precise or inoperative (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; map fusing, combination depth map, UAV’s navigation system with specific routes taken between two locations and update 3D models of the environment).
11: At least one non-transitory machine-readable storage medium having instructions stored thereon that, in response to execution by an unmanned aerial vehicle (UAV) delivery system, cause the UAV delivery system to perform operations comprising: acquiring aerial images of a terrain below a UAV of the UAV delivery system with an onboard camera system of the UAV while the UAV is flying a mission along a preplanned route over the terrain; generating a current optical flow map based upon image pixel motion between consecutive images in a sequence of the aerial images; comparing the current optical flow map to reference optical flow maps stored on board the UAV, wherein the reference optical flow maps are precomputed from a model of the terrain along the preplanned route; and determining a position of the UAV in at least two lateral dimensions based on the comparing (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 1 above).
12: wherein the model comprises a geo-registered point cloud of the terrain along the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 2 above).
13: wherein the geo-registered point cloud is derived from a lidar scan flown over the terrain (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 3 above).
14: wherein the reference optical flow maps comprise collections of the reference optical flow maps where the collections are associated with candidate paths that each align with the preplanned route or a corresponding one of a plurality of lateral offsets of the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 4 above).
15: wherein comparing the current optical flow map to the reference optical flow maps comprises: dividing the terrain along the preplanned route into tiles of a predetermined size; and searching the reference optical flow maps corresponding to one of the tiles over which the UAV is currently flying to identify one of the candidate paths that matches a current flight path of the UAV (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 5 above).
16: wherein some of the candidate paths correspond to vertically offsets of the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 6 above).
17: wherein the operations further comprise: scaling either the reference or current optical flow maps to identify a candidate path that is vertically offset from the preplanned route (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 7 above).
18: wherein the operations further comprise: determining when the UAV is flying straight and level; and limiting localization of the UAV using the reference optical flow maps when the UAV is flying straight and level (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 8 above).
19: wherein the operations further comprise: semantically segmenting the aerial images to identify pixels within the aerial images associated with either moving objects or transitory objects; and masking any portion of the current optical flow map that aligns with an instance of either the moving objects or the transitory objects (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 1 above).
20: wherein determining the position based on the comparing comprises a backup localization for the UAV when a global navigation satellite system (GNSS)-based localization is insufficiently precise or inoperative (See Shoeb at least fig. 1-6B and in particular Abstract & ¶ 18-22, 46 fig. 1-4C, 5-5D; see claim 10 above).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MACEEH ANWARI whose telephone number is 571-272-7591. The examiner can normally be reached on Monday-Friday 7:30-5:00 PM ES.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Angela Ortiz can be reached on 571-272-1206. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MACEEH ANWARI/Primary Examiner, Art Unit 3663
Prosecution Insights