DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The preliminary amendment filed on 02/06/2026 has been entered and fully considered.
Claims 1-44 have been canceled.
Claims 45-64 have been newly added.
Claims 45-64 are pending in Instant Application.
Priority
Examiner acknowledges Applicant’s claim to priority benefits of 62/683,971 filed 06/12/2018, 16/439,448 filed 06/12/2019, and 18/351,599 filed 07/13/2023.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 12/20/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered if signed and initialed by the Examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 45-50, 52-60, and 62-64 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (USPGPub 2019/0220002) in view of Millard et al. (USPGPub 2022/0365532). As per claim 45, Huang discloses a method for controlling an autonomous aerial vehicle during flight, the method comprising: while the autonomous aerial vehicle is autonomously executing a planned trajectory through a physical environment (see at least paragraph 0124; wherein generate a motion path traversing through passable (open) space within an environmental map such as a 3D map), receiving, by a computer system, a modification input indicative of an object-level change to a virtual navigation object located in a shared virtual environment that is representative of the physical environment (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment), the shared virtual environment having been generated based at least in part on fused perception inputs from one or more sensors (see at least paragraph 0098; wherein the environmental sensing unit may include multiple imaging devices, or an imaging device with multiple lenses and/or image sensors. The multiple images may aid in the creation of a 3D scene, a 3D virtual environment, a 3D map, or a 3D model); updating, by the computer system and during continued execution of the planned trajectory, the shared virtual environment to reflect the object-level change to the virtual navigation object (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment. Examples of those actions may include selecting one or more points or objects, drag-and-drop, translate, rotate, spin, push, pull, zoom-in, zoom-out, etc. Any type of movement action of the points or objects in a three-dimensional virtual space may be contemplated. A user may use the user terminal to manipulate the points or objects in the virtual environment to control a flight path of the UAV and/or motion characteristic(s) of the UAV. A user may also use the user terminal to manipulate the points or objects in the virtual environment to control motion characteristic(s) and/or different functions of the imaging device); using a continually updated three-dimensional occupancy map of the physical environment (see at least paragraph 0151; wherein a user can receive updated flight information through the augmented FPV as the user is navigating the movable object in an environment). Huang does not explicitly mention regenerating, by the computer system, a planned trajectory through the shared virtual environment relative to the modified virtual navigation object, wherein regenerating the planned trajectory comprises applying one or more built-in safety objectives and constraining the planned trajectory; and causing, by the computer system, the autonomous aerial vehicle to transition from the planned trajectory to the regenerated planned trajectory without interrupting autonomous flight. However Millard does disclose: regenerating, by the computer system, a planned trajectory through the shared virtual environment relative to the modified virtual navigation object, wherein regenerating the planned trajectory comprises applying one or more built-in safety objectives and constraining the planned trajectory (see at least paragraphs 0064-0065; wherein the robot may comply with unique safety and social considerations for various classes of objects during its travel. At stage 510, the planning engine determines a path for the robot using the selected planning policies. The path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated. The planning engine can determine a trajectory cost function that is based on the individual cost functions defined by the respective planning policies of all or a subset of obstacles represented in the virtual environment); and causing, by the computer system, the autonomous aerial vehicle to transition from the planned trajectory to the regenerated planned trajectory without interrupting autonomous flight (see at least paragraph 0065; wherein the path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Millard with the teachings as in Huang. The motivation for doing so would have been to improving reactivity of the robot to objects of different semantic classes by reducing latency, see Millard paragraph 0021. As per claim 46, Huang discloses wherein the modification input is received from a network- connected user device that is concurrently displaying a rendered view of the shared virtual environment (see at least paragraph 0072; wherein the image data may be provided in a 3D virtual environment that is displayed on the user terminal (e.g., virtual reality system or augmented reality system)). As per claim 47, Huang discloses wherein the object-level change comprises repositioning the virtual navigation object to a different virtual location corresponding to a different physical location in the physical environment (see at least paragraph 0072; wherein the virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment. Examples of those actions may include selecting one or more points or objects, drag-and-drop, translate, rotate, spin, push, pull, zoom-in, zoom-out, etc). As per claim 48, Huang discloses wherein the object-level change comprises resizing, re- orienting, or re-shaping the virtual navigation object within the shared virtual environment (see at least paragraph 0072; wherein the virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment. Examples of those actions may include selecting one or more points or objects, drag-and-drop, translate, rotate, spin, push, pull, zoom-in, zoom-out, etc). As per claim 49, Huang discloses wherein the virtual navigation object is associated with a behavioral objective, and wherein regenerating the planned trajectory preserves the behavioral objective while modifying a spatial relationship between the autonomous aerial vehicle and the virtual navigation object (see at least paragraphs 0072-0073; wherein a user may use the user terminal to manipulate the points or objects in the virtual environment to control a flight path of the UAV and/or motion characteristic(s) of the UAV. In some cases, the selection may extend to a portion of the target. The point may be located on or proximate to the target in the image. The UAV may then fly towards and/or track the target). As per claim 50, Huang discloses wherein the behavioral objective includes at least one of: hover, circle, fly-over, capture images, or track a physical object (see at least paragraphs 0072-0073; wherein a user may use the user terminal to manipulate the points or objects in the virtual environment to control a flight path of the UAV and/or motion characteristic(s) of the UAV. In some cases, the selection may extend to a portion of the target. The point may be located on or proximate to the target in the image. The UAV may then fly towards and/or track the target). As per claim 52, Millard discloses wherein regenerating the planned trajectory is performed over a time horizon while the autonomous aerial vehicle continues executing a prior segment of the planned trajectory (see at least paragraph 0065; wherein the path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated). As per claim 53, Millard discloses wherein updating the shared virtual environment and regenerating the planned trajectory are performed without pausing or terminating execution of the planned trajectory (see at least paragraph 0065; wherein the path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated). As per claim 54, Huang discloses wherein the modification input is received via an augmented-reality interface that presents the virtual navigation object as a graphical augmentation registered to the physical environment (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment). As per claim 55, Huang discloses a system comprising: one or more processors (see at least paragraph 0075; wherein one or more processors); and one or more memory units storing instructions that, when executed by the one or more processors (see at least paragraph 0249; wherein the processing unit can have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU)). The processing unit can be operatively coupled to a non-transitory computer readable medium. The non-transitory computer readable medium can store logic, code, and/or program instructions executable by the processing unit for performing one or more steps), cause the system to: while an autonomous aerial vehicle is autonomously maneuvering along a planned trajectory (see at least paragraph 0124; wherein generate a motion path traversing through passable (open) space within an environmental map such as a 3D map), receive a modification input corresponding to an object-level adjustment of a virtual navigation object positioned in a shared virtual environment (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment) generated based at least in part on fused perception inputs (see at least paragraph 0098; wherein the environmental sensing unit may include multiple imaging devices, or an imaging device with multiple lenses and/or image sensors. The multiple images may aid in the creation of a 3D scene, a 3D virtual environment, a 3D map, or a 3D model); update the shared virtual environment to reflect the object-level adjustment during continued execution of the planned trajectory (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment. Examples of those actions may include selecting one or more points or objects, drag-and-drop, translate, rotate, spin, push, pull, zoom-in, zoom-out, etc. Any type of movement action of the points or objects in a three-dimensional virtual space may be contemplated. A user may use the user terminal to manipulate the points or objects in the virtual environment to control a flight path of the UAV and/or motion characteristic(s) of the UAV. A user may also use the user terminal to manipulate the points or objects in the virtual environment to control motion characteristic(s) and/or different functions of the imaging device); using a continually updated three-dimensional occupancy map (see at least paragraph 0151; wherein a user can receive updated flight information through the augmented FPV as the user is navigating the movable object in an environment). Huang does not explicitly mention regenerate a planned trajectory through the shared virtual environment relative to the adjusted virtual navigation object by applying one or more safety objectives and constraining the planned trajectory; and cause the autonomous aerial vehicle to transition to the regenerated planned trajectory without interrupting autonomous maneuvering. However Millard does disclose: regenerate a planned trajectory through the shared virtual environment relative to the adjusted virtual navigation object by applying one or more safety objectives and constraining the planned trajectory (see at least paragraphs 0064-0065; wherein the robot may comply with unique safety and social considerations for various classes of objects during its travel. At stage 510, the planning engine determines a path for the robot using the selected planning policies. The path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated. The planning engine can determine a trajectory cost function that is based on the individual cost functions defined by the respective planning policies of all or a subset of obstacles represented in the virtual environment); and cause the autonomous aerial vehicle to transition to the regenerated planned trajectory without interrupting autonomous maneuvering (see at least paragraph 0065; wherein the path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Millard with the teachings as in Huang. The motivation for doing so would have been to improving reactivity of the robot to objects of different semantic classes by reducing latency, see Millard paragraph 0021. As per claim 56, Huang discloses wherein the fused perception inputs are received from a plurality of devices including at least one sensor carried by the autonomous aerial vehicle (see at least paragraph 0098; wherein the environmental sensing unit may include multiple imaging devices, or an imaging device with multiple lenses and/or image sensors. The multiple images may aid in the creation of a 3D scene, a 3D virtual environment, a 3D map, or a 3D model). As per claim 57, Huang discloses wherein the modification input is received from a user device distinct from the autonomous aerial vehicle (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment). As per claim 58, Millard discloses wherein regeneration of the planned trajectory occurs concurrently with execution of a previously generated trajectory segment (see at least paragraphs 0064-0065; wherein the robot may comply with unique safety and social considerations for various classes of objects during its travel. At stage 510, the planning engine determines a path for the robot using the selected planning policies. The path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated. The planning engine can determine a trajectory cost function that is based on the individual cost functions defined by the respective planning policies of all or a subset of obstacles represented in the virtual environment). As per claim 59, Huang discloses wherein the virtual navigation object remains selectable and modifiable throughout autonomous flight of the autonomous aerial vehicle (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment). As per claim 60, Millard discloses wherein the safety objectives include obstacle avoidance that is enforced independently of the modification input (see at least paragraphs 0064-0065; wherein the robot may comply with unique safety and social considerations for various classes of objects during its travel. At stage 510, the planning engine determines a path for the robot using the selected planning policies. The path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated. The planning engine can determine a trajectory cost function that is based on the individual cost functions defined by the respective planning policies of all or a subset of obstacles represented in the virtual environment). As per claim 62, Huang discloses an apparatus comprising: one or more non-transitory computer-readable storage media (see at least paragraph 0249; wherein the processing unit can have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU)). The processing unit can be operatively coupled to a non-transitory computer readable medium. The non-transitory computer readable medium can store logic, code, and/or program instructions executable by the processing unit for performing one or more steps); and program instructions that, when executed by one or more processors (see at least paragraph 0249; wherein the processing unit can have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU)). The processing unit can be operatively coupled to a non-transitory computer readable medium. The non-transitory computer readable medium can store logic, code, and/or program instructions executable by the processing unit for performing one or more steps), cause the one or more processors to: while an autonomous aerial vehicle is autonomously executing a flight trajectory (see at least paragraph 0124; wherein generate a motion path traversing through passable (open) space within an environmental map such as a 3D map), receive input modifying a virtual navigation object within a shared virtual environment (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment) generated from fused perception inputs (see at least paragraph 0098; wherein the environmental sensing unit may include multiple imaging devices, or an imaging device with multiple lenses and/or image sensors. The multiple images may aid in the creation of a 3D scene, a 3D virtual environment, a 3D map, or a 3D model); update the shared virtual environment to reflect the modification during continued flight of the autonomous aerial vehicle (see at least paragraph 0072; wherein the 3D virtual environment may optionally correspond to a 3D map. The virtual environment may comprise a plurality of points or objects that can be manipulated by a user. The user can manipulate the points or objects through a variety of different actions in the virtual environment. Examples of those actions may include selecting one or more points or objects, drag-and-drop, translate, rotate, spin, push, pull, zoom-in, zoom-out, etc. Any type of movement action of the points or objects in a three-dimensional virtual space may be contemplated. A user may use the user terminal to manipulate the points or objects in the virtual environment to control a flight path of the UAV and/or motion characteristic(s) of the UAV. A user may also use the user terminal to manipulate the points or objects in the virtual environment to control motion characteristic(s) and/or different functions of the imaging device); using a continually updated three-dimensional occupancy map and one or more safety objectives (see at least paragraph 0151; wherein a user can receive updated flight information through the augmented FPV as the user is navigating the movable object in an environment). Huang does not explicitly mention regenerate a flight trajectory relative to the modified virtual navigation object by constraining the trajectory; and control the autonomous aerial vehicle to transition to the regenerated flight trajectory without interrupting autonomous flight. However Millard does disclose: regenerate a flight trajectory relative to the modified virtual navigation object by constraining the trajectory (see at least paragraphs 0064-0065; wherein the robot may comply with unique safety and social considerations for various classes of objects during its travel. At stage 510, the planning engine determines a path for the robot using the selected planning policies. The path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated. The planning engine can determine a trajectory cost function that is based on the individual cost functions defined by the respective planning policies of all or a subset of obstacles represented in the virtual environment); and control the autonomous aerial vehicle to transition to the regenerated flight trajectory without interrupting autonomous flight (see at least paragraph 0065; wherein the path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Millard with the teachings as in Huang. The motivation for doing so would have been to improving reactivity of the robot to objects of different semantic classes by reducing latency, see Millard paragraph 0021. As per claim 63, Huang discloses wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to cause display of the modified virtual navigation object at a network-connected user device during flight (see at least paragraph 0072; wherein the image data may be provided in a 3D virtual environment that is displayed on the user terminal (e.g., virtual reality system or augmented reality system)). As per claim 64, Millard discloses wherein regenerating the flight trajectory is based on both the modified virtual navigation object and real- time perception inputs received during flight (see at least paragraphs 0064-0065; wherein the robot may comply with unique safety and social considerations for various classes of objects during its travel. At stage 510, the planning engine determines a path for the robot using the selected planning policies. The path identifies where the robot should travel over an upcoming period of time, and the robot may continue to follow the path until it is either completed or a new, updated path is generated. The planning engine can determine a trajectory cost function that is based on the individual cost functions defined by the respective planning policies of all or a subset of obstacles represented in the virtual environment).
Claims 51 and 61 are rejected under 35 U.S.C. 103 as being unpatentable over Huang et al. (USPGPub 2019/0220002), in view of Millard et al. (USPGPub 2022/0365532), and further in view of Atalla (USPGPub 2019/0162856). As per claim 51, Huang and Millard do not explicitly mention wherein constraining the regenerated planned trajectory using the three-dimensional occupancy map comprises preventing intersection of the regenerated planned trajectory with occupied voxels of the occupancy map. However Atalla does disclose: wherein constraining the regenerated planned trajectory using the three-dimensional occupancy map comprises preventing intersection of the regenerated planned trajectory with occupied voxels of the occupancy map (see at least paragraph 0048; wherein the VAM application is configured to generate a voxel map from the voxel mapping virtual construct by defining occupancy of voxels (i.e. cubes) in the voxel mapping virtual construct based on information provided by LiDAR sensors mounted on a mapping vehicle or on an autonomously operating entity…the image frames may be processed by object recognition algorithms to identify objects or identify visually similar objects and, such information may be overlaid on the voxel map to generate high-definition 3D maps). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Atalla with the teachings as in Huang and Millard. The motivation for doing so would have been to provide a method and system for positioning of autonomously operating entities, see Atalla paragraph 0006. As per claim 61, Huang and Millard do not explicitly mention wherein obstacle avoidance is enforced by constraining motion of the autonomous aerial vehicle relative to occupied voxels of the three-dimensional occupancy map. However Atalla does disclose: wherein obstacle avoidance is enforced by constraining motion of the autonomous aerial vehicle relative to occupied voxels of the three-dimensional occupancy map (see at least paragraph 0048; wherein the VAM application is configured to generate a voxel map from the voxel mapping virtual construct by defining occupancy of voxels (i.e. cubes) in the voxel mapping virtual construct based on information provided by LiDAR sensors mounted on a mapping vehicle or on an autonomously operating entity…the image frames may be processed by object recognition algorithms to identify objects or identify visually similar objects and, such information may be overlaid on the voxel map to generate high-definition 3D maps). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Atalla with the teachings as in Huang and Millard. The motivation for doing so would have been to provide a method and system for positioning of autonomously operating entities, see Atalla paragraph 0006.
Relevant Art
The prior art made of record and not relied upon are considered pertinent to applicant’s disclosure: USPGPub 2019/0287302 – Provide a system and method of controlling a virtual camera. The method comprises receiving a camera path of the virtual camera, the camera path defining movement of the virtual camera over a period of time to capture video data of a scene; and determining a plurality of control points on the camera path using a plurality of time markers within the period of time, each of the time markers corresponding to at least one event of a predetermined type being identified in the scene, wherein the control points at least partially define the camera path. The method further comprises controlling the virtual camera using the camera path modified based on the plurality of control points in response to a user input in relation to at least one of the control points. USPGPub 2016/0349835 – Provide a “Tactile Autonomous Drone” (TAD) (e.g., flying drones, mobile robots, etc.) supplies real-time tactile feedback to users immersed in virtual reality (VR) environments. TADs are not rendered into the VR environment, and are therefore not visible to users immersed in the VR environment. In various implementations, one or more TADs track users as they move through a real-world space while immersed in the VR environment. One or more TADs apply tracking information to autonomously position themselves, or one or more physical surfaces or objects carried by the TADs, in a way that enables physical contact between those surfaces or objects and one or more portions of the user's body
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHMOUD S ISMAIL whose telephone number is (571)272-1326. The examiner can normally be reached M - F: 8:00AM- 4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at 571-270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHMOUD S ISMAIL/Primary Examiner, Art Unit 3662
Prosecution Insights