DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 07/13/2026.
Claims 1-22 are presented for examination.
IDS Considerations
The information disclosure statement (IDS) submitted on 7/29/25, 6/13/25, 3/19/25, 12/17/24 and 6/28/23 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Response to Arguments
Applicant's arguments filed 07/13/2026 with respect to claims 22 have been considered but are moot in view of the new ground(s) of rejection. In addition applicant's arguments filed 07/13/2026 with respect to claims 22 has been considered but are not persuasive.
Applicant argued in page 11 that claim 22 should be allowable because it includes elements from previous dependent claim 18. Examiner disagree on this because applicant did not claim complete scope of the previously indicated allowable subject matter which was combined teaching of claim 1 and claim 18.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chan (U.S. Pub. No. 20170301109 A1), further in view of Kuhara (U.S. Pub. No. 20180108260 A1).
Regarding to claim 22:
22. Chan teach a flight control method for a movable platform with a photographing device, comprising: (Chan Fig. 1 [0042] FIG. 1 illustrates a small autonomous system 100 in accordance with various embodiments of the present disclosure. The system 100 can include a chassis 105, an imaging system 120, and a vision-based guidance system 150. The chassis 105 can include one or more motors 110 to enable and control movement of the chassis 105. The system 100 can use the vision-based guidance system 150 to autonomously plan and control movement of the chassis 105, imaging system 120, or both, to follow, surveille, or obtain additional information related to an object of interest without feedback from a human operator. [0043] In accordance with various embodiments, the chassis 105 can include a land-, sea-, or air-based vehicle body. For example, the chassis 105 can include a car or truck body, a boat, a submarine or a plane body. In an exemplary embodiment, the chassis 105 can be a small unmanned aerial vehicle (sUAV) body such as that of a quadcopter or drone. In some embodiments, the chassis 105 can include an altitude sensor 107 such as an altimeter. [0045] The imaging system 120 can include a still image camera or a video camera.)
obtaining at least one of a type of a target photographing object or a distance between the target photographing object and the movable platform;
(Chan Fig. 1 [0042] [0056] In some embodiments, the vision-based guidance system 150 can acquire images of an object of interest using the imaging system 120. For example, the vision-based guidance system 150 can obtain a sequence of images or video from the imaging system 120. The vision processing module 154 of the vision-based guidance system 150 can process the images acquired by the imaging system 120 to determine the object position or size of the object in camera view coordinates. In some embodiments, the object position can be represented as the XY position in the camera image. In some embodiments, the vision processing module 154 can assess whether the object of interest continues to be in the field of view, whether object observations have sufficient quality to support identification, or the location of the object of interest in the field of view as described below. [0113] if the system identifies the class of object of interest (e.g., a vehicle, an individual, a building, or a natural feature), the vision-based navigation system 150 can utilize a template or expected VI response based on the object class. [0119] In some embodiments, a desired distance or altitude between the object of interest and the system can be calculated, for example, to improve identification of the object of interest. In accordance with various embodiments, the vision-based navigation system 150 can compute or access in memory the desired distance or altitude without external input from remote operators. The desired distance or altitude can be computed by estimating the size of the object in one or more acquired images using computer vision algorithms in an attempt to adjust the relative size of the object of interest in the image.)
determining a target flight trajectory among a plurality of flight trajectories based on at least one of the type of the target photographing object or the distance between the target photographing object and the movable platform; (Chan Fig. 1 [0077] Controllers supporting relative navigation modes can include a hover controller, a flyover controller, an orbit controller, a spiral controller, or a view-angle controller. In some embodiments, the vision-based navigation system 150 can autonomously control motion of the system 100 to track the object of interest using relative navigation modes implemented by controllers to obviate the need for external control by a remote operator. See [0113] and [0119])
and photographing the target photographing object with the photographing device. (Chan Fig. 1 [0045] The imaging system 120 can include a still image camera or a video camera. [0056] In some embodiments, the vision-based guidance system 150 can acquire images of an object of interest using the imaging system 120. For example, the vision-based guidance system 150 can obtain a sequence of images or video from the imaging system 120. The vision processing module 154 of the vision-based guidance system 150 can process the images acquired by the imaging system 120 to determine the object position or size of the object in camera view coordinates. In some embodiments, the object position can be represented as the XY position in the camera image. In some embodiments, the vision processing module 154 can assess whether the object of interest continues to be in the field of view, whether object observations have sufficient quality to support identification, or the location of the object of interest in the field of view as described below. [0057] In some embodiments, the vision-based guidance system 150 can automatically detect the object of interest in the sequence of images. In some embodiments, the vision-based guidance system 150 can apply background subtraction performed on registered images or can use specialized object detectors. In other embodiments, the vision-based guidance system 150 can utilize a convolutional neural network to detect the object of interest in one or more images)
Chan [0061] The low-level controller module 158 receives the 3D velocity commands from the short-term planning module 156 and can convert the commands into appropriate servo control signals to the motors 110. In some embodiments, the low-level controller module 158 can perform state estimation based on the use of, for example, Kalman filters. The low-level controller module 158 can feedback state or motor information back to the short-term planning module 156. In some embodiments, the low-level controller module 158 can be implemented in the vision-based navigation system 150 as either a software package or a dedicated hardware controller. Chan do not teach controlling the movable platform to fly according to the target flight trajectory; establishing a communication between the movable platform and a terminal device; sending the target flight trajectory to the terminal device to enable a display device of the terminal device to superimpose and display the target flight trajectory, a flight area corresponding to the target flight trajectory, and a map corresponding to the target flight trajectory;
However Kuhara teach establishing a communication between the movable platform and a terminal device; (Kuhara [0068] FIG. 1 is a diagram illustrating the configuration of a flight control system according to a first embodiment of the present disclosure. The flight control system illustrated in FIG. 1 includes a communication terminal 10, a flight route generating server 20, an external server 30, and an unmanned aerial vehicle 40. [0070] The flight route generating server 20 generates a flight route for autonomous flight of the unmanned aerial vehicle 40. The flight route generating server 20 is communicably connected to the communication terminal 10, external server 30, and unmanned aerial vehicle 40 via the network 50. [0078] The communication unit 1003 includes electronic circuits for performing wireless communication with other unmanned aerial vehicles and external communication terminals. The communication unit 1003 receives wireless communication of commands or the like relating to flight control and so forth, from other unmanned aerial vehicles, external terminals, and so forth, and transmits wireless signals to other unmanned aerial vehicles, external terminals, and so forth.)
sending the target flight trajectory to the terminal device to enable a display device of the terminal device to superimpose and display the target flight trajectory, (Kuhara [0003] For example, Japanese Unexamined Patent Application Publication No. 2010-152834 describes a remote operation device and an unmanned moving object. The remote operation device includes a travel instruction unit that gives travel instructions by superimposing and setting a turning position regarding which an unmanned moving object is to turn, and an operating icon correlated to a travel direction at this turning position, at an instruction position of an image on a display unit. The unmanned moving object includes a route planning unit that plans a travel route for autonomous travel based on the turning position instructed by the travel instruction unit and the travel direction at the turning position, a speed planning unit that plans travel speed of the unmanned moving object in accordance with the travel route, and an autonomous travel unit that causes the unmanned moving object to travel according to the planned travel route and travel speed. However, further improvement has been necessary in the above-described conventional technology. [0104] The display control unit 122 displays the received map information on the display unit 106 in step S6. [0110] Next, in step S12, the display control unit 122 displays an icon, representing the waypoint input by the user, on the display unit 106. The icon representing the waypoint is not restricted in particular, and is expressed by various shapes, such as a circle, a filled-in circle, etc., for example. The icon representing the waypoint preferably is different from the icon representing the departure point. The display control unit 122 may connect the icon representing the departure point and the icon representing the waypoint using a straight line or arrow, thereby explicitly indicating the flight route of the unmanned aerial vehicle 40. Kuhara teach trajectory map displayed on display unit 106. Fig. 8-25 show the departure waypoint icon and Fig. 24-25 show the flight range circle. These layered information is considered superimpose and display the target flight trajectory)
a flight area corresponding to the target flight trajectory, and a map corresponding to the target flight trajectory; (Kuhara Fig. 24 [0219] The communication unit 103 of the communication terminal 10 receives the maximum flight distance transmitted by the flight route generating server 23. The display control unit 122 of the communication terminal 10 displays a circular maximum flight range that is centered on the input waypoint and has a radius that is the maximum flight distance received by the communication unit 103. When the first waypoint is input, the display unit 106 displays the range that the unmanned aerial vehicle 40 can cover from the first planned arrival time at which the unmanned aerial vehicle 40 reaches the first waypoint, until the end time (the sundown time). [0231] FIG. 24 is a diagram illustrating an example of a display screen displayed on the display unit of the communication terminal according to the present fourth embodiment. As illustrated in FIG. 24, the display unit 106 displays the icon 501 that indicates the departure point, and the icon 502 that indicates the waypoint that is passed next after the departure point, each having been input by the user. The departure time (“16:30″ in FIG. 24) is displayed near the icon 501.
[0232] The planned arrival time at which the unmanned aerial vehicle 40 will reach the waypoint (“17:00” in FIG. 24) is displayed near the icon 502. A circular maximum flight range 511 having a radius that is the maximum flight distance indicated by the maximum flight distance information, and centered on the icon 502 indicating the waypoint, is further displayed on the display unit 106. [0233] Thus, the maximum flight range that can be covered from a waypoint to the sundown time is displayed when setting a flight route, so the user can be prompted to set the flight route to return by the sundown time. Accordingly, the unmanned aerial vehicle 40 can be prevented from flying after the sundown time.)
controlling the movable platform to fly according to the target flight trajectory; (Kuhara [0137] Next, in step S37, the flight route setting unit 224 transmits flight route information to the unmanned aerial vehicle 40 via the first communication unit 203, indicating the flight route over which the unmanned aerial vehicle 40 will perform autonomous flight. The flight route information includes the departure point information indicating the position of the departure point, the waypoint information indicating the positions of waypoints, and the departure time information indicating the departure time. The unmanned aerial vehicle 40 receives the flight route information, and when the departure time indicated by the departure time information arrives, starts autonomous flight. The unmanned aerial vehicle 40 then passes the waypoints indicated by the waypoint information, and finally returns to the departure point.)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Chan, further incorporating Kuhara in video/camera technology. One would be motivated to do so, to incorporate controlling the movable platform to fly according to the target flight trajectory; establishing a communication between the movable platform and a terminal device. This functionality will improve efficiency with predictable results.
Allowable subject matter
Regarding to claim 1 and 20:
Chan (U.S. Pub. No. 20170301109 A1) and Wang (U.S. Pub. No. 9056676 B1) are closest prior art.
Regarding to claim 1 and 20:
1. Chan teach a flight control method for a movable platform with a photographing device, comprising: (Chan [0046] autonomous surveillance by UAVs and is illustrated in FIG. 2. For example, a typical scenario might include conducting a surveillance mission searching for objects of interest. Once an object of interest is detected based on visual cues of the tracked object in a sequence of images, the system can self-direct itself to follow the object and can improve the observation distance and viewing angles with respect to the object of interest if there are insufficient image details to identify the object of interest) obtaining target information, (Chan [0038] In some embodiments, systems and methods described herein address the general problem of automated surveillance of objects of interest, which involves multiple phases of a response chain including object detection, tracking, identification, and engagement. Automating surveillance of objects of interest is challenging for a number of reasons. First, the object of interest can be uncooperative and potentially evasive, so following one requires dynamic replanning in a continuous manner. In contested environments with limited communications, the sUAV are not able to rely on a sustained video downlink to perform the necessary replanning from a remote ground station. [0039] As used herein, “autonomous” refers to a system or module that is self-directing or self-determining without intervention from external persons or systems. For example, autonomous systems and methods described herein can perform one or more of image analysis, trajectory planning, object tracking, object reacquisition, and navigation without input from an independent operator) wherein the target information comprises at least one of a type of a target object to be photographed by the photographing device (Chan Fig. 8 [0046] a typical scenario might include conducting a surveillance mission searching for objects of interest. Once an object of interest is detected [type of a target object] based on visual cues of the tracked object in a sequence of images, the system can self-direct itself to follow the object and can improve the observation distance and viewing angles with respect to the object of interest if there are insufficient image details to identify the object of interest. [0072] In some embodiments, the processor 157 can adjust the images to compensate for motion [target information] of the chassis 105, imaging system 120, or both. Any motion of the object of interest [type of a target object is moving object] observed in a sequence of images is a combination of motions of the chassis and imaging system with respect to the object of interest. Chassis or imaging system motion can be intentional (such as when the chassis is moving to follow an object of interest) or unintentional (such as when the chassis is moved by external forces such as wind).) or a distance between the target object and the movable platform; (Chan Fig. 12 [0059] The long-term planning module 152, short-term planning module 156, and low-level controller module 158 can cooperate to plan and control the motion of the system 100. The long-term planning module 152 can produce a trajectory that includes desired altitudes, desired distances from the object of interest, and desired viewing angles with consideration of the current tracking mode (i.e., whether a detected object is being tracked or not) to improve the diversity of observations of the object of interest. The long-term planning module 152 is described below in greater detail with reference to FIGS. 13-23.)
wherein the target flight trajectory corresponds to at least one preset style of video post-photographing editing template; (Chan [0084] The spiral function can control the motion of the system 100 to spiral [video editing template] around the object of interest at a desired distance and circling speed but at a changing altitude. There are two ways to think of how this controller works. First, a spiral is just a circle with a varying altitude. Thus, the spiral controller design is similar to that of the orbit controller but with changing altitude and a decrease in the radius of the circle as the system ascends such that the system 100 maintains the same distance from the object of interest. Second, a spiral (in the sense of the spiral controller) is a path along the surface of a hemisphere. Thus, the motion commands from this controller are analogous to starting on the equator and walking north while circling the hemisphere. In some embodiments, the spiral controller can achieve essentially every possible view of the object of interest in a discrete sense. Chan [0077] To control the motion of the chassis 105 in an object-relative mode using vision-based guidance, the short-term planning module 156 can implement one or more relative navigation modes that are chosen based upon what information is to be gathered about the object of interest. Relative navigation modes are distinguishable from absolute navigation modes [plurality of flight trajectories] that rely upon information provided by a global positioning system (GPS). Object-relative navigation modes implement motion relative to an origin centered on the object, which can be moving. In some embodiments, each relative navigation mode can be supported by a corresponding controller implemented as software or code in the short-term planning module 156. Each controller can act as a building block for a more complex control scheme implemented by the short-term planning module 156. Controllers supporting relative navigation modes can include a hover controller, a flyover controller, an orbit controller, a spiral controller, or a view-angle controller. In some embodiments, the vision-based navigation system 150 can autonomously control motion of the system 100 to track the object of interest using relative navigation modes implemented by controllers to obviate the need for external control by a remote operator.) controlling the movable platform to fly according to the target flight; (Chan [0066] The method includes determining a trajectory relative to the object of interest (step 506). For example, the trajectory can be determined using the long-term planning module 152 of the vision-based navigation system 150 described above with reference to FIGS. 1 and 4. The method also includes controlling the one or more motors to move the chassis along the trajectory while keeping the object of interest in view of the imaging system (step 508). For example, the short-term planning module 156 and low-level controller module 158 of the vision-based navigation system 150 can control the motors 110 to move the chassis 105 as described above with reference to FIGS. 1 and 4.)
However Wang teach determining, based on the target information, among a plurality of flight trajectories, a target flight trajectory with respect to the target object, (Wang col. 38 line 53-col 39 line 3: any description herein of the distance may also refer to any permissible zone relative to the companion vehicle. Any permissible zone may have any shape and/or may refer to boundaries within which the UAV will need to stay relative to the companion vehicle. The permissible zone may have different distance thresholds for different directions relative to the companion vehicle. For example, a UAV may need to remain within 4 km to the front of the companion vehicle, 1 km of a side of the companion vehicle and 2 km of the rear of the companion vehicle. (185) As the companion vehicle moves, the threshold distance [based on the target information] and/or permissible zone may move with the companion vehicle. For example, if the permissible zone is a circle around the companion vehicle, as the companion vehicle moves, the permissible zone may move [plurality of flight trajectories] with the companion vehicle so that the center of the circle remains with the vehicle. Thus, the region within which UAV in flight may fly may change over time. As vehicle moves UAV keeps circling around vehicle makes many new UAV trajectory) and obtaining a video of the target object, by the photographing device, while flying according to the target flight trajectory. (Wang col. 52 line 23-30 a user on-board the vehicle may be able to see the images displayed on the monitor. The user on-board the vehicle may advantageously be able to see images captured by the UAV, which may show images of objects or locations that may otherwise not be viewable from the user while on-board the vehicle. The user may have a bird's eye view of the user's surrounding environment on the monitor. Wang col. 51 line 17-24 The UAV may send data to the companion vehicle. The data may include data from a payload of the vehicle and/or one or more sensors of the vehicle. In one example, the payload may be a camera or other type of image capturing device. The camera may capture static images (e.g., stills) and/or dynamic images (e.g., video). The images captured by the camera may be streamed to the companion vehicle.)
Prior art does not explicitly teach editing the video captured by the photographing device by modifying the video based on the at least one preset style of video post-photographing editing template.
This claim limitation is interpreted based on FIGS. 18 to 20 and PGPUB paragraph [0180-0181].
Dependent claims are allowed because of dependency.
Closely related prior art
Examiner notes teaching U.S. Pub. No. 20210375147 A1 is/are pertinent to the independent claim(s), however is not used because primary reference is more relevant.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896
Prosecution Insights