DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This communication is responsive to the correspondence filled on 1/26/2026.
Claims 1-22 are presented for examination.
Applicant is requested to file foreign priority document.
Response to Arguments
Applicant's arguments filed 1/26/2026 with respect to claims 1-22 have been considered but are moot in view of the new ground(s) of rejection. To reduce the number of reference Hu (U.S. Pub. No. 20200389593 A1) is replaced with Qian (U.S. Pub. No. 20200104598 A1) because of claim amendment. In this process some of the limitations has been remapped because of significant claim amendments.
Applicant argued in page 17 that prior art does not teach wherein the preliminary-flight trajectory and the control information of the gimbal are separately configured.
Examiner disagree on this because Qian [0129] the method may also include: if the joystick operation signal transmitted by the external is received, controlling at least one of [separately configured] the flight of the UAV or the attitude of the gimbal based on the joystick operation signal. In this embodiment, the joystick operation signal is generated through the user operating a remote-control device that controls the UAV. In some embodiments, the joystick operation signal may include at least one of a signal controlling the UAV to ascend or descend perpendicular to the ground, a signal controlling the UAV to move away or closer to the target object, a signal for controlling the flight velocity of the UAV, a signal for controlling the yaw angle of the gimbal, a signal for controlling the rotation of the aircraft body of the UAV, or a signal controlling other UAV parameters or gimbal parameters.
IDS Considerations
The information disclosure statement (IDS) submitted on 10/16/2024 is/are being considered by the examiner as the submission is in compliance with the provisions of 37 CFR 1.97.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 10-12, 17 and 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Qian (U.S. Pub. No. 20200104598 A1), in view of Enke (U.S. Pub. No. 20190215451 A1).
Regarding to claim 1, 19 and 21:
1. Qian teach a control method of an aerial vehicle including a gimbal carrying an image acquisition device, comprising: (Qian [0030] The imaging control method and device may be used for controlling the aerial photography of the UAV or the imaging of other aerial photography devices, such as an unmanned vehicle having a gimbal, a movable robot, etc.)
obtaining a preliminary-flight trajectory of the aerial vehicle (Qian [0041] A control signal may be generated based on at least a portion of the deviation (e.g., through one or multiple processors of the UAV), and an adjustment to roughly correct the deviation may be executed based on the control signal. The adjustment may be used in the images captured by the UAV to roughly maintain one or multiple expected features of the target object (e.g., the location or size of the target object). In some embodiments, when the UAV execute a flight instruction (e.g., circling or moving instructions) provided by a user or a predetermined flight path, the adjustment may be performed in real time) and control information of the gimbal, the preliminary-flight trajectory and the control information of the gimbal being set by a user; (Qian [0129] the method may also include: if the joystick operation signal transmitted by the external is received, controlling at least one of the flight of the UAV or the attitude of the gimbal based on the joystick operation signal.)
automatically controlling the aerial vehicle to move according to the preliminary-flight trajectory; and (Qian [0142] the user instruction may include: at least one of a flight distance and a flight velocity, which may be used to instruct the UAV to automatically accomplish a flight in each flight mode based on the flight distance and/or the flight velocity. The flight distance and flight velocity corresponding to each flight mode may be set based on actual needs, thereby satisfying diversified demands of the user. [0143] Step S802: generating a starting instruction based on the user instruction. The starting instruction may include a flight mode of the UAV. The starting instruction may be configured to trigger the autonomous flight of the UAV based on the flight mode)
automatically controlling the gimbal and the image acquisition device to photograph a target object according to the control information of the gimbal: (Qian [0123] during the image composition, if the target object need not be displayed at the horizontal (i.e., X axis) center of the captured image, then to accomplish the desired image composition, the attitude of the gimbal may be controlled, including: obtaining a horizontal field of view of the imaging device, a second total pixel distance of the captured image in a second direction, where the second direction corresponds to the yaw direction of the gimbal; determining a second pixel deviation distance of the target object to a center line of the second direction of the captured image; determining a yaw angle of the gimbal based on the second total pixel distance, the horizontal field of view, and the second pixel deviation distance; controlling the attitude of the gimbal based on the yaw angle. In some embodiments, as shown in FIG. 5, the second pixel deviation distance of the target object relative to the X axis may be d_col, the second total pixel distance of the image in the X axis direction may be col_size, the horizontal field of view of the camera may be HFOV, then the yaw angle of the gimbal may be d_col/col_size*HFOV. [0124] In some embodiments, the image composition may use the location in the captured image where the target object or the background identification is to be displayed as the basis for image composition. In some embodiments, background identifications such as sky, building, or sea surface, etc., may be recognized through a classification algorithm such as convolutional neural network (“CNN”), thereby achieving better image composition)
wherein the preliminary-flight trajectory and the control information of the gimbal are separately configured, (Qian [0129] the method may also include: if the joystick operation signal transmitted by the external is received, controlling at least one of [separately configured] the flight of the UAV or the attitude of the gimbal based on the joystick operation signal. In this embodiment, the joystick operation signal is generated through the user operating a remote control device that controls the UAV. In some embodiments, the joystick operation signal may include at least one of a signal controlling the UAV to ascend or descend perpendicular to the ground, a signal controlling the UAV to move away or closer to the target object, a signal for controlling the flight velocity of the UAV, a signal for controlling the yaw angle of the gimbal, a signal for controlling the rotation of the aircraft body of the UAV, or a signal controlling other UAV parameters or gimbal parameters.)
Qian do not explicitly teach the control information of the gimbal is determined based on a target photographing mode of the gimbal selected by the user from a plurality of preset photographing modes, and each of the plurality of preset photographing modes includes an attitude and an attitude change of the gimbal during photographing by the image acquisition device.
However Enke teach the control information of the gimbal is determined based on a target photographing mode of the gimbal selected by the user from a plurality of preset photographing modes, (Enke [0007] The input may be at least one of a device setting, a configuration, data associated with an image, or a user input. The device setting may be associated with the imaging device and may include at least one of an image resolution, an image mode, or a field of view of the imaging device. The image mode may be a portrait mode or a landscape mode) and each of the plurality of preset photographing modes includes an attitude and an attitude change of the gimbal during photographing by the image acquisition device. (Enke [0062] FIG. 4 The system 400 is configured to dynamically adjust the pitch, yaw, and roll of the stabilization mechanism 420 based on device setting parameters, configuration parameters, user input, and/or imaging system data. Example device setting aspects may be any device setting of the host 410 including, and not limited to, resolution, portrait/landscape mode, and/or field of view)
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Qian, further incorporating Enke in video/camera technology. One would be motivated to do so, to incorporate the control information of the gimbal is determined based on a target photographing mode of the gimbal selected by the user from a plurality of preset photographing modes. This functionality will improve efficiency with predictable results.
Regarding to claim 2:
2. Qian teach the method according to claim 1, Qian do not explicitly teach wherein obtaining the control information of the gimbal includes: obtaining the photographing mode of the gimbal, and including an attitude of the gimbal when the image acquisition device is photographing; and determining the control information of the gimbal according to the target photographing mode of the gimbal.
However Enke teach wherein obtaining the control information of the gimbal includes: obtaining the photographing mode of the gimbal, and (Enke [0062] FIG. 4 The system 400 is configured to dynamically adjust the pitch, yaw, and roll of the stabilization mechanism 420 based on device setting parameters, configuration parameters, user input, and/or imaging system data. Example device setting aspects may be any device setting of the host 410 including, and not limited to, resolution, portrait/landscape mode, and/or field of view) including an attitude of the gimbal when the image acquisition device is photographing; and determining the control information of the gimbal according to the target photographing mode of the gimbal. (Enke [0007] The input may be at least one of a device setting, a configuration, data associated with an image, or a user input. The device setting may be associated with the imaging device and may include at least one of an image resolution, an image mode, or a field of view of the imaging device. The image mode may be a portrait mode or a landscape mode)
Regarding to claim 10:
10. Qian teach the method according to claim 2, wherein determining the control information of the gimbal according to the target photographing mode of the gimbal includes: (Qian [0104] Step S105: based on the orientation information, controlling attitude [photographing mode] of a gimbal to render the target object to appear in an image captured by an imaging device) obtaining preliminary-flight distance information of the aerial vehicle; (Qian [0078] if the distance to the target object is relatively far away after the UAV moves, the features of the target object in the captured image such as the grayscale, texture, etc., may be insufficient for recognizing the target object from the image, which may result in the loss of the target object.) and determining the control information of the gimbal based on the preliminary-flight distance information of the aerial vehicle and the target photographing mode of the gimbal. (Qian [0123] In some embodiments, during the image composition, if the target object need not be displayed at the horizontal (i.e., X axis) center of the captured image, then to accomplish the desired image composition, the attitude of the gimbal may be controlled, including: obtaining a horizontal field of view of the imaging device, a second total pixel distance of the captured image in a second direction, where the second direction corresponds to the yaw direction of the gimbal; determining a second pixel deviation distance of the target object to a center line of the second direction of the captured image; determining a yaw angle of the gimbal based on the second total pixel distance, the horizontal field of view, and the second pixel deviation distance; controlling the attitude of the gimbal based on the yaw angle. In some embodiments, as shown in FIG. 5, the second pixel deviation distance of the target object relative to the X axis may be d_col, the second total pixel distance of the image in the X axis direction may be col_size, the horizontal field of view of the camera may be HFOV, then the yaw angle of the gimbal may be d_col/col_size*HFOV.)
Regarding to claim 11:
11. Qian teach the method according to claim 2, wherein determining the control information of the gimbal according to the target photographing mode of the gimbal includes: obtaining proportion information of the target object in an image containing the target object; (Qian [0081] The yaw angle in the imaging angle information may be a sum of the yaw angle of the gimbal and the deviation angle relative to the Y axis of the image. Specifically, FIG. 2a and FIG. 2b show the image physical coordinate system, the horizontal field of view (“HFOV”) and the vertical field of view (“VFOV”) of the imaging device. The deviation angle relative to the X axis of the image and the deviation angle relative to the Y axis of the image may be obtained based on a pixel distance ratio between the pixel distances of the center point of the target object relative to the X axis and the Y axis and the corresponding field of view. In addition, combining FIG. 6a, FIG. 6b, and FIG. 6c, which shows the location relationship between the imaging device and the imaging scene, the relationship between the target object and the field of view (“FOV”) of the imaging device may be learned by a person having ordinary skills in the art.) and determining the control information of the gimbal based on the proportion information and the target photographing mode of the gimbal. (Qian [0104] Step S105: based on the orientation [proportion] information, controlling attitude of a gimbal to render the target object to appear in an image captured by an imaging device. [0106] In some embodiments, to obtain better image composition effect, the attitude of the gimbal may be controlled such that the size of the target object in the captured images maintains a predetermined size. The orientation information may be the center location information of the predetermined size or other location information (e.g., vertex angle information) in a region corresponding to the predetermined size. In this embodiment, the size of the target object in the captured images is a production of a pixel height and a pixel width of the target object in the captured images. In some embodiments, the predetermined size may be a size frame directly input by the user at the user interface of the smart terminal 2. In some embodiments, the predetermined size may be a default size frame. Regardless of whether the predetermined size is a size frame set by the user or a default size frame, in subsequent imaging processes, the target object is located in the size frame in the captured images. In some embodiments, the size of the size frame may be configured to be just enough to surround the target object in the captured images, thereby satisfying an image composition demand [target photographing mode] of the user. In some embodiments, the size frame may have a shape of a rectangle, a square, etc. [0119] the elevation angle and/or the horizontal angle of the imaging location may be directly set by the user. Assuming the imaging location information corresponding to the imaging location is (x, y, z), a directional angle may be defined by the imaging location pointing to the target object. The elevation angle may be defined as arctan(z/x), the horizontal angle may be defined as arctan (y/x). the user setting the elevation angle is to set the ratio between x and z. The user setting the horizontal angle is to set the ratio between x and y. in some embodiments, determining the deviation angle of the target object relative to the center line of the first direction of the captured image may include: obtaining a first total pixel distance of the captured image in the first direction and the vertical field of view and/or horizontal field of view of the imaging device; determining a first deviation pixel distance of the target object to the center line of the first direction of the captured image; determining the deviation angle of the target object relative to the center line of the first direction of the captured image based on the first total pixel distance, the vertical field of view and/or the horizontal field of view, and the first deviation pixel distance.)
Regarding to claim 12:
12. Qian teach the method according to claim 2, Qian do not explicitly teach wherein the photographing mode includes at least one of horizontal photographing, vertical photographing, horizontal-vertical switching photographing, or preset angle photographing.
However Enke teach wherein the photographing mode includes at least one of horizontal photographing, vertical photographing, horizontal-vertical switching photographing, or preset angle photographing. (Enke [0007] The image mode may be a portrait mode or a landscape mode. [0062] FIG. 4 Example device setting aspects may be any device setting of the host 410 including, and not limited to, resolution, portrait [vertical]/landscape [horizontal] mode, and/or field of view. The configuration information may include a device type to which the stabilization mechanism is mounted, a type of situation that the stabilization mechanism is being used. [0040] FIG. 1-2 the imaging device 200 of FIG. 2A to provide still and video images by means of a communication link (not shown) to a ground-based user)
Regarding to claim 17:
17. Qian teach the method according to claim 1, wherein automatically controlling the gimbal and the image acquisition device to photograph the target object according to the control information of the gimbal includes: (Qian [0123] during the image composition, if the target object need not be displayed at the horizontal (i.e., X axis) center of the captured image, then to accomplish the desired image composition, the attitude of the gimbal may be controlled, including: obtaining a horizontal field of view of the imaging device, a second total pixel distance of the captured image in a second direction, where the second direction corresponds to the yaw direction of the gimbal; determining a second pixel deviation distance of the target object to a center line of the second direction of the captured image; determining a yaw angle of the gimbal based on the second total pixel distance, the horizontal field of view, and the second pixel deviation distance; controlling the attitude of the gimbal based on the yaw angle. In some embodiments, as shown in FIG. 5, the second pixel deviation distance of the target object relative to the X axis may be d_col, the second total pixel distance of the image in the X axis direction may be col_size, the horizontal field of view of the camera may be HFOV, then the yaw angle of the gimbal may be d_col/col_size*HFOV. [0124] In some embodiments, the image composition may use the location in the captured image where the target object or the background identification is to be displayed as the basis for image composition. In some embodiments, background identifications such as sky, building, or sea surface, etc., may be recognized through a classification algorithm such as convolutional neural network (“CNN”), thereby achieving better image composition)
Qian do not explicitly teach controlling a roll axis of the gimbal according to the control information of the gimbal such that the image acquisition device reaches a corresponding photographing mode of the plurality of preset photographing modes; and controlling the image acquisition device to perform a photographing operation based on the corresponding photographing mode.
However Enke teach controlling a roll axis of the gimbal according to the control information of the gimbal such that the image acquisition device reaches a corresponding photographing mode; (Enke [0030] Each of the motors 112, 114, 116 may control orientation of the imaging device 104 about a single axis (e.g., pitch, roll, or yaw), for example, when orientation of the rotational axes of the motors 112, 114, 116 is orthogonal. The axis controlled by each of the motors 112, 114, 116 may depend on the type of platform to which the gimbal 100 is attached. For example, when the gimbal 100 is attached to a UAV, the first motor 112 may rotate the imaging device 104 about a roll axis, the second motor 114 may rotate the imaging device 104 about a yaw axis, and the third motor 116 may rotate the imaging device 104 about a pitch axis. In another example, when the gimbal 100 is attached to a handheld grip, the first motor 112 may rotate the imaging device 104 about a yaw axis, the second motor 114 may rotate the imaging device 104 about a roll axis, and the third motor 116 may rotate the imaging device 104 about a pitch axis.) of the plurality of preset photographing modes; (Enke [0062] FIG. 4 The system 400 is configured to dynamically adjust the pitch, yaw, and roll of the stabilization mechanism 420 based on device setting parameters, configuration parameters, user input, and/or imaging system data. Example device setting aspects may be any device setting of the host 410 including, and not limited to, resolution, portrait/landscape mode, and/or field of view)
and controlling the image acquisition device to perform a photographing operation based on the corresponding photographing mode. (Enke [0007] The image mode may be a portrait mode or a landscape mode. [0062] FIG. 4 Example device setting aspects may be any device setting of the host 410 including, and not limited to, resolution, portrait/landscape mode, and/or field of view. The configuration information may include a device type to which the stabilization mechanism is mounted, a type of situation that the stabilization mechanism is being used. [0040] FIG. 1-2 the imaging device 200 of FIG. 2A to provide still and video images by means of a communication link (not shown) to a ground-based user)
Regarding to claim 20:
20. Qian teach an aerial vehicle system comprising: the aerial vehicle according to claim 19 and a gimbal control. (Qian [0129] the method may also include: if the joystick operation signal transmitted by the external is received, controlling at least one of [separately configured] the flight of the UAV or the attitude of the gimbal based on the joystick operation signal. In this embodiment, the joystick operation signal is generated through the user operating a remote control device that controls the UAV. In some embodiments, the joystick operation signal may include at least one of a signal controlling the UAV to ascend or descend perpendicular to the ground, a signal controlling the UAV to move away or closer to the target object, a signal for controlling the flight velocity of the UAV, a signal for controlling the yaw angle of the gimbal, a signal for controlling the rotation of the aircraft body of the UAV, or a signal controlling other UAV parameters or gimbal parameters. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art use the algorithm of Enke [0053] virtual touch screen buttons to control gimbal)
Qian do not explicitly teach a control terminal including a screen configured to display a trajectory selection control.
However Enke teach and a control terminal including a screen configured to display a trajectory selection control (Enke [0053] The computing device 300 may include or be in communication with a user interface (UI) 310. The UI 310 may include a display configured to provide information related to operation modes (e.g., camera modes, flight modes), connection status (e.g., connected, wireless, wired), power modes (e.g., standby, sensor, video), metadata sources 306 (e.g., heart rate, GPS, barometric), and/or other information associated with the gimbal 100, the imaging device 200, the handheld grip 206, the UAV 210, and/or the remote controller 214. In some implementations, the UI 310 may include virtually any device capable of registering inputs from and communicating outputs to a user. These may include, without limitation, display, touch, gesture, proximity, light, sound receiving/emitting, wired/wireless, and/or other input/output devices. The UI 310 may include a display, one or more tactile elements (e.g., joysticks, switches, buttons, and/or virtual touch screen buttons), lights such as light emitting diodes (LED)s, speaker, and/or other interface elements.)
Claims 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Qian (U.S. Pub. No. 20200104598 A1), in view of Enke (U.S. Pub. No. 20190215451 A1), further in view of Hu (U.S. Pub. No. 20200389593 A1).
Regarding to claim 8:
8. Qian teach the method according to claim 2, Qian do not explicitly teach wherein determining the control information of the gimbal according to the target photographing mode of the gimbal includes: obtaining a preliminary-photographing duration; and determining the control information of the gimbal according to the preliminary-photographing duration and the target photographing mode of the gimbal.
However Hu teach wherein determining the control information of the gimbal according to the target photographing mode of the gimbal includes: obtaining a preliminary-photographing duration; and determining the control information of the gimbal according to the preliminary-photographing duration (Hu [0037] In the embodiments of the present disclosure, only the prediction of a future flight trajectory is described, and the attitude of the gimbal may be adjusted according to the predicted flight trajectory. It can be known that the aircraft continuously predicts the flight trajectory within a preset time period from a current time at a certain frequency during the entire flight, and then adjusts the attitude of the gimbal according to each predicted flight trajectory thereby adjusting the attitude of the gimbal during the entire flight. For example, the aircraft performs steps S201 and S202 every several hundred milliseconds during the flight. [0038] The embodiments of the present disclosure bind the predicted velocity direction of a certain point on the flight trajectory with the attitude of the gimbal, to adjust the attitude of the gimbal, and then adjusts the direction of the camera device mounted on the gimbal to capture image information, so that the user can have a more realistic flight experience when controlling the UAV) and the target photographing mode of the gimbal. (Hu [0035] the control data may be configured to adjust one or more operation parameters of the load, such as capturing a static or dynamic image, varying a focus of the lens, start/shutdown, switching imaging modes, changing imaging resolution, changing focus, changing depth of field, changing exposure time, changing lens speed, changing viewable angle [photographing mode] or field of view, etc. [0026] The UAV 101 may have multiple operation modes. In one embodiment, it may include a first operation mode, a second operation mode, etc. In the first operation mode, the user can separately control a flight of the UAV 101 and the attitude of the gimbal 102, such that the UAV 101 flies according to the user's desire, and the gimbal 102 completes tasks such as recording in the direction that the user desires. In the second operation mode, the flight trajectory of the UAV 101 can be predicted, and the attitude of the gimbal 102 can be controlled in advance based on the flight trajectory to complete a special flight experience. For example, in the second operation mode, the flight trajectory of the UAV 101 can be better predicted for a certain period of time in the future, and the attitude of the gimbal 102 can be adjusted based on the predicted velocity direction at a point on the flight trajectory. The camera device mounted on the gimbal 102 may capture images in the predicted velocity direction, so that users can have a realistic flight experience)
The motivation for combining Qian and Enke as set forth in claim 1 is equally
applicable to claim 8. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Qian, further incorporating Enke and Hu in video/camera technology. One would be motivated to do so, to incorporate determining the control information of the gimbal according to the target photographing mode of the gimbal includes: obtaining a preliminary-photographing duration; and determining the control information of the gimbal according to the preliminary-photographing duration. This functionality will improve user experience with predictable results.
Regarding to claim 18:
Cancelled.
Regarding to claim 6:
Cancelled.
Claims 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Qian (U.S. Pub. No. 20200104598 A1), in view of Enke (U.S. Pub. No. 20190215451 A1), further in view of Takami (U.S. Pub. No. 20190238744 A1).
Regarding to claim 13:
13. Qian teach the method according to claim 1, Qian do not explicitly teach further comprising: generating a video based on content captured by the image acquisition device; obtaining an image type of a video frame in the video; and determining display information of the video frame according to the image type of the video frame.
However Takami teach further comprising: generating a video based on content captured by the image acquisition device; obtaining an image type of a video frame in the video; and determining display information of the video frame according to the image type of the video frame. (Takami [0065] FIG. 6 illustrates an example of the profile setting screen. A profile setting screen 40 illustrated in FIG. 6 is displayed on the display unit 23 of the client apparatus 20. A profile identifier (ID) list display area 41 displays a list of identifiers of the profiles set to the monitoring camera 10. The user operates the input unit 24 of the client apparatus 20 and selects a video source setting that indicates the distribute target video type of an image of the corresponding profile setting, from a selection box 42 of a video source setting video type.)
The motivation for combining Qian and Enke as set forth in claim 1 is equally
applicable to claim 13. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Qian, further incorporating Enke and Takami in video/camera technology. One would be motivated to do so, to incorporate generating a video based on content captured by the image acquisition device; obtaining an image type of a video frame in the video; and determining display information of the video frame according to the image type of the video frame. This functionality will improve quality with predictable results.
Claims 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Qian (U.S. Pub. No. 20200104598 A1), in view of Enke (U.S. Pub. No. 20190215451 A1), further in view of Zuckerman (U.S. Pub. No. 11565807 B1).
Regarding to claim 16:
16. Qian teach the method according to claim 1, Qian do not explicitly teach further comprising: obtaining a real-time position of the aerial vehicle; and displaying the real-time position of the aerial vehicle and the preliminary-flight trajectory on a map.
However Zuckerman teach further comprising: obtaining a real-time position of the aerial vehicle; and displaying the real-time position of the aerial vehicle. (Zuckerman Fig. 1J illustrates one embodiment of a drone operative to utilize multiple vehicles together with temporary landing sites and designated on-road stopping areas to move between different locations; Col. 30 line 15-32 The system can also detect on-road traffic in general, and use the traffic itself as a positioning reference, and then fly just above the traffic detected, therefore, again, avoiding other undetected or difficult-to-detect objects. The system can use the drone to create a real-time 3D map 1-RT-model of the environment, and use such real-time map to either detect the road and place the drone at a certain height above road, or compare/correlate structures 1-structure appearing in the real-time model with structures 1-structure-PE appearing in a pre-existing model 1-PE-model, thereby determining position of the drone relative to the pre-defined virtual flight corridor 1-corridor′ and make course corrections accordingly, or the system can utilize the real-time map 1-RT-model to facilitate both techniques at the same time, thereby staying above road traffic and below suspended road-related objects.) and the preliminary-flight trajectory on a map. (Zuckerman Col. 33 line 40-60 (ii) visual-simultaneous-localization-and-mapping (VSLAM) technique using imagery data captured by cameras (e.g., at least one of 4-cam-1, 4-cam-2, 4-cam-3, 4-cam-4, 4-cam-5, 4-cam-6 in FIG. 1A) onboard the drones 10a, 10b, 10c, 10d, 10e, 10f; and the system is further configured to use the three-dimensional data generated in order to map exact locations of the static objects 1-obstacles-1, 1-obstacles-2, 1-obstacles-3 suspended above the roads 1-road-1, 1-road-2, and use said exact locations to adjust the flight corridors 1-corridor (e.g., the adjustment 1-corridor-adjust in FIG. 1N) so as to account for the actual heights of the suspended static objects 1-obstacles, in which said adjustment comprises at least one of: (i) expanding relevant parts of the flight corridors upwards 1-corridor-adjust (FIG. 1N), so as to still pass safely below at least some of the suspended static objects 1-obstacles along the flight path 10-path-a, 10-path-b, 10-path-c, 10-path-d, but so as to now pass higher and closer to the actual suspended static objects as permitted by said mapping, and (ii) shrinking relevant parts of the flight corridors 1-corridor downwards, so as to pass even safer below at least some of the suspended static objects 1-obstacles along the flight path 10-path-a, 10-path-b, 10-path-c, 10-path-d as required by said mapping.)
The motivation for combining Qian and Enke as set forth in claim 1 is equally
applicable to claim 16. It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to modify Qian, further incorporating Enke and Zuckerman in video/camera technology. One would be motivated to do so, to incorporate obtaining a real-time position of the aerial vehicle; and displaying the real-time position of the aerial vehicle. This functionality will improve speed with predictable results.
Allowable subject matter
Regarding to claim 3-5, 7, 9 and 14-15:
Claims 3-5, 7, 9 and 14-15 is/are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims because the limitations of these dependent claims are not obvious from the prior art search when all the limitations of independent and intervening claims are taken into account.
Claim 22 is allowed because:
Regarding to claim 22:
22. Qian teach a control method of an aerial vehicle including a gimbal carrying an image acquisition device, comprising: obtaining a preliminary-flight trajectory of the aerial vehicle and control information of the gimbal, the preliminary-flight trajectory and the control information of the gimbal being set by a user, and obtaining the control information of the gimbal including: obtaining a photographing mode of the gimbal, the photographing mode of the gimbal being determined according to a user selection and including an attitude of the gimbal when the image acquisition device is photographing, automatically controlling the aerial vehicle to move according to the preliminary-flight trajectory; automatically controlling the gimbal and the image acquisition device to photograph a target object according to the control information of the gimbal; (Please see the rejection of claim 1)
and the matching photographing mode of the gimbal is related to a length and a width of the target object on the screen; (Qian [0104] Step S105: based on the orientation information, controlling attitude of a gimbal [matching photographing mode] to render the target object to appear in an image captured by an imaging device. [0106] In some embodiments, to obtain better image composition effect, the attitude of the gimbal may be controlled such that the size of the target object in the captured images maintains a predetermined size. The orientation information may be the center location information of the predetermined size or other location information (e.g., vertex angle information) in a region corresponding to the predetermined size. In this embodiment, the size of the target object in the captured images is a production of a pixel height and a pixel width of the target object in the captured images. In some embodiments, the predetermined size may be a size frame directly input by the user at the user interface of the smart terminal 2. In some embodiments, the predetermined size may be a default size frame. Regardless of whether the predetermined size is a size frame set by the user or a default size frame, in subsequent imaging processes, the target object is located in the size frame in the captured images. In some embodiments, the size of the size frame may be configured to be just enough to surround the target object in the captured images, thereby satisfying an image composition demand of the user. In some embodiments, the size frame may have a shape of a rectangle, a square, etc.) and determining the control information of the gimbal according to the photographing mode of the gimbal; (Qian [0104] Step S105: based on the orientation information, controlling attitude [photographing mode] of a gimbal to render the target object to appear in an image captured by an imaging device) obtaining preliminary-flight distance information of the aerial vehicle; Qian [0078] if the distance to the target object is relatively far away after the UAV moves, the features of the target object in the captured image such as the grayscale, texture, etc., may be insufficient for recognizing the target object from the image, which may result in the loss of the target object.)
Enke teach wherein the user selection is determined based on a user operation on a screen, (Enke [0053] The computing device 300 may include or be in communication with a user interface (UI) 310. The UI 310 may include a display configured to provide information related to operation modes (e.g., camera modes, flight modes), connection status (e.g., connected, wireless, wired), power modes (e.g., standby, sensor, video), metadata sources 306 (e.g., heart rate, GPS, barometric), and/or other information associated with the gimbal 100, the imaging device 200, the handheld grip 206, the UAV 210, and/or the remote controller 214. In some implementations, the UI 310 may include virtually any device capable of registering inputs from and communicating outputs to a user. These may include, without limitation, display, touch, gesture, proximity, light, sound receiving/emitting, wired/wireless, and/or other input/output devices. The UI 310 may include a display, one or more tactile elements (e.g., joysticks, switches, buttons, and/or virtual touch screen buttons), lights such as light emitting diodes (LED)s, speaker, and/or other interface elements.)
the screen is configured to display a matching photographing mode of the gimbal that match the target object, (Enke [0007] The device setting may be associated with the imaging device and may include at least one of an image resolution, an image mode, or a field of view of the imaging device. The image mode may be a portrait mode or a landscape mode)
Prior art do not teach and at least one of: determining the matching photographing mode of the gimbal based on an aspect ratio of the target object; or determining the matching photographing mode of the gimbal based on a relative proportion between the length and the width; wherein: determining the matching photographing mode of the gimbal based on the aspect ratio of the target object includes: in response to the aspect ratio being greater than a first threshold, determining that the matching photographing mode of the gimbal includes horizontal photographing; in response to the aspect ratio being less than a second threshold, determining that the matching photographing mode of the gimbal includes vertical photographing;
or in response to the aspect ratio being greater than or equal to the second threshold and less than or equal to the first threshold, determining that the matching photographing mode of the gimbal includes horizontal-vertical switching photographing; and determining the matching photographing mode of the gimbal based on the relative proportion between the length and the width includes: in response to the length of the target object being greater than the width, determining that the matching photographing mode of the gimbal includes horizontal photographing, vertical photographing, or horizontal-vertical switching photographing; or in response to the length of the target object being less than or equal to the width, determining that the matching photographing mode of the gimbal includes vertical photographing.
Closely related prior art
Examiner notes teaching of Fig, 3 of U.S. Pub. No. 20200389593 A1 is/are pertinent to the independent claim(s), however this is not used in independent claim rejection because this does not teach does not disclose or suggest that "the preliminary-flight trajectory and the control information of the gimbal are separately configured, the control information of the gimbal is determined based on a target photographing mode of the gimbal selected by the user from a plurality of preset photographing modes, and each of the plurality of preset photographing modes includes an attitude and an attitude change of the gimbal during photographing by the image acquisition device.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NASIM N NIRJHAR whose telephone number is (571) 272-3792. The examiner can normally be reached on Monday - Friday, 8 am to 5 pm ET.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William F Kraig can be reached on (571) 272-8660. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NASIM N NIRJHAR/Primary Examiner, Art Unit 2896