Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claim 16 objected to because of the following informalities: Claim 16 recites “The method according to claim 5,…” when claim 5 has now been cancelled. For sake of examination, the examiner assumes the dependency of claim 16 to be upon claim 1 based on other amendments made to other pending claims. Appropriate correction is required.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-4 and 6-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 6, 16-17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Li, et al., hereinafter Li (U.S. Patent Application Pub. No. 2018/0241936) in view of Townsend, et al., hereinafter Townsend (U.S. Patent No. 2019/0273837).
Regarding Claim 1, Li teaches: A control method for an aerial vehicle (Li, Para. 0010 – “a method for operating an unmanned aerial vehicle (UAV)”) comprising:
obtaining a control operation value sent by a control device in communication connection with the aerial vehicle (Li, Para. 0053, 0089, 0131, and 0230-0231 – where an operator triggers “simulated-joystick operation commands”, or control operation value, on a “mobile terminal”, or control device, and “the simulated-joystick operation command is used to operate the aircraft to move vertically and change an attitude of the aircraft”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue operation commands);
where a “target locking page” provides a “joystick control” which may be operated by the user to “adjust the photographing view of the UAV”, or target image region, where a “found target” is determined based on searching the photographing view; where the UAV captures a “horizontal panoramic landscape image” of the photographing view using a “camera or video camera attached to the UAV”),
sending the target image region to the control device, to enable the control device to display the target image region (Li, Fig. 8 and Para. 0088 and 0135 – where the “mobile terminal” may “display the preview image and the mark on the found target”; where Fig. 8 shows a “mark on the found target” 802 which is a region around a target in the photographing view of the UAV).
PNG
media_image1.png
643
558
media_image1.png
Greyscale
Li, Fig. 8
While Li teaches the control operation value, and determining a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, Li does not teach determining, according to the control operation value, a virtual attitude angle to which the control operation value maps, and determining, according to the determined virtual attitude angle, a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, a position or attitude of the one or more photographing devices relative to the aerial vehicle not changing irrespective of the determined virtual attitude angle.
However, Townsend teaches determining, according to the control operation value, a virtual attitude angle to which the control operation value maps (Townsend, Para. 0069-0074 – a device which receives “input from a user of the device”, where the user may “specify or modify an angle associated with the output video data”, and where the device may determine an “angle between the projected vector and a reference vector on the reference plane” which “is called the azimuth”, or virtual attitude angle, “at a reference location relative to the image capture device”), and determining, according to the determined virtual attitude angle, a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, a position or attitude of the one or more photographing devices relative to the aerial vehicle not changing irrespective of the determined virtual attitude angle (Townsend, Fig. 2B and Para. 0069-0074, 0081, 0163-0164 – where based on “the angle of view (e.g., azimuth)”, or virtual attitude angle, the device generates a “cropped image” within a “panoramic image” that is recorded by an “image capture device” at “a fixed location”, such that the user inputted angle does not affect the position/attitude of the “image capture device”, where the “cropped image” may be include “a point of interest”, or target image region; Fig. 2B shows an example of a cropped image, or target image region, of a full panoramic image).
PNG
media_image2.png
1116
882
media_image2.png
Greyscale
Townsend, Fig. 2B
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include determining, according to the control operation value, a virtual attitude angle to which the control operation value maps, and determining, according to the determined virtual attitude angle, a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, a position or attitude of the one or more photographing devices relative to the aerial vehicle not changing irrespective of the determined virtual attitude angle, as taught by Townsend, in order to provide a method of easily highlighting subject matter which is interesting within a panoramic image (Townsend, Para. 0031).
In regards to Claim 3, Li in view of Townsend teaches the method of Claim 1, and Li further teaches wherein:
the control device includes a remote controller and a terminal device (Li, Para. 0088-0089, 0230-0231 and 0249 – where the “mobile terminal”, or control device, controls the “remote UAV” through “simulated-joystick operation commands” and where the “mobile terminal” also includes a “display” for displaying a “preview image”, such that the mobile terminal acts as both the remote controller and the terminal device);
obtaining the control operation value sent by the control device includes obtaining the control operation value sent by the remote controller (Li, Para. 0053, 0131, and 0230-0231 – where an operator triggers “simulated-joystick operation commands” on the “mobile terminal”, or control device, and “the simulated-joystick operation command is used to operate the aircraft to move vertically and change an attitude of the aircraft”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue operation commands); and
sending the target image region to the control device to enable the control device to display the target image region includes sending the target image region to the terminal device, to enable the terminal device to display the target image region (Li, Fig. 8 and Para. 0088 and 0135 – where the “mobile terminal” may “display the preview image and the mark on the found target”; where Fig. 8 shows a “mark on the found target” 802 which is a region around a target in the photographing view of the UAV).
In regards to Claim 4, Li teaches the method of Claim 1, and Li further teaches wherein:
the control device includes a control area and a display area (Li, Fig. 8, Para. 0088-0089, 0230-0231, and 0249 – where the “mobile terminal”, or control device, includes “a preview image” in a portion of the mobile termina, such that it is a display area, and the display includes a “joystick control”, located on the bottom of the display, or a control area);
obtaining the control operation value sent by the control device includes obtaining the control operation value sent generated based on a user operation in the control area (Li, Para. 0053, 0131, and 0230-0231 – where an operator triggers “simulated-joystick operation commands” on the “mobile terminal”, or control device, and “the simulated-joystick operation command is used to operate the aircraft to move vertically and change an attitude of the aircraft”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue operation commands); and
sending the target image region to the control device to enable the control device to display the target image region includes sending the target image region to the control device, to enable the display area of the control device to display the target image region (Li, Fig. 8 and Para. 0088-0089 and 0135 – where the “mobile terminal” may “display the preview image and the mark on the found target”; where Fig. 8 shows a “mark on the found target” 802 which is a region around a target in the photographing view of the UAV in “a preview image 801”, or display area).
PNG
media_image3.png
614
671
media_image3.png
Greyscale
Li, Annotated Fig. 8
In regards to Claim 6, Li in view of Wang teaches the method of Claim 1, and Li in view of Townsend further teaches wherein determining the target image region according to the virtual attitude angle includes determining the target image region according to a preset field of view and the virtual attitude angle (Townsend, Fig. 2B and Para. 0069-0074 and 0221 – where based on “the angle of view (e.g., azimuth)”, or virtual attitude angle, the device generates a “cropped image” within a “panoramic image” that is recorded by an “image capture device”, where the “cropped image” may be include “a point of interest”, or target image region, and wherein the panoramic image is cropped to a “16:9 aspect ratio”, or “narrow field of view” focused on the target, or preset field of view, as shown on Fig. 2B).
PNG
media_image2.png
1116
882
media_image2.png
Greyscale
Townsend, Fig. 2B
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend to further include wherein determining the target image region according to the virtual attitude angle includes determining the target image region according to a preset field of view and the virtual attitude angle, as taught by Townsend, in order to adjust the field of view to include the target image region to display interesting subject matter.
In regards to Claim 16, Li in view of Townsend teaches the method of Claim 1, and Li in view of Townsend further teaches wherein determining, according to the control operation value, the virtual attitude angle to which the control operation value maps includes: determining the virtual attitude angle in a virtual camera coordinate system according to the control operation value (Townsend, Para. 0065 and 0069-0074 – a device which receives “input from a user of the device”, where the user may “specify or modify an angle associated with the output video data”, and where the device may determine an “angle between the projected vector and a reference vector on the reference plane” which “is called the azimuth”, or virtual attitude angle, “at a reference location relative to the image capture device”; where “the portion of the panoramic video data displayed on the display 104 (e.g., cropped image, image data, etc.) may be associated with a position (e.g., x and y pixel coordinates) within the panoramic video data”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend to further include wherein determining, according to the control operation value, the virtual attitude angle to which the control operation value maps includes: determining the virtual attitude angle in a virtual camera coordinate system according to the control operation value, as taught by Townsend, in order to provide a reference frame for the virtual attitude of the camera to better define the position of the camera.
In regards to Claim 17, Li in view of Townsend teaches the method of Claim 1, but Li does not teach wherein the one or more photographing devices include a plurality of photographing devices.
However, Townsend teaches wherein the one or more photographing devices include a plurality of photographing devices (Townsend, Para. 0064 – where the “image capture device” may include “a plurality of cameras”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend to further include wherein the one or more photographing devices include a plurality of photographing devices, as taught by Townsend, in order to utilize a plurality of photographing device to collect more images or videos of a target object and provide multiple views to a user.
Regarding Claim 19, Li teaches: An aerial vehicle (Li, Para. 0010 – “an unmanned aerial vehicle (UAV)”) comprising:
one or more photographing devices configured to capture a panoramic image (Li, Para. 0095 and 0163 – where the UAV captures a “horizontal panoramic landscape image” of a photographing view using a “camera or video camera attached to the UAV”);
a memory storing a computer program (Li, Para. 0054 and 0304 – “a memory” which is “configured to provide a high-speed cache to the operating system and for the computer readable instructions”, where the memory is a storage medium which stores “a computer program instructing relevant hardware”); and
a processor configured to execute the computer program (Li, Para. 0053 – “a processor” configured to “perform a control method for photographing using a UAV”, which executes the “computer readable instructions” stored in the memory) to:
obtain a control operation value sent by a control device in communication connection with the aerial vehicle (Li, Para. 0053, 0089, 0131, and 0230-0231 – where an operator triggers “simulated-joystick operation commands” on a “mobile terminal”, or control device, and “the simulated-joystick operation command is used to operate the aircraft to move vertically and change an attitude of the aircraft”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue operation commands);
determining, where a “target locking page” provides a “joystick control” which may be operated by the user to “adjust the photographing view of the UAV”, or target image region, where a “found target” is determined based on searching the photographing view; where the UAV captures a “horizontal panoramic landscape image” of the photographing view),
send the target image region to the control device, to enable the control device to display the target image region (Li, Fig. 8 and Para. 0088 and 0135 – where the “mobile terminal” may “display the preview image and the mark on the found target”; where Fig. 8 shows a “mark on the found target” 802 which is a region around a target in the photographing view of the UAV).
PNG
media_image1.png
643
558
media_image1.png
Greyscale
Li, Fig. 8
While Li teaches the control operation value, and determining a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, Li does not teach determining, according to the control operation value, a virtual attitude angle to which the control operation value maps, and determining, according to the determined virtual attitude angle, a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, a position or attitude of the one or more photographing devices relative to the aerial vehicle not changing irrespective of the determined virtual attitude angle.
However, Townsend teaches determining, according to the control operation value, a virtual attitude angle to which the control operation value maps (Townsend, Para. 0069-0074 – a device which receives “input from a user of the device”, where the user may “specify or modify an angle associated with the output video data”, and where the device may determine an “angle between the projected vector and a reference vector on the reference plane” which “is called the azimuth”, or virtual attitude angle, “at a reference location relative to the image capture device”), and determining, according to the determined virtual attitude angle, a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, a position or attitude of the one or more photographing devices relative to the aerial vehicle not changing irrespective of the determined virtual attitude angle (Townsend, Fig. 2B and Para. 0069-0074, 0081, 0163-0164 – where based on “the angle of view (e.g., azimuth)”, or virtual attitude angle, the device generates a “cropped image” within a “panoramic image” that is recorded by an “image capture device” at “a fixed location”, such that the user inputted angle does not affect the position/attitude of the “image capture device”, where the “cropped image” may be include “a point of interest”, or target image region; Fig. 2B shows an example of a cropped image, or target image region, of a full panoramic image).
PNG
media_image2.png
1116
882
media_image2.png
Greyscale
Townsend, Fig. 2B
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include determining, according to the control operation value, a virtual attitude angle to which the control operation value maps, and determining, according to the determined virtual attitude angle, a target image region in a panoramic image captured by one or more photographing devices carried by the aerial vehicle, a position or attitude of the one or more photographing devices relative to the aerial vehicle not changing irrespective of the determined virtual attitude angle, as taught by Townsend, in order to provide a method of easily highlighting subject matter which is interesting within a panoramic image (Townsend, Para. 0031).
Claim(s) 2 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Townsend, and further in view of Huang, et al., hereinafter Huang (U.S. Patent Application Pub. No. 2019/0220002).
In regards to Claim 2, Li in view of Townsend teaches the method of Claim 1, and Li further teaches wherein:
the control device includes a remote controller (Li, Para. 0230-0231 and 0249 – where the “mobile terminal”, or control device, controls the “remote UAV” through “simulated-joystick operation commands”)
obtaining the control operation value sent by the control device includes obtaining the control operation value sent by the remote controller (Li, Para. 0053, 0131, and 0230-0231 – where an operator triggers “simulated-joystick operation commands” on the “mobile terminal”, or control device, and “the simulated-joystick operation command is used to operate the aircraft to move vertically and change an attitude of the aircraft”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue operation commands); and
sending the target image region to the control device to enable the control device to display the target image region where the “mobile terminal” may “display the preview image and the mark on the found target”; where Fig. 8 shows a “mark on the found target” 802 which is a region around a target in the photographing view of the UAV).
PNG
media_image1.png
643
558
media_image1.png
Greyscale
Li, Fig. 8
Li does not teach a head-mounted display device and sending the target image region to the head-mounted display device, to enable the head-mounted display device to display the target image region.
However, Huang teaches a head-mounted display device (Huang, Para. 0014 – a “terminal”, or control device, including a “head-mounted display (HMD)”) and sending the target image region to the head-mounted display device, to enable the head-mounted display device to display the target image region (Huang, Para. 0014, 0045, 0067-0068, and 0080 – where “terminal” receives “imaging data from a movable object”, such that the movable object sends the imaging data, and displays the image on a “display”; where the display may be a “head-mounted display” which displays “a plurality of FPV stereoscopic images or video captured by an imaging device on a movable object”; where the “movable object” is “an unmanned aerial vehicle (UAV)” and the image includes “a portion of the image (e.g., point, region, and/or object)” defined as a “target” by the user).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend to include a head-mounted display device and sending the target image region to the head-mounted display device, to enable the head-mounted display device to display the target image region, as taught by Huang, in order to provide a method of presenting a first person view to a user by a head-mounted display device to provide an immersive experience and to better engage the user.
In regards to Claim 20, Li in view of Townsend teaches the aerial vehicle of Claim 19, and Li further teaches wherein:
the control device includes a remote controller (Li, Para. 0230-0231 and 0249 – where the “mobile terminal”, or control device, controls the “remote UAV” through “simulated-joystick operation commands”)
and the processor is further configured to execute the computer program (Li, Para. 0053 – “a processor” configured to “perform a control method for photographing using a UAV”, which executes the “computer readable instructions” stored in the memory) to:
obtain the control operation value sent by the remote controller (Li, Para. 0053, 0131, and 0230-0231 – where an operator triggers “simulated-joystick operation commands” on the “mobile terminal”, or control device, and “the simulated-joystick operation command is used to operate the aircraft to move vertically and change an attitude of the aircraft”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue operation commands); and
send the target image region to the where the “mobile terminal” may “display the preview image and the mark on the found target”; where Fig. 8 shows a “mark on the found target” 802 which is a region around a target in the photographing view of the UAV).
PNG
media_image1.png
643
558
media_image1.png
Greyscale
Li, Fig. 8
Li does not teach a head-mounted display device and to send the target image region to the head-mounted display device, to enable the head-mounted
However, Huang teaches a head-mounted display device (Huang, Para. 0014 – a “terminal”, or control device, including a “head-mounted display (HMD)”) and to send the target image region to the head-mounted display device, to enable the head-mountedwhere “terminal” receives “imaging data from a movable object”, such that the movable object sends the imaging data, and displays the image on a “display”; where the display may be a “head-mounted display” which displays “a plurality of FPV stereoscopic images or video captured by an imaging device on a movable object”; where the “movable object” is “an unmanned aerial vehicle (UAV)” and the image includes “a portion of the image (e.g., point, region, and/or object)” defined as a “target” by the user).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the aerial vehicle including the above limitations of Li in view of Townsend to include a head-mounted display device and to send the target image region to the head-mounted display device, to enable the head-mounted.
Claim(s) 7-9 and 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Townsend, and further in view of Wang, et al., hereinafter Wang (U.S. Patent Application Pub. No. 2015/0142213).
In regards to Claim 7, Li in view of Townsend teaches the method of Claim 1, Li in view of Townsend teaches the virtual attitude angle (Townsend, Para. 0069-0074 – a device which receives “input from a user of the device”, where the user may “specify or modify an angle associated with the output video data”, and where the device may determine an “angle between the projected vector and a reference vector on the reference plane” which “is called the azimuth”, or virtual attitude angle, “at a reference location relative to the image capture device”), does not teach wherein the virtual attitude angle to which the control operation value maps is related to a flight control quantity of the aerial vehicle to which the control operation value maps.
However, Wang teaches wherein the virtual attitude angle to which the control operation value maps (Wang, Para. 0201 and 0271-0274 – where “operation of the carrier and/or the movable object” is achieved by “a virtual joystick” operated by the user to generate “a state control signal”; where left and right virtual control sticks control “the payload's pitch” and the “aircraft's yaw”, or the virtual attitude) is related to a flight control quantity of the aerial vehicle to which the control operation value maps (Wang, Para. 0271-0273 and 0322-0323 – where the virtual joysticks control pitch and yaw, and a method is provided to “limit the extent the terminal and/or payload may be rotated about an axis”, for example a “mechanical limit”, or flight control quantity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method including the above limitations of Li in view of Townsend to include wherein the virtual attitude angle to which the control stick value maps is related to a flight control quantity of the aerial vehicle to which the control stick value maps, as taught by Wang, in order to provide a method of adjusting an attitude angle of the aerial vehicle within the mechanical limitations of the vehicle to provide steady movements when filming with the vehicle’s camera.
In regards to Claim 8, Li in view of Townsend teaches the method of Claim 1, and Li in view of Townsend teaches wherein: the aerial vehicle is an unmanned aerial vehicle (Li, Para. 0010 – “operating an unmanned aerial vehicle (UAV)”); but Li in view of Townsend does not teach determining the virtual attitude angle to which the control operation value maps according to the control operation value includes: determining, according to the control operation value, a flight control quantity of the unmanned aerial vehicle to which the control operation value maps, and determining the virtual attitude angle to which the control operation value maps according to the control operation value and the flight control quantity of the unmanned aerial vehicle.
However Wang teaches determining the virtual attitude angle to which the control operation value maps according to the control operation value includes: determining, according to the control operation value, a flight control quantity of the unmanned aerial vehicle to which the control operation value maps (Wang, Para. 0271-0273 and 0322-0323 – where the virtual joysticks control pitch and yaw, and a method is provided to “limit the extent the terminal and/or payload may be rotated about an axis”, for example a “mechanical limit”, or flight control quantity); and determining the virtual attitude angle to which the control operation value maps according to the control operation value and the flight control quantity of the unmanned aerial vehicle (Wang, Para. 0201, 0271-0273, and 0322-0323 – where “operation of the carrier and/or the movable object” is achieved by “a virtual joystick” operated by the user; where left and right virtual control sticks control “the payload's pitch” and the “aircraft's yaw”, or the virtual attitude; where the output attitude is based on the control of the payloads pitch and aircraft’s yaw and the mechanical limit the “terminal and/or payload may be rotated about an axis”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method including the above limitations of Li in view of Townsend to include determining the virtual attitude angle to which the control stick value maps according to the control operation value includes: determining, according to the control operation value, a flight control quantity of the unmanned aerial vehicle to which the control operation value maps; and determining the virtual attitude angle to which the control operation value maps according to the control operation value and the flight control quantity of the unmanned aerial vehicle, as taught by Wang, in order to provide a method of adjusting an attitude angle of the aerial vehicle within the mechanical limitations of the vehicle to provide steady movements when filming with the vehicle’s camera.
In regards to Claim 9, Li in view of Townsend and Wang teaches the method of Claim 8, and Li in view of Wang further teaches wherein determining, according to the control operation value, the flight control quantity of the unmanned aerial vehicle to which the control operation value maps includes: determining the flight control quantity of the unmanned aerial vehicle according to the control operation value and a preset virtual aircraft control model (Wang, Para. 0213 and 0318-0319 – where a terminal controls the aircraft, and the terminal controls may be “traditional mechanical sticks (i.e. joystick)”; where it is determined whether an “inclination angle” input by the terminal is “within a certain range”, and if so, “the pitch inclination of the terminal corresponds to the pitch inclination of the payload”, and similarly for other angles, and if the angle is “outside the range”, the angle input is adjusted to indicate “rotational speed of the payload”, such that the range control acts as a control model), the preset virtual aircraft control model including a correspondence relationship between the flight control quantity of the unmanned aerial vehicle and the control operation value (Wang, Para. 0318-0319 – where if the angle is “within a certain range”, a “linear relationship may be provided between the angle measurement of the terminal and the payload when within the predetermined range”, and if outside the range, the angle may indicate a “speed of the payload”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend and Wang to further include wherein determining, according to the control stick value, the flight control quantity of the unmanned aerial vehicle to which the control stick value maps includes: determining the flight control quantity of the unmanned aerial vehicle according to the control stick value and a preset virtual aircraft control model, the preset virtual aircraft control model including a correspondence relationship between the flight control quantity of the unmanned aerial vehicle and the control stick value, as taught by Wang, in order to model how the input control stick values relate the movement of the aerial vehicle, to allow smooth control of the aerial vehicle.
In regards to Claim 11, Li in view of Wang teaches the method of Claim 8, and Li in view of Townsend and Wang further teaches wherein: the control operation value includes a first control operation value and a second control operation value (Li, Fig. 8 and Para. 0089 – “a joystick control 803”, where Fig. 8 depicts a first and second control stick; Wang, Para. 0271-0272 – “touch-screen-implemented virtual control sticks 601 and 602 to control the aircraft”); and determining, according to the control operation value, the flight control quantity of the unmanned aerial vehicle to which the control operation value maps includes: determining, according to the first control operation value, a first flight control quantity of the unmanned aerial vehicle for flying upwards or downwards in a vehicle body coordinate system (Wang, Para. 0271-0272 – “touch-screen-implemented virtual control sticks 601 and 602 to control the aircraft”, which may control “the aircraft's multiple movement dimensions, such as… up/down”); and determining, according to the second control operation value, a second flight control quantity of the unmanned aerial vehicle for flying forwards or backwards in the vehicle body coordinate system (Wang, Para. 0271-0272 – “touch-screen-implemented virtual control sticks 601 and 602 to control the aircraft”, which may control “the aircraft's multiple movement dimensions, such as back/forth”).
PNG
media_image1.png
643
558
media_image1.png
Greyscale
Li, Fig. 8
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend and Wang to further include wherein: the control stick value includes a first control stick value and a second control stick value; and determining, according to the control stick value, the flight control quantity of the unmanned aerial vehicle to which the control stick value maps includes: determining, according to the first control stick value, a first flight control quantity of the unmanned aerial vehicle for flying upwards or downwards in a vehicle body coordinate system; and determining, according to the second control stick value, a second flight control quantity of the unmanned aerial vehicle for flying forwards or backwards in the vehicle body coordinate system, as taught by Wang, in order to allow the aerial vehicle to fly in different directions and adjust the altitude by flying upwards or downwards in order to better image a target object or region.
In regards to Claim 12, Li in view of Townsend and Wang teaches the method of Claim 11, and Li in view of Townsend and Wang further teaches wherein: the control operation value further includes a third control operation value (Wang, Para. 0271-0272 – “touch-screen-implemented virtual control sticks 601 and 602 to control the aircraft”, which may control “the aircraft's multiple movement dimensions, such as back/forth, left/right, up/down and the pointing direction (e.g., attitude of the aircraft)”, including “pitch” and “yaw”); and determining, according to the control operation value, the flight control quantity of the unmanned aerial vehicle to which the control operation value maps further includes: determining a yaw angle in the virtual attitude angle according to the third control operation value (Wang, Para. 0271 – “the left/right movement of the left visual control stick 601 can control aircraft's yaw”); and determining a pitch angle in the virtual attitude angle according to the first flight control quantity and the second flight control quantity (Wang, Para. 0271-0272 – “the up and down movement of the left virtual control stick 601 can control the payload's pitch”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend and Wang to further include wherein: the control stick value further includes a third control stick value; and determining, according to the control stick value, the flight control quantity of the unmanned aerial vehicle to which the control stick value maps further includes: determining a yaw angle in the virtual attitude angle according to the third control stick value; and determining a pitch angle in the virtual attitude angle according to the first flight control quantity and the second flight control quantity., as taught by Wang, in order to adjust the attitude of the aerial vehicle to better capture a view of a target object or image region.
In regards to Claim 13, Li in view of Townsend and Wang teaches the method of Claim 12, and Li in view of Townsend and Wang further teaches wherein the first control operation value and the second control operation value are speed control quantities (Wang, Para. 0273 – “the speed of the joystick can control the speed of the aircraft”, for example “the faster the movement of the virtual joystick, the faster the payload's speed in the corresponding direction”; where the joysticks may control “back/forth, left/right, up/down and the pointing direction (e.g., attitude of the aircraft)””).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Wang to further include wherein the first control stick value and the second control stick value are speed control quantities, as taught by Wang, in order to allow the user the adjust the position of the aerial vehicle at different speed in order to achieve steady camera movement or speed up the operation of the aerial vehicle to quickly achieve a certain view.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Li, in view of Townsend and Wang, and further in view of Huang.
In regards to Claim 10, Li in view of Townsend and Wang teaches the method of Claim 9, but Li in view of Townsend and Wang does not teach wherein the preset virtual aircraft control model includes a preset virtual first-person view aircraft control model.
However, Huang teaches wherein the preset virtual aircraft control model includes a preset virtual first-person view aircraft control model (Huang, Para. 0004-0005 and 0014 – generating a “first person view (FPV) of an environment” from “video data of the environment” generated by a movable object, or aircraft; where a terminal allows a user to “control and navigate the movable object from the first person view (FPV)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method including the above limitations of Li in view of Townsend and Wang to include wherein the preset virtual aircraft control model includes a preset virtual first-person view aircraft control model, as taught by Huang, in order to provide a method of presenting a first person view to a user to provide an immersive experience and to better engage the user.
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Li, in view of Townsend and Wang, and further in view of Hu, et al., hereinafter Hu (Chinese Patent Application Pub. No. 110297500).
In regards to Claim 14, Li in view of Townsend and Wang teaches the method of Claim 12, and Li in view of Townsend and Wang teaches determining the yaw angle in the virtual attitude angle according to the third control operation value (Wang, Para. 0271 – “the left/right movement of the left visual control stick 601 can control aircraft's yaw”) but Li in view of Townsend and Wang does not teach further comprising, after determining the yaw: performing prediction on a movement trajectory of the unmanned aerial vehicle to obtain a predicted trajectory of the unmanned aerial vehicle; determining a yaw offset angle according to the predicted trajectory; and adjusting the yaw angle in the virtual attitude angle according to the yaw offset angle.
However, Hu teaches further comprising, after determining the yaw: performing prediction on a movement trajectory of the unmanned aerial vehicle to obtain a predicted trajectory of the unmanned aerial vehicle (Hu, Para. 0007, 0017, and 0079 – predicting an output vector at a future time using a predictive control method for trajectory planning to obtain an optimal trajectory); determining a yaw offset angle according to the predicted trajectory; and adjusting the yaw angle in the virtual attitude angle according to the yaw offset angle (Hu, Para. 0057-0072 and 0112 – determining the state of the aerial vehicle at a predicted next waypoint, or trajectory point, and the heading, or yaw, between the current and next waypoints is used to determine a “corresponding angle” and the angle is constrained, or offset, to minimize a “swing angle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method including the above limitations of Li in view of Townsend and Wang to include further comprising, after determining the yaw: performing prediction on a movement trajectory of the unmanned aerial vehicle to obtain a predicted trajectory of the unmanned aerial vehicle; determining a yaw offset angle according to the predicted trajectory; and adjusting the yaw angle in the virtual attitude angle according to the yaw offset angle, as taught by Hu, in order to minimize a swing of an aerial vehicle at a future time in order to maintain stability of the aerial vehicle when imaging a target object or image region.
In regards to Claim 15, Li in view of Townsend, Wang and Hu teaches the method of Claim 14, and Li in view of Townsend, Wang and Hu further teaches wherein determining the yaw offset angle according to the predicted trajectory (Hu, Para. 0057-0072 and 0112 – determining the state of the aerial vehicle at a predicted next waypoint, or trajectory point, and the heading, or yaw, between the current and next waypoints is used to determine a “corresponding angle” and the angle is constrained, or offset, to minimize a “swing angle”) includes: obtaining a preset forward-looking time; determining a target trajectory point in the predicted trajectory according to the preset forward-looking time (Hu, Para. 0057 and 0108-0112 – determining the current waypoint Pi+1 and determining the next waypoint; where the control input is defined at current time k and the state of the aerial vehicle at a future waypoint at time k+1 is determined); and determining the yaw offset angle according to the target trajectory point (Hu, Para. 0057-0072 and 0112 – determining the state of the aerial vehicle at future time k+1, where the heading is a state determined at a waypoint, or trajectory point, and the heading, or yaw, between two waypoints is used to determine a “corresponding angle” and the angle is constrained, or offset, to minimize a “swing angle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Townsend, Wang and Hu to further include wherein determining the yaw offset angle according to the predicted trajectory includes: obtaining a preset forward-looking time; determining a target trajectory point in the predicted trajectory according to the preset forward-looking time; and determining the yaw offset angle according to the target trajectory point., as taught by Hu, in order to minimize a swing of an aerial vehicle at a future time in order to maintain stability of the aerial vehicle when imaging a target object or image region.
Claim(s) 18 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Townsend, and further in view of Choe (Korean Patent No. 10-1848690).
In regards to Claim 18, Li in view of Townsend teaches the method of Claim 17, but Li in view of Townsend does not teach wherein: the unmanned aerial vehicle includes a first arm and a second arm connected to each other through a rotation shaft and the one or more photographing devices include fisheye photographing devices disposed at two ends of the rotation shaft.
However, Choe teaches wherein: the unmanned aerial vehicle includes a first arm and a second arm connected to each other through a rotation shaft (Choe, Fig. 2 and Para. 0012-0017 – where Fig. 2 shows “a plurality of arms (55)” radially provided around a “central axis (32)” which “rotates in place”); and the one or more photographing devices include fisheye photographing devices disposed at two ends of the rotation shaft (Choe, Fig. 2 and Para. 0010-0018 – where the “camera (40)” includes a cameras installed “on the upper and lower parts of the central axis (32)”, as shown on Fig. 2; where the cameras are “360-degree cameras”, or fisheye).
PNG
media_image4.png
494
694
media_image4.png
Greyscale
Choe, Fig. 2
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method including the above limitations of Li in view of Townsend to include wherein: the unmanned aerial vehicle includes a first arm and a second arm connected to each other through a rotation shaft and the photographing devices include fisheye photographing devices disposed at two ends of the rotation shaft, as taught by Choe, in order to utilize a fisheye photographing device to capture a wide view of a target object or image to obtain more image information than a standard lens.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Doron (U.S. Patent No. 10666941) teaches an apparatus which receives video streams each from a respective capture device, receive data from a playback device to determine a region of interest and output one or more upcoming video frames of an encoded panoramic video stream to a communication device.
Day (U.S. Patent No. 10110814) teaches an interface which receive a panoramic video stream from a capture device and present one or more enhanced region of interest video frames to a communication device.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HELEN LI whose telephone number is (703)756-4719. The examiner can normally be reached Monday through Friday, from 9am to 5pm eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at (571) 272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.L./Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights