Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Response to Amendment
The amendment filed 1/27/2026, has been entered. Claims 1-4, 6, 8-17, and 19-21 are pending in the application. Applicant’s amendments to the claims have overcome each and every objection previously set forth in the Final Action mailed 11/28/2025.
Response to Arguments
Applicant's arguments filed 1/27/2026 have been fully considered but they are not persuasive. The applicant has amended the independent claims to better reflect and clarify the intended invention, and argues that the previously cited prior arts Li in view of Townsend do not teach the amended limitations, see applicant’s remarks pages 12-14. While the examiner agrees that the previously cited paragraphs of prior art Li teach an embodiment and method which differs from the current pending limitations of the independent claims, upon review of the pending application and the prior art, other embodiments taught in Li read on the pending limitations under the broadest reasonable interpretation of the pending claims.
The amended limitations of independent claims 1, 17, and 19 teach obtaining a “control operation instruction”, determining a “control mode” of the “control device”, and in response to the control device being in either a “first mode” or a “second mode”, obtaining either a first or second image region corresponding to either a first virtual attitude angle or a second virtual attitude angle & a flight control information, respectively, and outputting the obtained image region (see the current amended limitations).
Prior art Li teaches an embodiment (see paragraphs 0172-0179 and Figs. 16-19) in which “two control modes are set, so that the movement of the aircraft may be operated, and the rotation of the lens may be operated”, where in the embodiment, an “operation gesture” from the user is obtained, and a corresponding “operation command may be generated according to an operation gesture”, where the operation command generated depends on “a current control mode”, which may be either a “camera control mode” or an “aircraft control mode”. In the camera control mode, based on the “operation gesture”, which is a swipe on a screen corresponding to a first virtual angle (for example, “a 30-degree up-right swipe”), a lens of a camera rotates. In the aircraft control mode, the “operation gesture”, or second virtual angle, the aircraft receives an operation command, or flight control information, to move. The captured picture, or image, is outputted, regardless of the mode. Therefore, Li teaches the newly amended limitations. The rejections below contain specific citations of the prior art corresponding to the amended limitations, as well as adjustments to the dependent claims to reflect the amended scope of the invention under broadest reasonable interpretation.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-4 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites the limitation "the target image region" in line 5. Claim 2 is dependent on independent claim 1, which has been amended to teach the limitations “a first image region” and “a second image region”, and no longer teaches “a target image region”. There is insufficient antecedent basis for this limitation in the claim. If claim 2 were amended to recite the amended language of claim 1 and/or substantially similar claim 21, there would be sufficient antecedent basis.
Similarly, claims 3 and 4 both recite the limitation “the target image region”, in lines 5 and 6 respectively. There is insufficient antecedent basis for this limitation in the claims.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 4, 8, 9, 11, 17, 19, and 21 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li, et al. (U.S. Patent Application Pub. No. 2018/0241936).
Regarding Claim 1, Li teaches: A control method (Li, Para. 0010 – “a method for operating an unmanned aerial vehicle (UAV)”) comprising:
obtaining a control operation instruction generated by a control device in communication connection with an aerial vehicle (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on a “UAV-operation interface” of a “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command”);
determining whether the control device is in a first control mode or a second control mode (Li, Para. 0174, 0179 – “monitoring a current control mode”, where the current control mode may be either “a camera control mode”, a first control mode, or “an aircraft control mode”, a second control mode);
in response to the control device being in the first control mode, obtaining a first image region (Li, Para. 0172-0179 – “if the current control mode is a camera control mode”, or the first control mode, “generating, according to the operation gesture, a corresponding operation command for controlling a lens to rotate”; where “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time” corresponding to the “rotating direction of the lens”, or first image region) corresponding to a first virtual attitude angle determined by the control operation instruction (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the camera lens, or first virtual attitude angle, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command” sent to the camera for “controlling a lens to rotate” in a “camera control mode”);
in response to the control device being in the second control mode, obtaining a second image region (Li, Para. 0172-0179 – “if the current control mode is an aircraft control mode”, or the second control mode, “generating, according to the operation gesture, a corresponding operation command for controlling an aircraft to move”; where “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time” corresponding to the “movement of the aircraft”, or second image region) corresponding to a second virtual attitude angle determined by the control operation and flight control information, the flight control information being indicative of a flight motion status of the aerial vehicle (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the UAV, or second virtual attitude angle, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command to be sent to the UAV” for “controlling an aircraft to move” in a “aircraft control mode”; where in the “aircraft control mode”, according to an “operation gesture”, “a corresponding operation command for controlling an aircraft to move”, or flight control information, is generated); and
outputting the first image region or the second image region (Li, Fig. 16 and Para. 0177-0178 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, where Fig. 16 shows a picture, or image region, captured based on the UAV or camera control).
PNG
media_image1.png
446
658
media_image1.png
Greyscale
Li, Fig. 16
In regards to Claim 3, Li teaches the method of Claim 1, and Li further teaches wherein:
the control device includes a remote controller and a terminal device (Li, Para. 0053 and 0174-0179 – where the “mobile terminal”, or control device, controls the “remote UAV” and where the “mobile terminal” includes a “UAV-operation interface” to “control the movement of the aircraft” and an “image transmission interface” for displaying a “picture captured by the UAV”, such that the mobile terminal acts as both the remote controller and the terminal device);
obtaining the control operation instruction generated by the control device includes obtaining the control operation instruction sent by the remote controller (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on the “UAV-operation interface”, or remote controller, of the “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command”); and
sending the target image region to the control device to enable the control device to display the target image region includes sending the target image region to the terminal device, to enable the terminal device to display the target image region (Li, Para. 0177-0179 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, or terminal device).
In regards to Claim 4, Li teaches the method of Claim 1, and Li further teaches wherein:
the control device includes a control area and a display area (Li, Para. 0053 and 0174-0179 – where the “mobile terminal”, or control device, controls the “remote UAV” and where the “mobile terminal” includes a “UAV-operation interface”, or control area, to “control the movement of the aircraft” and an “image transmission interface”, or display area, for displaying a “picture captured by the UAV”);
obtaining the control operation instruction generated by the control device includes obtaining the control operation instruction sent generated based on a user operation in the control area (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on the “UAV-operation interface”, or control area, of the “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command”); and
sending the target image region to the control device to enable the control device to display the target image region includes sending the target image region to the control device, to enable the display area of the control device to display the target image region (Li, Para. 0177-0179 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, or display area).
In regards to Claim 8, Li teaches the method of Claim 1, and Li further teaches wherein: the aerial vehicle is an unmanned aerial vehicle (Li, Para. 0010 – “operating an unmanned aerial vehicle (UAV)”); and the control operation instruction maps to the flight control information (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the UAV, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command to be sent to the UAV” for “controlling an aircraft to move” in a “aircraft control mode”; where in the “aircraft control mode”, according to an “operation gesture”, “a corresponding operation command for controlling an aircraft to move”, or flight control information, is generated, for example “controlling the UAV 104 to move in the left direction”).
In regards to Claim 9, Li teaches the method of Claim 8, and Li further teaches further comprising: determining the flight control information according to the control operation instruction and a preset virtual aircraft control model, the preset virtual aircraft control model including a correspondence relationship between the flight control information and the control operation instruction (Li, Fig. 17 and Para. 0172-0179, 0181-0182 – where the operation gesture, or control operation instruction, corresponds to a “type of movement” and “magnitude” of the UAV, or flight control information; for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command to be sent to the UAV” for “controlling an aircraft to move” in an “upward” and a “left direction”, such that the correlation between a swipe, or instruction, and a generated command is known and modeled; Fig. 17 shows a “diagram of operating an aircraft”).
PNG
media_image2.png
559
658
media_image2.png
Greyscale
Li, Fig. 17
In regards to Claim 11, Li teaches the method of Claim 8, and Li further teaches wherein: the control operation instruction includes a first control operation instruction and a second control operation instruction (Li, Para. 0181-0185 – where an “operation gesture” may include a “single touch point” operation, such as a swipe, or first control operation instruction, and/or a “two touch point” operation, such as a “pinch” or “de-pinch finger gesture”, or second control operation instruction); and the method further comprises:
determining, according to the first control operation instruction, a first flight control information for flying upwards or downwards in a vehicle body coordinate system (Li, Para. 0181-0183 – the “user makes a swipe up, down…” and a “generated operation command correspondingly controls an aircraft to move as follows: the aircraft moves up, the aircraft moves down…” according to a “single touch point” gesture/swipe, or first control operation instruction); and
determining, according to the second control operation instruction, a second flight control information for flying forwards or backwards in the vehicle body coordinate system (Li, Para. 0184-0185 – “the user may use two fingers to perform the touch control operation”, or second control instruction, and the “UAV 104 can be controlled to move forward” or “the UAV 104 is controlled to move backward”, based on the touch control operation).
Regarding Claim 17, Li teaches: A control method for an aerial vehicle (Li, Para. 0010 – “a method for operating an unmanned aerial vehicle (UAV)”) comprising:
obtaining a control operation instruction generated by a control device in communication connection with an aerial vehicle (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on a “UAV-operation interface” of a “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command”);
determining whether the control device is in a first control mode or a second control mode (Li, Para. 0174, 0179 – “monitoring a current control mode”, where the current control mode may be either “a camera control mode”, a first control mode, or “an aircraft control mode”, a second control mode);
in response to the control device being in the first control mode, obtaining a first image region (Li, Para. 0172-0179 – “if the current control mode is a camera control mode”, or the first control mode, “generating, according to the operation gesture, a corresponding operation command for controlling a lens to rotate”; where “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time” corresponding to the “rotating direction of the lens”, or first image region) corresponding to a first virtual attitude angle determined by the control operation instruction (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the camera lens, or first virtual attitude angle, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command” sent to the camera for “controlling a lens to rotate” in a “camera control mode”);
in response to the control device being in the second control mode, obtaining a second image region (Li, Para. 0172-0179 – “if the current control mode is an aircraft control mode”, or the second control mode, “generating, according to the operation gesture, a corresponding operation command for controlling an aircraft to move”; where “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time” corresponding to the “movement of the aircraft”, or second image region) corresponding to a second virtual attitude angle determined by the control operation and flight control information, the flight control information being indicative of a flight motion status of the aerial vehicle (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the UAV, or second virtual attitude angle, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command to be sent to the UAV” for “controlling an aircraft to move” in a “aircraft control mode”; where in the “aircraft control mode”, according to an “operation gesture”, “a corresponding operation command for controlling an aircraft to move”, or flight control information, is generated); and
outputting the first image region or the second image region (Li, Fig. 16 and Para. 0177-0178 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, where Fig. 16 shows a picture, or image region, captured based on the UAV or camera control).
PNG
media_image1.png
446
658
media_image1.png
Greyscale
Li, Fig. 16
Regarding Claim 19, Li teaches: An aerial vehicle (Li, Para. 0010 – “an unmanned aerial vehicle (UAV)”) comprising:
one or more photographing devices configured to capture a panoramic image (Li, Para. 0095 and 0163 – where the UAV captures a “horizontal panoramic landscape image” of a photographing view using a “camera or video camera attached to the UAV”);
a memory storing a computer program (Li, Para. 0054 and 0304 – “a memory” which is “configured to provide a high-speed cache to the operating system and for the computer readable instructions”, where the memory is a storage medium which stores “a computer program instructing relevant hardware”); and
a processor configured to execute the computer program (Li, Para. 0053 – “a processor” configured to “perform a control method for photographing using a UAV”, which executes the “computer readable instructions” stored in the memory) to:
obtain a control operation instruction generated by a control device in communication connection with the aerial vehicle (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on a “UAV-operation interface” of a “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command”);
determine whether the control device is in a first control mode or a second control mode (Li, Para. 0174, 0179 – “monitoring a current control mode”, where the current control mode may be either “a camera control mode”, a first control mode, or “an aircraft control mode”, a second control mode);
in response to the control device being in the first control mode, obtain a first image region (Li, Para. 0172-0179 – “if the current control mode is a camera control mode”, or the first control mode, “generating, according to the operation gesture, a corresponding operation command for controlling a lens to rotate”; where “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time” corresponding to the “rotating direction of the lens”, or first image region) corresponding to a first virtual attitude angle determined by the control operation instruction (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the camera lens, or first virtual attitude angle, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command” sent to the camera for “controlling a lens to rotate” in a “camera control mode”);
in response to the control device being in the second control mode, obtain a second image region (Li, Para. 0172-0179 – “if the current control mode is an aircraft control mode”, or the second control mode, “generating, according to the operation gesture, a corresponding operation command for controlling an aircraft to move”; where “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time” corresponding to the “movement of the aircraft”, or second image region) corresponding to a second virtual attitude angle determined by the control operation and flight control information, the flight control information being indicative of a flight motion status of the aerial vehicle (Li, Para. 0172-0179 – where the operation gesture corresponds to a “type of movement” of the UAV, or second virtual attitude angle, for example, “an up-left swipe on the image transmission interface is decomposed into a left swipe and an up swipe, each trigging a corresponding command to be sent to the UAV” for “controlling an aircraft to move” in a “aircraft control mode”; where in the “aircraft control mode”, according to an “operation gesture”, “a corresponding operation command for controlling an aircraft to move”, or flight control information, is generated); and
output the first image region or the second image region (Li, Fig. 16 and Para. 0177-0178 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, where Fig. 16 shows a picture, or image region, captured based on the UAV or camera control).
PNG
media_image1.png
446
658
media_image1.png
Greyscale
Li, Fig. 16
In regards to Claim 21, Li teaches the method of Claim 1, and Li further teaches wherein the first image region or the second image region is outputted to the control device, to enable the control device to display the first image region or the second image region in real time during a flight process of the aerial vehicle (Li, Para. 0173, 0177-0179 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602” at “the same time when a user operates the UAV”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 10, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Huang, et al. (U.S. Patent Application Pub. No. 2019/0220002).
In regards to Claim 2, Li teaches the method of Claim 1, and Li further teaches wherein:
the control device includes a remote controller (Li, Para. 0053 and 0174 – where the “mobile terminal”, or control device, controls the “remote UAV”)
obtaining the control operation instruction generated by the control device includes obtaining the control operation instruction sent by the remote controller (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on a “UAV-operation interface” of a “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command” to the “remote UAV”); and
sending the target image region to the control device to enable the control device to display the target image region and Para. 0177-0178 – “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, where Fig. 16 shows a picture, or image region, captured based on the UAV or camera control, on the “image transmission interface” of the “mobile terminal”).
PNG
media_image1.png
446
658
media_image1.png
Greyscale
Li, Fig. 16
Li does not teach a head-mounted display device, and sending the target image region to the head-mounted display device, to enable the head-mounted display device to display the target image region.
However, Huang teaches a head-mounted display device (Huang, Para. 0014 – a “terminal”, or control device, including a “head-mounted display (HMD)”), and sending the target image region to the head-mounted display device, to enable the head-mounted display device to display the target image region (Huang, Para. 0014, 0045, 0067-0068, and 0080 – where “terminal” receives “imaging data from a movable object”, such that the movable object sends the imaging data, and displays the image on a “display”; where the display may be a “head-mounted display” which displays “a plurality of FPV stereoscopic images or video captured by an imaging device on a movable object”; where the “movable object” is “an unmanned aerial vehicle (UAV)” and the image includes “a portion of the image (e.g., point, region, and/or object)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include a head-mounted display device, and sending the target image region to the head-mounted display device, to enable the head-mounted display device to display the target image region, as taught by Huang, in order to provide a method of presenting a first person view to a user by a head-mounted display device to provide an immersive experience and to better engage the user.
In regards to Claim 10, Li teaches the method of Claim 9, but Li does not teach wherein the preset virtual aircraft control model includes a preset virtual first-person view aircraft control model.
However, Huang teaches wherein the preset virtual aircraft control model includes a preset virtual first-person view aircraft control model (Huang, Para. 0004-0005 and 0014 – generating a “first person view (FPV) of an environment” from “video data of the environment” generated by a movable object, or aircraft; where a terminal allows a user to “control and navigate the movable object from the first person view (FPV)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include wherein the preset virtual aircraft control model includes a preset virtual first-person view aircraft control model, as taught by Huang, in order to provide a method of presenting a first person view to a user to provide an immersive experience and to better engage the user.
In regards to Claim 20, Li teaches the aerial vehicle of Claim 19, and Li further teaches wherein:
the control device includes a remote controller (Li, Para. 0053 and 0174 – where the “mobile terminal”, or control device, controls the “remote UAV”)
the processor is further configured to execute the computer program (Li, Para. 0053 – “a processor” configured to “perform a control method for photographing using a UAV”, which executes the “computer readable instructions” stored in the memory) to:
obtain the control operation instruction generated by the remote controller (Li, Para. 0010, 0053, and 0172-0179 – where an operator triggers an “operation gesture” on a “UAV-operation interface” of a “mobile terminal”, or control device, and the mobile terminal generates “a corresponding operation command”, or control operation instruction, “according to the operation gesture”; where the mobile terminal includes a “communications apparatus” that is “configured to perform wireless communication with a UAV” to issue the “operation command” to the “remote UAV”); and
send the first image region or the second image region to the “a picture captured by the UAV 104 is displayed on the UAV-operation interface in real time by using the image transmission interface 1602”, where Fig. 16 shows a picture, or image region, captured based on the UAV or camera control modes, on the “image transmission interface” of the “mobile terminal”).
PNG
media_image1.png
446
658
media_image1.png
Greyscale
Li, Fig. 16
Li does not teach a head-mounted display device, and to send the first image region or the second image region to the head-mounted display device, to enable the head-mounted
However, Huang teaches a head-mounted display device (Huang, Para. 0014 – a “terminal”, or control device, including a “head-mounted display (HMD)”), and to send the first image region or the second image region to the head-mounted display device, to enable the head-mountedwhere “terminal” receives “imaging data from a movable object”, such that the movable object sends the imaging data, and displays the image on a “display”; where the display may be a “head-mounted display” which displays “a plurality of FPV stereoscopic images or video captured by an imaging device on a movable object”; where the “movable object” is “an unmanned aerial vehicle (UAV)” and the image includes “a portion of the image (e.g., point, region, and/or object)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the aerial vehicle of Li to include a head-mounted display device, and to send the first image region or the second image region to the head-mounted display device, to enable the head-mounted.
Claim(s) 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Townsend, et al., hereinafter Townsend (U.S. Patent No. 2019/0273837).
In regards to Claim 6, Li teaches the method of Claim 1, and Li does not teach wherein obtaining the first image region corresponding to the first virtual attitude angle includes obtaining the first image region according to a preset field of view and the first virtual attitude angle.
However, Townsend teaches wherein obtaining the first image region corresponding to the first virtual attitude angle includes obtaining the first image region according to a preset field of view and the first virtual attitude angle (Townsend, Fig. 2B and Para. 0069-0074 and 0221 – where based on “an angle of view (e.g., azimuth)”, or first virtual attitude angle, input by a user on the “user interface”, the device generates a “cropped image” within a “panoramic image” that is recorded by an “image capture device”, where the “cropped image” may be include “a point of interest”, or target image region, and wherein the panoramic image is cropped to a “16:9 aspect ratio”, or “narrow field of view” focused on the target, or preset field of view, as shown on Fig. 2B).
PNG
media_image3.png
1004
882
media_image3.png
Greyscale
Townsend, Fig. 2B
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include wherein determining the target image region according to the virtual attitude angle includes determining the target image region according to a preset field of view and the virtual attitude angle, as taught by Townsend, in order to adjust a field of view to include an image region having interesting subject matter.
In regards to Claim 16, Li teaches the method of Claim 1, but Li does not teach further comprising: determining the virtual attitude angle in a virtual camera coordinate system according to the control operation instruction.
However, Townsend teaches further comprising: determining the virtual attitude angle in a virtual camera coordinate system according to the control operation instruction (Townsend, Fig. 18 and Para. 0065, 0069-0074, 0171 – a device which receives “input from a user of the device”, where the user may “specify or modify an angle associated with the output video data”, or virtual attitude angle, and where the device may determine an “angle between the projected vector and a reference vector on the reference plane” which “is called the azimuth” at “a reference location relative to the image capture device”; where “the portion of the panoramic video data displayed on the display 104 (e.g., cropped image, image data, etc.) may be associated with a position (e.g., x and y pixel coordinates) within the panoramic video data”; where Fig. 18 shows a “swipe motion”, or control operation instruction, of the user via three “contact points”, and a change in the angle “from 0 degrees to 90 degrees”).
PNG
media_image4.png
483
849
media_image4.png
Greyscale
Townsend, Fig. 18
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include further comprising: determining the virtual attitude angle in a virtual camera coordinate system according to the control operation instruction, as taught by Townsend, in order to provide a reference frame for the virtual attitude of the camera to better define the position of the camera.
Claim(s) 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Wang, et al. (U.S. Patent Application Pub. No. 2015/0142213).
In regards to Claim 12, Li teaches the method of Claim 11, but Li does not teach wherein: the control operation instruction further includes a third control operation instruction; and the method further comprises: determining a yaw angle in the virtual attitude angle according to the third control operation instruction; and determining a pitch angle in the virtual attitude angle according to the first flight control information and the second flight control information.
However, Wang teaches wherein: the control operation instruction further includes a third control operation instruction (Wang, Para. 0271-0272 – “touch-screen-implemented virtual control sticks 601 and 602 to control the aircraft”, which may control “the aircraft's multiple movement dimensions, such as back/forth, left/right, up/down and the pointing direction (e.g., attitude of the aircraft)”, including “pitch” and “yaw”); and the method further comprises:
determining a yaw angle in the virtual attitude angle according to the third control operation instruction (Wang, Para. 0271 – “the left/right movement of the left visual control stick 601 can control aircraft's yaw”); and
determining a pitch angle in the virtual attitude angle according to the first flight control information and the second flight control information (Wang, Para. 0271-0272 – “the up and down movement of the left virtual control stick 601 can control the payload's pitch”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Li to include wherein: the control operation instruction further includes a third control operation instruction; and the method further comprises: determining a yaw angle in the virtual attitude angle according to the third control operation instruction; and determining a pitch angle in the virtual attitude angle according to the first flight control information and the second flight control information, as taught by Wang, in order to adjust the attitude of the aerial vehicle to better capture a view of a target object or image region.
In regards to Claim 13, Li in view of Wang teaches the method of Claim 12, and Li in view of Wang further teaches wherein the first control operation instruction and the second control operation instruction are speed control quantities (Wang, Para. 0273 – “the speed of the joystick can control the speed of the aircraft”, for example “the faster the movement of the virtual joystick, the faster the payload's speed in the corresponding direction”; where the joysticks may control “back/forth, left/right, up/down and the pointing direction (e.g., attitude of the aircraft)””).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Wang to further include wherein the first control operation instruction and the second control operation instruction are speed control quantities, as taught by Wang, in order to allow the user the adjust the position of the aerial vehicle at different speed in order to achieve steady camera movement or speed up the operation of the aerial vehicle to quickly achieve a certain view.
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Wang, and further in view of Hu, et al. (Chinese Patent Application Pub. No. 110297500).
In regards to Claim 14, Li in view of Wang teaches the method of Claim 12, and Li in view of Wang teaches determining the yaw angle in the virtual attitude angle according to the third control operation instruction (Wang, Para. 0271 – “the left/right movement of the left visual control stick 601 can control aircraft's yaw”) but Li in view of Wang does not teach further comprising, after determining the yaw angle: performing prediction on a movement trajectory of the unmanned aerial vehicle to obtain a predicted trajectory of the unmanned aerial vehicle; determining a yaw offset angle according to the predicted trajectory; and adjusting the yaw angle in the virtual attitude angle according to the yaw offset angle.
However, Hu teaches further comprising, after determining the yaw angle: performing prediction on a movement trajectory of the unmanned aerial vehicle to obtain a predicted trajectory of the unmanned aerial vehicle (Hu, Para. 0007, 0017, and 0079 – predicting an output vector at a future time using a predictive control method for trajectory planning to obtain an optimal trajectory); determining a yaw offset angle according to the predicted trajectory; and adjusting the yaw angle in the virtual attitude angle according to the yaw offset angle (Hu, Para. 0057-0072 and 0112 – determining the state of the aerial vehicle at a predicted next waypoint, or trajectory point, and the heading, or yaw, between the current and next waypoints is used to determine a “corresponding angle” and the angle is constrained, or offset, to minimize a “swing angle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the method including the above limitations of Li in view of Wang to include further comprising, after determining the yaw angle: performing prediction on a movement trajectory of the unmanned aerial vehicle to obtain a predicted trajectory of the unmanned aerial vehicle; determining a yaw offset angle according to the predicted trajectory; and adjusting the yaw angle in the virtual attitude angle according to the yaw offset angle, as taught by Hu, in order to minimize a swing of an aerial vehicle at a future time in order to maintain stability of the aerial vehicle when imaging a target object or image region.
In regards to Claim 15, Li in view of Wang and Hu teaches the method of Claim 14, and Li in view of Wang and Hu further teaches wherein determining the yaw offset angle according to the predicted trajectory (Hu, Para. 0057-0072 and 0112 – determining the state of the aerial vehicle at a predicted next waypoint, or trajectory point, and the heading, or yaw, between the current and next waypoints is used to determine a “corresponding angle” and the angle is constrained, or offset, to minimize a “swing angle”) includes: obtaining a preset forward-looking time; determining a target trajectory point in the predicted trajectory according to the preset forward-looking time (Hu, Para. 0057 and 0108-0112 – determining the current waypoint Pi+1 and determining the next waypoint; where the control input is defined at current time k and the state of the aerial vehicle at a future waypoint at time k+1 is determined); and determining the yaw offset angle according to the target trajectory point (Hu, Para. 0057-0072 and 0112 – determining the state of the aerial vehicle at future time k+1, where the heading is a state determined at a waypoint, or trajectory point, and the heading, or yaw, between two waypoints is used to determine a “corresponding angle” and the angle is constrained, or offset, to minimize a “swing angle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method including the above limitations of Li in view of Wang and Hu to further include wherein determining the yaw offset angle according to the predicted trajectory includes: obtaining a preset forward-looking time; determining a target trajectory point in the predicted trajectory according to the preset forward-looking time; and determining the yaw offset angle according to the target trajectory point., as taught by Hu, in order to minimize a swing of an aerial vehicle at a future time in order to maintain stability of the aerial vehicle when imaging a target object or image region.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Zang (U.S. Patent Application Pub. No. 2017/0023938) teaches systems, methods, and devices related to target tracking by UAVs including receiving target information from a control terminal related to a target to be tracked by an imaging device coupled to the UAV, where the target information may be used by the UAV to automatically track the target so as to maintain predetermined position and/or size of the target within one or more images captured by the imaging device.
Bachrach, et al. (U.S. Patent Application Pub. No. 2016/0327950) teaches methods and systems for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HELEN LI whose telephone number is (703)756-4719. The examiner can normally be reached Monday through Friday, from 9am to 5pm eastern.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at (571) 272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.L./Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665
Prosecution Insights