Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of Claims
The following is a final office action in response to the communication filed on 05/05/2026.
Claims 1-28 are pending and have been examined.
Claims 10, 14, and 23 are amended.
Claims 1-28 are rejected.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 02/12/2026, 05/11/20206, and 05/14/2026 were filed. The submissions are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the Examiner.
Response to Arguments
Applicant’s respectful arguments and corresponding amendments, see pages 12-17, filed on 05/05/2026, have been fully considered and are addressed as follows.
Regarding the Objection to the Specification: The amendment to the Abstract has rendered the objection moot. Accordingly, the objection has been withdrawn.
Regarding the Objections to the Claims: The amendments to claims 10 and 23 have rendered the objections moot. Accordingly, the objections have been withdrawn.
Regarding the Claims Rejections under 35 § USC 102: Applicant’s arguments and corresponding amendments, see pages 12-14 filed on 05/05/2026, have been fully considered and are addressed as follows:
Regarding the argument that, (Page 13, Lines 5-7) “Li does at least not disclose the following feature, which is to be examined in its context: adapting the live-view by digitally scaling the view based on the pitch point progression,” Examiner respectfully disagrees and will particularly address the following arguments.
First, regarding the argument, (Page 13, Lines 12-15) “that cited paragraph [0074] of Li at most discloses adapting the live-view by digitally scaling the view based on a pitch point progression,” and that, (Page 13, Lines “the pinch and zoom gesture mentioned in para. [0074] is not related to a movement (instruction) of the UAV as claimed,” Examiner respectfully disagrees. Li teaches that, (Paragraph [0074], Lines 15-20) “The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture,” and therefore the pinch and zoom gesture is capable of additionally implementing movement control of a movable object such as a UAV.
Second, regarding the argument that, (Page 13, Lines 20-24) “Even the fact that a zoom can be implemented by moving the object such as an UAV (instead of a zoom by digital or mechanical camera means) does not prove otherwise. This is not disclosed as a combination with a digital scaling as claimed but as an alternative,” Examiner respectfully disagrees and interprets Paragraph [0074] as disclosing both a combination or alternative of the UAV movement and digital scaling features. However, given that the Applicant has interpreted Paragraph [0074] as only teaching the features as alternatives, which the Examiner does not concede, Paragraph [0064] has been cited below to provide evidence towards a reasonable interpretation of the pinch and zoom gesture present in Paragraph [0074] being applicable towards a combination of the UAV movement and digital scaling features.
Li teaches, (Paragraph [0064]) “As another example, an adjustment corresponding to a pinch and zoom gesture, e.g., to zoom in or zoom out of the image view, may be achieved by controlling the zoom in/out of the imaging device (e.g., if the imaging device supports the zoom level required), by controlling the movement of the movable object (e.g., so as to get closer to or farther away from a given feature region), or by a combination of zoom in/out of the imaging device and the movement of the movable object. A processor onboard the movable object may make the determination as to which object or combination of objects to adjust. For example, if the imaging device does not support a zoom level required, may be controlled to move instead of or in addition to adjusting the zoom of the imaging device.”
Regarding the argument that Li does not teach, (Page 13, Line 27 & Page 14, Lines 1-6) “receiving and identifying a two-finger pinch touch input with two pitch points, indicative of moving the UAV in the physical environment along a pinch direction, and based thereon instruction the UAV to move, deriving a pitch point progression start state and a pitch point progression end state, and instructing the UAV to move along the pinch direction based on the pitch point progression start state and end state,” Examiner respectfully disagrees and will particularly address the following arguments.
First, regarding the argument that, (Page 14, Lines 10-11) “Li’s zoom gesture is not indicative of moving the UAV,” Examiner respectfully disagrees, as Li teaches, (Paragraph [0074], Lines 15-20) “The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.”
Second, regarding the argument that, “there is no disclosure in Li that a pinch direction is determined let alone that the UAV is moved along the pinch direction,” Examiner respectfully disagrees. Li teaches, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” A person of ordinary skill in the art would understand that a system which monitors a change in position of touch points in a gesture would derive the pitch points start and end states, such as touch points 612 and 614 shown in Fig. 6.
PNG
media_image1.png
359
436
media_image1.png
Greyscale
In order to more explicitly address the UAV movement along a pinch direction, Paragraph [0077] provides additional context behind the perpendicular pitch movement derived by touch points 612 and 614 that a person of ordinary skill in the art would understand. Li teaches, (Paragraph [0077], Lines 18-23) “Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions. Similarly, as shown in image view 708, for a multi-touch gesture, such as a pinch and zoom gesture, a velocity control signal for the movable object in the z direction.” Therefore, the rejections have been maintained.
Regarding the Claims Rejections under 35 § USC 103: Applicant’s arguments and corresponding amendments, see pages 15-16 filed on 05/05/2026, have been fully considered and are addressed as follows:
First, regarding the argument, (Page 15, Lines 20-21) “that because Li is deficient, the combination of references are deficient,” Examiner respectfully disagrees based upon the response to arguments made in the preceding “Regarding the Claims Rejections under 35 § USC 102,” section.
Second, regarding the argument, (Page 15, Lines 23-25) “Regarding claim 14, Li does not disclose the claimed feature of: instructing the UAV to move transverse to the first direction and along the stroke direction, based on the stroke progression start state and end state,” and that,“ Li is not only silent about instruction to move transverse to the view direction – therefore is no referral thereto at all – but also along the stroke direction,” Examiner respectfully disagrees.
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308,” and therefore a swipe gesture is being mapped as an example of a stroke input under broadest reasonable interpretation. Notably, various points in between are determined along the gesture, which are being mapped to stroke progression under broadest reasonable interpretation.
Li additionally teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and therefore the movable object motion may be controlled based upon inputted gestures as discussed in the preceding arguments.
Li additionally teaches, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions,” the velocity control signal of which may effectuate movable object movement. Most notably, the x and y directions being an example of traverse directions to the image view (Z- Direction) a UAV may travel along under broadest reasonable interpretation. Therefore, the rejections have been maintained.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6-7, 9, and 13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Li et al. (US 2021/0018910 A1, hereinafter Li)
Claim 1 Discloses: (Original)
“A computer implemented method for controlling the flight of a UAV in a physical environment,”
Li teaches, (Abstract, Lines 1-2) “A control terminal for controlling an unmanned aerial vehicle (UAV).”
“the method including: continuously generating a view of the physical environment of the UAV based on image data from a camera system of the UAV, continuously displaying the view of the physical environment in a live-view by a touch sensitive display,”
Li teaches, (Paragraph [0051]) “In some embodiments, the movable object 302 or carrier 310 can include one or more sensors. Examples of such sensors may include … camera …The sensors can provide static sensing data (e.g., a photograph) or dynamic sensing data (e.g., a video). The sensors may capture sensing data continuously in real time or at high frequencies,” and that, (Paragraph [0054], Lines 1-3 and 16-20) “The control terminal 304 can be configured to display data received from the movable object 302 via a display … In some embodiments, the images and the tracking indicator are displayed in substantially real -time as the image data and tracking information are received from the movable object and/or as the image data is acquired.”
“receiving and identifying a "two-finger pinch" touch input with two pitch points,
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308. A pinch and zoom gesture may be determined to have multiple touch points and multiple release points.” An example of a pinch and zoom gesture can be seen in Figure 6.
PNG
media_image1.png
359
436
media_image1.png
Greyscale
“indicative of moving the UAV in the physical environment along a pinch direction, and based thereon instructing the UAV to move, characterized by while receiving the "two-finger pinch" touch input determining a pitch point progression of the pitch points … and deriving a pitch point progression start state and a pitch point progression end state, and instructing the UAV to move along the pinch direction based on the pitch point progression start state and end state.”
Li teaches, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” A person of ordinary skill in the art would understand that a system which monitors a change in position of touch points in a gesture would derive the pitch points start and end states, such as touch points 612 and 614 shown in Fig. 6.
Li additionally teaches with regards to the direction the UAV travels during a pinch gesture, (Paragraph [0077], Lines 18-23) “Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions. Similarly, as shown in image view 708, for a multi-touch gesture, such as a pinch and zoom gesture, a velocity control signal for the movable object in the z direction.”
“and adapting the live-view by digitally scaling the view based on the pitch point progression,”
Li teaches, (Paragraph [0074], Lines 15-20) “The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” Therefore, a digital zoom may occur in addition to movement of the UAV. For the purposes of mapping towards the preceding limitation under broadest reasonable interpretation, the Examiner is interpreting a digital zoom as a simultaneous cropping and scaling of the presented image data.
Claim 6 Discloses: (Original)
“The method according to claim 1, including deriving a pitch point progression start state and a pitch point progression end state, and instructing the UAV to move along the pinch direction based on the pitch point progression start state and end state, while receiving the "two-finger pinch" touch input.”
Li teaches, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” A person of ordinary skill in the art would understand that a system which monitors a change in position of touch points in a gesture would derive the pitch points start and end states, such as touch points 612 and 614 shown in Fig. 6.
Li additionally teaches with regards to the direction the UAV travels during a pinch gesture, (Paragraph [0077], Lines 18-23) “Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions. Similarly, as shown in image view 708, for a multi-touch gesture, such as a pinch and zoom gesture, a velocity control signal for the movable object in the z direction.”
Claim 7 Discloses: (Original)
“The method according to claim 1, including instructing the UAV to move along the pinch direction based on the pitch point progression start state and end state, after the "two-finger pinch" touch input has been received.”
Li teaches, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” A person of ordinary skill in the art would understand that a system which monitors a change in position of touch points in a gesture would derive the pitch points start and end states, such as touch points 612 and 614 shown in Fig. 6.
Li additionally teaches with regards to the direction the UAV travels during a pinch gesture, (Paragraph [0077], Lines 18-23) “Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions. Similarly, as shown in image view 708, for a multi-touch gesture, such as a pinch and zoom gesture, a velocity control signal for the movable object in the z direction.”
Claim 9 Discloses: (Original)
“The method according to claim 1, including while the UAV is moving along the pinch direction, the view being a simulated view of the physical environment, and continuously displaying the simulated view by the touch sensitive display.”
Li teaches, (Paragraph [0051]) “In some embodiments, the movable object 302 or carrier 310 can include one or more sensors. Examples of such sensors may include … camera …The sensors can provide static sensing data (e.g., a photograph) or dynamic sensing data (e.g., a video). The sensors may capture sensing data continuously in real time or at high frequencies,” and that, (Paragraph [0054], Lines 1-3 and 16-20) “The control terminal 304 can be configured to display data received from the movable object 302 via a display … In some embodiments, the images and the tracking indicator are displayed in substantially real -time as the image data and tracking information are received from the movable object and/or as the image data is acquired.”
Li additionally teaches, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.”
Claim 13 Discloses: (Original)
“A computer program product comprising machine readable program code stored in a non-transitory medium, which when executed by processing units related to a mobile control device having a touch sensitive display and/or a UAV enables controlling the flight of a UAV including a camera system, according to the method of claim 1.”
Li teaches, (Paragraphs [0108-0110]) “Consequently, features of the present disclosure may be implemented using a processing system (e.g., including one or more processors) … Features of the present disclosure can be implemented in, using, or with the assistance of a computer program product which is a storage medium (media) or computer readable medium (media) having instructions stored thereon/in which can be used to program a processing system to perform any of the features presented herein … Stored on any one of the machine readable medium (media), features of the present disclosure can be incorporated in software and/or firmware for controlling the hardware of a processing system, and for enabling a processing system to interact with other mechanism utilizing the results of the present disclosure. Such software or firmware may include, but is not limited to, application code, device drivers, operating systems and execution environments/containers.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Zhou. (US 2020/0320886 A1, hereinafter Zhou)
Claim 2 Discloses: (Original)
“The method according to claim 1, including deriving a first distance between the two pitch points in the pitch point progression start state and a second distance between the two pitch points in the pitch point progression end state, and instructing the UAV to move along the pinch direction based on the derived first distance and second distance.”
Li does not explicitly teach determining the claimed first and second distances between the two pitch points.
However, Li does teach a pinching gesture where the locations of the two points are continuously identified.
Li teaches, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.”
PNG
media_image1.png
359
436
media_image1.png
Greyscale
Additionally, Fig. 6 portrays a dashed line between touch points 612 and 614, alluding do, but not explicitly teaching receiving the distance between the two points.
Zhou does teach the preceding first and second claimed distances.
Zhou teaches, (TECHNICAL FIELD, Paragraph [0002]) “unmanned aerial vehicle technology and, more particularly, to an information processing device, a flight control instruction method, a program, and a recording medium that instruct flight control for multiple aircrafts,” wherein, (Paragraph [0104]) “In the pinch zoom-out operation or the pinch zoom-in operation, for example, the terminal controller 81 obtains information inputted at two positions of the touch-control panel (TP) at two time points, and calculates a distance between the two inputted positions at the two time points. When the distance at the latter of the two time points is greater than the distance at the former of the two tie points, the terminal controller 81 detects the pinch zoom-in operation. When the distance at the latter of the two time points is smaller than the distance at the former of the two tie points, the terminal controller 81 detects the pinch zoom-out operation,” and that, (Paragraph [0019], Lines 1-4) “FIG. 9 is a schematic diagram showing an example of calculating a distance that each of the plurality of UAVs moves when a user performs the pinch zoom-in operation or the pinch zoom-out operation.”
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the UAV touch display control systems of Li and Zhou to comprise the explicit disclosure of monitoring the distance between two points in a pinching operation to effectuate UAV motion, in order to yield predictable results.
Combining the references would yield the well-known technical effect of monitoring the distance between two pinching points in order to determine whether, for example, the motion is a pinch in/zoom command or a pinch out/ zoom out command. As Zhou describes, (Paragraph [0014]) “When the distance at the latter of the two time points is greater than the distance at the former of the two tie points, the terminal controller 81 detects the pinch zoom-in operation. When the distance at the latter of the two time points is smaller than the distance at the former of the two tie points, the terminal controller 81 detects the pinch zoom-out operation.”
Claims 3-4 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Li.
Claim 3 Discloses: (Original)
“The method according to claim 1, including deriving an end-midpoint between the two pitch points in the pitch point progression end state, and instructing the UAV to move along the pinch direction based on the derived end-midpoint.”
Li teaches, (Paragraph [0066], Lines 13-19) “As used herein, the coordinates of a feature region, or point within a gesture (including a touch point and a release point) may represent a center point of that feature region or gesture point. The center point may be determined geometrically or through other techniques, based on the client device that detects the gesture,” and that, (Paragraph [0074], Lines 22-27) “the pinch and zoom gesture can be combined with a rotate gesture, where touch points 612, 614 are rotated in a substantially circular rotation about a center point between the touch points 612, 614. The view may then be zoomed in and rotated based on the combined gesture,” as well as, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.”
Therefore, the center point is identified after identifying a gesture, the gesture of pinching of which includes a touch point and release point. Due to the nature of having identified a release point, the disclosure of Li must be capable of identifying the center point at the timeframe of release, which under broadest reasonable interpretation is an example of an end-midpoint for the gesture.
Therefore, it would have been obvious to a person of ordinary skill in the art to arrive at a UAV touch screen control system wherein the explicit end-midpoint dictates the UAV movement direction, in order to yield predictable results.
This is due to the fact that an end-midpoint identification is the last identified point of a display unit which continuously measures user pinching inputs. As Li describes, (Paragraph [0078], Lines 12-17) “the difference between the coordinates of the feature region and current reference finger position can be used as real-time feedback, enabling the position and orientation of the movable object and/or image capture device to be adjusted in real time to follow the gesture-based input.”
Claim 4 Discloses: (Original)
“The method according to claim 1, including deriving a start-midpoint between the two pitch points in the pitch point progression start state, and instructing the UAV to move along the pinch direction based on the derived start-midpoint.”
Li teaches, (Paragraph [0066], Lines 13-19) “As used herein, the coordinates of a feature region, or point within a gesture (including a touch point and a release point) may represent a center point of that feature region or gesture point. The center point may be determined geometrically or through other techniques, based on the client device that detects the gesture,” and that, (Paragraph [0074], Lines 22-27) “the pinch and zoom gesture can be combined with a rotate gesture, where touch points 612, 614 are rotated in a substantially circular rotation about a center point between the touch points 612, 614. The view may then be zoomed in and rotated based on the combined gesture,” as well as, (Paragraph [0074], Lines 11-20) “in application 610, a user selects multiple touch points 612, 614 in a pinch and zoom gesture. Feature regions for each touch point 612, 614 can be analyzed and corresponding feature points for each feature region can be identified. The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.”
Therefore, the center point is identified after identifying a gesture, the gesture of pinching of which includes a touch point and release point. Due to the nature of having identified a touch point, the disclosure of Li must be capable of identifying the center point at the timeframe of touching, which under broadest reasonable interpretation is an example of an start-midpoint for the gesture.
Therefore, it would have been obvious to a person of ordinary skill in the art to arrive at a UAV touch screen control system wherein the explicit start-midpoint dictates the UAV movement direction, in order to yield predictable results.
This is due to the fact that a start-midpoint identification is the first identified point of a display unit which continuously measures user pinching inputs. As Li describes, (Paragraph [0078], Lines 12-17) “the difference between the coordinates of the feature region and current reference finger position can be used as real-time feedback, enabling the position and orientation of the movable object and/or image capture device to be adjusted in real time to follow the gesture-based input.”
Claim 11 Discloses: (Original)
“The method according to claim 1, including while the UAV is moving along the pinch direction, the view being a blank view, and continuously displaying the blank view by the touch sensitive display.”
Li does not teach the preceding limitations. However, Li does teach the following.
Li teaches, (Paragraph [0054], Lines 16-21) “In some embodiments, the images and the tracking indicator are displayed in substantially real-time as the image data and tracking information are received from the movable object and/or as the image data is acquired. In other embodiments, the display may be provided after some delay.”
However, it would have been obvious to a person of ordinary skill in the art to continuously display a blank view during UAV movement in order to save power, as the UAV would be effectively loading a new field of view at the UAV’s new location. Turning a display off temporality to save power is a well-known technical relationship. See at least Hosaka (JP-2002247477-A) wherein a, (Title) “Audio/visual device” describes a situation, (DETAILED DESCIPRTION OF THE PREFFERED EMBODIMENTS, Page 12, Bottom 2 Lines of Page) “when the user has no intention to view the image, the display unit 30 consuming a large amount of power is turned off … so that energy can be saved.” A person in the art would recognize that as the new UAV POV loads, there is no reason to keep the display on (which is a blank screen under broadest reasonable interpretation) without any additional claim limitations which apply a specific technical effect or inventive concept that would render the combination non-obvious, as the user does not mandatorily need to view the display during this timeframe.
Therefore, it would have been obvious to a person of ordinary skill in the art to apply a blank/ turned off screen during the imaging delay of Li, which occurs for example during a pinching operation, in order to yield predictable results.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Donahoe et al., (US 2019/0250601 A1, hereinafter Donahoe) further in view of Wan et al. (US 2018/0186472 A1, hereinafter Wan)
Claim 5 Discloses: (Original)
“The method according to claim 1, the camera system including a plurality of cameras arranged peripherally at the UAV, with each camera having a field of view with a fixed orientation in relation to the UAV and directed away from the UAV, one front camera facing forward, one top camera facing up, one bottom camera facing down, and at least one side camera facing sideways, wherein the cameras are arranged such that each field of view overlaps to a predefined degree at least one adjacent field of view, and the camera system provides an all-round view to the physical environment, characterized by including while receiving the "two-finger pinch" touch input determining, based on the pitch point progression, at least one of the plurality of cameras, based on the image data of which the view is continuously generated and displayed.”
Li does not teach the explicit camera system of the preceding claim.
Donahoe does teach the limitations of the preceding claim with regards to front and sideways cameras with overlapping field of views.
Donahoe teaches, (Abstract, Lines 1-2) “A graphical user interface (GUI) for controlling the flight of an aircraft such as an unmanned aerial vehicle (UAV),” wherein, (Paragraph [0108]) “The UI can also be configured to receive multi-touch gestures such as pinch -to-zoom, two-fingered scroll, two-fingered rotate, etc. FIG. 18 shows a screen 1800 of the example GUI in which a user is applying a multi-touch pinch to zoom input. As shown at screen 1800, as the user drags two fingers together or apart (i.e., pinches), the UAV 100 may move towards or away from a point corresponding with a displayed portion of the view.”
PNG
media_image2.png
466
382
media_image2.png
Greyscale
Donahoe teaches, (Paragraph [0030], Lines 1-6) “In the example depicted in FIG. 1, the image capture devices 114 and/or 115 are depicted capturing an object 102 in the physical environment that happens to be a person. In some cases, the image capture devices may be configured to capture images for display to users (e.g., as an aerial video platform),” and that, (Paragraph [0031], Lines 10-15) “the example configuration of UAV 100 depicted in FIG. 1 includes an array of multiple stereoscopic image capture devices 114 placed around a perimeter of the UAV 100 so as to provide stereoscopic image capture up to a full 360 degrees around the UAV 100,” as well as, (Paragraph [0215], Lines 32-35) “the cameras of an image capture device 4134 may be arranged such that at least two cameras are provided with overlapping FOV at multiple angles around the UAV 100.”
Therefore, Donahoe portrays outwardly facing cameras which may serve under broadest reasonable interpretation as cameras facing forward and sideways.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the display which is capable of receiving a user pinching input to control a UAV of Li, with the display which is capable of receiving a user pinching input to control a UAV and has forward facing and sideways facing cameras of Donahoe, in order to yield predictable results.
Combining the references would result in the well-known benefits of having more viewing angles/and or an improved FOV from introducing additional cameras facing different directions into the UAV. As Donahoe describes, (Paragraph [0215], Lines 32-44) “the cameras of an image capture device 4134 may be arranged such that at least two cameras are provided with overlapping FOV at multiple angles around the UAV 100, thereby allowing for stereoscopic (i.e., 3D) image/video capture and depth recovery (e.g., through computer vision algorithms) at multiple angles around UAV 100. For example, UAV 100 may include four sets of two cameras each positioned so as to provide a stereoscopic view at multiple angles around the UAV 100. In some embodiments, a UAV 100 may include some cameras dedicated for image capture of a subject and other cameras dedicated for image capture for visual navigation (e.g., through visual inertial odometry).”
However, Donahoe does not teach the bottom camera facing down and the top camera facing up present in the preceding claim.
Wan does teach the bottom camera facing down and the top camera facing up.
Wan teaches, (Paragraph [0021], Lines 1-9) “In both exemplary UAVs, the 360-degree camera system comprises a top lens 212 coupled to a top camera 213 and mounted to a top portion of the UAV body 210 and a bottom lens 214 coupled to a bottom camera 215 and mounted to a bottom portion of the UAV body 210. In one embodiment, the top lens 212 and bottom lens 214 have angles of view α and β, respectively, and the collective angle of view for these lenses is equal to or greater than 360 degrees.”
Wan additionally teaches, (Paragraph [0022], Lines 1-4) “Once images are captured with the top lens 112 and the bottom lens 114, they are then stitched together to form a composite image showing the entire 360-degree spherical space surrounding the UAV 200.”
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the UAV system capable of generating an all-round view, with the top and bottom cameras of Wan, in order to yield predictable results.
The rationale for combining the references would be to acquire top and bottom viewing angle of the UAV which could be applied to an all-round view. As Wan describes, (Paragraph [0021], Lines 5-9) “In one embodiment, the top lens 212 and bottom lens 214 have angles of view α and β, respectively, and the collective angle of view for these lenses is equal to or greater than 360 degrees.”
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Bachrach et al. (US 2016/0327950 A1, hereinafter Bachrach), further in view of Henry et al. (US 2022/0014675 A1, hereinafter Henry)
Claim 8 Discloses: (Original)
“The method according to claim 1, including digitally scaling the view to a digitally scaled end-view in the pitch point progression end state,”
Li teaches, (Paragraph [0074], Lines 15-20) “The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” Therefore, a digital zoom may occur in addition to movement of the UAV. For the purposes of mapping towards the preceding limitation under broadest reasonable interpretation, the Examiner is interpreting a digital zoom as a simultaneous cropping and scaling of the presented image data.
“and while the UAV is moving along the pinch direction, digitally scaling the view from the digitally scaled end view smoothly to a digitally un-scaled view, and continuously displaying the digitally un-scaled view of the physical environment in the live-view by the touch sensitive display.”
Li does not teach smoothly undoing the digital scaled view during motion of the UAV. However, it would have been obvious to arrive at the claimed invention in light of Bachrach and Henry.
Bachrach teaches, (Paragraph [0093]) “As illustrated in FIGS. 12A-12D, using predefined multitouch gestures, the user 102 may control the FDA 100 and video captured by the FDA 100. For example, as illustrated in FIGS. 12A-12B, in order to :zoom in or out,” the user 102 may apply a “pinch to zoom” gesture using two fingers. This pinch to zoom gesture may cause the FDA 100 to adjust its altitude and/or the image capture device to adjust its focal length (i.e. optically zoom), and/or a digital image processor to adjust the captured image (i.e. digital zoom).”
Bachrach additionally teaches, (Paragraph [0068], Lines 1-14) “One solution to the problem of delay is to compensate for the delay through the use of image manipulation (optical zoom and or digital image processing). For example in the above example of a motion in one direction X, as the user 102 begins to move the PMD 104 forward along direction X, the view 910a provided via the live feed from FDA 100 may initially be adjusted by zooming in (either through optical or digital zoom) to compensate for any delay caused by the limited speed of the FDA 100. The view 910a may further be adjusted to compensate as the FDA 100 catches up and arrives at the location indicated by the by the motion of the PMD 104. Similarly, images may be captured at a wider viewing angle than as displayed via view 910a at PMD 104.”
Therefore, it would have been obvious to a person of ordinary skill before the effective filling date of the claimed invention to apply a methodology of view adjustment as the UAV moves as taught by Bachrach, in order to yield predictable results.
Combining the references would allow a better user viewing experience by compensating the image as the UAV catches up to a desired location. As Bachrach describes, (Paragraph [0068], Lines 9-16) “The view 910a may further be adjusted to compensate as the FDA 100 catches up and arrives at the location indicated by the by the motion of the PMD 104. Similarly, images may be captured at a wider viewing angle than as displayed via view 910a at PMD 104.”
However, Bachrach does not teach an explicit smoothing during the UAV movement.
Henry teaches an imaging system for autonomous drones wherein, (Paragraph [0123]) “In some examples, the user 112 may use the controller 104 to control an image presented on the display 124 associated with the controller 104, such as to zoom in or zoom out a presented image. For instance, the user 112 may use the controller 104 to make one or more inputs to a touchscreen associated with the display 124 that may cause the UAV 102 to aim the longer-focal-length camera 106 at a desired target, change the aim of the longer-focal-length camera 106 dynamically, zoom-in or zoom-out on a desired target (optical and/or digital zoom) and the like. The UAV 102 may dynamically update the composite virtual image 105 as the focal point of the longer-focal-length camera 106 is changed by the user 112 manipulating the controller 104 or, alternatively, as the focal point is changed under autonomous control of the UAV 102,” and that, (Paragraph [0038], Lines 13-18) “Interpolation may be used to enable a smooth transition between the zoomed-in image provided by the longer-focal-length camera 106 and an un-zoomed image presented based on a composite of the images provided by the wider-angle cameras 108.”
Despite the embodiment of importance of Henrry being directed to smoothing a transition between a digitally zoomed in camera and a composite virtual image, Henry is relevant to the Applicant’s disclosure due to teaching the motivation to apply interpolation to a digitally scaled image to arrive at a zoomed in or zoomed out view. The technical effect of interpolation for smoothing would be easily conceived by a person of ordinary skill in the art to a device such as Li, wherein the UAV physically moves to achieve its zoomed in/ zoomed out view during a pinching operation.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the UAV display control system of Li with a step of interpolation to smooth an image while moving in light of Henry, in order to yield predictable results.
Combining the references to arrive at the claimed limitations would yield the benefits of a smooth viewing experience for an operator without any disorientation which could be caused from adjusting the image. As Henry describes, (Paragraph [0049], Lines 3-8) “since 360 degree panoramic images may be disorienting in motion, some implementations herein may perform two smooth interpolations when generating an un-zoomed virtual image, i.e., FOV interpolation and orientation interpolation.”
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Tofte et al. (US 9,927,809 B1, hereinafter Tofte)
Claim 10 Discloses: (Currently Amended)
“The method according to claim 1, including while the UAV is moving along the pinch direction, the view being a frozen view of the physical environment, and continuously displaying the frozen view by the touch sensitive display.”
Li does not teach the preceding limitations. However, Li does teach the following.
Li teaches, (Paragraph [0054], Lines 16-21) “In some embodiments, the images and the tracking indicator are displayed in substantially real-time as the image data and tracking information are received from the movable object and/or as the image data is acquired. In other embodiments, the display may be provided after some delay.”
Tofte does teach the preceding limitations.
Tofte teaches, (Abstract, Lines 1-4) “Various techniques are described to facilitate the control of an unmanned aerial vehicle (UAV). A graphical user interface (GUI) is provided that allows a user to control a UAV using familiar gestures,” wherein, (Page 21, Column 18, Lines 1-15 & Page 22, Column 19, Lines 1-3) “To provide an illustrative example, display 316 may display a window including a live video feed recorded and transmitted by a UAV. A user may perform a “pinch-to-zoom” gesture within the video feed window, thereby causing the displayed live video to initially change the displayed zoom level to zoom in by the amount indicated by the gesture (e.g., 25%, 50%, etc.). In other words, upon the zoom level changing, the live video window is adjusted to indicate a view from the UAVs perspective and to provide feedback to the user regarding the gesture that was performed. That is, regardless of whether the UAV camera actually performs a camera adjustment (e.g., a camera zoom, rotation, etc.) some embodiments include the live video being initially adjusted in any suitable manner such that the gesture performed within the live video window is conveyed to the user. In other embodiments, however, the UAV may move as each gesture is received without the live video being initially adjusted.” The Examiner is interpreting a lack of initial adjustment to be the equivalent of a current frozen view of the physical environment under broadest reasonable interpretation.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the system of Li which can delay the screen being updated upon drone movement with the implementation of a frozen view during UAV movement, as taught by Tofte, in order to yield predictable results.
Doing so would mitigate unnecessary overall power consumption of the system, by only showing a desired end result view after the UAV has moved to a desired location from a pinch motion. Turning a display off temporality to save power is a well-known technical relationship. See at least Hosaka (JP-2002247477-A) wherein a, (Title) “Audio/visual device” describes a situation, (DETAILED DESCRIPTION OF THE PREFFERED EMBODIMENTS, Page 12, Bottom 2 Lines of Page) “when the user has no intention to view the image, the display unit 30 consuming a large amount of power is turned off … so that energy can be saved.”
Claims 12, 14-18, 20-22, and 24-25 are rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Bachrach.
Claim 12 Discloses: (Original)
“The method according to claim 9, including, after a movement of the UAV along the pinch direction,”
Li teaches, (Paragraph [0074], Lines 15-20) “The pinch and zoom gesture can indicate that the movable object and/or camera should be moved to zoom in or zoom out (e.g., using a mechanical or digital zoom on a camera, or by moving the movable object from its current position) based on the change in position of the touch points in the gesture.” Therefore, a digital zoom may occur in addition to movement of the UAV. For the purposes of mapping towards the preceding limitation under broadest reasonable interpretation, the Examiner is interpreting a digital zoom as a simultaneous cropping and scaling of the presented image data.
“… and continuously displaying the … view in the live-view by the touch sensitive display.”
Li teaches, (Paragraph [0054], Lines 1-3 and 16-20) “The control terminal 304 can be configured to display data received from the movable object 302 via a display … In some embodiments, the images and the tracking indicator are displayed in substantially real -time as the image data and tracking information are received from the movable object and/or as the image data is acquired.”
“passing from the view to a digitally un-scaled view of the physical environment … displaying the digitally unscaled view”
Li teach not explicitly teach arriving at a digitally unscaled view after motion of the UAV. However, it would have been obvious to arrive at the claimed invention in light of Bachrach.
Bachrach teaches, (Paragraph [0068]) “One solution to the problem of delay is to compensate for the delay through the use of image manipulation (optical zoom and or digital image processing). For example in the above example of a motion in one direction X, as the user 102 begins to move the PMD 104 forward along direction X, the view 910a provided via the live feed from FDA 100 may initially be adjusted by zooming in (either through optical or digital zoom) to compensate for any delay caused by the limited speed of the FDA 100. The view 910a may further be adjusted to compensate as the FDA 100 catches up and arrives at the location indicated by the by the motion of the PMD 104. Similarly, images may be captured at a wider viewing angle than as displayed via view 910a at PMD 104. The additional captured image data, outside the field of view 910a, may be utilized for digital panning of the field of view. For example, if a user 102 quickly applies a rotational acceleration along axis Z (indicating an input to apply a horizontal pan of the camera at FDA 100), a digital pan may be applied until the FDA 100 catches up with a yaw maneuver about its Z axis. Also, if the camera onboard the FDA 100 is gimbaled to allow for rotation about this axis, such rotation of the camera may be applied as well.” Therefore, the digital scaling is undone once the drone catches up with its intended location.
Therefore, it would have been obvious to a person of ordinary skill before the effective filling date of the claimed invention to apply a methodology of view adjustment as the UAV moves until it reaches its desired destination as taught by Bachrach, in order to yield predictable results.
Combining the references would allow a better user viewing experience by compensating the image as the UAV catches up to a desired location. As Bachrach describes, (Paragraph [0068]) “The view 910a may further be adjusted to compensate as the FDA 100 catches up and arrives at the location indicated by the by the motion of the PMD 104. Similarly, images may be captured at a wider viewing angle than as displayed via view 910a at PMD 104 … For example, if a user 102 quickly applies a rotational acceleration along axis Z (indicating an input to apply a horizontal pan of the camera at FDA 100), a digital pan may be applied until the FDA 100 catches up with a yaw maneuver about its Z axis.”
Claim 14 Discloses: (Currently Amended)
“A computer implemented method for controlling the flight of a UAV having a main view direction in a physical environment,”
Li teaches, (Abstract, Lines 1-2) “A control terminal for controlling an unmanned aerial vehicle (UAV).”
“the method including: continuously generating a view of the physical environment of the UAV based on image data from a camera system of the UAV, continuously displaying the view of the physical environment, in a first view direction, in a live-view by a touch sensitive display,”
Li teaches, (Paragraph [0051]) “In some embodiments, the movable object 302 or carrier 310 can include one or more sensors. Examples of such sensors may include … camera …The sensors can provide static sensing data (e.g., a photograph) or dynamic sensing data (e.g., a video). The sensors may capture sensing data continuously in real time or at high frequencies,” and that, (Paragraph [0054], Lines 1-3 and 16-20) “The control terminal 304 can be configured to display data received from the movable object 302 via a display … In some embodiments, the images and the tracking indicator are displayed in substantially real -time as the image data and tracking information are received from the movable object and/or as the image data is acquired.”
“receiving and identifying a "two-finger stroke" touch input with a stroke progression, indicative of moving the UAV in the physical environment along a stroke direction, and based thereon instructing the UAV to move, characterized by while receiving the "two-finger stroke" touch input determining the stroke progression, deriving a stroke progression start state and a stroke progression end state, and instructing the UAV to move transverse to the first view direction and along the stroke direction
Li does not explicitly teach the preceding two-finger stroke input. However Li does teach a one-finger stroke input.
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308.
Li additionally teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and that, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions.”
Bachrach does explicitly teach a two-finger stroke input.
Bachrach teaches, (Abstract, Lines 1-4) “Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone.”
Bachrach additionally teaches, (Paragraph [0051], Lines 13-14) “a user 102 viewing a live video feed from FDA 100 through the touch display of PMD 104,” and that, (Paragraph [0069], Lines 16-20) “changes in the position/orientation of the PMD 104, applied by user 102, may be translated into changes in the position/orientation of the virtual camera in the virtual 3D space (the real time rendering of the physical environment surrounding the FDA 100.”
Bachrach additionally teaches, (Paragraph [0094], Lines 1-4) “As illustrated in FIG. 12C, the user 102 may drop the FDA 100 to a lower altitude and off to the side of the user by applying a “two finger scroll” gesture, as shown in FIG. 12C.”
Claim 15 Discloses: (Original)
“The method according to claim 14, including determining a first location of the stroke progression start state, and a second location of the stroke progression end state, determining the stroke direction based on the first location and the second location, and instructing the UAV to move transverse to the first view direction and along the stroke direction, based on the first location and the second location.”
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308.
Li additionally teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and that, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions.”
PNG
media_image3.png
425
610
media_image3.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the touch screen UAV control system capable of receiving both stroke and multi point touch inputs with the explicit two-finger stroke input of Bachrach, as a person of ordinary skill would recognize using two fingers as one of a finite number of identified predictable solutions, in order to achieve a reasonable expectation of success.
Combining the references would utilize the well-known methodology of receiving a two-finger stroke input on a touchscreen to move a UAV, such as the once evidenced by Bachrach.
Claim 16 Discloses: (Original)
“The method according to claim 15, including determining a distance between the first location and the second location, and instructing the UAV to move transverse to the first view direction and along the stroke direction, based on the distance.”
Li teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and that, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions.”
Claim 17 Discloses: (Original)
“The method according to claim 16, including determining a flight-velocity based on the distance, and instructing the UAV to move based on the flight-velocity.”
Li teaches, (Paragraph [0086], Lines 1-11) “In some embodiments, the coordinates of the reference point (e.g., a current touch point, release point, or any other touch point along the gesture) and the coordinates of the feature region can be received by an image orientation controller 914. As discussed further below, the image orientation controller can determine a difference between the reference touch point and the feature region and determine control data corresponding to the difference. For example, the control data can include velocity control signals to control the speed of the movable object along one or more axes.”
Claim 18 Discloses: (Original)
“The method according to claim 14, including instructing the UAV to move at a constant distance to an identified object surface.”
Li does not explicitly teach the preceding limitation.
Bachrach does explicitly teach the preceding limitation.
Bachrach teaches, (Paragraph [0076]) “An example embodiment, in which the drawn path user interaction paradigm may be applied, is roof inspection using an FDA 100 and associated PMD 104. In such an example, a view of a roof may be displayed to the user 102 via a display of a PMD 104. The view of the roof corresponds to image capture by an FDA 100 in a hover over the roof. Via the touch screen display of PMD 104, the user 102 may select the roof of the building as the reference surface. Once selected, the user 102 may draw a pattern or provide a gesture indicating a path for the FDA 100 to take over the roof. In response, the FDA 100 will fly the defined path while maintaining a constant height above the roof, even if the roof is not flat relative to the ground.”
Therefore, it would have been obvious to a person of ordinary kill in the art before the effective filling date of the claimed invention to combine the UAV system of Li, with the UAV system Bachrach which can maintain a constant distance to a viewed object, in order to yield predictable results.
Combining the references allows the user to image a subject, even if they are moving, from a safe distance without having a collision. As Bachrach describes, (Paragraph [0083], Lines 16-18) “For example, the FDA may be preset to track the skier down the mountain while orbiting at a constant distance.”
Claim 20 Discloses: (Original)
“The method according to claim 14, including deriving a stroke progression start state and a stroke progression end state, and instructing the UAV to move transverse to the first view direction and along the stroke direction, based on the stroke progression start state and end state,”
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308.
Li additionally teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and that, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions.”
“while receiving the "two-finger stroke" touch input.”
Li does not explicitly teach the preceding two-finger stroke input.
Bachrach does explicitly teach a two-finger stroke input.
Bachrach teaches, (Abstract, Lines 1-4) “Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone.”
Bachrach additionally teaches, (Paragraph [0051], Lines 13-14) “a user 102 viewing a live video feed from FDA 100 through the touch display of PMD 104,” and that, (Paragraph [0069], Lines 16-20) “changes in the position/orientation of the PMD 104, applied by user 102, may be translated into changes in the position/orientation of the virtual camera in the virtual 3D space (the real time rendering of the physical environment surrounding the FDA 100.”
Bachrach additionally teaches, (Paragraph [0094], Lines 1-4) “As illustrated in FIG. 12C, the user 102 may drop the FDA 100 to a lower altitude and off to the side of the user by applying a “two finger scroll” gesture, as shown in FIG. 12C.”
PNG
media_image3.png
425
610
media_image3.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the touch screen UAV control system capable of receiving both stroke and multi point touch inputs with the explicit two-finger stroke input of Bachrach, as person of ordinary skill would recognize using two fingers as one of a finite number of identified predictable solutions, in order to achieve a reasonable expectation of success of applying an input to a touchscreen.
Combining the references would utilize the well-known methodology of receiving a two-finger stroke input on a touchscreen to move a UAV, such as the once evidenced by Bachrach.
Claim 21 Discloses: (Original)
“The method according to claim 14, including instructing the UAV to move transverse to the first view direction and along the stroke direction, based on the stroke progression start state and end state,”
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308.
Li additionally teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and that, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions.”
“after the "two-finger stroke" touch input has been received.”
Li does not explicitly teach the preceding two-finger stroke input.
Bachrach does explicitly teach a two-finger stroke input.
Bachrach teaches, (Abstract, Lines 1-4) “Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone.”
Bachrach additionally teaches, (Paragraph [0051], Lines 13-14) “a user 102 viewing a live video feed from FDA 100 through the touch display of PMD 104,” and that, (Paragraph [0069], Lines 16-20) “changes in the position/orientation of the PMD 104, applied by user 102, may be translated into changes in the position/orientation of the virtual camera in the virtual 3D space (the real time rendering of the physical environment surrounding the FDA 100.”
Bachrach additionally teaches, (Paragraph [0094], Lines 1-4) “As illustrated in FIG. 12C, the user 102 may drop the FDA 100 to a lower altitude and off to the side of the user by applying a “two finger scroll” gesture, as shown in FIG. 12C.”
PNG
media_image3.png
425
610
media_image3.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the touch screen UAV control system capable of receiving both stroke and multi point touch inputs with the explicit two-finger stroke input of Bachrach, as person of ordinary skill would recognize using two fingers as one of a finite number of identified predictable solutions, in order to achieve a reasonable expectation of success of applying an input to a touchscreen.
Combining the references would utilize the well-known methodology of receiving a two-finger stroke input on a touchscreen to move a UAV, such as the once evidenced by Bachrach.
Claim 22 Discloses: (Original)
“The method according to claim 14, including while the UAV is moving transverse to the first view direction and along the stroke direction, the view being a simulated view of the physical environment, and continuously displaying the simulated view by the touch sensitive display.”
Li teaches, (Paragraph [0058], Lines 13-15) “A gesture-based input (e.g., a swipe, pinch, tap, or other gesture) may be received through application 306,” and that, (Paragraph [0059], Lines 1-9) “In some embodiments, when a gesture is detected the system can determine a type of gesture. For example, a swipe gesture may be determined to include a touch point 316 (e.g., the beginning of a gesture), a release point 318 (e.g., the end of a gesture), and various points in between along the gesture 320, within the coordinate system of the image view 308.
Li additionally teaches, (Paragraph [0026], Lines 1-4) “In accordance with various embodiments of the present disclosure, a movable object can be controlled in-flight based on interactions with image data received from the movable object,” and that, (Paragraph [0077], Lines 1-25) “In image view 706, a gesture-based input is received within the image view 706. The initial touch point is represented by point f, and the release point is represented by point t … Accordingly, the greater the distance between the feature region and the release point, the greater the corresponding velocity control signals in the x and y directions.”
Li additionally teaches, (Paragraph [0079], Lines 6-10) “Movable object 802 can include an image capture device 806 that is configured to capture image data in a movable object environment and send the data to control terminal 808. The image data can be displayed in image view 810 of application 804,” and that, (Paragraph [0054], Lines 1-3 and 16-20) “The control terminal 304 can be configured to display data received from the movable object 302 via a display … In some embodiments, the images and the tracking indicator are displayed in substantially real -time as the image data and tracking information are received from the movable object and/or as the image data is acquired.”
Claim 24 Discloses: (Original)
“The method according to claim 14, including while the UAV is moving transverse to the first view direction and along the stroke direction, the view being a blank view, and continuously displaying the blank view by the touch sensitive display.”
Li does not teach the preceding limitations. However, Li does teach the following.
Li teaches, (Paragraph [0054], Lines 16-21) “In some embodiments, the images and the tracking indicator are displayed in substantially real-time as the image data and tracking information are received from the movable object and/or as the image data is acquired. In other embodiments, the display may be provided after some delay.”
Bachrach does not teach the preceding limitations.
However, it would have been obvious to a person of ordinary skill in the art to continuously display a blank view during UAV movement in order to save power, as the UAV would be effectively loading a new field of view at the UAV’s new location. Turning a display off temporality to save power is a well-known technical relationship. See at least Hosaka (JP-2002247477-A) wherein a, (Title) “Audio/visual device” describes a situation, (DETAILED DESCIPRTION OF THE PREFFERED EMBODIMENTS, Page 12, Bottom 2 Lines of Page) “when the user has no intention to view the image, the display unit 30 consuming a large amount of power is turned off … so that energy can be saved.” A person in the art would recognize that as the new UAV POV loads, there is no reason to keep the display on (which is a blank screen under broadest reasonable interpretation) without any additional claim limitations which apply a specific technical effect or inventive concept that would render the combination non-obvious, as the user does not mandatorily need to view the display during this timeframe.
Therefore, it would have been obvious to a person of ordinary skill in the art to apply a blank/ turned off screen during the imaging delay of Li which occurs for example during a pinching operation, in order to yield predictable results.
Claim 25 Discloses: (Original)
“A computer program product comprising machine readable program code stored in a non-transitory machine readable medium, which when executed by processing units related to a mobile control device having a touch sensitive display and/or a UAV enables controlling the flight of a UAV including a camera system, according to the method of claim 14.”
Li teaches, (Paragraphs [0108-0110]) “Consequently, features of the present disclosure may be implemented using a processing system (e.g., including one or more processors) … Features of the present disclosure can be implemented in, using, or with the assistance of a computer program product which is a storage medium (media) or computer readable medium (media) having instructions stored thereon/in which can be used to program a processing system to perform any of the features presented herein … Stored on any one of the machine readable medium (media), features of the present disclosure can be incorporated in software and/or firmware for controlling the hardware of a processing system, and for enabling a processing system to interact with other mechanism utilizing the results of the present disclosure. Such software or firmware may include, but is not limited to, application code, device drivers, operating systems and execution environments/containers.”
Li additionally teaches, (Paragraph [0048]) “As shown in FIG. 3, system 300 may enable a movable object 302 to be controlled based on inputs received through application 306 executing on control terminal 304. Application 306 can include a live image view 308. Although the movable object 302 is depicted as an unmanned aerial vehicle (UAV), this depiction is not intended to be limiting, and any suitable type of movable object can be used, as described herein. One of skill in the art would appreciate that any of the embodiments described herein in the context of aircraft systems can be applied to any suitable movable object. Similarly, although control terminal 304, application 306, and image view 308 are described with respect to a tablet computer or smartphone and touchscreen implementation, any client device capable of displaying or otherwise relaying visual data and receiving gesture-based inputs related to the visual data may be used.”
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Li in view of Bachrach, further in view of Tofte.
Claim 23 Discloses: (Currently Amended)
“The method according to claim 14, including while the UAV is moving transverse to the first view direction and along the stroke direction, the view being a frozen view of the physical environment, and continuously displaying the frozen view by the touch sensitive display.”
Li does not teach the preceding limitations. However, Li does teach the following.
Li teaches, (Paragraph [0054], Lines 16-21) “In some embodiments, the images and the tracking indicator are displayed in substantially real-time as the image data and tracking information are received from the movable object and/or as the image data is acquired. In other embodiments, the display may be provided after some delay.”
Bachrach does not teach the preceding limitations.
Tofte does teach the preceding limitations.
Tofte teaches, (Abstract, Lines 1-4) “Various techniques are described to facilitate the control of an unmanned aerial vehicle (UAV). A graphical user interface (GUI) is provided that allows a user to control a UAV using familiar gestures,” wherein, (Page 21, Column 18, Lines 1-15 & Page 22, Column 19, Lines 1-3) “To provide an illustrative example, display 316 may display a window including a live video feed recorded and transmitted by a UAV. A user may perform a “pinch-to-zoom” gesture within the video feed window, thereby causing the displayed live video to initially change the displayed zoom level to zoom in by the amount indicated by the gesture (e.g., 25%, 50%, etc.). In other words, upon the zoom level changing, the live video window is adjusted to indicate a view from the UAVs perspective and to provide feedback to the user regarding the gesture that was performed. That is, regardless of whether the UAV camera actually performs a camera adjustment (e.g., a camera zoom, rotation, etc.) some embodiments include the live video being initially adjusted in any suitable manner such that the gesture performed within the live video window is conveyed to the user. In other embodiments, however, the UAV may move as each gesture is received without the live video being initially adjusted.” The Examiner is interpreting a lack of initial adjustment to be the equivalent of a current frozen view of the physical environment under broadest reasonable interpretation.
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the system of Li which can delay the screen being updated upon drone movement with the implementation of a frozen view during UAV movement, as taught by Tofte, in order to yield predictable results.
Doing so would mitigate unnecessary overall power consumption of the system, by only showing a desired end result view after the UAV has moved to a desired location from a pinch motion. Turning a display off or not updating it temporality to save power is a well-known technical relationship. See at least Hosaka (JP-2002247477-A) wherein a, (Title) “Audio/visual device” describes a situation, (DETAILED DESCRIPTION OF THE PREFFERED EMBODIMENTS, Page 12, Bottom 2 Lines of Page) “when the user has no intention to view the image, the display unit 30 consuming a large amount of power is turned off … so that energy can be saved.”
Claims 19 and 26-28 are rejected under 35 U.S.C. 103 as being unpatentable over Li view of Bachrach, further in view of Wan.
Claim 19 Discloses: (Original)
“The method according to claim 14, the camera system including a plurality of cameras arranged peripherally at the UAV, with each camera having a field of view with a fixed orientation in relation to the UAV and directed away from the UAV, one front camera facing forward, one top camera facing up, one bottom camera facing down, and at least one side camera facing sideways, wherein the cameras are arranged such that each field of view overlaps to a predefined degree at least one adjacent field of view, and the camera system provides an all-round view to the physical environment, characterized by including while receiving the "two-finger stroke" touch input determining, based on the stroke progression, at least one of the plurality of cameras, based on the image data of which the view is continuously generated and displayed.”
Li does not teach the explicit camera system of the preceding claim.
Bachrach teaches, (Paragraph [0044]) “According to some embodiments, FDA 100 may comprise multiple high resolution image capture devices 602 (“cameras”) with spatial offsets from each other, thereby providing the capability to capture a full view of the world in all directions. The cameras may be arranged such that at least two cameras are capable of viewing every angle, thereby allowing for 3D image/video capture and depth recovery (e.g. through computer vision algorithms) at every angle. According to some embodiments each camera may include a “fisheye” lens. For example, FIG. 6 shows a high-level illustration of the concept of multiple cameras with overlapping fields of view as represented by the dotted lines. FIG. 6 is provided to illustrate the concept, but does not indicate a particular configuration or geometry as a limitation. According to some embodiments, an FDA in accordance with the present teachings may include more or fewer cameras.” Upon viewing Figure 6, Bachrach teach at least the forward and sideways facing camera.
PNG
media_image4.png
563
453
media_image4.png
Greyscale
Bachrach additionally teaches, (Paragraph [0051], Lines 13-14) “a user 102 viewing a live video feed from FDA 100 through the touch display of PMD 104,” and that, (Paragraph [0069], Lines 16-20) “changes in the position/orientation of the PMD 104, applied by user 102, may be translated into changes in the position/orientation of the virtual camera in the virtual 3D space (the real time rendering of the physical environment surrounding the FDA 100.”
Bachrach additionally teaches, (Paragraph [0094], Lines 1-4) “As illustrated in FIG. 12C, the user 102 may drop the FDA 100 to a lower altitude and off to the side of the user by applying a “two finger scroll” gesture, as shown in FIG. 12C.”
PNG
media_image3.png
425
610
media_image3.png
Greyscale
However, Bachrach does not teach the bottom camera facing down and the top camera facing up present in the preceding claim.
Wan does teach the bottom camera facing down and the top camera facing up.
Wan teaches, (Paragraph [0021], Lines 1-9) “In both exemplary UAVs, the 360-degree camera system comprises a top lens 212 coupled to a top camera 213 and mounted to a top portion of the UAV body 210 and a bottom lens 214 coupled to a bottom camera 215 and mounted to a bottom portion of the UAV body 210. In one embodiment, the top lens 212 and bottom lens 214 have angles of view α and β, respectively, and the collective angle of view for these lenses is equal to or greater than 360 degrees.”
Wan additionally teaches, (Paragraph [0022], Lines 1-4) “Once images are captured with the top lens 112 and the bottom lens 114, they are then stitched together to form a composite image showing the entire 360-degree spherical space surrounding the UAV 200.”
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the UAV system capable of generating an generating an all-round view, with the top and bottom cameras of Wan, in order to yield predictable results.
The rationale for combining the references would be to acquire top and bottom viewing angles of the UAV which could be applied to an all-round view. As Wan describes, (Paragraph [0021], Lines 5-9) “In one embodiment, the top lens 212 and bottom lens 214 have angles of view α and β, respectively, and the collective angle of view for these lenses is equal to or greater than 360 degrees.”
Claim 26 Discloses: (Original)
“A system for controlling the flight of a UAV in a physical environment, the system including: a UAV having a camera system providing image data, the camera system including a plurality of cameras arranged peripherally at the UAV, with each camera having a field of view with a fixed orientation in relation to the UAV and directed away from the UAV, one front camera facing forward, one top camera facing up, one bottom camera facing down, and at least one side camera facing sideways, wherein the cameras are arranged such that each field of view overlaps to a predefined degree at least one adjacent field of view, and the camera system provides an all-round view to the physical environment, and a computer program product for performing the method of claim 1.”
Li does not teach the explicit camera system of the preceding claim.
Bachrach does teach the limitations of the preceding claim with regards to front and sideways cameras with overlapping field of views.
Bachrach teaches, (Paragraph [0044]) “According to some embodiments, FDA 100 may comprise multiple high resolution image capture devices 602 (“cameras”) with spatial offsets from each other, thereby providing the capability to capture a full view of the world in all directions. The cameras may be arranged such that at least two cameras are capable of viewing every angle, thereby allowing for 3D image/video capture and depth recovery (e.g. through computer vision algorithms) at every angle. According to some embodiments each camera may include a “fisheye” lens. For example, FIG. 6 shows a high-level illustration of the concept of multiple cameras with overlapping fields of view as represented by the dotted lines. FIG. 6 is provided to illustrate the concept, but does not indicate a particular configuration or geometry as a limitation. According to some embodiments, an FDA in accordance with the present teachings may include more or fewer cameras.” Upon viewing Figure 6, Bachrach teach at least the forward and sideways facing camera.
PNG
media_image4.png
563
453
media_image4.png
Greyscale
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the UAV system capable of generating an all-round view, with the front and sideways cameras of Bachrach, in order to yield predictable results.
Combining the references would improve the desired capability to view a larger FOV. As Bachrach describes, (Paragraph [0044], Lines 5-9) “at least two cameras are capable of viewing every angle, thereby allowing for 3D image/video capture and depth recovery.”
However, Bachrach does not teach the bottom camera facing down and the top camera facing up present in the preceding claim.
Wan does teach the bottom camera facing down and the top camera facing up.
Wan teaches, (Paragraph [0021], Lines 1-9) “In both exemplary UAVs, the 360-degree camera system comprises a top lens 212 coupled to a top camera 213 and mounted to a top portion of the UAV body 210 and a bottom lens 214 coupled to a bottom camera 215 and mounted to a bottom portion of the UAV body 210. In one embodiment, the top lens 212 and bottom lens 214 have angles of view α and β, respectively, and the collective angle of view for these lenses is equal to or greater than 360 degrees.”
Wan additionally teaches, (Paragraph [0022], Lines 1-4) “Once images are captured with the top lens 112 and the bottom lens 114, they are then stitched together to form a composite image showing the entire 360-degree spherical space surrounding the UAV 200.”
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filling date of the claimed invention to combine the UAV system capable of generating an all-round view, with the top and bottom cameras of Wan, in order to yield predictable results.
The rationale for combining the references would be to acquire top and bottom viewing angles of the UAV which could be applied to an all-round view. As Wan describes, (Paragraph [0021], Lines 5-9) “In one embodiment, the top lens 212 and bottom lens 214 have angles of view α and β, respectively, and the collective angle of view for these lenses is equal to or greater than 360 degrees.”
Claim 27 Discloses: (Original)
“The system according to claim 26, further including a mobile control device having a touch sensitive display.”
Li teaches, (Paragraph [0048]) “As shown in FIG. 3, system 300 may enable a movable object 302 to be controlled based on inputs received through application 306 executing on control terminal 304. Application 306 can include a live image view 308. Although the movable object 302 is depicted as an unmanned aerial vehicle (UAV), this depiction is not intended to be limiting, and any suitable type of movable object can be used, as described herein. One of skill in the art would appreciate that any of the embodiments described herein in the context of aircraft systems can be applied to any suitable movable object. Similarly, although control terminal 304, application 306, and image view 308 are described with respect to a tablet computer or smartphone and touchscreen implementation, any client device capable of displaying or otherwise relaying visual data and receiving gesture-based inputs related to the visual data may be used.”
Claim 28 Discloses: (Original)
“The system according to claim 26, wherein the UAV includes: a body extending along an axis from a front end to a back end and having a housing, a first mounting structure attached to the body and extending away from the body in a direction to a left side of the axis, a second mounting structure attached to the body and extending away from the body in a direction to a right side of the axis being an opposite direction to the direction to the left side, four propulsion units, in particular rotor assemblies, two of which are mounted to the first mounting structure and two of which are mounted to the second mounting structure,”
Li does not teach the physical drone structure of the preceding limitations. Bachrach does teach the preceding limitations.
Figure 6 of Bachrach teaches a body extending from front to back in the form of the circular center, a left triangular side comprising two rotor assemblies, and a right triangular side comprising two other rotor assemblies, both of which attach to the center, circular body.
PNG
media_image4.png
563
453
media_image4.png
Greyscale
“a directional distance measuring module including a measuring field of view with a main view direction, within which measuring field of view directions and distances to surfaces in the physical environment are measurable by directionally emitting distance measurement radiation into the field of view, a detector unit for detecting distance measurement radiation reflected from a surface, and a distance measurement radiation source, characterized in that the directional distance measuring module is integrated in the front end of the body inside the housing, and the distance measurement radiation is directionally emittable by the directional distance measuring module through the housing out of the front end of the body.”
PNG
media_image5.png
540
392
media_image5.png
Greyscale
Bachrach teaches, (Paragraph [0036], Lines 1-10) “According to some embodiments, computer vision may include remote sensing technologies such as laser illuminated detection and ranging (LIDAR or LIDAR). For example, an FDA 100 equipped with LIDAR may emit one or more laser beams in a continuous scan up to 360 degrees in all directions around the FDA 100. Light received by the FDA 100 as the laser beams reflect off physical objects in the surrounding physical world may be analyzed to construct a real time 3D computer model of the surrounding physical world.”
Therefore, it would have been obvious to a person of ordinary skill in the art to combine the UAV system of LI with that of UAV system with a directional LIDAR in a convention drone configuration known in the art, in order to yield predictable results.
Combining the reference would yield the benefits of 360 degree three-dimensional imaging in order to provide a wide breadth, detailed image. As Bachrach describes, (Paragraph [0036], Lines 6-17) “Light received by the FDA 100 as the laser beams reflect off physical objects in the surrounding physical world may be analyzed to construct a real time 3D computer model of the surrounding physical world. Such 3D models may be analyzed to identify particular physical objects (e.g. a user 102) in the physical world for tracking. Further, images captured by cameras (e.g., as described earlier) may be combined with the laser constructed 3D models to form textured 3D models that may be further analyzed in real time or near real time for physical object recognition (e.g. by using computer vision algorithms).”
RELEVANT, BUT NOT CITED PRIOR ART
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Yang et al., (US 2016/0139595 A1) teaches, (Abstract) “A mobile terminal and controlling method thereof are disclosed. The present invention includes a communication module configured to transceive data with at least one drone, a display module configured to output a screen for controlling a motion of the drone, a touch interface module configured to receive a random touch drag within the displayed screen, and a controller controlling the communication module, the display module and the touch interface module, the controller generating a 1.sup.st control data for controlling an altitude of the drone if the received touch drag is recognized as a 1.sup.st direction, the controller generating a 2.sup.nd control data for controlling right and left directions of the drone if the received touch drag is recognized as a 2.sup.nd direction.”
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALEXANDER V. GENTILE whose telephone number is (703)756-1501. The examiner can normally be reached Monday - Friday 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito R. Robinson can be reached at (571)270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALEXANDER V GENTILE/Examiner, Art Unit 3664
/KITO R ROBINSON/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights