DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending regarding this application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
A method for tracking multiple targets, comprising:
identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier;
determining a target group including two or more targets from the plurality of targets, the primary target being always in the target group, and determining the target group including:
determining one or more remaining targets in the target group based on a spatial relationship or a relative distance between the primary target and each target of the plurality of targets other than the primary target; and
controlling at least one of the movable object or the carrier to track the target group as a whole.
Step 1: Evaluating whether the claim belongs to one of the statutory categories.
Claim 1 recites at least one step or act. Thus, the claim is directed to a process, which is one of the statutory categories of invention (Step 1: YES)
Step 2A Prong One: Evaluating whether the claim recites a judicial exception (an abstract idea enumerated in 2019 PEG, a law of nature, or a natural phenomenon). If no exception is recited, the claim is eligible. This concludes the eligibility analysis. If the claim recites an exception, go to Step 2A Prong Two.
Claim 1 recites an abstract idea of a mental process. At least steps a-c are recited at a high level of generality such that they could be practically performed by a human (The courts consider a mental process (thinking) that “can be performed in the human mind, or be a human using a pen and paper” to be an abstract idea.). These concepts fall into the “mental processes” group of abstract ideas, which is observation, evaluation and/or judgement. See MPEP, 2106.04 (a) (2) III (C): Performing a mental process on a generic computer. An example of a case identifying a mental process performed on a generic computer as an abstract idea is Voter Verified, Inc. v. Election Systems & Software, LLC, 887 F.3d 1376, 1385, 126 USPQ2d 1498, 1504 (Fed. Cir. 2018). The limitations, interpreted under their broadest reasonable interpretation and in consistence with the specification, cover performance of the limitations in the mind or by generic computer components. See MPEP 2106.04 and the 2019 PEG. (Step 2A Prong One YES)
Step 2A Prong Two: Evaluating whether the claim recites additional elements that integrate the exception into a practical application of the exception. This evaluation is performed by (a) identifying whether there are any additional elements recited in the claim beyond judicial exception, and (b) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into practical application. If the answer to (a) is YES and (b) is NO, go to Step 2B; if the answer to (a) and (b) is YES, go to PATHWAY B, i.e., the claim is not directed to a judicial exception and the claim is eligible.
The 2019 PEG defines the phrase “integration into a practical application” to require an additional element or a combination of additional elements in the claim to apple, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that it is more than a drafting effort designed to monopolize the exception.
Limitations that are indicative of integration into a practical application when recited in a claim with a judicial exception include:
Improvements to the functioning of a computer, or to any other technology or technical field, as discussed in MPEP 2106.05(a);
Applying or using a judicial exception to affect a particular treatment or prophylaxis for disease or medical condition – see Vanda Memo
Applying the judicial exception with, or by use of, a particular machine, as discussed in MPEP 2106.05(b);
Effecting a transformation or reduction of a particular article to a different state or thing, as discussed in MPEP 2106.05©; and
Applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception, as discussed in MPEP 2106.05(e) and the Vanda Memo issued in June 2018.
Limitations that are not indicative of integration into a practical application when recited in a claim with a judicial exception include:
Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea, as discussed in MPEP 2106.05(f);
Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP 2106.05(g); and
Generally linking the use of the judicial exception to a particular technological environment or field of use, as discussed in MPEP 2106.05(h).
[Examiners should note that revised Step 2A excludes consideration of whether claim elements represent well-understood, routine, conventional activity. The question of whether claim elements represent only well-understood, routine, conventional activity is considered at Step 2B and is not a consideration in Step 2A.]
Step d can be regarded as an additional element recited in claim 1. This additional element, i.e., controlling at least one of the movable object or the carrier to track the target group as a whole, does not integrate the exception into a practical application of the exception. Note even if the specification discloses that the invention pertains to an improvement in the technology, the claim must be evaluated to ensure the claim itself reflects the improvement in technology. It is also important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. Therefore, the additional elements do not recite an improvement. (Step 2A Prong Two NO)
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
Step d can be regarded as an additional element recited in claim 1. This additional element, i.e., controlling at least one of the movable object or the carrier to track the target group as a whole is considered insignificant extra-solution activities which amounts to automating a manual human activity.
In the instant case, all the recited functional limitations steps a-c can be performed by a photographer (organizing human activity/ Mere automation of manual processes).
Step d recites the features of “controlling at least one of the movable object or the carrier to track the target group as a whole”. Controlling a UAV/carrier to track objects is a well-understood, routine, conventional activity in the field. Using the broadest reasonable interpretation of the claim, the additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the abstract idea itself. See MPEP 2106.05. (Step 2B: NO) The claim is not eligible.
Claim 2 recites
“wherein determining the target group further includes selecting the two or more targets from the plurality of targets based on a state associated with each of the two or more targets.”
Step a is directed to the abstract idea of mental processes which are mere automation of manual processes. The similar examination analysis as applied to claim 1 is applied to step a of claim 2. No additional elements are recited. Accordingly, claim 2 does not have eligible subject matter.
Claim 3 recites “wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling, based on a change of a bounding box of the target group, the at least one of the movable object or the carrier to track the target group as a whole”. This contains steps that are directed to the abstract idea of mental processes which are mere automation of manual processes and the additional element of controlling the movable object/carrier to track objects. The similar examination analysis as applied to claim 1 is applied to the steps of claim 3. Accordingly, claim 3 does not have eligible subject matter.
Claim 4 recites “wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling, based on a change of a bounding box of the target group, the at least one of the movable object or the carrier to track the target group as a whole”. This contains steps that are directed to the abstract idea of mental processes which are mere automation of manual processes and the additional element of controlling the movable object/carrier to track objects. The similar examination analysis as applied to claim 1 is applied to the steps of claim 4. Accordingly, claim 4 does not have eligible subject matter.
Claim 5 recites “detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets”. This contains steps that are directed to the abstract idea of mental processes which are mere automation of manual processes. The similar examination analysis as applied to claim 1 is applied to the steps of claim 5. No additional elements are recited. Accordingly, claim 5 does not have eligible subject matter.
Claim 6 recites “determining a group state of the target group based on states of the two or more targets”. This contains steps that are directed to the abstract idea of mental processes which are mere automation of manual processes. The similar examination analysis as applied to claim 1 is applied to the steps of claim 6. No additional elements are recited. Accordingly, claim 6 does not have eligible subject matter.
Claim 7 recites “wherein the group state of the target group or the states of the two or more targets includes at least one of a position, a size, a velocity, or an orientation”. This contains steps that are directed to the abstract idea of mental processes. The similar examination analysis as applied to claim 1 is applied to the steps of claim 7. No additional elements are recited. Accordingly, claim 7 does not have eligible subject matter.
Claim 8 recites “wherein the group state of the target group is determined as a weighted average of the states of the two or more targets in the target group”. This contains steps that are directed to the abstract idea of mental processes and mathematical calculations which are mere automation of manual processes. The similar examination analysis as applied to claim 1 is applied to the steps of claim 8. No additional elements are recited. Accordingly, claim 8 does not have eligible subject matter.
Claim 9 recites “controlling the movable object and the carrier to track the target group as a whole”. This contains steps that are directed to the abstract idea of mental processes which are mere automation of manual processes and the additional element of controlling the movable object/carrier to track objects. The similar examination analysis as applied to claim 1 is applied to the steps of claim 9. Accordingly, claim 9 does not have eligible subject matter.
Claim 10 recites “updating the target group and controlling the at least one of the movable object or the carrier to track the updated target group”. This contains steps that are directed to the abstract idea of mental processes which are mere automation of manual processes and the additional element of controlling the movable object/carrier to track objects. The similar examination analysis as applied to claim 1 is applied to the steps of claim 10. Accordingly, claim 10 does not have eligible subject matter.
Independent claim 11 is directed to a machine, which is a statutory category of invention. Similar analysis is applicable as applied above to the method of claim 1. Claim 11 further recites other additional elements of “a memory that stores one or more computer-executable instructions; and one or more processors configured to access the memory and execute the computer- executable instructions to perform a method”. These elements are recited at a high level of generality such that they amount to no more than mere generic computer system elements. The remainder of the claim is identical to claim 1. The similar examination analysis as applied to claim 1 is applied to the remaining steps of claim 11. Accordingly claim 11 does not have eligible subject matter.
Similar analysis as applied to claims 2, 3, 4, 5, 6, 7, 9, and 10 can be applied to corresponding claims 12, 13, 14, 15, 16, 17, 18, and 19, respectively.
Independent claim 20 is directed to a machine, which is a statutory category of invention. Similar analysis is applicable as applied above to the method of claim 1. Claim 20 further recites other additional elements of “a non-transitory computer readable storage medium, wherein a computer program is stored thereon, and the computer program is executed by a processor to cause the processor to implement a method”. These elements are recited at a high level of generality such that they amount to no more than mere generic computer system elements. The remainder of the claim is identical to claim 1. The similar examination analysis as applied to claim 1 is applied to the remaining steps of claim 20. Accordingly claim 20 does not have eligible subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6, 7, 9, 10-14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (CN 106161953 A, see attached English translation for citations), hereinafter Wang, in view of Zang (U.S. Publication No. 2016/0031559 A1).
Regarding claim 1, Wang teaches a method for tracking multiple targets (Wang teaches a method for tracking multiple candidate targets in para. [0080]-[0092]), comprising:
identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object (Wang teaches “the selected target is selected by the user based on the multiple candidate targets received by the mobile terminal” in para. [0073]. Here, the selected target is interpreted as equivalent to the claimed primary target. Here, the targets are identified within a plurality of images captured by an imaging device attached to a drone/UAV as shown in para. [0010]-[0011] and [0087]-[0092]);
determining a target group including two or more targets from the plurality of targets, the primary target being always in the target group (Wang teaches “after receiving the selected target fed back by the mobile terminal, acquiring an alternative target whose distance from the selected target is within a preset range is an associated target” in para. [0081]. Here, the selected target and alternative target are interpreted as equivalent to the claimed target group), and determining the target group including:
determining one or more remaining targets in the target group based on a spatial relationship or a relative distance between the primary target and each target of the plurality of targets other than the primary target (Wang teaches “the method for obtaining the associated target is to select an candidate target whose distance from the selected target is within a preset range, wherein the distance may include a center point coordinate distance or an edge coordinate distance” in para. [0082]); and
controlling at least one of the movable object or the carrier to track the target group as a whole (Wang teaches “after the tracking device generates a plurality of candidate targets, the plurality of candidate targets are merged into a single target as a tracking target” in para. [0051]. See also that “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein “the tracking device of the drone generates a new tracking target according to the selected target fed back by the mobile terminal, and the flight controller controls the drone to track and capture the tracking target” as shown in para. [0088]).
Wang fails to teach identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier (emphasis added).
However, Zang teaches identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier (Zang teaches “the imaging device is coupled to the UAV via a carrier configured to permit the imaging device to move relative to the UAV” in para. [0022], wherein “the user may provide target type information by selecting one or more targets from within one or more images” as shown in para. [0205]. Here, the selected target is interpreted as equivalent to the claimed primary target).
Wang and Zang are both considered to be analogous to the claimed invention because they are in the same field of tracking targets using a UAV. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Wang to incorporate the teachings of Zang and include “identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier”. The motivation for doing so would have been “to permit the imaging device to rotate around at least two axes relative to the UAV”, as suggested by Zang in para. [0023]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Wang with Zang to obtain the invention specified in claim 1.
Regarding claim 2, Wang and Zang teach the method of claim 1,
wherein determining the target group further includes selecting the two or more targets from the plurality of targets based on a state associated with each of the two or more targets (Wang teaches selecting targets and alternate targets from a group of candidate targets in para. [0079], wherein the selected targets are selected based on finding targets within a preset distance from a primary selected target as shown in para. [0081]-[0082]. The position of the targets is interpreted as equivalent to the claimed state associated with the targets).
Regarding claim 3, Wang and Zang teach the method of claim 1,
wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling, based on a change of a bounding box of the target group, the at least one of the movable object or the carrier to track the target group as a whole (Wang teaches determining a fusion frame based on the edge coordinates of the multiple targets within the target group that represents the tracking target, as shown in para. [0067]-[0068]. This fusion frame is interpreted as equivalent to the claimed bounding box of the target group. Wang furthermore teaches “after the tracking device of the drone generates an candidate target, all the candidate targets are automatically merged into a tracking target, and the flight controller controls the drone body to track the tracking target” as shown in para. [0069]).
Regarding claim 4, Wang and Zang teach the method of claim 1,
wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling, based on a group state of the target group, the at least one of the movable object or the carrier to track the target group as a whole (Wang teaches that, “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein the method further involves “sending a tracking instruction to the flight controller, so that the flight controller controls the drone to track and capture the tracking target” as shown in para. [0087]).
Regarding claim 6, Wang and Zang teach the method of claim 4, further comprising determining a group state of the target group based on states of the two or more targets (Wang teaches “the tracking shooting method provided in this embodiment preferably combines multiple candidate targets that need to be tracked and shot into a single tracking target, which can automatically place multiple targets in the center position or near center position of the screen, thereby achieving better implementation” in para. [0069], wherein fusion information of the plurality of targets is used to generate the tracking target as shown in para. [0076]-[0078]).
Regarding claim 7, Wang and Zang teach the method of claim 6,
wherein the group state of the target group or the states of the two or more targets includes at least one of
a position (Wang teaches “the tracking shooting method provided in this embodiment preferably combines multiple candidate targets that need to be tracked and shot into a single tracking target, which can automatically place multiple targets in the center position or near center position of the screen” in para. [0069]. Here, tracking the multiple targets as a single target involves identifying a position of the target group such that the target group is centered within the frame),
a size,
a velocity, or
an orientation.
Regarding claim 9, Wang and Zang teach the method of claim 1,
wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling the movable object and the carrier to track the target group as a whole (Wang teaches that, “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein the method further involves “sending a tracking instruction to the flight controller, so that the flight controller controls the drone to track and capture the tracking target” as shown in para. [0087]. Zang additionally teaches controlling the movable object and the carrier to track a target object as shown in claim 1 and para. [0114]. Here, the teachings of Wang can be combined with Zang to teach the above subject matter of claim 9. Similar motivations as applied to claim 1 can be applied here to claim 9).
Regarding claim 10, Wang and Zang teach the method of claim 1,
further comprising updating the target group and controlling the at least one of the movable object or the carrier to track the updated target group (Wang teaches a process of updating the selected target and generated a new tracking target according to the updated selected target as shown in para. [0074]-[0078] wherein the drone is configured to be controlled to track the new tracking target as shown in para. [0088]-[0090]).
Regarding claim 11, Wang teaches a tracking system (Wang, see the system as taught in para. [0118]-[0123] and FIG. 10), comprising:
a memory that stores one or more computer-executable instructions (Wang teaches “the UAV includes a memory 1001 and a processor 1002. The memory 1001 is electrically connected to the processor 1002” in para. [0117]); and
one or more processors configured to access the memory and execute the computer- executable instructions (Wang teaches “the UAV includes a memory 1001 and a processor 1002. The memory 1001 is electrically connected to the processor 1002” in para. [0117]) to perform a method including:
identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object (Wang teaches “the selected target is selected by the user based on the multiple candidate targets received by the mobile terminal” in para. [0073]. Here, the selected target is interpreted as equivalent to the claimed primary target. Here, the targets are identified within a plurality of images captured by an imaging device attached to a drone/UAV as shown in para. [0010]-[0011] and [0087]-[0092]);
determining a target group including two or more targets from the plurality of targets, the primary target being always in the target group (Wang teaches “after receiving the selected target fed back by the mobile terminal, acquiring an alternative target whose distance from the selected target is within a preset range is an associated target” in para. [0081]. Here, the selected target and alternative target are interpreted as equivalent to the claimed target group), and determining the target group including:
determining one or more remaining targets in the target group based on a spatial relationship or a relative distance between the primary target and each target of the plurality of targets other than the primary target (Wang teaches “the method for obtaining the associated target is to select an candidate target whose distance from the selected target is within a preset range, wherein the distance may include a center point coordinate distance or an edge coordinate distance” in para. [0082]); and
controlling at least one of the movable object or the carrier to track the target group as a whole (Wang teaches “after the tracking device generates a plurality of candidate targets, the plurality of candidate targets are merged into a single target as a tracking target” in para. [0051]. See also that “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein “the tracking device of the drone generates a new tracking target according to the selected target fed back by the mobile terminal, and the flight controller controls the drone to track and capture the tracking target” as shown in para. [0088]).
Wang fails to teach identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier (emphasis added).
However, Zang teaches identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier (Zang teaches “the imaging device is coupled to the UAV via a carrier configured to permit the imaging device to move relative to the UAV” in para. [0022], wherein “the user may provide target type information by selecting one or more targets from within one or more images” as shown in para. [0205]. Here, the selected target is interpreted as equivalent to the claimed primary target).
Wang and Zang are both considered to be analogous to the claimed invention because they are in the same field of tracking targets using a UAV. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Wang to incorporate the teachings of Zang and include “identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier”. The motivation for doing so would have been “to permit the imaging device to rotate around at least two axes relative to the UAV”, as suggested by Zang in para. [0023]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Wang with Zang to obtain the invention specified in claim 11.
Regarding claim 12, Wang and Zang teach the system of claim 11,
wherein determining the target group further includes selecting the two or more targets from the plurality of targets based on a state associated with each of the two or more targets (Wang teaches selecting targets and alternate targets from a group of candidate targets in para. [0079], wherein the selected targets are selected based on finding targets within a preset distance from a primary selected target as shown in para. [0081]-[0082]. The position of the targets is interpreted as equivalent to the claimed state associated with the targets).
Regarding claim 13, Wang and Zang teach the system of claim 11,
wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling, based on a change of a bounding box of the target group, the at least one of the movable object or the carrier to track the target group as a whole (Wang teaches determining a fusion frame based on the edge coordinates of the multiple targets within the target group that represents the tracking target, as shown in para. [0067]-[0068]. This fusion frame is interpreted as equivalent to the claimed bounding box of the target group. Wang furthermore teaches “after the tracking device of the drone generates an candidate target, all the candidate targets are automatically merged into a tracking target, and the flight controller controls the drone body to track the tracking target” as shown in para. [0069]).
Regarding claim 14, Wang and Zang teach the system of claim 11,
wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling, based on a group state of the target group, the at least one of the movable object or the carrier to track the target group as a whole (Wang teaches that, “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein the method further involves “sending a tracking instruction to the flight controller, so that the flight controller controls the drone to track and capture the tracking target” as shown in para. [0087]).
Regarding claim 16, Wang and Zang teach the system of claim 14,
wherein the method further includes determining a group state of the target group based on states of the two or more targets (Wang teaches “the tracking shooting method provided in this embodiment preferably combines multiple candidate targets that need to be tracked and shot into a single tracking target, which can automatically place multiple targets in the center position or near center position of the screen, thereby achieving better implementation” in para. [0069], wherein fusion information of the plurality of targets is used to generate the tracking target as shown in para. [0076]-[0078]).
Regarding claim 17, Wang and Zang teach the system of claim 16,
wherein the group state of the target group or the states of the two or more targets includes at least one of
a position (Wang teaches “the tracking shooting method provided in this embodiment preferably combines multiple candidate targets that need to be tracked and shot into a single tracking target, which can automatically place multiple targets in the center position or near center position of the screen” in para. [0069]. Here, tracking the multiple targets as a single target involves identifying a position of the target group such that the target group is centered within the frame),
a size,
a velocity, or
an orientation.
Regarding claim 18, Wang and Zang teach the system of claim 11,
wherein controlling the at least one of the movable object or the carrier to track the target group as a whole includes controlling the movable object and the carrier to track the target group as a whole (Wang teaches that, “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein the method further involves “sending a tracking instruction to the flight controller, so that the flight controller controls the drone to track and capture the tracking target” as shown in para. [0087]. Zang additionally teaches controlling the movable object and the carrier to track a target object as shown in claim 1 and para. [0114]. Here, the teachings of Wang can be combined with Zang to teach the above subject matter of claim 9. Similar motivations as applied to claim 1 can be applied here to claim 9).
Regarding claim 19, Wang and Zang teach the system of claim 11,
wherein the method further includes updating the target group and controlling the at least one of the movable object or the carrier to track the updated target group (Wang teaches a process of updating the selected target and generated a new tracking target according to the updated selected target as shown in para. [0074]-[0078] wherein the drone is configured to be controlled to track the new tracking target as shown in para. [0088]-[0090]).
Regarding claim 20, Wang teaches a non-transitory computer readable storage medium, wherein a computer program is stored thereon (Wang teaches “the memory 202 can be, but not limited to, a random access memory (RAM), a read only memory (ROM), and a programmable read-only memory (PROM). Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (EEPROM), and the like” in para. [0034]. Here, at least ROM is an example of non-transitory CRM), and the computer program is executed by a processor to cause the processor to implement a method for tracking multiple targets (Wang teaches “the memory 202 is used to store a program, and the processor 204 executes the program after receiving the execution instruction” in para. [0034]) comprising:
identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object (Wang teaches “the selected target is selected by the user based on the multiple candidate targets received by the mobile terminal” in para. [0073]. Here, the selected target is interpreted as equivalent to the claimed primary target. Here, the targets are identified within a plurality of images captured by an imaging device attached to a drone/UAV as shown in para. [0010]-[0011] and [0087]-[0092]);
determining a target group including two or more targets from the plurality of targets, the primary target being always in the target group (Wang teaches “after receiving the selected target fed back by the mobile terminal, acquiring an alternative target whose distance from the selected target is within a preset range is an associated target” in para. [0081]. Here, the selected target and alternative target are interpreted as equivalent to the claimed target group), and determining the target group including:
determining one or more remaining targets in the target group based on a spatial relationship or a relative distance between the primary target and each target of the plurality of targets other than the primary target (Wang teaches “the method for obtaining the associated target is to select a candidate target whose distance from the selected target is within a preset range, wherein the distance may include a center point coordinate distance or an edge coordinate distance” in para. [0082]); and
controlling at least one of the movable object or the carrier to track the target group as a whole (Wang teaches “after the tracking device generates a plurality of candidate targets, the plurality of candidate targets are merged into a single target as a tracking target” in para. [0051]. See also that “after the selected target and the associated target are obtained according to the foregoing steps, the selected target and the associated target are merged into a new tracking target for tracking and shooting” as shown in para. [0086], wherein “the tracking device of the drone generates a new tracking target according to the selected target fed back by the mobile terminal, and the flight controller controls the drone to track and capture the tracking target” as shown in para. [0088]).
Wang fails to teach identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier (emphasis added).
However, Zang teaches identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier (Zang teaches “the imaging device is coupled to the UAV via a carrier configured to permit the imaging device to move relative to the UAV” in para. [0022], wherein “the user may provide target type information by selecting one or more targets from within one or more images” as shown in para. [0205]. Here, the selected target is interpreted as equivalent to the claimed primary target).
Wang and Zang are both considered to be analogous to the claimed invention because they are in the same field of tracking targets using a UAV. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Wang to incorporate the teachings of Zang and include “identifying a primary target from a plurality of targets based on a plurality of images obtained from an imaging device carried by a movable object via a carrier”. The motivation for doing so would have been “to permit the imaging device to rotate around at least two axes relative to the UAV”, as suggested by Zang in para. [0023]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Wang with Zang to obtain the invention specified in claim 20.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (CN 106161953 A, see attached English translation for citations), hereinafter Wang, in view of Zang (U.S. Publication No. 2016/0031559 A1) and Kim et al. (U.S. Publication No. 2018/0232891 A1), hereinafter Kim.
Regarding claim 5, Wang and Zang teach the method of claim 4.
While Wang teaches determining a group state (fusion frame) of the target group based on edge sampling coordinate data for each of the two or more targets in para. [0065]-[0067], Wang and Zang fail to teach detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets.
However, Kim teaches detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets (Kim teaches “the multi-object tracker 150 initializes multi-object tracking by allocating a bounding box (a tracking area) to each of the objects and then performs multi-object tracking” in para. [0068]. Additionally, Wang teaches combining edge information to generate fusion frame information that represents the group state of a target group (see above). As such, Wang’s teaching of determining the group state of the target group based on edge sampling information associated with each of the targets in view of Kim’s teaching of the edge information specifically being a bounding box can be combined to teach the above subject matter).
Wang, Zang, and Kim are all considered to be analogous to the claimed invention because they are in the same field of tracking targets using a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Wang (as modified by Zang) to incorporate the teachings of Kim and include “detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets”. The motivation for doing so would have been to improve the center location error and increase the success rate for multi-object tracking, as suggested by Kim in para. [0121]-[0122]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Wang and Zang with Kim to obtain the invention specified in claim 5.
Regarding claim 15, Wang and Zang teach the system of claim 14.
While Wang teaches determining a group state (fusion frame) of the target group based on edge sampling coordinate data for each of the two or more targets in para. [0065]-[0067], Wang and Zang fail to teach detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets.
However, Kim teaches detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets (Kim teaches “the multi-object tracker 150 initializes multi-object tracking by allocating a bounding box (a tracking area) to each of the objects and then performs multi-object tracking” in para. [0068]. Additionally, Wang teaches combining edge information to generate fusion frame information that represents the group state of a target group (see above). As such, Wang’s teaching of determining the group state of the target group based on edge sampling information associated with each of the targets in view of Kim’s teaching of the edge information specifically being a bounding box can be combined to teach the above subject matter).
Wang, Zang, and Kim are all considered to be analogous to the claimed invention because they are in the same field of tracking targets using a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Wang (as modified by Zang) to incorporate the teachings of Kim and include “detecting bounding box information of a bounding box associated with each of the two or more targets; and determining the group state of the target group based on the bounding box information associated with each of the two or more targets”. The motivation for doing so would have been to improve the center location error and increase the success rate for multi-object tracking, as suggested by Kim in para. [0121]-[0122]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Wang and Zang with Kim to obtain the invention specified in claim 15.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (CN 106161953 A, see attached English translation for citations), hereinafter Wang, in view of Zang (U.S. Publication No. 2016/0031559 A1) and Sasaki et al. (U.S. Publication No. 2014/0245367 A1), hereinafter Sasaki.
Regarding claim 8, Wang and Zang teach the method of claim 6.
Wang and Zang fail to teach wherein the group state of the target group is determined as a weighted average of the states of the two or more targets in the target group.
However, Sasaki teaches wherein the group state of the target group is determined as a weighted average of the states of the two or more targets in the target group (Sasaki teaches “if the viewed subjects of interest are prioritized, a weighted average value of the plurality of pieces of positional information may be used, rather than the average value of the plurality of pieces of viewed subject position information” in para. [0215]. Here, the positional information of the multitude of subjects of interest is interpreted as equivalent to the claimed “state”).
Wang, Zang, and Sasaki are all considered to be analogous to the claimed invention because they are in the same field of tracking targets using camera system. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Wang (in view of Zang) to incorporate the teachings of Sasaki and include “wherein the group state of the target group is determined as a weighted average of the states of the two or more targets in the target group”. The motivation for doing so would have been that “the user can enjoy a video focusing on the plurality of viewed subjects”, as suggested by Sasaki in para. [0215]. Therefore, it would have been obvious to one of ordinary skill at the time the invention was filed to combine Wang and Zang with Sasaki to obtain the invention specified in claim 8.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sheng et al. (U.S. Publication No. 2013/0272570 A1) teaches a reduction of computational complexity associated with object tracking, and/or a robust and efficient integration of multiple object tracking modules into a single object tracker.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to KYLA G ALLEN whose telephone number is (703)756-5315. The examiner can
normally be reached M-F 7:30am - 4:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use
the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
John Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this
application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users. To
file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or
571-272-1000.
/Kyla Guan-Ping Tiao Allen/
Examiner, Art Unit 2661
/AARON W CARTER/Primary Examiner, Art Unit 2661
Prosecution Insights