DETAILED ACTION
Claims 1-6, 8-20 and 61 are currently presented for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/25/2026 has been entered.
Response to Arguments
Following Applicants arguments and amendments, and in light of the 2019 Patent Eligibility guidance, the 101 rejection of the Claims is Withdrawn.
The claim now clarifies that the operation of the claim is done with an image sensor and imaging device connected to a moveable object. These limitations cannot be performed mentally, making the claim as a whole, not directed to an abstract idea.
Following Applicants arguments and amendments, the 102 rejection of the claims is Withdrawn.
See Updated 103 rejection below that is necessitated by Applicant’s Amendment.
Following Applicants arguments and amendments, the 103 rejection of the claims is Maintained.
Applicant’s Argument: Applicant’s arguments directed the 103 rejection are based on newly amended subject matter.
Examiner’s Response: All arguments are addressed in the 103 rejection of the claims below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 8-20 and 61 are rejected under 35 U.S.C. 103 as being unpatentable over Mao et al. USPPN 2018/0001480 in view of Zhou et al. USPPN 2017/0083748.
Regarding claim 1, Mao teaches A method for operating a movable object, comprising: obtaining image data based on one or more images captured by an imaging sensor on board the movable object, ([0083], [0112], an intelligent photographing UAV robot is used; Figures 2 and 3, [0089]-[0090] the images are captured by the robot)
each of the one or more images including at least a portion of a first human body; (Figures 2 and 3, [0089]-[0093], an image of the operator is taken)
identifying a first indication of the first human body in a field of view of the imaging sensor based on the image data, wherein the first indication comprises a predefined body (Figures 2 and 3, [0088]-[0093] a gesture of the operator is captured in the image)
prior to causing the movable object to operate, confirming that the identified first indication is intended to operate the movable object, including waiting a predetermined threshold time period and verifying that the predefined body pose persists for at least the predetermined threshold time period; ([0096], a gesture password is used to determine who the operator is; [0107] a gesture template is used to determine if the gesture results in an action; [0126], a static photo or a dynamic video gesture is used for determining drone instructions; [0101], a follow up image is captured after a predetermined period of time to ensure that the gesture persists)
in response to confirming that the identified first indication is intended to operate the movable object, designating the first human body as an operator to operate the movable object Figures 2 and 3, [0088]-[0096] a gesture of the operator is captured in the image and confirmed to intend to move the drone; [0126], the gesture for control is acted upon and the drone is moved)
after designating the first human body as the operator, tracking or following the operator in the field of view of the imaging sensor, (Figures 1 and 2, [0088], [0093], [0096]-[0098], the user is tracked as multiple images are taken while the user is walking)
wherein, during tracking or following the operator, body indications of any other human body in the field of view of the imaging sensor are not tracked or recognized as control commands to control the movable object; and ([0096]-[0110], the operator is chosen based on who knows the password; Figures 1 and 2, [0088], [0093], [0096]-[0098], the operator is tracked as multiple images are taken while the operator is walking)
causing the movable object to operate in response to a second indication of the first human body in the field of view of the imaging sensor. (Figures 1-3, [0096], [0100], [0110], a gesture is converted to an operation instruction; Figure 3, [0074], [0126]-[0127], the operation instruction moves the robot)
Mao does not explicitly teach detecting, based on the image data, a plurality of human bodies in a field of view of the imaging sensor, the plurality of human bodies including the first human body; from among the plurality of human bodies in the field of view of the imaging sensor;
Zhou teaches detecting, based on the image data, a plurality of human bodies in a field of view of the imaging sensor, the plurality of human bodies including the first human body; (Figures 2 and 4, [0097] and [0110], a plurality of people are tracked)
from among the plurality of human bodies in the field of view of the imaging sensor; (Abstract, [0003] one or more objects is tracked; Figures 2 and 4, [0097] and [0110], from a plurality of tracked objects)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Mao and Zhou as the references deal with imaging, in order to implement a system that distinguishes a person out of a plurality of people and displaying the image. Zhou would modify Mao by distinguishing a person out of a plurality of people and displaying the image. The benefit of doing so is the improved tracking capabilities may allow an imaging device to automatically detect one or more moving objects and to autonomously track the moving objects, without requiring manual input and/or operation by a user. The improved tracking capabilities may be particularly useful when the imaging device is used to precisely track a fast-moving group of objects, whereby the size and/or shape of the group may be amorphous and change over time as the objects move. The improved tracking capabilities can be incorporated into an aerial vehicle, such as an unmanned aerial vehicle (UAV). Also, the user device can be used to receive an analyzed image from the image analyzer (Zhou [0003], [0091])
Regarding claim 2, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches detecting one or more human bodies including the first human body in each of the one or more images; and (Figures 2 and 3, [0089]-[0093], an image of the operator is taken, and the operator is detected)
determining indications associated with the one or more human bodies respectively based on the one or more images. (Figures 2 and 3, [0089]-[0093], the gestures of the operator are determined from the image)
Regarding claim 3, the combination of Mao and Zhou teach the limitations of claim 2. Mao also teaches determining that the first indication satisfies a predefined criterion; and (Figures 1-3, [0089]-[0093], the gestures are matched against predefined gestures)
in accordance with determining that the first indication of the first human body satisfies the predefined criterion, determining the first human body is associated with an operator to operate the movable object. and (Figures 1-3, [0089]-[0093], system confirms that the person in the picture is the operator and that the operator is performing a gesture)
Regarding claim 4, the combination of Mao and Zhou teach the limitations of claim 2. Mao also teaches determining that the first human body is associated with a registered user by performing facial recognition on the one or more images; and (Figure 2, [0095]-[0096], [0098], facial recognition is performed)
in accordance with determining that the first human body is associated with the registered user, determining the registered user is an operator to operate the movable object. (Figure 2, [0095]-[0096], [0098], the face of the operator, who is able to operate the robot, is determined)
Regarding claim 5, the combination of Mao and Zhou teach the limitations of claim 2. Mao also teaches wherein the indications associated with the one or more human bodies are determined by applying a machine learning model to the image data obtained from the one or more images. ([0099], [0104]-[0108], a support vector machine (machine learning model) sis applied to the one or more images)
Regarding claim 8, the combination of Mao and Zhou teach the limitations of claim 2. Mao does not explicitly teach causing display of one or more bounding boxes respectively surrounding the one or more detected human bodies on a display device.
Zhou teaches causing display of one or more bounding boxes respectively surrounding the one or more detected human bodies on a display device. ([0091], the frame including annotations and detected human bodies is displayed on the output device)
See motivation of claim 1
Regarding claim 9, the combination of Mao and Zhou teach the limitations of claim 2. Mao also teaches further comprising: determining that a plurality of indications associated with a plurality of human bodies satisfy predefined criteria, and ([0088], [0097], the gesture is matched with predetermined gestures, [0100], this can come from multiple operators, Figures 1-3, [0089]-[0097], the gesture from multiple images can be determined)
causing the movable object to operate in response to the plurality of indications. (Figures 1-3, [0096], [0100], [0110], a gesture is converted to an operation instruction; [0126], the operation instruction moves the robot)
Regarding claim 10, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches generating an operation instruction to operate the movable object in accordance with predefined criteria associated with the identified first indication. (Figures 1-3, [0096], [0100], [0110], a gesture is converted to an operation instruction; [0126], the operation instruction moves the robot; [0088], [0097], the gesture is matched with predetermined gestures)
Regarding claim 11, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches further comprising: in response to identifying the first indication of the first human body, causing the movable object and the imaging sensor to track the first human body in the field of view of the imaging sensor. (Figures 1 and 2, [0088], [0093], [0096]-[0098], the user is tracked as multiple images are taken while the user is walking)
Regarding claim 12, Mao anticipates the limitations of claim 1. Mao teaches determining that the first indication of the first human body satisfies a predefined criterion, (Figures 1-3, [0083]-[0093], it is determined that the human body is the operator and the operator is performing a gesture)
Mao does not explicitly teach causing display on a display device of a first bounding box surrounding the first human body.
Zhou teaches causing display on a display device of a first bounding box surrounding the first human body. ([0091], the frame including annotations and detected human bodies is displayed on the output device)
See motivation of claim 1
Regarding claim 13, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches determining that the second indication of the first human body satisfies a predefined criterion, and causing the movable object to autonomously land. (Figures 1-3, [0096], [0100], [0110], a gesture is converted to an operation instruction; [0126], the operation instruction moves the robot; [0112], the instruction is for the robot to autonomously land)
Regarding claim 14, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches further comprising: determining that the first indication of the first human body satisfies predefined criteria, and (Figures 1-3, [0089]-[0093], the gestures are matched against predefined gestures)
causing the imaging sensor to autonomously capture one or more images of the first human body. ([0083], [0093], multiple images are taken; [0088]-[0098] for dynamic gestures more than one image is required of the same body)
Regarding claim 15 the combination of Mao and Zhou teach the limitations of claim 1. Mao teaches determining that the second indication of the first human body satisfies predefined criteria, and (Figures 1-3, [0083]-[0093], it is determined that the human body is the operator and the operator is performing a gesture)
and causing autonomous adjustment of one or more parameters of the imaging sensor to change from a first photography mode to a second photography mode. ([0093] the sensor changes from photo to video mode when static and dynamic gestures are detected)
Regarding claim 16, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches determining one or more characteristics associated with the first indication of the first human body; and (Figures 1-3, [0083]-[0097], gestures associated with the operator are determined including static and dynamic gestures)
causing the movable object to operate in accordance with the determined one or more characteristics. (Figures 1-3, [0096], [0100], [0110], a gesture is converted to an operation instruction; [0126], the operation instruction moves the robot)
Regarding claim 17, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches wherein the first indication of the first human body includes a body movement identified based on a plurality of images, ([0088]-[0097], body movement of limbs is detected)
the body movement including at least one of a hand movement, a finger movement, a palm movement, a facial expression, a head movement, an arm movement, a leg movement, or a torso movement. ([0088], [0097], waving and walking is identified)
Regarding claim 18, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches wherein the first indication of the first human body includes a body pose associated with a stationary bodily attitude or position that is identified based on one image. (Figures 2 and 3, [0088], [0096], static poses are identified)
Regarding claim 19, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches further comprising: prior to causing the movable object to operate, confirming that the first indication of the first human body is intended to operate the movable object. (Figures 1-3, [0021]-[0022], [0030], the gesture is checked with the preconfigured gestures to make sure the gesture is one to operate the robot)
Regarding claim 20, the combination of Mao and Zhou teach the limitations of claim 1. Mao also teaches wherein the movable object is an unmanned aerial vehicle (UAV). ([0083], [0112], an intelligent photographing UAV robot is used)
Regarding claim 61, Mao teaches a processor; and a memory coupled to the processor, the memory storing instructions which, when executed by the processor, cause the processor to perform a method comprising: (Figure 4, a processor and memory are used)
obtaining image data based on one or more images captured by an imaging sensor on board the movable object, ([0083], [0112], an intelligent photographing UAV robot is used; Figures 2 and 3, [0089]-[0090] the images are captured by the robot)
each of the one or more images including at least a portion of a first human body; (Figures 2 and 3, [0089]-[0093], an image of the operator is taken)
identifying a first indication of the first human body in a field of view of the imaging sensor based on the image data; (Figures 2 and 3, [0088]-[0093] a gesture of the operator is captured in the image)
prior to causing the movable object to operate, confirming the identified first indication of the first human body by verifying that the identified first indication of the first human body persists for a predetermined time period; ([0096], a gesture password is used to determine who the operator is; [0107] a gesture template is used to determine if the gesture results in an action; [0126], a static photo or a dynamic video gesture is used for determining drone instructions; [0101], a follow up image is captured after a predetermined period of time to ensure that the gesture persists)
generating an operation instruction based on the identified first indication of the first human body and a stored set of rules that associate respective body indications with corresponding operation instructions for the movable object; ([0096], a gesture password is used to determine who the operator is; [0107] a gesture template is used to determine if the gesture results in an action; Figures 1-4, [0096], [0100], [0110], a gesture is converted to an operation instruction; Figure 3, [0074], [0126]-[0127], the operation instruction moves the robot)
generating one or more control signals for one or more propulsion devices of the movable object based on the generated operation instruction; (Figures 1-3, [0096], [0100], [0110], a gesture is converted to an operation instruction; Figure 3, [0074], [0126]-[0127], the operation instruction moves the robot)
transmitting the generated one or more control signals to a controller of the movable object for execution, to cause the movable object to operate in response to the identified first indication of the first human body in the field of view of the imaging sensor, comprising causing the one or more propulsion devices of the movable object to operate in accordance with the generated one or more control signals; and ([0096], a gesture password is used to determine who the operator is; [0107] a gesture template is used to determine if the gesture results in an action; Figures 1-4, [0096], [0100], [0110], a gesture is converted to an operation instruction; Figure 3, [0074], [0126]-[0127], the operation instruction moves the robot)
determining that the first indication of the first human body satisfies predefined criteria, and ([0096], a gesture password is used to determine who the operator is; [0107] a gesture template is used to determine if the gesture results in an action)
causing autonomous adjustment of one or more parameters of the imaging sensor to change from a first photography mode to a second photography mode, ([0093] the sensor changes from photo to video mode when static and dynamic gestures are detected)
Mao does not explicitly teach the causing the autonomous adjustment comprising adjusting at least one of a focal length, a shutter speed, or an ISO setting of the imaging sensor.
Zhou teaches the causing the autonomous adjustment comprising adjusting at least one of a focal length, a shutter speed, or an ISO setting of the imaging sensor. ([0065], exposure setting are changed including exposure (e.g., exposure time, shutter speed, aperture, film speed), gain, gamma, area of interest, binning/subsampling, pixel clock, offset, triggering, and ISO.
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Mao and Zhou as the references deal with imaging, in order to implement a system that changes exposure settings. Zhou would modify Mao by changing exposure settings. The benefit of doing so is the amount of light that hits the sensor at a given time can be modified to achieve the desired image. (Zhou [0065])
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Mao in view of Zhou and in further view of Xiao et al. "Tracking small targets in infrared image sequences under complex environmental conditions".
Regarding claim 6, the combination of Mao and Zhou teach the limitations of claim 2. Mao also teaches wherein determining the indications associated with the one or more human bodies further comprises: determining respective locations of a plurality of key physical points on each of the one or more human bodies, ([0088], the hand and arm in relation to each other (key points) are determined so the arm can be identified as bent; [0097] walking is determined by looking at the respective locations of the legs (key points))
Mao does not explicitly recite causing display of … the plurality of key physical points for at least one of the one or more human bodies on a display device.
Zhou teaches causing display of … the plurality of key physical points for at least one of the one or more human bodies on a display device. ([0091], the frame including annotations and detected human bodies is displayed on the output device)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Mao and Zhou as the references deal with imaging, in order to implement a system that distinguishes a person out of a plurality of people and displaying the image. Zhou would modify Mao by distinguishing a person out of a plurality of people and displaying the image. The benefit of doing so is the improved tracking capabilities may allow an imaging device to automatically detect one or more moving objects and to autonomously track the moving objects, without requiring manual input and/or operation by a user. The improved tracking capabilities may be particularly useful when the imaging device is used to precisely track a fast-moving group of objects, whereby the size and/or shape of the group may be amorphous and change over time as the objects move. The improved tracking capabilities can be incorporated into an aerial vehicle, such as an unmanned aerial vehicle (UAV). Also, the user device can be used to receive an analyzed image from the image analyzer (Zhou [0003], [0091])
The combination of Mao and Zhou does not explicitly teach a confidence map
Xiao teaches a confidence map (Abstract, Section 2, Figure 1, a confidence map is created)
It would have been obvious to one of ordinary skill in the art, before the effective filing date, to combine the teachings of Mao and Zhou with Xiao as the references deal with imaging, in order to implement a system that uses a confidence map. Xiao would modify Mao and Zhou by using a confidence map. The benefit of doing so is it can enhance the reliability and robustness of tracking for complex environmental conditions including camera ego-motion, competing background clutter, low contrast, and intense noise. (Xiao Section 2)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zadeh et al. USPPN 2020/0184278: Also teaches the use of an imaging sensor that can change exposure settings to produce the best image.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL COCCHI whose telephone number is (469)295-9079. The examiner can normally be reached 7:15 am - 5:15 pm CT Monday - Thursday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571-272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL EDWARD COCCHI/Primary Examiner, Art Unit 2188
Prosecution Insights