Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on August 27, 2025 has been entered.
Response to Amendment
Applicant’s amendment filed on August 27, 2025 amends claims 1, 13, and 20-21. Claims 1-5 and 7-21 are pending.
Response to Arguments
Applicant's arguments filed on August 27, 2025 regarding the newly presented claim limitations have been fully considered and are unpersuasive and/or moot as shown in the rejections that follow. Examiner disagrees with Applicant’s argument that “the sensor section 102, assuming that it could correspond to the claimed image sensor including an image capturing unit does not also include the first recognition unit as claimed” and that “at best, the recognition section 104 of Suzuki would necessarily correspond to the claimed first recognition unit and not the claimed second recognition unit arranged outside of the image sensor.” Examiner used Suzuki to teach “a second subject recognition unit arranged outside the image sensor” as recited in the third clause of claim 1. Examiner used Liu to show a teaching of an image sensor including an image capturing unit that also includes the first subject recognition as claimed. As explained in detail in the rejections that follow, the newly presented claims are taught by the combination of Liu and Suzuki..
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 1 is rejected under 35 U.S.C. 112(b), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Each of claims 1 and 20-21 recites “a second subject recognition unit arranged outside the image sensor”. The term “arranged outside” is a relative and/or subjective term which renders the claim indefinite. The phrase “arranged outside the image sensor” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Examiner interprets “a second subject recognition unit arranged outside the image sensor” to be any recognition unit, module, circuitry, or sectional processing unit that is separated from the image sensor.
Claims 2-5 and 7-19 depend on claim 1 and are also rejected under 35 U.S.C. 112(b), second paragraph, because they inherit the limitations and fail to resolve the deficiencies of independent claim 1.
Appropriate amendments are required for the above-identified issues. No new matter should be added for any amendment.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5, 7-11, 14-15, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577).
Regarding claim 1, Liu teaches a drone flight control device that performs flight control of a drone having a plurality of propellers, the drone flight control device comprising: an image sensor including an image capturing unit that has a plurality of pixels arrayed and generates an image signal, (see Liu at [0054] which discloses that the moveable object 102 is a UAV and includes components to enable flight and/or flight control and that in some embodiments, the moveable object 102 includes communication system 206 with one or more network or other communications interfaces (e.g., via which flight control instructions are received), one or more movement mechanisms 114 (e.g., 114a, 114b), and/or one or more moveable object actuators 210 (e.g., 210a, 210b) and that the moveable object actuators (e.g., 210a, 210b) cause movement of movement mechanisms (e.g., 114a, 114b), e.g., in response to received control instructions. Furthermore, see Liu at [0055] which discloses that in some embodiments, the moveable object 102 includes movement mechanisms 114 (e.g., propulsion mechanisms) and that movement mechanism includes single or multiple rotors; also, see Liu at [0059] which discloses that in some embodiments, the sensing system 208 of the moveable object 102 includes one or more sensors and that in some embodiments, the one or more sensors of moveable object sensing system 208 includes image sensor 220 (e.g., an imaging sensor of an imaging device 218, such as a digital camera); further see Liu at [0059] which discloses that the image sensor 220 includes, e.g., semiconductor charge-coupled devices (CCD), active pixel sensors using complementary metal-oxide-semiconductor (CMOS) and/or N-type metal-oxide-semiconductors (NMOS, Live MOS) and that in some embodiments, the image sensor 220 includes one or more arrays of photo sensors. Examiner maps moveable object 102 to the recited drone. Examiner maps the collection of the foregoing components of the moveable object, such as a memory 204, a communication system 206, a moveable object sensing system 208, and one or more communication buses 212, one or more moveable object actuators 210, as well as the one or more processors to the flight control device, for example. Examiner maps multiple rotors to plurality of propellers. Examiner maps image sensor 220 to the recited image sensor. Examiner further maps the semiconductor charge-coupled device (CCD) and/or active pixel sensors using complementary metal-oxide semiconductor (CMOS) and/or N-type metal-oxide-semiconductors to the image capturing unit that generates an image signal. Examiner maps active pixel sensors and/or one or more arrays of photo sensors to a plurality of pixels arrayed.)
and a first subject recognition unit that executes subject recognition processing by using the image signal; (see Liu at [0037] which discloses the case where an obstacle is detected in the original movement path of the UAV, or where an object of interest is detected in the field of view of the UAV, instructions can be generated at the control device or onboard the UAV for the UAV to execute appropriate obstacle avoidance maneuvers or to head toward the object of interest that has been detected; see Liu at [0052] which discloses that in some embodiments, the one or more processor(s) 202 include one or more image signal processors (ISPs); see Liu at [0062] which discloses that in some embodiments, the controlling system includes instructions and/or functional units for initiating and/or ceasing storage of the image data output of the image sensor 220 and that in some embodiments, the controlling system includes image processing instructions and/or functional units for processing high quality image data to generate raw format image data and/or to generate reduced-size image data. Thus, Examiner notes that processing of an image is performed by one or more image signal processors (ISPs). Liu at [0059] discloses that in some embodiments, one or more processors(s) 202, memory 204, and/or ISP(s) 216 are components of imaging device 218. Liu at Fig. 2 further illustratively depicts imaging device 218 comprising image sensor 220. Liu at [0062] further discloses that in some embodiments, the image processing instructions include one or more compression algorithms, such as are well-known in the art, that in some embodiments, the controlling system includes instructions and/or functional units for preprocessing the high-quality image data or the reduced-size image data in preparation of the image-based object detection procedures, that in some embodiments, the controlling system includes instructions and/or functional units for processing the image data to extract feature points and track the feature points across multiple images that have been taken at different locations along the moveable objects' movement path, that in some embodiments, the controlling system includes instructions and/or functional units for detecting objects and estimating depths of the objects represented in the captured images (or in the field of view of the moveable object) and that in some embodiments, the controlling system includes instructions and/or functional units for characterizing (e.g., estimating the sizes, shapes, and locations of) the objects detected in the captured image (or in the field of view of the moveable object). Based on the teachings of Liu, Liu teaches an image sensor which includes one or more image signal processors (ISPs) which perform subject recognition processing. Examiner notes that the processing of image data and the tracking of feature points across images and the detection of objects in a captured image corresponds to subject recognition processing by using the image signal. Examiner maps one of the functional units for pre-processing or processing the image data in preparation of the image-based object detection procedures or for detecting objects and estimating depths of objects to the recited first subject recognition unit.)
a first generation unit configured to generate a first control signal for drive control of the plurality of propellers on a basis of a recognition result by the first subject recognition unit; (see Liu at [0055] which discloses that the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; see Liu at [0062] which discloses that in some embodiments, the controlling system includes instructions and/or functional units for generating specific instructions for adjusting the movement of the moveable object in accordance with the characterization of the objects that have been detected. Examiner notes that an electronic speed controller and/or any one of functional units for adjusting movement of the moveable object may be mapped to the first generation unit configured to generate a control signal for drive control of the propeller.)
a drive unit configured to drive the plurality of propellers on a basis of the first control signal from the first generation unit and on a basis of the second control signal from the second generation unit; and (see Liu at [0055] which discloses that the moveable object 102 includes movement mechanisms 114 (e.g., propulsion mechanisms), and that although the plural term “movement mechanisms” is used herein for convenience of reference, “movement mechanisms 114” refers to a single movement mechanism (e.g., a single propeller) or multiple movement mechanisms (e.g., multiple rotors) and that the movement mechanisms 114 include one or more movement mechanism types such as rotors, propellers, blades, engines, motors, wheels, axles, magnets, nozzles, and so on. Liu at [0055] further discloses that the movement mechanisms 114 are coupled to the moveable object 102 using any suitable means, such as support elements (e.g., drive shafts) and/or other actuating elements (e.g., the moveable object actuators 210). Examiner maps movement mechanisms or propulsion mechanisms, such as a drive shaft or actuators, to drive unit configured to drive the propeller on a basis of the control signal from the first generation unit. See Liu at [0055] which discloses that the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; Examiner notes that processor(s) include more than one processor and that a second processor may provide another control signal to the moveable object actuator. Thus, Examiner maps another of the control signals to the second control signal, while another of the processor(s) may be mapped to the second generation unit. Also, see Liu at [0062] which discloses that in some embodiments, the controlling system includes instructions and/or functional units for generating specific instructions for adjusting the movement of the moveable object in accordance with the characterization of the objects that have been detected.)
wherein the first generation unit, the second subject recognition unit, the second generation unit, and the switching unit are implemented in a processor (see Liu at Fig. 2 which illustratively discloses processor(s) 202 within a moveable object 102; see Liu at [0168] which discloses that many features of the technology disclosed herein can be performed in, using, or with the assistance of hardware, software, firmware, or combinations thereof, that consequently, features of the present technology may be implemented using a processing system, that exemplary processing systems (e.g., processor(s) 202, 302) include, without limitation, one or more general purpose microprocessors (for example, single or multi-core processors), application-specific integrated circuits, application-specific instruction-set processors, field-programmable gate arrays, graphics processors, physics processors, digital signal processors, coprocessors, network processors, audio processors, encryption processors, and the like. Examiner maps processing system to the recited processor.)
Liu does not expressly disclose a second subject recognition unit arranged outside the image sensor; which in a related art Suzuki teaches (see Suzuki at [0108] in conjunction with Fig. 1 which discloses that the recognition processing section 104 performs a recognition process (e.g., person detection, face identification, and image classification) corresponding to pixel data for recognizing an object within an image, on the basis of pixel data received from the sensor control section 103 and that a recognition result obtained by the recognition processing section 104 is given to the output control section 107; see Suzuki at [0376] in conjunction with Figs. 1 and 54, for example, which discloses that feature value accumulation control section 2112 also gives the feature value given from the feature value calculation section 2111, to the recognition process execution section 2115 and that the recognition process execution section 2115 executes a recognition process on the basis of the integrated feature value given from the feature value accumulation control section 2112 (step S5106) and that the recognition process execution section 2115 outputs a recognition result obtained by the recognition process to the output control section 107 (step S5107). Per Suzuki at Fig. 1, for example, the Examiner notes that the recognition processing section 104 is located “outside” the sensor section 102. In other words, the Examiner notes that the recognition processing section 104 is performed outside of the sensor section processing 102. Furthermore, per Suzuki at Fig. 54, for example, the Examiner maps the feature value calculation section or feature value accumulation control section to the second subject recognition unit arranged “outside” the image sensor. Examiner notes that the specification is silent with respect to the term “outside the image sensor”. Examiner has shown a teaching based on a broadest reasonable interpretation of the claimed language.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to include a second subject recognition unit arranged outside the image sensor, as taught by Suzuki.
One would have been motivated to make such a modification to speed-up calculation of a determination basis for moving images as applicable to imaging devices mounted on various mobile bodies, such as drones, for example, as suggested by Suzuki at [0004] and at [0468].
a second generation unit configured to generate a second control signal for drive control of each of the plurality of propellers on a basis of a recognition result of the second subject recognition unit; and a switching unit configured to switch and inputs, to the drive unit, a control signal from the first generation unit and a control signal from the second generation unit (see Suzuki at [0393-0395] which discloses switching from the recognition process for each readout unit to the ordinary recognition process which is based on pixel data of pixels read from an entire frame by readout from the entire frame and that the ordinary recognition process may switch a mode of an operation clock of a device (a processor that executes the recognition process (the program of the learning model trained)) to a higher-speed mode; see Suzuki at [0396] which discloses that when reliability of the recognition process for each readout unit lowers, or when a determination basis presented for a recognition result is not understandable, the recognition process for each readout unit may be switched to the ordinary recognition process and that thereafter, the recognition process may be returned to the recognition process for each readout unit when high reliability of the recognition process is recovered; see Suzuki at [0486] which discloses using automatic avoidance from obstacles and that the microcomputer 6801 classifies obstacles around the vehicle 6900 into obstacles visible for the driver of the vehicle 6900 and obstacles difficult to view for the driver, that in addition, the microcomputer 6801 determines a collision risk indicating a level of danger of collision with the respective obstacles, and that when a collision risk has a setting value or higher and indicates a possibility of collision, the microcomputer 6801 is capable of offering driving assistance for avoiding collision with the obstacles. Examiner maps the processor disclosed in Suzuki at [0395] to the switching unit. Examiner notes that the microcomputer, by way of providing driving assistance, teaches a second generation unit configured to generate a control signal for drive control. Examiner maps the microcomputer which avoids collision with the obstacles and is capable of offering driving assistance to the second generation unit.)
Regarding claim 2, the modified Liu teaches the drone flight control device according to claim 1, wherein a recognition result of a subject is output from the first subject recognition unit in a frame period in which the image signal is read out from the image capturing unit (see Suzuki at the Abstract which discloses a determination basis calculation section that calculates a determination basis of a recognition process performed by the recognition section; see Suzuki at [0020] which discloses that the recognition section executes a machine learning process using an RNN (recurrent neural network) for pixel data of a plurality of the readout units in an identical frame image, to execute the recognition process on the basis of a result of the machine learning process; see Suzuki at [0108] which discloses that the recognition processing section 104 performs a recognition process (e.g., person detection, face identification, and image classification) corresponding to pixel data for recognizing an object within an image, on the basis of pixel data received from the sensor control section 103; see Suzuki at [0279] which discloses that for example, the recognition process for image data by using a CNN executes a recognition or detection process such as face detection, face authentication, visual line detection, facial expression recognition, face direction detection, object detection, object recognition, movement (mobile object) detection, pet detection, scene recognition, state detection, and avoidance target recognition; also, see Suzuki at [0264] in conjunction with Fig. 29 which depicts recognition results and determination processes within a frame cycle (1/30 second). Suzuki at [0393] further discloses that ordinary recognition process is a recognition process based on pixel data of pixels read from an entire frame by readout from the entire frame. Examiner maps calculating a determination basis of a recognition process performed by a recognition processing section corresponds to outputting a recognition result of a subject form a first subject recognition unit. Examiner maps any one of face detection, face authentication, visual line detection, facial expression recognition, face direction detection, object detection, object recognition, movement (mobile object) detection, pet detection, scene recognition, state detection, and avoidance target recognition to a recognition result of a subject. Examiner maps recognition processing section to subject recognition unit, for example.)
Regarding claim 3, the modified Liu teaches the drone flight control device according to claim 1, wherein a recognition result of a subject is output from the first subject recognition unit before readout of the image signal from the image capturing unit ends (see Suzuki at [0267] which discloses that as described above, a recognition result and a determination basis for this recognition result can be sequentially obtained without a necessity of input of all image data of a frame to the recognizer (recognition processing section 104), by sequentially executing the recognition process and the determination basis calculation process for the recognition result for each readout unit (line in this example) and that accordingly, a delay produced until acquisition of the recognition result and the determination basis for the recognition result can be reduced. Examiner notes that determining a recognition result for each line readout corresponds to wherein a recognition result of a subject is output from the first subject recognition unit before readout of the image signal from the image capturing unit ends.)
Regarding claim 5, Liu teaches the drone flight control device according to claim 1, wherein the image sensor used for the subject recognition processing is used as an image sensor for capturing an image (see Liu at [0051] which discloses that the moveable object 102 typically includes one or more processor(s) 202, a memory 204, a communication system 206, a moveable object sensing system 208, and one or more communication buses 212 for interconnecting these components; see Liu at [0052] which discloses that in some embodiments, the one or more processor(s) include at least one Field Programmable Gate Array (FPGA) and/or at least one Application Specific Integrated Circuit (ASIC) and that in some embodiments, the one or more processor(s) 202 include one or more image signal processors (ISPs) 216 (e.g., implemented in the at least one FPGA and/or the at least one ASIC); see Liu at [0052] in conjunction with Fig. 2 which discloses one or more image signal processors (ISPs); furthermore, see Liu at [0059] which discloses that in some embodiments, one or more processors(s) 202, memory 204, and/or ISP(s) 216 are components of imaging device 218; Liu at [0059] discloses that in some embodiments, one or more processors(s) 202, memory 204, and/or ISP(s) 216 are components of imaging device 218. Also, see Liu at [0059] which discloses that in some embodiments, the sensing system 208 of the moveable object 102 includes one or more sensors and that in some embodiments, the one or more sensors of moveable object sensing system 208 includes image sensor 220 (e.g., an imaging sensor of an imaging device 218, such as a digital camera); further see Liu at [0059] which discloses that the image sensor 220 includes, e.g., semiconductor charge-coupled devices (CCD), active pixel sensors using complementary metal-oxide-semiconductor (CMOS) and/or N-type metal-oxide-semiconductors (NMOS, Live MOS) and that in some embodiments, the image sensor 220 includes one or more arrays of photo sensors. Examiner notes that image signal processors are used for processing images which include any subjects or objects in the images. Examiner further notes that the image sensor used for capturing an image is also used in conjunction with providing images to the image signal processors (ISPs).)
Regarding claim 7, the modified Liu teaches the drone flight control device according to claim 1, wherein the second subject recognition unit and the second generation unit are arranged in an image processing unit (see Liu at [0055] which discloses that the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; see Liu at [0165-0166] which discloses that an electronic device 1500 includes a processing unit 1510 and that the processing unit is further configured to perform other operations described in FIGS. 13A-13E and FIGS. 14A-14G, and accompanying descriptions, using the various functional units of the device 1500. Examiner maps processing unit to the image processing unit.)
Regarding claim 8, the modified Liu teaches the drone flight control device according to claim 7, wherein the switching unit is arranged in the image processing unit (see Liu at [0165-0166] which discloses that an electronic device 1500 includes a processing unit 1510 and that the processing unit is further configured to perform other operations described in FIGS. 13A-13E and FIGS. 14A-14G, and accompanying descriptions, using the various functional units of the device 1500. Examiner maps processing unit to the image processing unit.); see Suzuki at [0393-0395] which discloses switching from the recognition process for each readout unit to the ordinary recognition process which is based on pixel data of pixels read from an entire frame by readout from the entire frame and that the ordinary recognition process may switch a mode of an operation clock of a device (a processor that executes the recognition process (the program of the learning model trained)) to a higher-speed mode; Examiner notes that the processor that performs switching may be included in the image processing unit which performs other operations such as operations described in conjunction with FIGS. 13A-13E and FIGS. 14A-14G, and accompanying descriptions.)
Regarding claim 9, the modified Liu teaches the drone flight control device according to claim 1, wherein the switching unit is arranged in the image sensor (see Suzuki at [0393-0395] which discloses switching from the recognition process for each readout unit to the ordinary recognition process which is based on pixel data of pixels read from an entire frame by readout from the entire frame and that the ordinary recognition process may switch a mode of an operation clock of a device (a processor that executes the recognition process (the program of the learning model trained)) to a higher-speed mode; see Suzuki at [0396] which discloses that when reliability of the recognition process for each readout unit lowers, or when a determination basis presented for a recognition result is not understandable, the recognition process for each readout unit may be switched to the ordinary recognition process and that thereafter, the recognition process may be returned to the recognition process for each readout unit when high reliability of the recognition process is recovered; see Liu at [0173] which discloses that the boundaries of these functional building blocks have often been arbitrarily defined herein for the convenience of the description and that alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Liu at [0173] further discloses that any such alternate boundaries are thus within the scope and spirit of the disclosure. Examiner notes that the boundaries of functional building blocks, for example, as depicted in Suzuki at Fig. 1 and/or in Liu at Fig. 2, may have alternate boundaries such that the processor that executes the switching and recognition process may be located in the sensor section 102 of Suzuki’s Fig. 1 or the imaging device 218 of Liu’s Fig. 2.)
Regarding claim 10, the modified Liu teaches the drone flight control device according to claim 9, wherein the second control signal generated by the second generation unit is transmitted to the switching unit arranged in the image sensor (see Liu at [0055] which discloses that the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; see Liu at [0173] which discloses that the boundaries of these functional building blocks have often been arbitrarily defined herein for the convenience of the description and that alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Liu at [0173] further discloses that any such alternate boundaries are thus within the scope and spirit of the disclosure. Based on the foregoing, Examiner notes that the second control signal may also perform its functions by way of alternatively being transmitted to the processor taught by Suzuki, which performs a switching function.)
Regarding claim 11, the modified Liu teaches the drone flight control device according to claim 10, wherein the image sensor further includes an output device that outputs the first control signal and the second control signal switched by the switching unit to the drive unit (see Suzuki at [0393-0395] which discloses switching from the recognition process for each readout unit to the ordinary recognition process which is based on pixel data of pixels read from an entire frame by readout from the entire frame and that the ordinary recognition process may switch a mode of an operation clock of a device (a processor that executes the recognition process (the program of the learning model trained)) to a higher-speed mode; see Suzuki at [0396] which discloses that when reliability of the recognition process for each readout unit lowers, or when a determination basis presented for a recognition result is not understandable, the recognition process for each readout unit may be switched to the ordinary recognition process and that thereafter, the recognition process may be returned to the recognition process for each readout unit when high reliability of the recognition process is recovered; Liu at [0062] further discloses that in some embodiments, the controlling system includes instructions and/or functional units for initiating and/or ceasing storage of the image data output of the image sensor 220. Also, see Liu at [0055] which discloses that the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; see Liu at [0173] which discloses that the boundaries of these functional building blocks have often been arbitrarily defined herein for the convenience of the description and that alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Liu at [0173] further discloses that any such alternate boundaries are thus within the scope and spirit of the disclosure. Examiner notes that a readout unit that switches between two processes corresponds to an output device that outputs the first control signal and the second control signal switched by the switching unit to the drive unit.)
Regarding claim 14, the modified Liu teaches the drone flight control device according to claim 1, wherein the first subject recognition unit and the second subject recognition unit further specify a type and speed of a subject (see Suzuki at [0163] which discloses that the memory 105 stores beforehand a program or a model parameter of a machine learning model so trained as to identify each of (achieve classification of) multiple types of objects, such as a car, on the basis of predetermined learning data; see Suzuki at [0399] which discloses that the external information may include vehicle information and ambient environment information, and that for example, the vehicle information is steering information or speed information; see Suzuki at [0514] which discloses that an information processing device including a recognition section that has a machine learning model trained on the basis of learning data.).
Regarding claim 15, the modified Liu teaches the drone flight control device according to claim 14, wherein the first subject recognition unit and second subject recognition unit determine whether or not avoidance is necessary according to the type or the speed of the specified subject (see Suzuki at [0179] which discloses that the recognition process for image data by using a CNN executes a recognition or detection process such as face detection, face authentication, visual line detection, facial expression recognition, face direction detection, object detection, object recognition, movement (mobile object) detection, pet detection, scene recognition, state detection, and avoidance target recognition; see Suzuki at [0486] which discloses that the microcomputer 6801 is capable of classifying three-dimensional object data indicating three-dimensional objects into a two-wheeled vehicle, a standard-sized vehicle, a large-sized vehicle, a pedestrian, an electric pole, and other three-dimensional objects on the basis of the distance information obtained from the imaging sections 6901 to 6904, and extracting the classified data to use the three-dimensional object data for automatic avoidance from obstacles, that for example, the microcomputer 6801 classifies obstacles around the vehicle 6900 into obstacles visible for the driver of the vehicle 6900 and obstacles difficult to view for the driver, that in addition, the microcomputer 6801 determines a collision risk indicating a level of danger of collision with the respective obstacles; see Suzuki at [0514] which discloses that an information processing device including a recognition section that has a machine learning model trained on the basis of learning data. Examiner notes that the information processing device utilizes a machine learning model based on learned information which includes type and speed of the specified subject, such as the car or vehicle disclosed by Suzuki.)
Claim 20 is directed toward a method that performs the steps recited in the computing device of claim 1. The cited portions of the reference(s) used in the rejection of claim 1 teach the steps recited in the method of claim 20. Therefore, claim 20 is rejected under the same rationale used in the rejection of claim 1.
Claim 21 is directed toward a computer-readable storage medium that performs the steps recited in the computing device of claim 1. The cited portions of the reference(s) used in the rejection of claim 1 teach the steps recited in the computer-readable storage medium of claim 21. Therefore, claim 21 is rejected under the same rationale used in the rejection of claim 1.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577) and further in view of Haukom (US 2016/0318622).
Regarding claim 12, the modified Liu teaches that a switching unit changes the first control signal and the second control signal (see Liu at [0055] which discloses that the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; see Suzuki at [0393-0395] which discloses switching from the recognition process for each readout unit to the ordinary recognition process which is based on pixel data of pixels read from an entire frame by readout from the entire frame and that the ordinary recognition process may switch a mode of an operation clock of a device (a processor that executes the recognition process (the program of the learning model trained)) to a higher-speed mode; see Suzuki at [0396] which discloses that when reliability of the recognition process for each readout unit lowers, or when a determination basis presented for a recognition result is not understandable, the recognition process for each readout unit may be switched to the ordinary recognition process and that thereafter, the recognition process may be returned to the recognition process for each readout unit when high reliability of the recognition process is recovered. As taught by Suzuki at [0395], the Examiner notes that the switching performed by the processor varies based on the recognition process and may further be based on the program of the learning model trained.
The modified Liu does not expressly disclose drone flight control device according to claim 1, wherein the switching unit changes the first control signal and the second control signal in such a manner that an angle of an airframe of the drone does not exceed a control limit which in a related art, Haukom teaches (see Haukom, at [0013], which discloses that in other examples, aircraft 14 can be an unmanned aerial vehicle, often referred to as a “drone,” that is adapted for autonomous and/or remotely piloted flight; see Haukom at [0026] which discloses that aircraft flight performance data can include aircraft angle of attack information received from aircraft avionics data acquisition system 26, flight display control unit 30, an air data computer (ADC) of aircraft 14 (not illustrated), or other system of aircraft 14 and that an aircraft interface device 12 can compare the angle of attack of aircraft 14 to a threshold angle of attack, and can determine that the aircraft anomaly condition is present in response to determining that the angle of attack exceeds a threshold angle of attack; see Haukom at [0027] which discloses that the aircraft flight performance data, in some examples, can include aircraft bank angle information received from aircraft avionics data acquisition system 26, FMC 28, an IRS of aircraft 14 (not illustrated), or other system or component and that an aircraft interface device 12 can compare the bank angle (i.e., roll angle) of aircraft 14 to a threshold bank angle, and can determine that the aircraft anomaly condition is present in response to determining that the bank angle exceeds a threshold bank angle, such as a threshold bank angle corresponding to a boundary of a performance envelope of aircraft 14. Examiner notes that determining that an anomaly condition is present when exceeding the threshold angle of attack or bank angle corresponds to performing in such a manner that an angle of an airframe of the drone does not exceed a control limit. Examiner maps angle of attack or bank angle to the angle of an airframe.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to include wherein the switching unit changes a first and second control signals in such a manner that an angle of an airframe of the drone does not exceed a control limit, as taught by Haukom.
One would have been motivated to make such a modification to provide aircraft operational anomaly detection, as suggested by Haukom, at [0002].
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577) and further in view of Dupray et al. (US 2017/0069214).
Regarding claim 16, the modified Liu does not expressly disclose the drone flight control device according to claim 15, wherein the first subject recognition unit and second subject recognition unit predict a motion according to the type or the speed of the specified subject and determine an avoidance direction which in a related art, Dupray teaches (see Dupray at [0215] which discloses that the actual location or flight path of the other object was not as anticipated or predicted (e.g., the location of the other object has been incorrectly observed or the calculated anticipated trajectory of the other object is incorrect, and the other object is now in a vicinity of the UAV, e.g., the operating zone B, before a new anticipated trajectory can be calculated and a new flight path adopted, e.g., the other object flies at a speed beyond the anticipated needed reaction time of the UAV), or other reasons endangering the operation of the UAV. Dupray at [0215] further discloses that in such cases, the emergency object avoidance procedure may be performed with at least a purpose of safe operation (e.g., to avoid a collision with the other object). Examiner maps calculating an anticipated trajectory to predicting a motion.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to include wherein the first and second subject recognition units predict a motion according to a type or speed of a specified subject and determine an avoidance direction, as taught by Dupray.
One would have been motivated to make such a modification to perform an emergency avoidance of another object, as suggested by Dupray at [0215].
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577) and further in view of Yoon (US 2023/0079308).
Regarding claim 4, the modified Liu teaches wherein a recognition result of a subject is output from the first subject recognition unit (see Liu at [0037] which discloses the case where an obstacle is detected in the original movement path of the UAV, or where an object of interest is detected in the field of view of the UAV, instructions can be generated at the control device or onboard the UAV for the UAV to execute appropriate obstacle avoidance maneuvers or to head toward the object of interest that has been detected. Examiner maps detection of an obstacle or object of interest to a recognition result, for example.)
The modified Liu does not expressly disclose the drone flight control device according to claim 1, wherein a recognition result of a subject is output from the first subject recognition unit without being synchronized with a cycle of reading out the image signal of one frame from the image capturing unit which in a related art, Yoon teaches (see Yoon at [0006] which discloses that an aspect of the present disclosure provides an action recognition device and an action recognition method for performing asynchronous action recognition of multiple objects using a separate edge device; see Yoon at [0008] which discloses that an action recognition device includes a camera for capturing an image, a target queue storing therein data about at least one target, a reasoner configured to perform action reasoning, and a computing device electrically connected to the camera, the target queue and the reasoner, wherein the computing device may update the target queue based on an object tracking result on the image, extract a target from the target queue, request the reasoner to perform action reasoning on the extracted target, and output an action recognition result based on an action reasoning result provided from the reasoner; see Yoon at [000] which discloses that the reasoner may be embodied as an edge device including a processor and a memory; see Yoon at [0127] which discloses that the asynchronous action recognition of multiple objects may be performed using the separate edge device, such that a constant processing speed and performance of the action recognition device may be maintained regardless of change in the number of action recognition targets. Examiner may map the camera used for capturing an image to the image capturing unit. Examiner notes that asynchronous action recognition corresponds to without being synchronized with a cycle of a reading out the image signal of one frame.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to include without being synchronized with a cycle of reading out the image signal of one frame from the image capturing unit, as taught by Yoon.
One would have been motivated to make such a modification to compensate for an increase in the number of action analysis targets, as suggested by Yoon at [0004].
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577) and further in view of Su (US 2019/0056725).
Regarding claim 13, the modified Liu does not expressly disclose the drone flight control device according to claim 1, wherein each of the first control signal and the second control signal is a serial signal for generating a PWM signal for driving a motor, which in a related art Su teaches (see Su at [0140] which discloses sending the signals in the form of PWM (Pulse Width Modulation) signal to a driving circuit to drive the electrical motors to rotate, to adjust the rotational speed of the rotor wing of the unmanned aerial vehicle and further realize controlling the unmanned aerial vehicle.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to include wherein each of first control signal and the second control signal is a serial signal for generating a PWM signal for driving a motor, as taught by Su.
One would have been motivated to make such a modification to provide controlling signals output to each of the electrical motors to realize controlling of the unmanned aerial vehicle, as suggested by Su at [0140].
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577) and further in view of Lombardini (US 11,079,752).
Regarding claim 17, the modified Liu teaches the first control signal for driving the plurality of propellers (see Liu at [0055] which discloses that for example, the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210. Examiner notes that a moveable object corresponds to propeller.) Liu does not expressly disclose the drone flight control device according to claim 1, wherein a control signal for driving the propeller is a signal for increasing a rotation speed of at least one propeller among the plurality of propellers on an opposite side of the drone with respect to a traveling direction of the drone in a case of forward travel, rearward travel, rightward travel, and leftward travel which, in a related art, Lombardini teaches (see Lombardini at col. 17 lines 42-60, which discloses, for example, that the UAV 300, may change its orientation, speed, altitude, direction of travel, or combinations thereof by changing the rotational speed of one or more of the propellers 315 and that the UAV 300 may hover in a fixed position by adjusting the speed of the propellers 315 to eliminate any pitch, yaw, roll, forward motion, backward motion, or lateral motion. Examiner notes that changing the rotational speed of one or more propellers includes increasing a rotation speed of a propeller on an opposite side with respect to a traveling direction. Examiner notes that changing the rotational speed of one or more propellers would facilitate travel in various directions including travel in the forward, rearward, rightward, and leftward directions. Examiner has shown a teaching based on a broadest reasonable interpretation of the claimed language in light of what is written in the specification.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Liu to include wherein a control signal for driving the propeller is a signal for increasing a rotation speed of a propeller on an opposite side with respect to a traveling direction in a case of forward travel, rearward travel, rightward travel, and leftward travel, as taught by Lombardini.
One would have been motivated to make such a modification to allow UAV to change its orientation, speed, altitude, direction of travel, as suggested by Lombardini, at col. 17, lines 45-48.
Claims 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 2019/0265734) in view of Suzuki et al. (US 2024/0089577) and further in view of Miyoshi (WO 2022107761) (English translation attached).
Regarding claim 18, the modified Liu teaches the first control signal and the second control signal for driving the plurality of propellers (see Liu at [0055] which discloses that the moveable object 102 includes movement mechanisms 114 (e.g., propulsion mechanisms) and that the plural term "movement mechanisms" is used herein for convenience of reference, "movement mechanisms 114" refers to a single movement mechanism (e.g., a single propeller) or multiple movement mechanisms (e.g., multiple rotors); see Liu at [0055] which discloses that the movement mechanisms 114 are coupled to the moveable object 102 using any suitable means, such as support elements (e.g., drive shafts) and/or other actuating elements (e.g., the moveable object actuators 210) and that for example, the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210. Examiner notes that a moveable object corresponds to propeller.) Liu does not expressly disclose the drone flight control device according to claim 1, wherein a control signal for driving the propeller is a signal for increasing a rotation speed of at least one propeller among the plurality of propellers rotating rightward in a case of rightward turning, which in a related art, Miyoshi teaches (see Miyoshi, at page 4 of the English translation, which discloses that the ESC219 controls the rotation speed of the motor according to the instruction from the CPU 211, that by controlling the rotation speed of the propeller, a pressure difference is generated above and below the propeller 220, and lift is generated by this pressure difference, and the unmanned airplane 2 flies, that lift is a force that works upward to push up the unmanned aerial vehicle 2, and that the unmanned aerial vehicle 2 can change the speed and the moving direction by changing the rotation speed of the propeller 220. Miyoshi at page 4 further discloses that by controlling the rotation speed of the propeller 220, the unmanned airplane 2 has hovering (lift and gravity become equal), ascending (the rotation speed of the four motors increases), and descending (the rotation speed of the four motors decreases), forward / backward / left / right movement (the number of rotations of the two propellers opposite to the direction of travel increases and moves in the direction of travel), left turn (the number of rotations of the propeller that rotates to the right increases), and right turn (rotation to the left), and that the number of rotations of the propeller will increase). Examiner maps one of the propellers opposite the direction of travel, such as one of the two propellers opposite to the right movement direction, to the propeller rotating rightward. Examiner interprets the propeller rotating rightward as a propeller that rotates such that the UAV is in a case for rightward turning, such as the one of the two propellers opposite to the right movement direction. Examiner has shown a teaching based on a broadest reasonable interpretation of the claimed language in light of what is written in the specification.)
Regarding claim 19, Liu teaches the first control signal and the second control signal for driving the plurality of propellers (see Liu at [0055] which discloses that the moveable object 102 includes movement mechanisms 114 (e.g., propulsion mechanisms) and that the plural term "movement mechanisms" is used herein for convenience of reference, "movement mechanisms 114" refers to a single movement mechanism (e.g., a single propeller) or multiple movement mechanisms (e.g., multiple rotors); see Liu at [0055] which discloses that the movement mechanisms 114 are coupled to the moveable object 102 using any suitable means, such as support elements (e.g., drive shafts) and/or other actuating elements (e.g., the moveable object actuators 210) and that for example, the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210; see Liu at [0054] which discloses that moveable object actuators (e.g., 210a, 210b) cause movement of movement mechanisms (e.g., 114a, 114b), e.g., in response to received control instructions; see Liu at [0055] which discloses that for example, the processor(s) 202 include an electronic speed controller that provides control signals to a moveable object actuator 210. Examiner notes that moveable object actuators and/or movement mechanisms correspond to propellers.) The modified Liu does not expressly disclose the drone flight control device according to claim 1, wherein a control signal for driving the propeller is a signal for increasing a rotation speed of at least one propeller among the plurality of propellers rotating leftward in a case of leftward turning, which in a related art, Miyoshi teaches (see Miyoshi, at page 4 of the English translation, which discloses that the ESC219 controls the rotation speed of the motor according to the instruction from the CPU 211, that by controlling the rotation speed of the propeller, a pressure difference is generated above and below the propeller 220, and lift is generated by this pressure difference, and the unmanned airplane 2 flies, that lift is a force that works upward to push up the unmanned aerial vehicle 2, and that the unmanned aerial vehicle 2 can change the speed and the moving direction by changing the rotation speed of the propeller 220. Miyoshi at page 4 further discloses that by controlling the rotation speed of the propeller 220, the unmanned airplane 2 has hovering (lift and gravity become equal), ascending (the rotation speed of the four motors increases), and descending (the rotation speed of the four motors decreases).), forward / backward / left / right movement (the number of rotations of the two propellers opposite to the direction of travel increases and moves in the direction of travel), left turn (the number of rotations of the propeller that rotates to the right increases), and right turn (rotation to the left), and that the number of rotations of the propeller will increase). Examiner maps one of the propellers opposite the direction of travel, such as one of the two propellers opposite to the left movement direction, to the propeller rotating leftward. Examiner interprets the propeller rotating leftward as a propeller that rotates such that the UAV is in a case for leftward turning, such as the one of the two propellers opposite to the left movement direction. Examiner has shown a teaching based on a broadest reasonable interpretation of the claimed language in light of what is written in the specification.).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROY RHEE whose telephone number is 313-446-6593. The examiner can normally be reached M-F 8:30 am to 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant may contact the Examiner via telephone or use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson, can be reached on 571-270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, one may visit: https://patentcenter.uspto.gov. In addition, more information about Patent Center may be found at https://www.uspto.gov/patents/apply/patent-center. Should you have questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROY RHEE/Examiner, Art Unit 3664
Prosecution Insights