DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/26/2026 has been entered.
Response to Arguments
Applicant Amendments and Remarks filed on 02/26/2026 in response to the Final office action mailed on 12/02/2025 have been fully considered and are addressed as follows:
Regarding the Claim Rejections under 35 USC § 103: With respect to the previous claim rejections under 35 U.S.C. § 103, Applicant has amended the independent claims and these amendments have changed the scope of the original application. Therefore, the Office has supplied new grounds for rejection attached below in the Non-Final office action and therefore the prior arguments are considered moot.
NON-FINAL OFFICE ACTION
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3, 5-7, 9-16, and 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Claims 1, 11, and 16 recite an abstract idea in the form of mental processes without significantly more.
Regarding eligibility step 1, the claimed invention of claims 1, 11, and 16 falls into at least one of the enumerated categories of apparatuses and processes. Therefore, claims 1, 11, and 16 pass step 1.
Proceeding to eligibility step 2A, the claimed invention of claims 1, 11, and 16 is directed to a judicial exception, such as an abstract idea. If a claim limitation under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the mental process grouping of an abstract idea. The claimed invention of claims 1, 11, and 16 is directed to processes that receive information, generate analysis by generating an image, determining a temperature and ranging information, and generate an evacuation route using generic computer components which can be performed in the human mind, or by a human using a pen and paper. The mere nominal recitation of a generic computer/generic computer components does not take the claim limitation out of the mental processes grouping. Accordingly, the claims recite an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claims 1, 11, and 16 further recites displaying, via a user interface, the image, the temperature, the ranging information, and the evacuation route. The generic computer components in the step are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using the generic computer components. Specifically, the displaying step does not require any dynamic or layered feature. Moreover, the displaying step does not require any interaction between various components being displayed. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea.
Proceeding to eligibility step 2B, claims 1, 11, and 16 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above, using generic computer components to perform displaying the components amounts to no more than mere instructions to apply the exception using the generic computer components. Mere instructions to apply the exception using the generic computer components cannot provide an inventive concept. Therefore, claims 1, 11, and 16 are not patent eligible.
Dependent claims 3, 5-7, 9, 10, 12-15, and 18-20 , when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because the additional recited limitations fail to establish that the claims are not directed to an abstract idea. The additional elements, if any, in the dependent claims are not sufficient to amount to significantly more than the judicial exception for the same reasons as with claims 1, 11, and 16.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5, 6 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Elhossini et al. (US 2022/0262263 A1, hereinafter “Elhossini”) in view of Sequeira et al. (US 2017 /0088261 A1, hereinafter “Sequeira”) further in view of Shin et al. (KR 102577564 B1, hereinafter “Shin”). The rejections below are based on the machine translation of Shin, a copy of which is attached to this Office Action as also indicated in the 892 form.
Regarding claim 1, Elhossini discloses a controller for generating an emergency event spatial analysis (Elhossini at para. [0021]: “an unmanned aerial vehicle (UAV) configured to facilitate search and rescue operations ( e.g., a search for a missing person in the wild for example)”), comprising:
a memory ; and
a processor configured to execute executable instructions stored in the memory to (Elhossini at para. [0025]: “The body 104 may be equipped with at least one logic device, such as a first logic device 120 and a second logic device 122. Each logic device, which may be referred to as an on-board computer, may be implemented as any appropriate logic device (e.g., processing device, microcontroller, processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), memory storage device, memory reader, or other device or combinations of devices) that may be adapted to execute, store, and/or receive appropriate instructions, such as software instructions implementing a control loop for controlling various operations of UAV 100 and/or other elements of a system”):
receive a visual input from a first sensor and (Elhossini at FIGS. 2-3 and para. [0030]: “UAV 100 may include a plurality of navigation cameras supporting a position estimation of the UAV 100”; para. [0033]: “Camera 140 may be configured to capture visible and non-visible images. For example, the camera 140 may be configured to capture visible, infrared, and/or thermal infrared images, among others”);
generate, based on the visual input and the spatial input, an emergency event spatial analysis (Elhossini at para. [0030]: “final position estimation may be performed by the first logic device 120, where first logic device 120 combines all the measurements from GPS module 126, IMU 128, and navigation cameras ( e.g., as processed by second logic device 122)”; para. [0051]: “process 1000 may include analyzing the one or more images to identify a target matching the target image. Block 1014 may include comparing the target image to at least portions of the one or more images of the search area using a template matching algorithm”) including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder (Elhossini at para. [0021]: “an unmanned aerial vehicle (UAV) configured to facilitate search and rescue operations ( e.g., a search for a missing person in the wild for example)”; para. [0026]: “first logic device 120 and/or second logic device 122 may be adapted to store sensor signals, sensor information, and/or operational parameters, over time, for example, and provide such stored data to a user”) by:
generating an image of an object that is radiating thermal energy in the space in which the UAV is located based on the visual input (Elhossini at para. [0033]: “camera 140 may be utilized to search for a target with a unique thermal signature” “both a visible and a thermal representation of a scene (e.g., a search area) may be captured and/or presented to the pilot or another user of the system”);
display, via a user interface, the emergency event spatial analysis (Elhossini at para. [0036]: “the grid path 506 may be overlaid on the search area 502 in the map view 500. The current position of UAV 100 may also be shown in map view 500”; para. [0052]: “process 1000 may include sending one or more notifications for display on the user interface based on a match between the target and the target image. For example, the target position may be provided to the pilot”) including the image of the object (Elhossini at para. [0033]: “camera 140 may be utilized to search for a target with a unique thermal signature” “both a visible and a thermal representation of a scene (e.g., a search area) may be captured and/or presented to the pilot or another user of the system”), the temperature of the object (Elhossini at para. [0038]: “the target image 600 may be provided as a thermal image”; The thermal image provides the temperature of the object),
However, Elhossini does not explicitly state:
a spatial input including time-of-flight distance and/or dimensional data about a space from a second sensor,
determining a temperature of the object that is radiating thermal energy in based on the visual input; and
determining ranging information of the space in which the UAV is located based on the spatial input, wherein determining the ranging information includes determining a distance from the second sensor to the object based on the object radiating the thermal energy in the space in which the UAV is located;
generate, based on the image of the object and the ranging information of the space, an evacuation route from the space; and
display the ranging information of the space, and the evacuation route.
In the same field of endeavor, Sequeira teaches:
a spatial input including time-of-flight distance and/or dimensional data about a space from a second sensor (Sequeira at para. [0028]: “The UAVs 12 have sensors such as ultra-wide band (UWB) transceiver sensors 26 for detecting survivors 40 in the search and rescue area 14” “the UWB transceiver sensors 26 can determine a distance between the UAVs 12 and survivors 40 based on the time difference between transmission of the UWB radio signals 30 and receipt of the reflected UWB radio signals 30R from survivors 40. This distance can be used in determining the survivor's location”),
determining ranging information of the space in which the UAV is located based on the spatial input, wherein determining the ranging information includes determining a distance from the second sensor to the object based on the object radiating the thermal energy in the space in which the UAV is located (Sequeira at para. [0028]: “the UWB transceiver sensors 26 can determine a distance between the UAVs 12 and survivors 40 based on the time difference between transmission of the UWB radio signals 30 and receipt of the reflected UWB radio signals 30R from survivors 40. This distance can be used in determining the survivor's location”; para. [0057]: “The UAV 12 includes an infrared (IR) camera 28 for supplementing detection of survivors 40 and navigation through search and rescue area 14. The IR camera 28 forms an image using infrared radiation” “the IR camera 28 is a thermal imaging camera used to detect heat signatures of survivors 40 and hotspots of fires”);
generate, based on the image of the object and the ranging information of the space, an evacuation route from the space (Sequeira at para. [0069]: “the mobile command center 16 determines an ingress/egress path (i.e., safe path info 84) to the survivor 40 and identifies hazardous conditions in the determined path (i.e., hazards in path 88). Specifically, the mobile command center 16 can use information received from the UAVs 12 to locate and identify hazardous conditions in the determined path to the survivor”); and
display the ranging information of the space, and the evacuation route (Sequeira at para. [0070]: “The mobile command center 16 forewarns first responders 82 of the search and rescue area landscape and survivor health conditions via a display of the configurator computer workstation 18. For example, as illustrated in FIG. 3, the first responder 82 can view survival vital signs telemetry data 62, a 3d location map 86, safe path info 84, and hazards in path 88”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini by adding the ranging information of Sequeira with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
However, Elhossini in view of Sequeira does not explicitly state:
determining a temperature of the object that is radiating thermal energy in based on the visual input.
Nevertheless, Elhossini at least suggests capturing and displaying thermal images to search for a target with a unique thermal signature (Elhossini at para. [0033]), which suggests that the temperature of the object is determined in some way.
In the same field of endeavor, Shin teaches:
determining a temperature of the object that is radiating thermal energy in based on the visual input (Shin at para. [0051]: “the temperature of the object is measured by detecting infrared rays emitted from the object using the thermal imaging unit (S110). The thermal imaging unit 110 detects infrared rays emitted from the object and its surroundings, converts the infrared detection amounts into temperature values, and then uses the temperature values to display the object 50 and its surroundings in different colors depending on the temperature”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini in view of Sequeira by adding determining the temperature of Shin with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira further in view of Shin is to improve the accuracy of temperature measurement.
Regarding claim 3, Elhossini in view of Sequeira further in view of Shin teaches the controller of claim 1.
Elhossini further discloses wherein the first sensor is an infrared (IR) camera that generates and transmits the visual input (Elhossini at para. [0033]: “Camera 140 may be configured to capture visible and non-visible images. For example, the camera 140 may be configured to capture visible, infrared, and/or thermal infrared images, among others”).
Regarding claim 5, Elhossini in view of Sequeira further in view of Shin teaches the controller of claim 1.
Sequeira further teaches wherein the ranging information includes dimensions of the space in which the UAV is located (Sequeira at para. [0043]: “The mobile command center 16 includes a 3-dimensional site mapping system 53 for generating a map of the search and rescue area 14. The 3-dimensional site mapping system 53 receives the mobile command center location from the GPS receiver 33 and/or the UAV location from the LPS command center module 34” “The 3-dimensional site mapping system 53 can generate a 3-dimensional contour map of the region relative to the UAV location”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini further in view of Sequeira further in view of Shin by adding the dimensions of the space of Sequeira with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira further in view of Shin is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
Regarding claim 6, Elhossini in view of Sequeira further in view of Shin teaches the controller of claim 1.
Sequeira further teaches wherein the ranging information includes distances from the second sensor to a plurality of objects in the space in which the UAV is located (Sequeira at para. [0028]: “the UWB transceiver sensors 26 can determine a distance between the UAVs 12 and survivors 40 based on the time difference between transmission of the UWB radio signals 30 and receipt of the reflected UWB radio signals 30R from survivors 40. This distance can be used in determining the survivor's location”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini in view of Sequeira further in view of Shin by adding the distances of Sequeira with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira further in view of Shin is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
Regarding claim 9, Elhossini in view of Sequeira further in view of Shin teaches the controller of claim 1.
Elhossini further discloses wherein the processor is configured to execute the instructions to display the emergency event spatial analysis by displaying (Elhossini et al. at para. [0029]: “navigational or environmental sensors providing measurements and/or other sensor signals that can be displayed to a user and/or used to provide operational control of UAV 100”).
However, Elhossini does not explicitly state displaying dimensions of a space in which the UAV is located on the user interface.
Sequeira further teaches displaying dimensions of the space in which the UAV is located on the user interface (Sequeira at para. [0043]: “The mobile command center 16 includes a 3-dimensional site mapping system 53 for generating a map of the search and rescue area 14. The 3-dimensional site mapping system 53 receives the mobile command center location from the GPS receiver 33 and/or the UAV location from the LPS command center module 34” “The 3-dimensional site mapping system 53 can generate a 3-dimensional contour map of the region relative to the UAV location”; para. [0070]: “The mobile command center 16 forewarns first responders 82 of the search and rescue area landscape and survivor health conditions via a display of the configurator computer workstation 18”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini in view of Sequeira further in view of Shin by adding the dimensions of the space of Sequeira with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira further in view of Shin is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Elhossini in view of Sequeira further in view of Shin and Gohl et al. (US 10,269,257 B1, hereinafter “Gohl”).
Regarding claim 7, Elhossini in view of Sequeira further in view of Shin teaches the controller of claim 1.
However, Elhossini in view of Sequeira further in view of Shin does not explicitly state wherein the second sensor is a light detection and ranging (LIDAR) sensor that generates and transmits the spatial input.
In the same field of endeavor, Gohl teaches wherein the second sensor is a light detection and ranging (LIDAR) sensor that generates and transmits the spatial input (Gohl at col. 23, ln. 8-10: “Depth information component 820 may obtain depth information from one or more sensors carried by the unmanned aerial vehicle”; col. 23, ln. 48-50: “distance sensors may include one or more of image sensors, infrared distance sensors, laser rangefinders, Lidar”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini in view of Sequeira further in view of Shin by adding the LIDAR sensor as taught by Gohl with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira further in view of Shin and Gohl is to provide improved perception of the surrounding environment.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Elhossini in view of Sequeira further in view of Shin and Zhou et al. (US 2021/0012520 A1, hereinafter “Zhou”).
Regarding claim 10, Elhossini in view of Sequeira further in view of Shin teaches the controller of claim 9.
However, Elhossini in view of Sequeira further in view of Shin does not explicitly state wherein the processor is configured to execute the instructions to display the emergency event spatial analysis by displaying a distance to a point of interest in the space in which the UAV is located on the user interface.
In the same field of endeavor, Zhou teaches wherein the processor is configured to execute the instructions to display the emergency event spatial analysis by displaying a distance to a point of interest in the space in which the UAV is located on the user interface (Zhou at para. [0069]: “the remote control may display captured images on the graphical user interface and mark the distance on an image currently displayed on the graphical user interface. Further, the image currently displayed on the graphical user interface may be the initial image with the identified to-be-measured object, or a live feed image containing the to-be-measured object”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini in view of Sequeira further in view of Shin by adding displaying the distance as taught by Zhou with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Sequeira further in view of Shin and Zhou is to provide improved perception of the surrounding environment.
Claims 11-13, 15, 16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Elhossini in view of Gohl further in view of Sequeira and Shin.
Regarding claim 11, Elhossini discloses a method for generating an emergency event spatial analysis (Elhossini at para. [0021]: “an unmanned aerial vehicle (UAV) configured to facilitate search and rescue operations ( e.g., a search for a missing person in the wild for example)”), comprising:
receiving, by a controller, a visual input from an infrared (IR) camera (Elhossini at para. [0033]: “Camera 140 may be configured to capture visible and non-visible images. For example, the camera 140 may be configured to capture visible, infrared, and/or thermal infrared images, among others”) and unmanned aerial vehicle (UAV) (Elhossini at para. [0030]: “UAV 100 may include a plurality of navigation cameras”);
generating, by the controller,
generating, based on the visual input, an image of an object that is radiating energy in a space in which the UAV is located (Elhossini at para. [0033]: “camera 140 may be utilized to search for a target with a unique thermal signature” “both a visible and a thermal representation of a scene (e.g., a search area) may be captured and/or presented to the pilot or another user of the system”);
displaying, via a user interface, the emergency event spatial analysis (Elhossini at para. [0036]: “the grid path 506 may be overlaid on the search area 502 in the map view 500. The current position of UAV 100 may also be shown in map view 500”; para. [0052]: “process 1000 may include sending one or more notifications for display on the user interface based on a match between the target and the target image. For example, the target position may be provided to the pilot”) including the image of the object (Elhossini at para. [0033]: “camera 140 may be utilized to search for a target with a unique thermal signature” “both a visible and a thermal representation of a scene (e.g., a search area) may be captured and/or presented to the pilot or another user of the system”), the temperature of the object (Elhossini at para. [0038]: “the target image 600 may be provided as a thermal image”; The thermal image provides the temperature of the object),
However, Elhossini does not explicitly state:
receiving a spatial input including time-of-flight distance and/or dimensional data about a space from a light detection and ranging (LIDAR) sensor,
the LIDAR sensor are located on an unmanned aerial vehicle (UAV),
generating an emergency event spatial analysis including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder,
determining a temperature of the object that is radiating the energy based on the visual input;
determining, based on the spatial input, ranging information of the space in which the UAV is located, wherein determining the ranging information includes determining a distance from the LIDAR sensor to the object based on the object radiating the energy in the space in which the UAV is located,
generating, by the controller, an evacuation route from the space based on the image of the object and the ranging information of the space, and
displaying the ranging information of the space, and the evacuation route.
In the same field of endeavor, Gohl teaches a spatial input including time-of-flight distance and/or dimensional data about a space from a light detection and ranging (LIDAR) sensor,
the LIDAR sensor are located on an unmanned aerial vehicle (UAV),
(Gohl at col. 1, ln. 31-36: “system configured to detect a moving object may include one or more of an image sensor, a motion and orientation sensor, one or more hardware-implemented processors and/or other components. Some or all of the system may be installed in a vehicle and/or be otherwise coupled with the vehicle”; col. 23, ln. 8-10: “Depth information component 820 may obtain depth information from one or more sensors carried by the unmanned aerial vehicle”; col. 23, ln. 48-50: “distance sensors may include one or more of image sensors, infrared distance sensors, laser rangefinders, Lidar”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the controller of Elhossini by adding the LIDAR sensor of Gohl with a reasonable expectation of success. The motivation to modify the controller of Elhossini in view of Gohl is to provide improved perception of the surrounding environment.
However, Elhossini in view of Gohl does not explicitly state generating an emergency event spatial analysis including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder,
determining a temperature of the object that is radiating the energy based on the visual input;
determining, based on the spatial input, ranging information of the space in which the UAV is located, wherein determining the ranging information includes determining a distance from the LIDAR sensor to the object based on the object radiating the energy in the space in which the UAV is located,
generating, by the controller, an evacuation route from the space based on the image of the object and the ranging information of the space, and
displaying the ranging information of the space, and the evacuation route.
In the same field of endeavor, Sequeira teaches generating an emergency event spatial analysis including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder (Sequeira at para. [0070]: “the first responder 82 can view survival vital signs telemetry data 62, a 3d location map 86, safe path info 84, and hazards in path 88. The first responder 82 can use this information in making informed decisions, assessing the response priority within the search and rescue area 14, and providing timely medical attention to the affected survivor 40”),
determining, based on the spatial input, ranging information of the space in which the UAV is located, wherein determining the ranging information includes determining a distance from the LIDAR sensor to the object based on the object radiating the energy in the space in which the UAV is located (Sequeira at para. [0028]: “the UWB transceiver sensors 26 can determine a distance between the UAVs 12 and survivors 40 based on the time difference between transmission of the UWB radio signals 30 and receipt of the reflected UWB radio signals 30R from survivors 40. This distance can be used in determining the survivor's location”; para. [0057]: “The UAV 12 includes an infrared (IR) camera 28 for supplementing detection of survivors 40 and navigation through search and rescue area 14. The IR camera 28 forms an image using infrared radiation” “the IR camera 28 is a thermal imaging camera used to detect heat signatures of survivors 40 and hotspots of fires”),
generating, by the controller, an evacuation route from the space based on the image of the object and the ranging information of the space (Sequeira at para. [0069]: “the mobile command center 16 determines an ingress/egress path (i.e., safe path info 84) to the survivor 40 and identifies hazardous conditions in the determined path (i.e., hazards in path 88). Specifically, the mobile command center 16 can use information received from the UAVs 12 to locate and identify hazardous conditions in the determined path to the survivor”), and
displaying the ranging information of the space, and the evacuation route (Sequeira at para. [0070]: “The mobile command center 16 forewarns first responders 82 of the search and rescue area landscape and survivor health conditions via a display of the configurator computer workstation 18. For example, as illustrated in FIG. 3, the first responder 82 can view survival vital signs telemetry data 62, a 3d location map 86, safe path info 84, and hazards in path 88”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl by adding the ranging information of Sequeira with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
However, Elhossini in view of Gohl further in view of Sequeira does not explicitly state:
determining a temperature of the object that is radiating the energy based on the visual input.
Nevertheless, Elhossini at least suggests capturing and displaying thermal images to search for a target with a unique thermal signature (Elhossini at para. [0033]), which suggests that the temperature of the object is determined in some way.
In the same field of endeavor, Shin teaches:
determining a temperature of the object that is radiating the energy based on the visual input (Shin at para. [0051]: “the temperature of the object is measured by detecting infrared rays emitted from the object using the thermal imaging unit (S110). The thermal imaging unit 110 detects infrared rays emitted from the object and its surroundings, converts the infrared detection amounts into temperature values, and then uses the temperature values to display the object 50 and its surroundings in different colors depending on the temperature”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl further in view of Sequeira by adding determining the temperature of Shin with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin is to improve the accuracy of temperature measurement.
Regarding claim 12, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the method of claim 11.
Gohl further teaches wherein generating the emergency event spatial analysis further includes generating a three-dimensional (3D) image of the space based on the spatial input from the LIDAR sensor (Gohl at col. 23, ln. 6-8: “Depth information component 820 may be configured to obtain depth information for an environment around the unmanned aerial vehicle and/or other information. Depth information component 820 may obtain depth information from one or more sensors carried by the unmanned aerial vehicle”; col. 23, ln. 48-50: “distance sensors may include one or more of image sensors, infrared distance sensors, laser rangefinders, Lidar”; col. 24, ln. 36-38: “to insert disparity data into a spherical depth map, the disparity data may be projected into 3D points”; col. 29, ln. 37-40: “A depth image may be transformed into a three-dimensional point-cloud. A statistical voxel map, an Octomap, a spherical depth map, and/or other depth map framework may be used”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin by adding the three-dimensional image as taught by Gohl with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin is to provide improved perception of the surrounding environment.
Regarding claim 13, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the method of claim 12.
Gohl further teaches wherein:
the ranging information includes dimensions of the space (Gohl at col. 23, ln. 8-10: “Depth information component 820 may obtain depth information from one or more sensors carried by the unmanned aerial vehicle”; at col 23, ln. 64-67: “The spherical depth map may represent distances to closest surfaces of the environment around the unmanned aerial vehicle as a function of longitude
angles and latitude angles ( e.g., around UAV 900, etc.)”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin by adding the dimensions as taught by Gohl with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin is to provide improved perception of the surrounding environment.
Sequeira further teaches the method further includes displaying the dimensions of the space on the 3D image (Sequeira at para. [0043]: “The mobile command center 16 includes a 3-dimensional site mapping system 53 for generating a map of the search and rescue area 14. The 3-dimensional site mapping system 53 receives the mobile command center location from the GPS receiver 33 and/or the UAV location from the LPS command center module 34” “The 3-dimensional site mapping system 53 can generate a 3-dimensional contour map of the region relative to the UAV location”; para. [0070]: “The mobile command center 16 forewarns first responders 82 of the search and rescue area landscape and survivor health conditions via a display of the configurator computer workstation 18”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin by adding displaying the dimensions of Sequeira with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
Regarding claim 15, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the method of claim 12.
Sequeira further teaches wherein the method further includes generating the evacuation route from the space on the 3D image (Sequeira at para. [0043]: “the 3-dimensional site mapping system 53 can generate a 3-dimensional contour map that marks the UAV location relative to the mobile command center location as well as the survivor location. The 3-dimensional site mapping system 53 can generate an ingress/egress path from the mobile command center 16 to the survivor”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin by adding the 3D image of Sequeira with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
Regarding claim 16, Elhossini discloses a system for generating an emergency event spatial analysis, including:
an unmanned aerial vehicle (UAV) (Elhossini at para. [0021]: “an unmanned aerial vehicle (UAV) configured to facilitate search and rescue operations ( e.g., a search for a missing person in the wild for example)”), the UAV including an infrared (IR) camera and (Elhossini et al. at para. [0033]: “Camera 140 may be configured to capture visible and non-visible images. For example, the camera 140 may be configured to capture visible, infrared, and/or thermal infrared images, among others”); and
a mobile device having a user interface and a controller (Elhossini at para. [0035]: “The pilot may have control of the UAV 100 and access to UAV data using user interface 400. For example, the user interface 400 may be connected to the UAV 100 using a wireless link”), wherein the controller is configured to:
wirelessly receive a visual input from the IR camera and a spatial input (Elhossini at FIGS. 2-3 and para. [0030]: “UAV 100 may include a plurality of navigation cameras supporting a position estimation of the UAV 100”; para. [0033]: “the camera 140 may be configured to capture visible, infrared, and/or thermal infrared images, among others”);
generate
generating, based on the visual input, an image of an object that is radiating energy in a space in which the UAV is located (Elhossini et al. at para. [0033]: “camera 140 may be utilized to search for a target with a unique thermal signature” “both a visible and a thermal representation of a scene (e.g., a search area) may be captured and/or presented to the pilot or another user of the system”);
determining a temperature of the object that is radiating the energy based on the visual input; and
cause the emergency event spatial analysis to be displayed by the user interface (Elhossini at para. [0036]: “the grid path 506 may be overlaid on the search area 502 in the map view 500. The current position of UAV 100 may also be shown in map view 500”; para. [0052]: “process 1000 may include sending one or more notifications for display on the user interface based on a match between the target and the target image. For example, the target position may be provided to the pilot”), the emergency event spatial analysis including the image of the object (Elhossini at para. [0033]: “camera 140 may be utilized to search for a target with a unique thermal signature” “both a visible and a thermal representation of a scene (e.g., a search area) may be captured and/or presented to the pilot or another user of the system”), the temperature of the object (Elhossini at para. [0038]: “the target image 600 may be provided as a thermal image”; The thermal image provides the temperature of the object),
However, Elhossini does not explicitly state:
a light detection and ranging (LIDAR) sensor,
a spatial input including time-of-flight distance and/or dimensional data about a space from the LIDAR sensor,
an emergency event spatial analysis including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder,
determining a temperature of the object that is radiating the energy based on the visual input,
determining, based on the spatial input, ranging information of the space in which the UAV is located, wherein determining the ranging information includes determining a distance from the LIDAR sensor to the object based on the object radiating the energy in the space in which the UAV is located,
generate an evacuation route form the space based on the image of the object and the ranging information of the space, and
the ranging information of the space, and the evacuation route.
In the same field of endeavor, Gohl teaches
a light detection and ranging (LIDAR) sensor (Gohl at col. 23, ln. 48-50: “distance sensors may include one or more of image sensors, infrared distance sensors, laser rangefinders, Lidar”), and
a spatial input including time-of-flight distance and/or dimensional data about a space from the LIDAR sensor (Gohl at col. 1, ln. 31-36: “system configured to detect a moving object may include one or more of an image sensor, a motion and orientation sensor, one or more hardware-implemented processors and/or other components. Some or all of the system may be installed in a vehicle and/or be otherwise coupled with the vehicle”; col. 23, ln. 8-10: “Depth information component 820 may obtain depth information from one or more sensors carried by the unmanned aerial vehicle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Elhossini by adding the LIDAR sensor of Gohl with a reasonable expectation of success. The motivation to modify the system of Elhossini in view of Gohl is to provide improved perception of the surrounding environment.
However, Elhossini in view of Gohl does not explicitly state:
an emergency event spatial analysis including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder,
determining a temperature of the object that is radiating the energy based on the visual input,
determining, based on the spatial input, ranging information of the space in which the UAV is located, wherein determining the ranging information includes determining a distance from the LIDAR sensor to the object based on the object radiating the energy in the space in which the UAV is located,
generate an evacuation route form the space based on the image of the object and the ranging information of the space, and
the ranging information of the space, and the evacuation route.
In the same field of endeavor, Sequeira teaches an emergency event spatial analysis including a collection of evaluated and focused information pertaining to the emergency event requiring a response from a first responder (Sequeira at para. [0070]: “the first responder 82 can view survival vital signs telemetry data 62, a 3d location map 86, safe path info 84, and hazards in path 88. The first responder 82 can use this information in making informed decisions, assessing the response priority within the search and rescue area 14, and providing timely medical attention to the affected survivor 40”),
determining, based on the spatial input, ranging information of the space in which the UAV is located, wherein determining the ranging information includes determining a distance from the LIDAR sensor to the object based on the object radiating the energy in the space in which the UAV is located (Sequeira at para. [0028]: “the UWB transceiver sensors 26 can determine a distance between the UAVs 12 and survivors 40 based on the time difference between transmission of the UWB radio signals 30 and receipt of the reflected UWB radio signals 30R from survivors 40. This distance can be used in determining the survivor's location”; para. [0057]: “The UAV 12 includes an infrared (IR) camera 28 for supplementing detection of survivors 40 and navigation through search and rescue area 14. The IR camera 28 forms an image using infrared radiation” “the IR camera 28 is a thermal imaging camera used to detect heat signatures of survivors 40 and hotspots of fires”),
generate an evacuation route form the space based on the image of the object and the ranging information of the space (Sequeira at para. [0069]: “the mobile command center 16 determines an ingress/egress path (i.e., safe path info 84) to the survivor 40 and identifies hazardous conditions in the determined path (i.e., hazards in path 88). Specifically, the mobile command center 16 can use information received from the UAVs 12 to locate and identify hazardous conditions in the determined path to the survivor”), and
the ranging information of the space, and the evacuation route (Sequeira at para. [0070]: “The mobile command center 16 forewarns first responders 82 of the search and rescue area landscape and survivor health conditions via a display of the configurator computer workstation 18. For example, as illustrated in FIG. 3, the first responder 82 can view survival vital signs telemetry data 62, a 3d location map 86, safe path info 84, and hazards in path 88”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Elhossini in view of Gohl by adding the ranging information of Sequeira with a reasonable expectation of success. The motivation to modify the system of Elhossini in view of Gohl further in view of Sequeira is to provide a solution that speeds up search and rescue operations to survivors while reducing or mitigating danger to first responders and survivors.
However, Elhossini in view of Gohl further in view of Sequeira does not explicitly state:
determining a temperature of the object that is radiating the energy based on the visual input.
Nevertheless, Elhossini at least suggests capturing and displaying thermal images to search for a target with a unique thermal signature (Elhossini at para. [0033]), which suggests that the temperature of the object is determined in some way.
In the same field of endeavor, Shin teaches:
determining a temperature of the object that is radiating the energy based on the visual input (Shin at para. [0051]: “the temperature of the object is measured by detecting infrared rays emitted from the object using the thermal imaging unit (S110). The thermal imaging unit 110 detects infrared rays emitted from the object and its surroundings, converts the infrared detection amounts into temperature values, and then uses the temperature values to display the object 50 and its surroundings in different colors depending on the temperature”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Elhossini in view of Gohl further in view of Sequeira by adding determining the temperature of Shin with a reasonable expectation of success. The motivation to modify the system of Elhossini in view of Gohl further in view of Sequeira and Shin is to improve the accuracy of temperature measurement.
Regarding claim 18, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the system of claim 16.
Elhossini further discloses wherein the controller is further configured to determine, based on the image of the object, whether the object is a human (para. [0036]: “The UAV 100 may fly the grid path 506 in search for a target 514 ( e.g., a person, vehicle, landmark, etc.)”; para. [0039]: “FIG. 7 illustrates a diagram of the UAY 100 identifying the target 514” “During the ground sweep, images are captured by the camera 140 and analyzed in real time to find a match for target 514. For instance, visible and/or thermal indicators (e.g., signatures) of the captured images may be compared to the target image 600 to determine a match”; FIG. 7 shows that the target 514 is a human).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Elhossini in view of Gohl further in view of Sequeira, Shin, and Fujimoto (JP 2007025890 A). The rejections below are based on the machine translation of Fujimoto.
Regarding claim 14, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the method of claim 12.
However, Elhossini in view of Gohl further in view of Sequeira and Shin does not explicitly state wherein:
the method further includes displaying the distance to the object on the 3D image.
In the same field of endeavor, Fujimoto teaches wherein:
the method further includes displaying the distance to the object on the 3D image (Fujimoto at para. [0025]: “when the map is displayed as a three-dimensional object, it has been described that the numerical correlation information such as distance, time, and fee related to the map is obtained and drawn as a scale object. Not only a map but also other three-dimensional objects. Also, the numerical correlation information is not limited to distance, time, fee, etc., as long as it represents a scale object as supplementary information related to the target three-dimensional object. Furthermore, the scale object may have a shape other than that shown in the above embodiment”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Elhossini in view of Gohl further in view of Sequeira and Shin by adding displaying the distance on the 3D image as taught by Fujimoto with a reasonable expectation of success. The motivation to modify the method of Elhossini in view of Gohl further in view of Sequeira, Shin, and Fujimoto is to provide improved perception of the surrounding environment.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Elhossini in view of Gohl further in view of Sequeira, Shin, and Ruedin (US 2008/0159591 A1).
Regarding claim 19, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the system of claim 18.
However, Elhossini in view of Gohl further in view of Sequeira and Shin does not explicitly state wherein the controller is configured to cause, in response to the object being determined to be human, the image of the object to be colored differently from other objects in the space.
In the same field of endeavor, Ruedin teaches wherein the controller is configured to cause, in response to the object being determined to be human, the image of the object to be colored differently from other objects in the space (Ruedin at para. [0063]: “As shown in FIG. 12, the algorithm may determine that object 1210 is "likely human." Based on this outcome the outline around person 1210 may be turned red. In addition, the algorithm may determine that objects 1211 and 1215 are "possible humans" and turn the outlines around each of these objects yellow. Moreover, the algorithm may determine that objects 1213 and 1217 are "unlikely human" and turn the outlines around each of those objects in image 1208 blue”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Elhossini in view of Gohl further in view of Sequeira and Shin by adding the object to be colored differently as taught by Ruedin with a reasonable expectation of success. The motivation to modify the system of Elhossini in view of Gohl further in view of Sequeira, Shin, and Ruedin is to provide intuitive assist for determining a human.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Elhossini in view of Gohl further in view of Sequeira, Shin, and Lulue et al. (US 2013/0208001 A1, hereinafter “Lulue”).
Regarding claim 20, Elhossini in view of Gohl further in view of Sequeira and Shin teaches the system of claim 16.
However, Elhossini in view of Gohl further in view of Sequeira and Shin does not explicitly state wherein the controller is configured to cause the emergency event spatial analysis to be displayed on user interfaces a plurality of mobile devices.
In the same field of endeavor, Lulue teaches wherein the controller is configured to cause the emergency event spatial analysis to be displayed on user interfaces of a plurality of mobile devices (Lulue at para. [0031]: “FIG. 1 shows a plurality 2 of C&C units, of which C&C units 4a-4e are representative, interconnected to each other in such a way as to enable two-way digital data links 6 between each C&C unit 4 of the group and the rest of the C&C 4 units of the group”; para. [0032]: “The digital reference map is constantly displayed and updated on a suitable display device in both the remote user's location and in at least one of the C&C units”; para. [0051]: “In tactical and emergency management situation, views are needed to explicitly support end-user (watch-stander) situational awareness (SA) and critical task lifecycle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Elhossini in view of Gohl further in view of Sequeira and Shin by adding the plurality of mobile devices as taught by Lulue with a reasonable expectation of success. The motivation to modify the system of Elhossini in view of Gohl further in view of Sequeira, Shin, and Lulue is to provide visual assistance for an emergency response.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and can be found in the attached PTO-892 form.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JISUN CHOI whose telephone number is (571)270-0710. The examiner can normally be reached Mon-Fri, 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Browne can be reached at (571)270-0151. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JISUN CHOI/Examiner, Art Unit 3666
/SCOTT A BROWNE/Supervisory Patent Examiner, Art Unit 3666
Prosecution Insights