DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
Figures 1 and 2 are objected to as depicting a block diagram without “readily identifiable” descriptors of each block, as required by 37 CFR 1.84(n). Rule 84(n) requires “labeled representations” of graphical symbols, such as blocks; and any that are “not universally recognized may be used, subject to approval by the Office, if they are not likely to be confused with existing conventional symbols, and if they are readily identifiable.” In the case of Figures 1 and 2, the blocks are not readily identifiable per se and therefore require the insertion of text that identifies the function of that block. That is, each vacant block should be provided with a corresponding label identifying its function or purpose.
Claim Interpretation
Paragraph 13 of the specification states, “The edge computing system may comprise a system at the edge of a network (that comprises the edge computing system and the remote component), i.e., it may be the most peripheral computing unit of a network”. Thus, “edge computing” is interpreted as a distributed computing model that processes data near the source of generation.
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “data processing unit” and “data storage unit” in claims 16, 18, 19-21, 25, and 34.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 16-29 and 32-35 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 16, 25, 32, 33, and 35 recite, “particularly”, such that one of ordinary skill in the art cannot know whether limitations following “particularly” are required in claim interpretation. Thus, the metes and bounds of the claims cannot be ascertained. For claim interpretation purposes, the limitations following “particularly” are taken to be optional.
Claims 16, 17, and 35 recite, “preferably”, such that one of ordinary skill in the art cannot know whether limitations following “preferably” are required in claim interpretation. Thus, the metes and bounds of the claims cannot be ascertained. For claim interpretation purposes, the limitations following “preferably” are taken to be optional.
Claim 25 recites the limitation "The aerial vehicle" in line 1. There is insufficient antecedent basis for this limitation in the claim. One of ordinary skill in the art cannot know which aerial vehicle is being referred to. Thus, the metes and bounds of the claims cannot be ascertained.
Listed depending claims do not remedy these deficiencies.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 25 and 26 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Claim 25 recites, “edge computing system, particularly the data processing unit thereof, is configured to receive a data stream from a sensor”, which is redundant relative to independent claim 16 from which claim 25 depends. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Depending claim 26 does not remedy this deficiency.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 16-35 are rejected under 35 U.S.C. 101 because the claimed invention is directed to mental processing abstract idea without significantly more.
Claim(s) 30 recite(s) “generating a status data element based on an image of an area… wherein the image comprises an aerial image of the area”, which can reasonably be interpreted as a human observer viewing a displayed aerial image of an area and mentally generating a status data element based on the image, via visual perception, wherein the status data element is information about the status of the image.
This judicial exception is not integrated into a practical application because additional elements of “sending the status data element to a remote component” are generically recited insignificant extra-solution activity of data outputting.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because additional elements of “sending the status data element to a remote component” are insignificant extra-solution activity of data outputting.
Depending claim 31 does not remedy this deficiency because it further recites, “generating the status data element comprises assigning a position to at least one pixel in the image”, which can reasonably be interpreted as a human observer viewing a displayed aerial image of an area and mentally assigning a position to a pixel point in the displayed image.
Claim(s) 16 recite(s) “generate a status data element based on an image of an area… wherein the image comprises an aerial image of the area”, which can reasonably be interpreted as a human observer viewing a displayed aerial image of an area and mentally generating a status data element based on the image, via visual perception, wherein the status data element is information about the status of the image.
This judicial exception is not integrated into a practical application because additional elements of:
“An edge computing system comprising a data processing unit, the data processing unit configured to” are generically recited computer elements that do not add a meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea on a computer;
“send the status data element to a remote component” are generically recited insignificant extra-solution activity of data outputting;
“wherein the edge computing system, particularly (the following limitations are not required) the data processing unit thereof, is configured to” are generically recited computer elements that do not add a meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea on a computer;
“receive a data stream from a sensor” are generically recited insignificant extra-solution activity of data gathering;
“a time interval between receiving the data stream and generating the status data element is less than 30 minutes, preferably (the following limitations are not required) less than 10 minutes, further preferably (the following limitations are not required) less than 1 minute” are generally linking the use of the judicial exception to a particular technological environment or field of use in that the “time interval” limitations are not recited as being a technical mechanism; i.e., the “time interval” related limitations do not specify how the speed is achieved and the improvement to the technology.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because additional elements of:
“An edge computing system comprising a data processing unit, the data processing unit configured to” are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f);
“send the status data element to a remote component” are insignificant extra-solution activity of data outputting;
“wherein the edge computing system, particularly (the following limitations are not required) the data processing unit thereof, is configured to” are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f);
“receive a data stream from a sensor” are insignificant extra-solution activity of data gathering;
“a time interval between receiving the data stream and generating the status data element is less than 30 minutes, preferably (the following limitations are not required) less than 10 minutes, further preferably (the following limitations are not required) less than 1 minute” are generally linking the use of the judicial exception to a particular technological environment or field of use in that the “time interval” limitations are not recited as being a technical mechanism; i.e., the “time interval” related limitations do not specify how the speed is achieved and the improvement to the technology.
Depending claims 17-29 and 32-35 do not remedy these deficiencies:
Claim 17 further recites limitations that are generally linking the use of the judicial exception to a particular technological environment or field of use in that the “time interval” limitations are not recited as being a technical mechanism; i.e., the “time interval” related limitations do not specify how the speed is achieved and the improvement to the technology;
Claim 18 further recites limitations that are generically recited and are well-understood, routine, conventional;
Claim 19 further recites limitations that are generically recited and are well-understood, routine, conventional and comprise insignificant extra-solution activity of data outputting;
Claim 20 further recites limitations that are a mental process abstract idea and generically recited computer elements that are mere instructions to implement an abstract idea on a computer (as stated above);
Claim 21 further recites limitations that are generically recited computer elements that are mere instructions to implement an abstract idea on a computer (as stated above);
Claim 22 further recites limitations that are a mental process abstract idea and generically recited computer elements that are mere instructions to implement an abstract idea on a computer (as stated above);
Claim 23 further recites limitations that are a mental process abstract idea and generically recited computer elements that are mere instructions to implement an abstract idea on a computer (as stated above);
Claim 24 further recites limitations that are insignificant extra-solution activity of data gathering;
Claim 25 further recites limitations that are insignificant extra-solution activity of data gathering and generically recited computer elements that are mere instructions to implement an abstract idea on a computer (as stated above);
Claim 26 further recites limitations that are insignificant extra-solution activity of data gathering;
Claim 27 further recites limitations that are a mental process abstract idea and generically recited computer elements that are mere instructions to implement an abstract idea on a computer (as stated above);
Claim 28 further recites limitations that are insignificant extra-solution activity of data gathering;
Claim 29 further recites limitations that are generally linking the use of the judicial exception to a particular technological environment;
Claim 32 further recites limitations that are generally linking the use of the judicial exception to a particular technological environment, insignificant extra-solution activity of data gathering, and insignificant extra-solution activity of data outputting.
Claim 33 further recites limitations that are generally linking the use of the judicial exception to a particular technological environment;
Claim 34 further recites limitations that are generally linking the use of the judicial exception to a particular technological environment and further recites limitations that are generically recited and are well-understood, routine, conventional; and
Claim 35 further recites limitations that are generally linking the use of the judicial exception to a particular technological environment or field of use in that the “time interval” limitations are not recited as being a technical mechanism; i.e., the “time interval” related limitations do not specify how the speed is achieved and the improvement to the technology.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 16-20, 25-27, and 30-35 are rejected under 35 U.S.C. 102(a)(1) and/or 102(a)(2) as being anticipated by US 20220055749 A1 (Stein).
As per claim 16, Stein teaches an edge computing system comprising a data processing unit (Stein:
para 11 (as referenced for claim 17 below): “data received from a drone”
para 13: “the invention relates to a UAV comprising, in various embodiments, a flight package; a navigation system; an image-acquisition device; a communication facility; a computer memory; and a computer including a processor and electronically stored instructions, executable by the processor, for using data received from the image-acquisition device as input to a predictor that has been computationally trained to identify and classify objects appearing in images acquired by the image-acquisition device during flight.”;
para 14: “the predictor is a neural network. The UAV may include a database of actions, with the computer configured to select and cause execution of an action from the database in response to a detected object classified by the predictor.”;
para 42 and Fig. 7 (both shown below);
Figs. 1-3 (shown below): 118;
PNG
media_image1.png
545
1323
media_image1.png
Greyscale
para 57: “If the real-time neural network 825 resident on the drone 118 is also a CNN ("RT-CNN"), the HAPSNN 800 may push these weights to the RT-CNN, which receives and loads them. That is, the HP-CNN and RT-CNN may be identical or substantially similar so that CNN weights generated for the HP-CNN may be propagated across a fleet of drones”;
“[0060] With reference to FIG. 10, an antenna 1000 may have a crack 1005, which is identified by a DINN-equipped drone.”
: note that the onboard computing of the drone is an edge computing system because it is on the periphery of a distributed, networked computing system), the data processing unit configured to
generate a status data element based on an image of an area (Stein:
para 11 (as referenced for claim 17 below): “data received from a drone”;
para 15 (as referenced below): “detected but unclassified object”;
para 16: “the invention pertains to a method of inspecting an asset using a UAV. In various embodiments, the method comprises the steps of acquiring digital images in real time during a flight of the UAV; computationally analyzing the acquired digital images with a predictor that has been computationally trained to identify and classify objects appearing in the images; and taking an action based on at least one classified object. The predictor may be a neural network, and the action may be determined based on database lookup in response to a detected object classified by the predictor. For example, the action may be altering a flight path of the drone.”;
para 42 and Fig. 7 (both shown below);
PNG
media_image2.png
905
1324
media_image2.png
Greyscale
PNG
media_image3.png
852
1323
media_image3.png
Greyscale
), and
send the status data element to a remote component (Stein:
para 11 (as referenced for claim 17 below): “the HAPSNN may process data received from a drone”;
para 15: “The computer may be further configured to communicate with a HAPS vehicle and, for example, to execute flight commands received from the HAPS vehicle, to communicate an altered flight to the HAPS vehicle for obtaining authorization from air-traffic control infrastructure, and/or to communicate a detected but unclassified object to the HAPS vehicle and receive, from the HAPS vehicle, a classification and associated action to be taken.”;
para 42 and Fig. 7 (both shown below);
Fig. 8A (shown below): Drone 118, image acquisition 827, communicate to 800 and to 805;
“[0045] The cloud neural network module 805 includes a classification neural network 815 that processes images and data received, via agile transceivers 808, from a drone in real time, and which may be passed to the cloud neural network 805.”;
Para 47: “communication links between drones operating in the airspace of the HAPSNN 800, with the terrestrial telecommunications network that some UTM systems utilize, or with backhaul communications channels to transmit data from the HAPS to the cloud-based neural network.”;
Paras 49, 51 (as shown above);
“[0053] The drone 118 transmits image data to the HAPSNN, which includes a high-precision CNN (in the compute engine 815 or even within the HAPS itself, if desired) capable of processing, for example, a 60 Megapixel (MP) photographic image each second. The CNN architecture is designed for speed and accuracy of classification by leveraging back-end logic that runs on the compute engine
825.”),
wherein the image comprises an aerial image of the area (Stein:
para 11 (as referenced for claim 17 below): “data received from a drone”;
para 42 and Fig. 7 (both shown below);
Fig. 8A (shown below): Drone 118, image acquisition 827;
Figs. 1-3 (shown below): 118
Para 49 (as shown above)),
wherein the edge computing system, particularly (the following limitations are not required) the data processing unit thereof, is configured to receive a data stream from a sensor (Stein:
para 8: “Such a drone inspection neural network ("DINN") may monitor, in real time, the data stream from a plurality of onboard sensors during navigation to an asset along a preprogrammed flight path and/or during its mission (e.g., as it scans and inspects an asset).”
PNG
media_image4.png
854
1540
media_image4.png
Greyscale
PNG
media_image5.png
811
1465
media_image5.png
Greyscale
PNG
media_image6.png
760
1324
media_image6.png
Greyscale
PNG
media_image7.png
733
993
media_image7.png
Greyscale
PNG
media_image8.png
690
1042
media_image8.png
Greyscale
Fig. 8A (shown below): Drone 118, image acquisition 827;
PNG
media_image9.png
989
1559
media_image9.png
Greyscale
Para 49, 51 (as shown above)), and wherein
a time interval between receiving the data stream and generating the status data element is less than 30 minutes, preferably (the following limitations are not required) less than 10 minutes, further preferably (the following limitations are not required) less than 1 minute (Stein:
para 8 (referenced above): “real time”;
para 47: “The neural network (NN) data link is a dedicated high-bandwidth backhaul channel that enables the HAPSNN 800 to communicate with DINN neural network compute engines 825, transmitting real-time data received from a plurality of drones operating in the monitored airspace and receiving predictions and action instructions obtained from the classification database 819.”;
para 51 (real time);
PNG
media_image10.png
789
1325
media_image10.png
Greyscale
“[0059] FIGS. 9A-13E illustrate various drone applications and the manner in which a DINN may be used to control and simplify drone operation. In FIG. 9A, the DINN 812 guides the drone 118 around an antenna 900 to be inspected in accordance with a flight pattern 910 that may change in real
time as the drone 118 detects anomalies or structures requiring closer inspection. For example, as shown in FIG. 9B, the flight path 910 may be altered to keep the drone 118 clear of power lines 915, which will have been recognized as an obstacle.”).
As per claim 17, Stein teaches the edge computing system according to claim 16, wherein a time interval between generating the status data element and sending the status data element to the remote component is less than 10 minutes, preferably (the following limitations are not required) less than 5 minutes, further preferably (the following limitations are not required) less than 1 minute (Stein: See arguments and citations offered in rejecting claim 16 above;
Para 11: “A HAPS platform may execute a neural network (a "HAPSNN") as it monitors air traffic; the neural network enables it to classify, predict, and resolve events in its airspace of coverage in real time as well as learn from new events that have never before been seen or detected. The HAPSNN-equipped HAPS platform may provide surveillance of nearly 100% of air traffic in its airspace of coverage, and the HAPSNN may process data received from a drone to facilitate safe and efficient drone operation within an airspace. The HAPSNN also enables bidirectional connection and real-time monitoring so drones can better execute their intended missions.”;
paras 51, 52 (shown above for claim 16): real time;
“[0053] The drone 118 transmits image data to the HAPSNN, which includes a high-precision CNN (in the compute engine 815 or even within the HAPS itself, if desired) capable of processing, for example, a 60 Megapixel (MP) photographic image each second. The CNN architecture is designed for speed and accuracy of classification by leveraging back-end logic that runs on the compute engine
825.”).
As per claim 18, Stein teaches the edge computing system according to claim 16, wherein the edge computing system comprises a data storage unit, wherein the data storage unit is configured to store at least one module configured to carry out a defined operation based on the image (Stein: See arguments and citations offered in rejecting claim 16 above;
Fig. 8A (shown above): mainly 118, 803, 804, 825, 830-835;
para 14: “the predictor is a neural network. The UAV may include a database of actions, with the computer configured to select and cause execution of an action from the database in response to a detected object classified by the predictor.”;
para 16: “the invention pertains to a method of inspecting an asset using a UAV. In various embodiments, the method comprises the steps of acquiring digital images in real time during a flight of the UAV; computationally analyzing the acquired digital images with a predictor that has been computationally trained to identify and classify objects appearing in the images; and taking an action based on at least one classified object. The predictor may be a neural network, and the action may be determined based on database lookup in response to a detected object classified by the predictor. For example, the action may be altering a flight path of the drone.”;
para 42 and Fig. 7 (both shown above for claim 16);
para 51 (shown above for claim 16);
para 57: “If the real-time neural network 825 resident on the drone 118 is also a CNN ("RT-CNN"), the HAPSNN 800 may push these weights to the RT-CNN, which receives and loads them. That is, the HP-CNN and RT-CNN may be identical or substantially similar so that CNN weights generated for the HP-CNN may be propagated across a fleet of drones”;
para 61: “This may prompt the drone 118 to perform a closer inspection (as indicated in the figure) in response to the classification of the condition and database lookup. Once again, a HAPSNN may recognize the condition and send new weights to the DINN of the drone 118, enabling it to make condition-specific classifications as it inspects the transformer 1200”).
As per claim 19, Stein teaches the edge computing system according to claim 18, wherein the data processing unit is configured to determine a result of executing any of the at least one module, and wherein the status data element comprises the result of executing any of the at least one module (Stein: See arguments and citations offered in rejecting claim 18 above;
para 42 and Fig. 7 (both shown above for claim 16)
para 51 (shown above)).
As per claim 20, Stein teaches the edge computing system according to claim 16, wherein the edge computing system, particularly (the following limitations are not required) the data processing unit thereof, is configured to identify an image object in the image (Stein: See arguments and citations offered in rejecting claim 16 above;
“[0060] With reference to FIG. 10, an antenna 1000 may have a crack 1005, which is identified by a DINN-equipped drone.”;
Para 62: “The DINN 812 analyzes images acquired by the drone's onboard camera and classifies the various components 1310-1335.”).
As per claim 25, Stein teaches the edge computing system according to claim 16, wherein the edge computing system, particularly (the following limitations are not required) the data processing unit thereof, is configured to receive a data stream from a sensor (Stein: See arguments and citations offered in rejecting claim 16 above).
As per claim 26, Stein teaches the edge computing system according to claim 25, wherein the sensor comprises a camera, wherein the camera comprises any of an optical camera, an infrared camera, or a hyperspectral camera (Stein: See arguments and citations offered in rejecting claim 16 above;
Para 49: videocamera;
Para 60: RGB camera, infrared, multispectral).
As per claim 27, Stein teaches the edge computing system according to claim 18, wherein the at least one module comprises a safety assessment module configured to determine a safety level of the area based on the image (Stein: See arguments and citations offered in rejecting claim 18 above
Para 46: “Emergency alerts may be issued to manned and/or umnanned traffic with instructions on which way to move to deconflict the airspace”;
Para 62: “In the preliminary inspection, the drone is farther away so the DINN 812 can only classify large items as the resolution of the image-acquisition device(s) is fixed. Once the closer flight path 1355 is executed, more asset detail will be detected, enabling the DINN 812 to classify new items and adjust the path of the drone again as needed. The ranging compute engine calculates the closest allowable approach distance between drone and an asset consistent with an acceptable safety margin”).
As per claim 30, Stein teaches a method comprising:
generating a status data element based on an image of an area (Stein: See arguments and citations offered in rejecting claim 16 above), and
sending the status data element to a remote component (Stein: See arguments and citations offered in rejecting claim 16 above),
wherein the image comprises an aerial image of the area (Stein: See arguments and citations offered in rejecting claim 16 above).
As per claim 31, Stein teaches the method according to claim 30, wherein generating the status data element comprises assigning a position to at least one pixel in the image (Stein: See arguments and citations offered in rejecting claim 16 above;
Para 62: “FIGS. 13C and 13D, the drone 118 may execute a preliminary flight pattern 1350 to gather images for analysis by the DINN, alone or in concert with a HAPSNN. The DINN 812 analyzes images acquired by the drone's onboard camera and classifies the various components 1310-1335. Based on these classifications, the navigation module 832 (see FIG. 8) or back-end code computes an optimized flight plan 1355 that permits all of the components 1310-1335 to be properly and efficiently inspected by the drone 118. In greater detail, each component present in the substation 1300 is classified during the preliminary inspection flight path 1350, which is farther away from the substation 1300 to accommodate GPS drift vectors and other unknowns relating to obstacles or inaccuracies about the initial flight path. Once a component is classified, its position in the image, as well as that of the drone, is registered. This process repeats multiple times throughout the preliminary inspection and enables the back-end code to triangulate and position each of the classified assets in a 3D space so a more precise inspection flight path, which will bring the drone and payload closer to the assets, can be calculated.”).
As per claim 32, Stein teaches the aerial vehicle comprising an edge computing system according to claim 16 and a sensor configured for (Stein: See arguments and citations offered in rejecting claim 16 above):
flying over an area (Stein: See arguments and citations offered in rejecting claim 16 above),
gathering data by means of the sensor (Stein: See arguments and citations offered in rejecting claim 16 above), and
sending the data to the edge computing system, wherein the aerial vehicle, particularly (the following limitations are not required) the edge computing system thereof, is further configured to communicate with a remote component (Stein: See arguments and citations offered in rejecting claim 16 above;
para 15: “The computer may be further configured to communicate with a HAPS vehicle and, for example, to execute flight commands received from the HAPS vehicle, to communicate an altered flight to the HAPS vehicle for obtaining authorization from air-traffic control infrastructure, and/or to communicate a detected but unclassified object to the HAPS vehicle and receive, from the HAPS vehicle, a classification and associated action to be taken.”).
As per claim 33, Stein teaches the aerial vehicle according to claim 32, wherein a flight path is configured to be loaded on to the aerial vehicle, particularly (the following limitations are not required) the edge computing system thereof (Stein: See arguments and citations offered in rejecting claim 32 above
para 12: “When the drone inspection is complete, the drone, using the DINN, may fly to the next
preprogrammed asset location and adapt its flight path in real time along the way to optimize its operation in the airspace”;
para 15: “The computer may be configured to cause the UAV to execute a preliminary flight plan around an asset to be inspected and, based on object classifications made by the predictor during the preliminary flight plan, compute and execute a revised flight plan around the asset.”;
para 16: “the invention pertains to a method of inspecting an asset using a UAV. In various embodiments, the method comprises the steps of acquiring digital images in real time during a flight of the UAV; computationally analyzing the acquired digital images with a predictor that has been computationally trained to identify and classify objects appearing in the images; and taking an action based on at least one classified object. The predictor may be a neural network, and the action may be determined based on database lookup in response to a detected object classified by the predictor. For example, the action may be altering a flight path of the drone.”;).
As per claim 34, Stein teaches the aerial vehicle according to the claim 32, wherein the edge computing system comprises a data storage unit, wherein the data storage unit is configured to store at least one module configured to carry out a defined operation based on the image, and wherein the at least one module loaded is based, at least in part, on the (only one of the following alternatives is required) area and/or on the flight path (Stein: See arguments and citations offered in rejecting claims 19, 32, and 33 above;
Para 51 (shown above for claim 16);
para 61: “This may prompt the drone 118 to perform a closer inspection (as indicated in the figure) in response to the classification of the condition and database lookup. Once again, a HAPSNN may recognize the condition and send new weights to the DINN of the drone 118, enabling it to make condition-specific classifications as it inspects the transformer 1200”).
As per claim 35, Stein teaches the aerial vehicle according to claim 32, wherein a time interval between the aerial vehicle gathering data by means of the sensor, and the aerial vehicle, particularly (the following limitations are not required) the edge computing system thereof, sending the status data element to the remote component is less than 1 hour, preferably (the following limitations are not required) less than 30 minutes, further preferably (the following limitations are not required) less than 2 minutes (Stein: See arguments and citations offered in rejecting claims 16, 17, and 32 above).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21-23, 28, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Stein as applied to claim 18 above, and further in view of US 9389084 B1 (Chen).
As per claim 21, Stein teaches the edge computing system according to claim 18, wherein the data storage unit is configured to store (Stein: See arguments and citations offered in rejecting claim 18 above). Stein does not teach store design data.
Chen teaches store design data (Chen: See arguments and citations offered in rejecting claim 23 below).
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Chen into Stein since both Stein and Chen suggest a practical solution and field of endeavor of drone aerial imaging and image analysis inspection of infrastructure sites in general and Chen additionally provides teachings that can be incorporated into Stein in that the image analysis involves comparing currently captured data with design data so that “By so doing, the change detection
system can provide a user ( e.g., a manager or owner of the target site) with helpful and accurate information regarding the progress ( e.g., construction progress) or other changes at the target site” (Chen: col 1, lines 55-61). The teachings of Chen can be incorporated into Stein in that the image analysis involves comparing currently captured data with design data. Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
As per claim 22, Stein in view of Chen teaches the edge computing system according to claim 21, wherein the at least one module comprises a comparison module configured to compare the image of the area with the design data (Chen: See arguments and citations offered in rejecting claim 23 below).
As per claim 23, Stein in view of Chen teaches the edge computing system according to claim 22, wherein the edge computing system is configured to identify, in the design data, a design object corresponding to the image object, wherein the comparison is based, at least in part, on a comparison of the image object and the corresponding design object (Stein: See arguments and citations offered in rejecting claim 18 above |
Chen:
Col 6, line 63 – col 7, line 11:
PNG
media_image11.png
647
885
media_image11.png
Greyscale
Col 8, lines 15-20: “if the target site is a building under construction, the first three-dimensional representation of the building may be a point cloud representing the building with only one floor completed. The second three-dimensional representation of the building may be a point cloud representing the building with two floors completed”;
Col 11, lines 36-55:
PNG
media_image12.png
872
1000
media_image12.png
Greyscale
Col 12, lines 33-38: “It will be understood that while the target site used in connection with FIGS. 4A-4E includes a simple shape (e.g., a box), this is for illustration purposes only and the same process can be performed with respect to any number of features ( e.g., buildings, terrain, equipment, etc.) of varying shapes and sizes present at a target site.”;
PNG
media_image13.png
801
1463
media_image13.png
Greyscale
PNG
media_image14.png
866
1312
media_image14.png
Greyscale
PNG
media_image15.png
825
1135
media_image15.png
Greyscale
).
As per claim 28, Stein teaches the edge computing system according to claim 16. Stein does not teach the area comprises a construction site. Chen teaches limitations (Chen: See arguments and citations offered in rejecting claim 23 above; Also see:
Col 1, lines 59-60;
Col 1, lines 18-26;
Col 2, lines 63-65
Col 5, lines 14-16;
Col 7, lines 35-37;
Col 8, lines 15-17).
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Chen into Stein since both Stein and Chen suggest a practical solution and field of endeavor of drone aerial imaging and image analysis inspection of infrastructure sites in general and Chen additionally provides teachings that can be incorporated into Stein in that the site is a construction site so that “it can be useful for construction site managers to utilize aerial photographs to track the progress of work at the construction site” (Chen: col 1, lines 20-25). The teachings of Chen can be incorporated into Stein in that the site is a construction site. Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
As per claim 29, Stein teaches the edge computing system according to claim 16, wherein the edge computing system is configured for photogrammetry. Chen teaches image photogrammetry (Chen: See arguments and citations offered in rejecting claim 23 above; Also see:
col 7, lines 11-27;
Col 12, lines 39-46;
Col 29, lines 22-36
: note that Photogrammetry is the science of extracting reliable measurements and creating detailed 3D models from ordinary photographs.).
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Chen into Stein since both Stein and Chen suggest a practical solution and field of endeavor of drone aerial imaging and image analysis inspection of infrastructure sites in general and Chen additionally provides teachings that can be incorporated into Stein in that the image analysis involves photogrammetry as to “to identify differences between the first three-dimensional representation and the second three-dimensional representation. Thereafter, the change detection system generates a report indicating changes at the target site based on the differences between the
first three-dimensional representation and the second three-dimensional representation. By so doing, the change detection system can provide a user ( e.g., a manager or owner of the target site) with helpful and accurate information regarding the progress ( e.g., construction progress) or other changes at the target site as will be explained in more detail below” (Chen: col 1, lines 50-61). The teachings of Chen can be incorporated into Stein in that the image analysis involves photogrammetry. Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
Claim(s) 24, 28, and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Stein as applied to claim 16 above, and further in view of Vacanas, Yiannis, et al. "The combined use of Building Information Modelling (BIM) and Unmanned Aerial Vehicle (UAV) technologies for the 3D illustration of the progress of works in infrastructure construction projects." Fourth international conference on remote sensing and geoinformation of the environment (RSCy2016). Vol. 9688. SPIE, 2016 (Vacanas).
As per claim 24, Stein teaches the edge computing system according to claim 16. Stein does not teach the image comprises an orthophoto map. Vacanas teaches these limitations (Vacanas:
Page 5, last 2 paras:
PNG
media_image16.png
437
1056
media_image16.png
Greyscale
Page 6, para 1:
PNG
media_image17.png
754
1061
media_image17.png
Greyscale
).
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Vacanas into Stein since both Stein and Vacanas suggest a practical solution and field of endeavor of UAV aerial imaging and image analysis inspection of infrastructure sites in general and Vacanas additionally provides teachings that can be incorporated into Stein in that the images are orthophotos as to “provide visual documentation of the progress of the progression of the road work over time” (Vacanas: page 6, para 1). The teachings of Vacanas can be incorporated into Stein in that the images are orthophotos. Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
As per claim 28, Stein teaches the edge computing system according to claim 16. Stein does not teach the area comprises a construction site. Vacanas teaches these limitations (Vacanas: abstract;
page 1, all paras;
Section 2, page, 2, all paras;
Section 4.1;
Section 5).
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Vacanas into Stein since both Stein and Vacanas suggest a practical solution and field of endeavor of UAV aerial imaging and image analysis inspection of infrastructure sites in general and Vacanas additionally provides teachings that can be incorporated into Stein in that the site is a construction site “in order to achieve efficient and accurate as-built data collection and 3D illustrations of the works progress during an infrastructure construction project” (Vacanas: abstract). The teachings of Vacanas can be incorporated into Stein in that the site is a construction site. Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
As per claim 29, Stein teaches the edge computing system according to claim 16, wherein the edge computing system is configured for image (Vacanas: abstract: “photogrammetry”;
Page 3, paras 1-2: “photogrammetry”;
Page 5, para 2 (shown above): “photogrammetry”).
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Vacanas into Stein since both Stein and Vacanas suggest a practical solution and field of endeavor of UAV aerial imaging and image analysis inspection of infrastructure sites in general and Vacanas additionally provides teachings that can be incorporated into Stein in that the image analysis involves photogrammetry since “By using photogrammetry, characteristics such as distances, areas, volumes, elevations, object sizes, and object shape can be determined within overlapping areas” (Vacanas: abstract). The teachings of Vacanas can be incorporated into Stein in that the image analysis involves photogrammetry. Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Atiba Fitzpatrick whose telephone number is (571) 270-5255. The examiner can normally be reached on M-F 10:00am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on (571) 270-5183. The fax phone number for Atiba Fitzpatrick is (571) 270-6255.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Atiba Fitzpatrick
/ATIBA O FITZPATRICK/
Primary Examiner, Art Unit 2677
Prosecution Insights