DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the response filed on February 4, 2026.
Claims 1-9 and 12-21 are currently pending and have been examined.
Claims 10-11 have been canceled by the applicant.
This action is made FINAL.
The examiner would like to note that this application is being handled by examiner Christine Huynh.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-18have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Because the claim now stands on its own and includes limitations not previously considered in the rejection of the independent claim, the prior rejection is no longer fully applicable as written. Accordingly, upon further search consideration, the prior rejection of claims 10-11 is withdrawn, and a new rejection of claims 1, 14, and 19 is entered under 35 U.S.C. 103 over Buczkowski et al. (US 20240005599 A1) in view of Papandreou et al. (US 20200302203 A1) and Asmari et al. (US 20210020073 A1). See detailed rejection below.
Dependent claims are rejected for the same reason as listed above due to dependency. See detailed rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-5, 14-16, 19, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buczkowski et al. (US 20240005599 A1) in view of Papandreou et al. (US 20200302203 A1) and Asmari et al. (US 20210020073 A1).
Regarding claims 1-5, 14-16, 19, and 21:
With respect to claims 1, 14, and 19, Buczkowski teaches:
processor, (“a system comprising a data-processing system…” [0015]).
control an image sensor to capture an image of a geographical region; (“UAVs are capable of capturing a large amount of high-quality aerial images that can then be processed to create high-resolution 3D maps.” [0003], “The Ground Sampling Distance can depend on properties of the optical sensor (e.g. sensor width/diameter, focal length, etc.) and a flight altitude of the aerial vehicle.” [0214]), where an optical sensor of a UAV can be used to capture an image.
Buczkowski does not teach, but Papandreou teaches:
determine count data associated with one or more cores of the processor, wherein the count data is indicative of a number of one or more available cores of the processor for detecting a first object in the captured image; (“According to embodiments, the total processing cost metric is based on a first configuration information of the distributed computing environment, the first configuration information comprising one or more of the following: a number of the computing nodes being available for performing the processing, a number of available processor cores for each available processing node…” [0055], “According to embodiments, the determination of the assignment is further based on a second configuration information of the distributed computing environment, the second configuration information comprising one or more of the following: a number of the computing nodes being available for performing the processing, a number of available processor cores for each available processing node…” [0056]), where the total processing cost metric for image segmenting for image analysis is based on a number of available processor cores.
segment the captured image into one or more segments based on the determined count data; (“In step S210, the control unit receives configuration information describing computing resources of the distributed computing environment. Such configuration information may comprise, without limitation, a number of computing nodes which are available for processing segments of the digital image as well as further configuration information about these available nodes.” [0073], “Based on the configuration information and properties of the digital image (which may include metadata such as dimensions of the image, number and configuration of channels (e.g. color channels), number and type of embedded magnifications, etc.) the control unit determines S212 a segmentation of the digital image into two or more segments. An example of configuration-dependent segmentation comprises dimensioning the image segments in a standard size which fits the smallest available memory capacity. In another example, the segment sizes are determined individually such that each segment can be loaded into the memory of at least one available computing node.” [0075]), where the image is divided into segments based on the determined available resources.
Buczkowski further teaches:
detect the first object of a set of objects in the captured image by using the one or more available cores to process the one or more segments; (“The error minimizing component, particularly the machine learning algorithm, may be configured for performing a nearest neighbour analysis step, wherein the nearest neighbour analysis step may comprise assigning to at least one object of one of the orthophoto maps (O1 or O2) a corresponding object of the same class in one of the other orthophoto maps (O2 or O1).” [0038], “The data-processing system, particularly the segmentation component, may be configured for determining classes for at least some of the part(s) of the at least two orthophoto maps (O1 or O2) by means of at least one convolutional neural network. In other words, the data-processing system, particularly the segmentation component, may comprise at least one convolutional neural network configured for determining classes for at least some of the part(s) of the at least two orthophoto map (O1, O2).” [0058]), where an object can be determined in a captured image using the image segmentation.
However, Buczkowski does not teach using the one or more available cores to process the one or more segments, but Papandreou teaches (“The workload of a given processor core is understood herein as the number of processing cycles to complete the processing of a said algorithm on a given image segment of the digital image. The workload is considered here to have a positive correlation to the processing time taken by the core to complete the processing of the workload (as measured from the first cycle).” [0034], “The classification decision for a given pixel as being either a background pixel or a non-background pixel depends on the classification algorithm used during the determination of the number of non-background pixels. Exemplary algorithms utilize a threshold criterion which relates to attributes such as (single-channel or combined) intensity, brightness, or signal strength encoded by each pixel.” [0036]), where one or more available cores to process the one or more segments for image analysis.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Papandreou’s resource allocation for (“iterative and distributed approaches to pre-processing and ML model training to enable an efficient handling of such large amounts of digital image data.” [0003]).
Buczkowski further teaches:
generate an association between the detected first object and the captured image; (“The error minimizing component, particularly the machine learning algorithm, may be configured for performing a nearest neighbour analysis step, wherein the nearest neighbour analysis step may comprise assigning to at least one object of one of the orthophoto maps (O1 or O2) a corresponding object of the same class in one of the other orthophoto maps (O2 or O1).” [0038], “The data-processing system, particularly the segmentation component, may be configured for assigning different classes to different portions of the at least two orthophoto maps (O1, O2) by the at least one convolutional neural network.” [0065), which the determined object and class is assigned in the image.
Buczkowski does not teach, but Asmari teaches:
transmit the generated association to a map database; (“The GPS information may be used to geotag the location of the detected assets and then store the data for integration with a mapping application. A computer processing unit on board of the vehicle may analyze the data and image information in real time, identify the assets, and generate a graphic information system (GIS) map of the assets using the information collected from the GPS and the IMU. This processing may occur on a server or a cloud-based system using stored sensor and image data.” [0007]), where the detected asset information can be stored and integrated with a mapping application.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Asmari’s map storage in because (“Data collected during one trip—which may act as a baseline or historical reference—may be compared to data from subsequent trips to identify changes in construction, orientation, and condition of assets. Repeat trips and analysis may be performed at some frequency to create a dynamic and up-to-date map of the assets.” See Asmari [0010]), in order to collect and compare information.
With respect to claim 2, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 1. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 1. Buczkowski further teaches:
determine location information associated with the geographical region based on the captured image; (“One further step in image analysis is the process of georeferencing that relates an internal coordinate system of a digital image to geographic locations in physical space and thus determine the geographic position for points on the digital image. Georeferencing on aerial photogrammetry data is typically done using so called Ground Control Points (GCPs) which are characteristic points on the site, e.g. characteristic points of the site, such as corners of buildings, corners of horizontal street sights, or artificial markings. Coordinates of these points are either known or are determined. There are a number of algorithms that can use these points to interpolate the geographical position of all points of the digital image based on the geographical position of these points.” [0008]), where the location information such as location coordinates of the image of the region can be determined.
Buczkowski does not teach, but Asmari teaches:
transmit the determined location information to the map database; (“Then, using the calculated distance of the assets to the vehicle, and based on the GPS location of the vehicle when each frame of the video was taken, the exact location—e.g., latitude, longitude, and altitude of each detected asset is calculated and marked on a GIS map. The location information and pictures of each detected asset may be integrated into a mapping system, such as an ArcGIS mapping system, for review and assessment.” [0009]), where the detected asset information, such as the asset location, can be stored and integrated with a mapping application.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Asmari’s map storage in because (“Data collected during one trip—which may act as a baseline or historical reference—may be compared to data from subsequent trips to identify changes in construction, orientation, and condition of assets. Repeat trips and analysis may be performed at some frequency to create a dynamic and up-to-date map of the assets.” See Asmari [0010]), in order to collect and compare information.
With respect to claim 3, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 1. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 1. Buczkowski further teaches:
control a location sensor to capture location information associated with the geographical region; (“One further step in image analysis is the process of georeferencing that relates an internal coordinate system of a digital image to geographic locations in physical space and thus determine the geographic position for points on the digital image. Georeferencing on aerial photogrammetry data is typically done using so called Ground Control Points (GCPs) which are characteristic points on the site, e.g. characteristic points of the site, such as corners of buildings, corners of horizontal street sights, or artificial markings. Coordinates of these points are either known or are determined. There are a number of algorithms that can use these points to interpolate the geographical position of all points of the digital image based on the geographical position of these points.” [0008], “The aerial vehicle may further comprise a sensing device configured for sensing a height of the area, e.g. a distance sensor, an altitude sensor and a corresponding processing unit.” [0211]), where the location information can be found using sensors.
Buczkowski does not teach, but Asmari teaches:
transmit the captured location information to the map database; (“Then, using the calculated distance of the assets to the vehicle, and based on the GPS location of the vehicle when each frame of the video was taken, the exact location—e.g., latitude, longitude, and altitude of each detected asset is calculated and marked on a GIS map. The location information and pictures of each detected asset may be integrated into a mapping system, such as an ArcGIS mapping system, for review and assessment.” [0009]), where the detected asset information, such as the asset location, can be stored and integrated with a mapping application.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Asmari’s map storage in because (“Data collected during one trip—which may act as a baseline or historical reference—may be compared to data from subsequent trips to identify changes in construction, orientation, and condition of assets. Repeat trips and analysis may be performed at some frequency to create a dynamic and up-to-date map of the assets.” See Asmari [0010]), in order to collect and compare information.
With respect to claims 4 and 16, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claims 1 and 14. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claims 1 and 14. Buczkowski further teaches:
apply a first machine learning (ML) model on the one or more segments of the captured image, wherein the ML model is trained using a training dataset; (“The error minimizing component, particularly the machine learning algorithm, may be configured for performing a nearest neighbour analysis step, wherein the nearest neighbour analysis step may comprise assigning to at least one object of one of the orthophoto maps (O1 or O2) a corresponding object of the same class in one of the other orthophoto maps (O2 or O1).” [0038], “The convolutional neural network may be trained with tiles comprising a lower resolution than the tiles processed by the data-processing system, particularly by the segmentation component. For example, the convolutional neural network may be obtained by training with tiles comprising image data corresponding to a bigger section of the area but a same number of pixels in comparison to the tiles processed by the semantic segmentation component.” [083]), where a machine learning algorithm can be applied on the segments of the captured image.
detect the first object in the captured image based on the application of the first ML model on the one or more segments; (“The error minimizing component, particularly the machine learning algorithm, may be configured for performing a nearest neighbour analysis step, wherein the nearest neighbour analysis step may comprise assigning to at least one object of one of the orthophoto maps (O1 or O2) a corresponding object of the same class in one of the other orthophoto maps (O2 or O1).” [0038]), where objects can be detected from the segments of the captured image.
With respect to claim 5, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 4. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 4. Buczkowski further teaches:
retrieve, for training the first ML model, the training dataset associated with the detection of the set of objects in a training image; (“The convolutional neural network may be trained with tiles comprising a lower resolution than the tiles processed by the data-processing system, particularly by the segmentation component. For example, the convolutional neural network may be obtained by training with tiles comprising image data corresponding to a bigger section of the area but a same number of pixels in comparison to the tiles processed by the semantic segmentation component.” [0083]), where the model is trained using image data for detection of objects in the training images.
train the first ML model based on the retrieved training dataset; (“the convolutional neural network may be trained with tiles comprising data corresponding to a bigger section of the area than the tiles processed in by the segmentation component (or the data-processing system), but however a same amount of data.” [0084]), where the model is trained on the image data.
With respect to claim 15, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 14. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 14. Buczkowski further teaches:
wherein the electronic device corresponds to an unmanned aerial vehicle (UAV); (“The aerial vehicle may be an unmanned aerial vehicle. The aerial vehicle, particularly the unmanned aerial vehicle, may be configured for generating at least one of the orthophoto maps (O1, O2) based on the aerial images.” [0105-0106]).
With respect to claim 21, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 19. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 19. Buczkowski further teaches:
wherein the one or more sensors comprises of an image capture sensor, and wherein the sensor data corresponds to an image of the geographical region; (“UAVs are capable of capturing a large amount of high-quality aerial images that can then be processed to create high-resolution 3D maps.” [0003], “The Ground Sampling Distance can depend on properties of the optical sensor (e.g. sensor width/diameter, focal length, etc.) and a flight altitude of the aerial vehicle.” [0214]), “One further step in image analysis is the process of georeferencing that relates an internal coordinate system of a digital image to geographic locations in physical space and thus determine the geographic position for points on the digital image. Georeferencing on aerial photogrammetry data is typically done using so called Ground Control Points (GCPs) which are characteristic points on the site, e.g. characteristic points of the site, such as corners of buildings, corners of horizontal street sights, or artificial markings. Coordinates of these points are either known or are determined. There are a number of algorithms that can use these points to interpolate the geographical position of all points of the digital image based on the geographical position of these points.” [0008]), where an optical sensor of a UAV can be used to capture an image and location information such as location coordinates of the image of the region can be determined.
Claim(s) 6, 17, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buczkowski et al. (US 20240005599 A1) in view of Papandreou et al. (US 20200302203 A1), Asmari et al. (US 20210020073 A1), and Goel (US 20200410225 A1).
Regarding claims 6, 17, and 20:
With respect to claims 6 and 17, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claims 1 and 14. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claims 1 and 14. Buczkowski does not teach, but Goel teaches:
apply a second machine learning (ML) model on the detected first object and the captured image; (“the use of the term “first output” can refer to information generated by analyzing the image data using the first machine learned model” [0116], “At operation 1006, the perception system 322 can analyze the first output using a second machine learned model, which can be trained to detect high variance regions, to generate a second output. In at least one example, the first output can be input into a second machine learned model for detecting high variance regions.” [0117]), where a second machine learning model can be applied on the analyzed captured image.
generate the association between the detected first object and the captured image based on the application of the second ML model on the detected first object and the captured image; (“At operation 506, the perception system 322 can analyze the first output using a second machine learned model, which can be trained to detect pedestrians, to generate a second output. In at least one example, the first output can be input into a machine learned model for detecting pedestrians. The machine learned model can be trained by the training system 340 as described above with reference to FIG. 3. In such an example, the pedestrian detector 112 can analyze the first output to generate a second output. The second output can include one or more indications that are associated with regions of interest corresponding to an identified pedestrian in the environment within which the vehicle 302 is positioned. In some examples, the indication can be a bounding box that surrounds a region of interest corresponding to pedestrian in the image data.” [0125]), which a determined object is further analyzed based on the application of the second machine learning model.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Goel’s second machine learning model because (“At operation 1106, the perception system 322 analyzes the image data using a second machine learned model trained to detect objects to generate a second output… In at least one example, the perception system 322 can output an intermediate output of features detected by the second machine learned model that are used to detect and/or classify objects in image data. For the purpose of the discussion of process 1100, the intermediate output of features can be referred to as the “second output.” In at least one example, the first output and the second output can be output at or near the same time. That is, in such an example, the first machine learned model and the second machine learned model can analyze the image data at the same time and/or in parallel.” See Goel [0133]) in order to further determine the object detected in captured image.
With respect to claim 20, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 19. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 19. Buczkowski does not teach, but Goel teaches:
wherein the one or more sensors comprises a galvanometer sensor, a hall effect magnetometer sensor, a corona discharge meter sensor, and a non-contact voltage tester sensor; (“the sensor system(s) 306 can include LIDAR sensors, RADAR sensors, ultrasonic transducers, sound navigation and ranging (SONAR) sensors, location sensors (e.g., global positioning system (GPS), compass, etc.), inertial sensors (e.g., inertial measurement units, accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, etc.), wheel encoders, microphones, environment sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), ToF sensors, etc. The sensor system(s) 306 can include multiple instances of each of these or other types of sensors.” [0061]), where Goel includes a sensor system including a plurality of environment sensors. It would have been obvious to a person of ordinary skill in the art to include a galvanometer sensor, a hall effect magnetometer sensor, a corona discharge meter sensor, and a non-contact voltage tester sensor in an attempt to provide an improved system or method, as a person with ordinary skill has good reason to pursue the known options within his or her technical grasp. In turn, because the product as claimed has the properties predicted by the prior art, it would have been obvious to make the system or product where the system includes a galvanometer sensor, a hall effect magnetometer sensor, a corona discharge meter sensor, and a non-contact voltage tester sensor.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Goel’s sensors because (“he sensor system(s) 306 can provide input to the vehicle computing device(s) 304. In some examples, the sensor system(s) 306 can preprocess at least some of the sensor data prior to sending the sensor data to the vehicle computing device(s) 304. In at least one example, the sensor system(s) 306 can send sensor data, via the network(s) 332, to the computing device(s) 334 at a particular frequency, after a lapse of a predetermined period of time, in near real-time, etc.” See Goel [0061]), therefore beneficial for collecting environmental information
Claim(s) 7, 8, and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buczkowski et al. (US 20240005599 A1) in view of Papandreou et al. (US 20200302203 A1), Asmari et al. (US 20210020073 A1), and Loveland et al. (US 20180130196 A1).
Regarding claims 7, 8, and 18:
With respect to claim 7, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 1. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 1. Buczkowski does not teach, but Loveland teaches:
detect a network event indicative of a disruption in a network connection of the UAV with the map database; (“UAVs and/or UGVs within a structure may lose network connections and/or control signals. In such situations, the UAVs and/or UGVs may be configured to return to a location of last connectivity and cache necessary information to finish the micro scans of regions lacking sufficient network availability.” [0106]), where a network connection of the UAV is disrupted.
store the generated association in the memory based on the detection of the network event; (“a default scanning pattern to ensure thorough micro scans includes an interior boustrophedonic flight plan. Specific location along the flight path may be annotated on the plan view map for additional or supplemental scanning. For example, an annotation on planview map, zoning parameter, or real-time UAV decision may instruct the UAV to perform specific navigational or scanning functions.” [0104], “In such situations, the UAVs and/or UGVs may be configured to return to a location of last connectivity and cache necessary information to finish the micro scans of regions lacking sufficient network availability. In some embodiments, the system may include a network of UAVs and/or UGVs. The network of UAVs may act as a mesh or relay network in which each UAV may utilize one or more other UAVs as an intermediary network node to maintain connectivity with a host or control device. Such networks may utilize self-healing algorithms to maintain communication between nodes (UAVs).” [0106]), where the location associated with network connection loss can be stored and used so that the UAV can return to a location of last connectivity.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Loveland’s detecting a network event because (“In such situations, the UAVs and/or UGVs may be configured to return to a location of last connectivity and cache necessary information to finish the micro scans of regions lacking sufficient network availability. In some embodiments, the system may include a network of UAVs and/or UGVs. The network of UAVs may act as a mesh or relay network in which each UAV may utilize one or more other UAVs as an intermediary network node to maintain connectivity with a host or control device. Such networks may utilize self-healing algorithms to maintain communication between nodes (UAVs).” [0106]), therefore improving UAV communication.
With respect to claims 8 and 18, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claims 1 and 14. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claims 1 and 14. Buczkowski further teaches:
receive one or more navigation commands associated with navigation of the UAV from a user device; (““The input user interface may e.g. a keyboard configured to allow the insertion of text and/or other keyboard commands (e.g. allowing the user to enter instructions to the unmanned aerial vehicle or parameters for the method) and/or a trackpad, mouse, touchscreen and/or joystick, e.g. configured for navigating the orthophoto map O or objects identified in the orthophoto map.” [0303]), which shows navigation commands can be received from a user device.
Buczkowski does not teach, but Loveland teaches:
control navigation of the UAV towards the geographical region based on the received one or more navigation commands; (“A standard flight plan may be saved on the server. The standard flight plan may be loaded on the UAV and altered based on information entered by the operator into the operator client interface. The UAV (e.g., via onboard or cloud-based processors) may also alter the standard flight plan based on the images captured and/or other sensor data.” [0086], “a user interface 1010 may include a site selection interface 1015 to receive an electronic input from an operator or other technician that identifies a location of a structure or other object to be assessed.” [0154]), where the UAV control navigation can be controlled based on a navigation command.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Loveland’s navigation control because (“The site selection interface may receive, from the operator, an electronic input identifying a location of a structure. The operator client may be a controller, computer, phone, tablet, or other electronic device. The operator may mark, via an electronic input on a boundary identification interface, one or more geographic boundaries associated with the structure and/or site. The operator may also identify, on the operator client, obstacles, boundaries, structures, and particular points of interest.” [0057]), for controlling UAV navigation.
Buczkowski further teaches:
control the image sensor to capture the image of the geographical region based on a determination that the UAV is over the geographical region; (“UAVs are capable of capturing a large amount of high-quality aerial images that can then be processed to create high-resolution 3D maps.” [0003], “The Ground Sampling Distance can depend on properties of the optical sensor (e.g. sensor width/diameter, focal length, etc.) and a flight altitude of the aerial vehicle.” [0214]), “One further step in image analysis is the process of georeferencing that relates an internal coordinate system of a digital image to geographic locations in physical space and thus determine the geographic position for points on the digital image. Georeferencing on aerial photogrammetry data is typically done using so called Ground Control Points (GCPs) which are characteristic points on the site, e.g. characteristic points of the site, such as corners of buildings, corners of horizontal street sights, or artificial markings. Coordinates of these points are either known or are determined. There are a number of algorithms that can use these points to interpolate the geographical position of all points of the digital image based on the geographical position of these points.” [0008]), where an optical sensor of a UAV can be used to capture an image and location information such as location coordinates of the image of the region can be determined.
Claim(s) 9 and 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Buczkowski et al. (US 20240005599 A1) in view of Papandreou et al. (US 20200302203 A1), Asmari et al. (US 20210020073 A1), and Hari et al. (US 20240087333 A1).
Regarding claims 9 and 12-13:
With respect to claim 9, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 1. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 1. Buczkowski further teaches:
segment, using a master process, the captured image into the one or more segments; (“The data-processing system may comprise a segmentation component. The segmentation component may be configured for generating the polygon(s) for the at least two input orthophoto maps (O1, O2), wherein each polygon approximates a part of the respective input orthophoto map.” [0030]), where the captured image is segmented.
Buczkowski does not teach, but Hari teaches:
allocate, using the master process, the one or more segments of the image to one or more cores of the processor of the UAV; (“The CPU(s) 906 may include a CPU cluster or CPU complex (alternatively referred to herein as a “CCPLEX”). The CPU(s) 906 may include multiple cores and/or L2 caches.” [0100], “Each streaming microprocessor may incorporate a number of mixed-precision processing cores partitioned into multiple blocks. For example, and without limitation, 64 PF32 cores and 32 PF64 cores may be partitioned into four processing blocks. In such an example, each processing block may be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two mixed-precision NVIDIA TENSOR COREs for deep learning matrix arithmetic, an L0 instruction cache, a warp scheduler, a dispatch unit, and/or a 64 KB register file” [0103], “The RISC cores may interact with image sensors (e.g., the image sensors of any of the cameras described herein), image signal processor(s), and/or the like.” [0114]), where the workload can be allocated to one or more cores.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Hari’s processor cores because (“The GPU(s) 908 may be power-optimized for best performance in automotive and embedded use cases” see Hari [0103]).
With respect to claim 12, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 1. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 1. Buczkowski further teaches:
wherein each of the one or more segments of the image is associated with a segment identifier, and wherein each of one or more cores of the processor is associated with a core identifier; (“The data-processing system, particularly the segmentation component, may be configured for may be configured for processing at least some of the tiles individually. This may be optionally advantageous, as it may allow the system to process orthophoto maps and/or digital elevation models that could not be processed as whole due to memory limitations.” [0069], “FIG. 3 shows classes assigned to individual parts in every set of parts 30a and 30b in the respective orthophoto map (O1, O2) of the area 10. Each part is located in one or more corresponding tiles of the respective orthophoto map (O1, O2). In the example of FIG. 3, an ID-variable comprises the class information. As can be seen in FIG. 3, generally, the classes may correspond to the class of the object corresponding to a part comprised by the respective set of parts (30a, 30b). In the example of FIG. 3, there is a heap of sand (ID1), an asphalt object (ID2) and again a heap of sand (ID1) within the orthophoto map O1. The same classes are also assigned to the corresponding objects in the orthophoto map O2.” [0227]), which shows that the different parts of the segmented image are labeled and can be processed individually. It would have been obvious to a person of ordinary skill in the art where the segments of the image are associated with a segment identifier, as the different parts of the image segments are labeled, in an attempt to provide an improved system or method, as a person with ordinary skill has good reason to pursue the known options within his or her technical grasp. In turn, because the product as claimed has the properties predicted by the prior art, it would have been obvious to make the system or product where the segments of the image are associated with a segment identifier.
However, Buczkowski does not teach using one or more cores, but Hari teaches, (“The CPU(s) 906 may include a CPU cluster or CPU complex (alternatively referred to herein as a “CCPLEX”). The CPU(s) 906 may include multiple cores and/or L2 caches.” [0100], “Each streaming microprocessor may incorporate a number of mixed-precision processing cores partitioned into multiple blocks. For example, and without limitation, 64 PF32 cores and 32 PF64 cores may be partitioned into four processing blocks. In such an example, each processing block may be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two mixed-precision NVIDIA TENSOR COREs for deep learning matrix arithmetic, an L0 instruction cache, a warp scheduler, a dispatch unit, and/or a 64 KB register file” [0103], “The RISC cores may interact with image sensors (e.g., the image sensors of any of the cameras described herein), image signal processor(s), and/or the like.” [0114]), where the workload can be allocated to one or more available cores.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Hari’s processor cores because (“The GPU(s) 908 may be power-optimized for best performance in automotive and embedded use cases” see Hari [0103]).
With respect to claim 13, Buczkowski in combination with Papandreou and Asmari, as shown in the rejection above, discloses the limitations of claim 12. The combination of Buczkowski, Papandreou, and Asmari teaches asset mapping using unmanned aerial vehicles of claim 12. Buczkowski further teaches:
wherein the processor is further configured to combine each of the one or more segments of the image based on the segment identifier and the core identifier; (“The data-processing system, particularly the segmentation component, may be configured for merging results from processing of the tiles.” [0070], “The data-processing system, particularly the segmentation component, may be configured for at least some tiles for, rotating the tiles; processing the rotated and the original tiles by means of the at least one convolutional network; for the results corresponding to the rotated tiles, inverting the rotation; and for each of the at least some tiles, merging the result of the original and the rotated tile.” [0075-0079], “The data-processing system, particularly the segmentation component, may be configured for may be configured for processing at least some of the tiles individually.” [0069]), where the segments of the image are combined.
However, Buczkowski does not teach using one or more cores, but Hari teaches, (“The CPU(s) 906 may include a CPU cluster or CPU complex (alternatively referred to herein as a “CCPLEX”). The CPU(s) 906 may include multiple cores and/or L2 caches.” [0100], “Each streaming microprocessor may incorporate a number of mixed-precision processing cores partitioned into multiple blocks. For example, and without limitation, 64 PF32 cores and 32 PF64 cores may be partitioned into four processing blocks. In such an example, each processing block may be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two mixed-precision NVIDIA TENSOR COREs for deep learning matrix arithmetic, an L0 instruction cache, a warp scheduler, a dispatch unit, and/or a 64 KB register file” [0103], “The RISC cores may interact with image sensors (e.g., the image sensors of any of the cameras described herein), image signal processor(s), and/or the like.” [0114]), where the workload can be allocated to one or more available cores.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the instant application to have combined Buczkowski’s UAV image analysis with Hari’s processor cores because (“The GPU(s) 908 may be power-optimized for best performance in automotive and embedded use cases” see Hari [0103]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Christine N Huynh whose telephone number is (571)272-9980. The examiner can normally be reached Monday - Friday 8 am - 4 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at (571)270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTINE NGUYEN HUYNH/Examiner, Art Unit 3662
/ANISS CHAD/Supervisory Patent Examiner, Art Unit 3662
Prosecution Insights