DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The instant application claims benefit to provisional application No. 62/622,654 filed on 01/26/2018. Domestic benefit is acknowledged.
Drawings
The 21 page drawings have been considered and placed on record in the file.
Status of Claims
Claims 1-20 are currently pending.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 3-10 and 12-19 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-3, 5, 7, 9-10, 13-16, 18-19 of U.S. Patent No. 11,138,706. Although the claims at issue are not identical, they are not patentably distinct from each other for the reasons set forth below.
Parent Patent No. 11,138,706
Instant Application No. 19/029,329
Claim 1: A method of generating a combined image from a plurality of images, the method comprising:
defining a geographic area; receiving the plurality of images of the geographic area, wherein at least a portion of each received image is within the defined geographic area
…determining a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the received plurality of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points,
…and stitching the plurality of images into the combined image based on the plurality of image regions and the geographical coordinates of each image of the plurality of images...
Claim 1: A method of generating a combined image from a plurality of images, the method comprising: defining a geographic area; receiving the plurality of images of the geographic area, wherein at least a portion of each received image is within the defined geographic area and each image of the plurality of images includes geographical coordinates of a corresponding region of the geographic area…
Claim 1: A method of generating a combined image from a plurality of images, the method comprising: …the plurality of images of the geographic area, wherein at least a portion of each received image is within the defined geographic area and each image of the plurality of images includes geographical coordinates of a corresponding region of the geographic area….
Claim 1: A method of generating a combined image from a plurality of images, the method comprising: … stitching the plurality of images into the combined image based on the plurality of image regions and the geographical coordinates of each image of the plurality of images, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 2: The method of claim 1 wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.
Claim 3: The method of claim 1 further comprising: capturing the plurality of images by an aerial vehicle.
Claim 5: The method of claim 1 further comprising: filtering one or more images of the received plurality of images.
Claim 6 (Original): The method of claim 5 wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 7: The method of claim 1 further comprising: applying one or more image enhancements to one or more images of the received plurality of images.
Claim 8 (Original): The method of claim 7 wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 9: A system for generating a combined image from a plurality of images, the system comprising: an unmanned aerial vehicle (UAV) comprising: a processor having addressable memory;
…capture a plurality of images; and wherein the processor is configured to: receive a geographic area; receive the plurality of images from the sensor of the geographic area, wherein each image of the plurality of images includes geographical coordinates of a corresponding region of the geographic area;
determine a plurality of image points, wherein each image point is a geographic location of a center field of view of each image;
…partition the geographic area into a plurality of image regions based on the determined plurality of image points,
…stitch the plurality of images into the combined image based on the plurality of image regions and the geographical coordinates of each image of the plurality of images, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 9: A system for generating a combined image from a plurality of images, the system comprising: … the sensor configured to capture a plurality of images; and wherein the processor is configured to: receive a geographic area…
Claim 10: The system of claim 9 wherein the UAV further comprises: a global positioning system (GPS) in communication with the processor, wherein the processor uses the GPS to determine the geographic location of each image point of the plurality of image points.
Claim 15: The system of claim 9 further comprising: a controller comprising: a processor having addressable memory, wherein the processor is configured to: define the geographic area; send the geographic area to the UAV; and receive the combined image from the UAV.
Claim 16: The system of claim 15 further comprising: a computing device comprising a processor and addressable memory, wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.
Claim 13: The system of claim 9 wherein the processor is further configured to: filter one or more images of the received plurality of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 14: The system of claim 9 wherein the processor is further configured to: apply one or more image enhancements to one or more images of the received plurality of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 19: The system of claim 16 wherein the processor of the computing device is further configured to: receive the plurality of images; and stitch the plurality of images into a high resolution combined image based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 18: The system of claim 16 wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.
Claim 9: A system for generating a combined image from a plurality of images, the system comprising: receive a geographic area; receive the plurality of images from the sensor of the geographic area, wherein each image of the plurality of images includes geographical coordinates of a corresponding region of the geographic area;
…stitch the plurality of images into the combined image based on the plurality of image regions and the geographical coordinates of each image of the plurality of images, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 1: A method comprising:
determining a set of images of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;
determining a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 3: The method of claim 1, further comprising: defining the geographic area, wherein the geographic area comprises an area to be imaged.
Claim 4: The method of claim 1, wherein each of the set of images includes geographical coordinates of the portion of the geographic area, and wherein the geographical coordinates of at least two of the set of images are configured to overlap within the geographic area.
Claim 5: The method of claim 4, wherein generating the combined image is configured to be further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 6: The method of claim 1, wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.
Claim 7: The method of claim 1, further comprising: capturing the set of images by an aerial vehicle.
Claim 8: The method of claim 1, further comprising: filtering one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 9: The method of claim 1, further comprising: applying one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 10: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
determine a set of images capturing of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;
determine a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partition the geographic area into a plurality of image regions based on the determined plurality of image points; and
generate a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 12: The system of claim 10, further comprising: a sensor configured to be in communication with the processor, wherein the sensor is configured to capture the set of images.
Claim 13: The system of claim 10, further comprising: a global positioning system (GPS) configured to be in communication with the processor, wherein the processor is configured to use the GPS to determine the geographic location of each image point of the plurality of image points.
Claim 14: The system of claim 10, further comprising: a controller comprising: a processor having addressable memory, wherein the processor is configured to: define the geographic area; send the geographic area to the UAV; and receive the combined image from the UAV; a computing device comprising a processor and addressable memory, wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.
Claim 15: The system of claim 14, wherein the processor of the computing device is further configured to: filter one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 16: The system of claim 14, wherein the processor of the computing device is further configured to: apply one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 17: The system of claim 14, wherein the processor of the computing device is further configured to: receive the set of images; and stitch the set of images into a high resolution combined image based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 18: The system of claim 14, wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.
Claim 19: The system of claim 10, wherein the processor is further configured to: receive the geographic area, wherein the geographic area comprises an area to be imaged; wherein each image of the set of images includes geographical coordinates of a corresponding region of the geographic area;
and wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-7 and 9-20 of U.S. Patent No. 11,741,571. Although the claims at issue are not identical, they are not patentably distinct from each other for the reasons set forth below.
Parent Patent No. 11,741,571
Instant Application No. 19/029,329
Claim 1: A method comprising:
…determining a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used,
…determining a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 1: A method comprising:
receiving a plurality of images of a geographic area,
wherein a portion of each received image is within the geographic area, and wherein at least two of the received plurality of images overlap within the geographic area;
determining a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used, the selecting based on whether the image falls within a set criteria and a desired overlap remains above a set amount…
Claim 2: The method of claim 1 further comprising: defining a geographic area, wherein the geographic area comprises an area to be imaged.
Claim 3: The method of claim 1, wherein each received image includes geographical coordinates of the portion of the geographic area.
Claim 4: The method of claim 3, wherein the geographical coordinates of at least two of the received plurality of images overlap within the geographic area;
Claim 5: The method of claim I, wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 6: The method of claim 1, wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.
Claim 7: The method of claim 1 further comprising: capturing the plurality of images by an aerial vehicle.
Claim 9: The method of claim 1 further comprising: filtering one or more images of the received plurality of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 10: The method of claim 1 further comprising: applying one or more image enhancements to one or more images of the received plurality of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 11: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
…determine a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used,
…determine a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the determined set of images;
partition the geographic area into a plurality of image regions based on the determined plurality of image points;
and generate a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 11: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
receive a plurality of images of a geographic area, wherein a portion of each received image is within the geographic area,
and wherein at least two of the received plurality of images overlap within the geographic area;
…the selecting based on whether the image falls within a set criteria and a desired overlap remains above a set amount…
Claim 12: The system of claim 11 further comprising: a sensor in communication with the processor, the sensor configured to capture the plurality of images.
Claim 13: The system of claim 11 further comprising: a global positioning system (GPS) in communication with the processor, wherein the processor uses the GPS to determine the geographic location of each image point of the plurality of image points.
Claim 14: The system of claim 11 further comprising: a controller comprising: a processor having addressable memory, wherein the processor is configured to: define the geographic area; send the geographic area to the UAV; and receive the combined image from the UAV.
Claim 15: The system of claim 14 further comprising: a computing device comprising a processor and addressable memory, wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.
Claim 16: The system of claim 15, wherein the processor of the computing device is further configured to: filter one or more images of the received plurality of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 17: The system of claim 15, wherein the processor of the computing device is further configured to: apply one or more image enhancements to one or more images of the received plurality of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 18: The system of claim 15, wherein the processor of the computing device is further configured to: receive the plurality of images, and stitch the plurality of images into a high resolution combined image based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 19: The system of claim 15, wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.
Claim 20: The system of claim 11, wherein the processor is further configured to: receive a geographic area, wherein the geographic area comprises an area to be imaged; wherein each image of the plurality of images includes geographical coordinates of a corresponding region of the geographic area;
and wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 1: A method comprising:
… determining a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used, the selecting based on whether the image falls within a set criteria and a desired overlap remains above a set amount;
determining a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 1: A method comprising:
determining a set of images of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;
determining a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 2: The method of claim 1, further comprising
receiving a plurality of images of the geographic area,
wherein a portion of each received image is configured to be within the geographic area, wherein at least two of the received plurality of images are configured to be overlap within the geographic area,
wherein determining the set of images includes selecting at least two images capture the overlapping geographic area from the received plurality of images, and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.
Claim 3: The method of claim 1, further comprising: defining the geographic area, wherein the geographic area comprises an area to be imaged.
Claim 4: The method of claim 1, wherein each of the set of images includes geographical coordinates of the portion of the geographic area, and wherein the geographical coordinates of at least two of the set of images are configured to overlap within the geographic area.
Claim 5: The method of claim 4, wherein generating the combined image is configured to be further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 6: The method of claim 1, wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.
Claim 7: The method of claim 1, further comprising: capturing the set of images by an aerial vehicle.
Claim 8: The method of claim 1, further comprising: filtering one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 9: The method of claim 1, further comprising: applying one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 10: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
determine a set of images capturing of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;
determine a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partition the geographic area into a plurality of image regions based on the determined plurality of image points; and
generate a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 11: The system of claim 10, wherein the processor is further configured to:
receive a plurality of images of the geographic area, wherein a portion of each received image is within the geographic area,
wherein the set of images are configured to be determined by selecting at least two images capturing the overlapping geographic area from the received plurality of images,
and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.
Claim 12: The system of claim 10, further comprising: a sensor configured to be in communication with the processor, wherein the sensor is configured to capture the set of images.
Claim 13: The system of claim 10, further comprising: a global positioning system (GPS) configured to be in communication with the processor, wherein the processor is configured to use the GPS to determine the geographic location of each image point of the plurality of image points.
Claim 14: The system of claim 10, further comprising: a controller comprising: a processor having addressable memory, wherein the processor is configured to: define the geographic area; send the geographic area to the UAV; and receive the combined image from the UAV; a computing device comprising a processor and addressable memory, wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.
Claim 15: The system of claim 14, wherein the processor of the computing device is further configured to: filter one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 16: The system of claim 14, wherein the processor of the computing device is further configured to: apply one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 17: The system of claim 14, wherein the processor of the computing device is further configured to: receive the set of images; and stitch the set of images into a high resolution combined image based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 18: The system of claim 14, wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.
Claim 19: The system of claim 10, wherein the processor is further configured to: receive the geographic area, wherein the geographic area comprises an area to be imaged; wherein each image of the set of images includes geographical coordinates of a corresponding region of the geographic area;
and wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 20: A method comprising:
determining a set of images configured to capture a geographic area, based on at least one of: whether the image falls within a set criteria and whether a desired overlap among the set of images remains above a set amount;
determining a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points; and
generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claims 1-20 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-7 and 9-20 of U.S. Patent No. 12,260,513. Although the claims at issue are not identical, they are not patentably distinct from each other for the reasons set forth below.
Parent Patent No. 12,260,513
Instant Application No. 19/029,329
Claim 1: A method comprising:
determining a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used;
determining a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 1: A method comprising:
receiving a plurality of images of a geographic area, wherein a portion of each received image is within the geographic area,
and wherein at least two of the received plurality of images overlap within the geographic area;
determining a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used…
Claim 2: The method of claim 1, wherein the selecting is based on whether the image falls within a set criteria and a desired overlap remains above a set amount.
Claim 3: The method of claim 1 further comprising: defining a geographic area, wherein the geographic area comprises an area to be imaged.
Claim 4: The method of claim 1, wherein each received image includes geographical coordinates of the portion of the geographic area, and wherein the geographical coordinates of at least two of the received plurality of images overlap within the geographic area.
Claim 5: The method of claim 1, wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 6: The method of claim 1, wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.
Claim 7: The method of claim 1 further comprising: capturing the plurality of images by an aerial vehicle.
Claim 9: The method of claim 1 further comprising: filtering one or more images of the received plurality of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 10: The method of claim 1 further comprising: applying one or more image enhancements to one or more images of the received plurality of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 11: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
…determine a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used;
determine a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the determined set of images;
partition the geographic area into a plurality of image regions based on the determined plurality of image points; and
generate a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 11: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
receive a plurality of images of a geographic area,
wherein a portion of each received image is within the geographic area,
…determine a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used…
Claim 12: The system of claim 11, wherein the selecting is based on whether the image falls within a set criteria and a desired overlap remains above a set amount.
Claim 13: The system of claim 11 further comprising: a sensor in communication with the processor, the sensor configured to capture the plurality of images.
Claim 14: The system of claim 11 further comprising: a global positioning system (GPS) in communication with the processor, wherein the processor uses the GPS to determine the geographic location of each image point of the plurality of image points.
Claim 15: The system of claim 11 further comprising: a controller comprising: a processor having addressable memory, wherein the processor is configured to: define the geographic area; send the geographic area to the UAV; and receive the combined image from the UAV; a computing device comprising a processor and addressable memory, wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.
Claim 16: The system of claim 15, wherein the processor of the computing device is further configured to: filter one or more images of the received plurality of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 17: The system of claim 15, wherein the processor of the computing device is further configured to: apply one or more image enhancements to one or more images of the received plurality of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 18: The system of claim 15, wherein the processor of the computing device is further configured to: receive the plurality of images; and stitch the plurality of images into a high resolution combined image based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 19: The system of claim 15, wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.
Claim 20: The system of claim 11, wherein the processor is further configured to: receive a geographic area, wherein the geographic area comprises an area to be imaged; wherein each image of the plurality of images includes geographical coordinates of a corresponding region of the geographic area;
and wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 1: A method comprising:
determining a set of images from the received plurality of images for further processing by selecting at least one image of the plurality of images capturing an overlapping geographic area to be used;
Claim 2: The method of claim 1, wherein the selecting is based on whether the image falls within a set criteria and a desired overlap remains above a set amount.
Claim 1: A method comprising: …determining a plurality of image points, wherein each image point is a geographic location of a center field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 1: A method comprising:
determining a set of images of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;
determining a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;
and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 2: The method of claim 1, further comprising
receiving a plurality of images of the geographic area, wherein a portion of each received image is configured to be within the geographic area,
wherein at least two of the received plurality of images are configured to be overlap within the geographic area,
wherein determining the set of images includes selecting at least two images capture the overlapping geographic area from the received plurality of images,
and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.
Claim 3: The method of claim 1, further comprising: defining the geographic area, wherein the geographic area comprises an area to be imaged.
Claim 4: The method of claim 1, wherein each of the set of images includes geographical coordinates of the portion of the geographic area, and wherein the geographical coordinates of at least two of the set of images are configured to overlap within the geographic area.
Claim 5: The method of claim 4, wherein generating the combined image is configured to be further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 6: The method of claim 1, wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.
Claim 7: The method of claim 1, further comprising: capturing the set of images by an aerial vehicle.
Claim 8: The method of claim 1, further comprising: filtering one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 9: The method of claim 1, further comprising: applying one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 10: A system comprising: an unmanned aerial vehicle (UAV) comprising a processor having addressable memory, wherein the processor is configured to:
determine a set of images capturing of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;
determine a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partition the geographic area into a plurality of image regions based on the determined plurality of image points; and
generate a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim 11: The system of claim 10, wherein the processor is further configured to:
receive a plurality of images of the geographic area,
wherein a portion of each received image is within the geographic area, wherein the set of images are configured to be
determined by selecting at least two images capturing the overlapping geographic area from the received plurality of images,
and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.
Claim 12: The system of claim 10, further comprising: a sensor configured to be in communication with the processor, wherein the sensor is configured to capture the set of images.
Claim 13: The system of claim 10, further comprising: a global positioning system (GPS) configured to be in communication with the processor, wherein the processor is configured to use the GPS to determine the geographic location of each image point of the plurality of image points.
Claim 14: The system of claim 10, further comprising: a controller comprising: a processor having addressable memory, wherein the processor is configured to: define the geographic area; send the geographic area to the UAV; and receive the combined image from the UAV; a computing device comprising a processor and addressable memory, wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.
Claim 15: The system of claim 14, wherein the processor of the computing device is further configured to: filter one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.
Claim 16: The system of claim 14, wherein the processor of the computing device is further configured to: apply one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.
Claim 17: The system of claim 14, wherein the processor of the computing device is further configured to: receive the set of images; and stitch the set of images into a high resolution combined image based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.
Claim 18: The system of claim 14, wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.
Claim 19: The system of claim 10, wherein the processor is further configured to: receive the geographic area, wherein the geographic area comprises an area to be imaged; wherein each image of the set of images includes geographical coordinates of a corresponding region of the geographic area;
and wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.
Claim 20: A method comprising:
determining a set of images configured to capture a geographic area,
based on at least one of: whether the image falls within a set criteria and whether a desired overlap among the set of images remains above a set amount;
determining a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;
partitioning the geographic area into a plurality of image regions based on the determined plurality of image points; and
generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Consider Independent Claim 1 is directed to determining a set of images of an overlapping geographic area and a plurality of common image points between the images to segment the geographic area and stitching the images together.
Step 1:
With regard to Step 1, the instant claims are directed to a method among the statutory categories of invention.
Step 2A – Prong 1:
With regard to Step 2A – Prong 1, for example in method Claim 1, the limitations: “determining a set of images of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;”, “determining a plurality of image points, wherein each image point is includes a selected geographic location of a field of view of each image of the determined set of images;”, “partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;” and “and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.”, as recited, is a method that, under its broadest reasonable interpretation, covers performance of the limitation in the mind/observation of a person inspecting a set of images to determined images with a common, overlapping geographic area and identifying a plurality of shared image points to segment the geographic area and stitch the images together. That is nothing in the claim steps preclude the limitations from practically being performed in the mind or through observation/judgement to determine an overlapping geographic area and common image points, and/or segmenting an image based on the overlapping geographic area to stitch images together. If a claim limitation, under its broadest reasonably interpretation covers performance of the limitation in the mind then it falls within the "Mental processes" grouping of the abstract idea, which include concepts performed in the human mind, including an observation, evaluation, judgement, opinion. Accordingly, the claim recites an abstract idea
Step 2A – Prong 2:
The 2019 PEG defines the phrase “integration into a practical application” to require an additional element or a combination of additional elements in the claim to apply, rely on, or use the judicial exception. In the instant case, the additional elements in the claims do not apply, rely on, or use the judicial exception.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Accordingly, these claims are not patent eligible.
Consider Independent Claim 10 is directed to a system for determining a set of images of an overlapping geographic area and a plurality of common image points between the images to segment the geographic area and stitching the images together. Claim 10 recites a system with elements corresponding to the steps recited in Claim 1. Therefore, the recited elements of this claim are rejected under 35 U.S.C. 101 in the same manner as the corresponding steps in its corresponding method claim 1.
Consider Independent Claim 20 is directed to determining a set of images of an overlapping geographic area, based on a minimum amount of overlap, and a plurality of common image points between the images to segment the geographic area and stitching the images together. Claim 20 recites a method with steps corresponding to the steps recited in Claim 1. Therefore, the recited steps of this claim are rejected under 35 U.S.C. 101 in the same manner as the corresponding steps in its corresponding method claim 1.
Further, with regard to dependent Claims 2-9 and 11-19 viewed individually, these additional elements are under their broadest reasonable interpretation, cover performance of the limitation in the mind and do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Accordingly, Claims 2-9 and 11-19 are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 6, 9 and 12 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009).
Consider Claim 1, Pan teaches “A method comprising: determining a set of images of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;” (Pan; Fig. 6 (See image below)) “determining a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;” (Pan; Section 2; “Let A={A1,A2,…,An} be a finite set of distinct areas in the 2-D Cartesian space R2. In A, the arbitrary area is not belonging to another area, but two arbitrary areas may overlap each other. Let Ai and Aj be the two arbitrary distinct areas in A. Let Ai∩Aj=Oj, Ai=A′i∪Oj, Aj=A′j∪Oj, p be an arbitrary point in Ai∪Aj, and da(p,Ai,Aj) be the Euclidean distance between point p and area Ai in the restriction of Aj. The da(p,Ai,Aj) is defined as the minimum Euclidean distance between point p and an arbitrary point in A′i, the nonoverlapping part of Ai between Ai and Aj.”) “partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;” (Pan; Fig. 7 (See image below)) “and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.” (Pan; Fig. 8 (See image below)).
PNG
media_image1.png
435
938
media_image1.png
Greyscale
PNG
media_image2.png
440
938
media_image2.png
Greyscale
PNG
media_image3.png
442
940
media_image3.png
Greyscale
Consider Claim 6, Pan teaches “The method of claim 1, wherein partitioning the geographic area into the plurality of image regions further comprises: generating a Voronoi diagram.” (Pan; Fig. 5 (See image below); Section 1; “In this paper, a novel area Voronoi diagrams with overlap (AVDO) is presented.” ).
PNG
media_image4.png
633
939
media_image4.png
Greyscale
Consider Claim 9, Pan teaches “The method of claim 1, further comprising: applying one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.” (Pan; Section 4; “Our algorithm was applied into the mosaicking. The mosaicking process used in this study includes the following steps: 1) to do the color balancing to minimize the radiometric differences between different orthoimages…”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in view of Simek et. al. (US 20180139431).
Consider Claim 2, Pan teaches “The method of claim 1, further comprising receiving a plurality of images of the geographic area,” (Pan; Fig. 6 (See image above)) “wherein a portion of each received image is configured to be within the geographic area, wherein at least two of the received plurality of images are configured to be overlap within the geographic area,” (Pan; Section 2; “In A, the arbitrary area is not belonging to another area, but two arbitrary areas may overlap each other.”) “wherein determining the set of images includes selecting at least two images capture the overlapping geographic area from the received plurality of images,” (Pan; Section 3.C; “Moreover, it is still necessary to construct the topological relation between seamlines and Voronoi polygons to benefit the mosaicking, e.g., to determine the adjacent Voronoi polygons to each seamline, the corresponding orthoimage to each Voronoi polygon, and the corresponding overlap to each seamline.”)
Pan does not explicitly disclose “and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.”. However, in an analogous field of endeavor, Simek teaches “and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.” (Simek; [0165]; “For example, in one or more exemplary embodiments, the depth-assisted stereo component 1302 can be configured to initially analyze the 2D images captured by the 2D/3D panoramic capture device to identify pairs of images depicting the same structure by counting keypoint matches that are visually similar and are geometrically consistent with the known poses of the cameras that captured the respective images….The depth-assisted stereo component 1302 can then filter out image pairs that have insufficient relative displacement relative to a threshold displacement, have insufficient overlap of viewing volumes relative to a threshold overlap volume, or whose difference in viewing angles are sufficiently large relative to a threshold angle.” (emphasis added)). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan with the teachings of Simek to further stitch two images based on the amount of overlap between the images. One of ordinary skill in the art would be motivated to combine Pan and Simek to more efficiently determine which corresponding images to stitch together accurately and reduce computational time. Accordingly, the combination of Pan and Simek discloses the invention of Claim 1.
Consider Claim 20, Pan teaches “A method comprising: determining a set of images configured to capture a geographic area,” (Pan; Fig. 6 (See image below)) “(Pan; Section 2; “Let A={A1,A2,…,An} be a finite set of distinct areas in the 2-D Cartesian space R2. In A, the arbitrary area is not belonging to another area, but two arbitrary areas may overlap each other. Let Ai and Aj be the two arbitrary distinct areas in A. Let Ai∩Aj=Oj, Ai=A′i∪Oj, Aj=A′j∪Oj, p be an arbitrary point in Ai∪Aj, and da(p,Ai,Aj) be the Euclidean distance between point p and area Ai in the restriction of Aj. The da(p,Ai,Aj) is defined as the minimum Euclidean distance between point p and an arbitrary point in A′i, the nonoverlapping part of Ai between Ai and Aj.”) “partitioning the geographic area into a plurality of image regions based on the determined plurality of image points;” (Pan; Fig. 7 (See image above)) “and generating a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.” (Pan; Fig. 8 (See image above)).
Pan does not explicitly disclose “based on at least one of: whether the image falls within a set criteria and whether a desired overlap among the set of images remains above a set amount;”. However, in an analogous field of endeavor, Simek teaches “based on at least one of: whether the image falls within a set criteria and whether a desired overlap among the set of images remains above a set amount;” (Simek; [0165]; “For example, in one or more exemplary embodiments, the depth-assisted stereo component 1302 can be configured to initially analyze the 2D images captured by the 2D/3D panoramic capture device to identify pairs of images depicting the same structure by counting keypoint matches that are visually similar and are geometrically consistent with the known poses of the cameras that captured the respective images….The depth-assisted stereo component 1302 can then filter out image pairs that have insufficient relative displacement relative to a threshold displacement, have insufficient overlap of viewing volumes relative to a threshold overlap volume, or whose difference in viewing angles are sufficiently large relative to a threshold angle.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan with the teachings of Simek to further stitch two images based on the amount of overlap between the images. One of ordinary skill in the art would be motivated to combine Pan and Simek to more efficiently determine which corresponding images to stitch together accurately and reduce computational time. Accordingly, the combination of Pan and Simek discloses the invention of Claim 1.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in view of Ezequiel et. al. ("UAV aerial imaging applications for post-disaster assessment, environmental management and infrastructure development” published on May 27-30, 2014).
Consider Claim 8, Pan does not explicitly disclose the limitations of Claim 8. However, in an analogous field of endeavor, Richman teaches “The method of claim 1, further comprising: filtering one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.” (Ezequiel; Section 2.B; “The transformation of aerial images into orthomosaics involves the following steps. First, the dataset of aerial images are manually filtered to remove take-off/landing, blurry and oblique (i.e. non-level flight) images.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan with the teachings of Ezequiel to further remove images due to blur. One of ordinary skill in the art would be motivated to combine Pan and Richman to remove blurry images in order to output a clear overall image of a geographic image. Accordingly, the combination of Pan and Richman discloses the invention of Claim 8.
Claims 3-5, 7, 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in view of Chao et. al. (“Band-reconfigurable multi-UAV-based cooperative remote sensing for real-time water management and distributed irrigation control” published on July 6-11, 2008).
Consider Claim 3, Pan does not explicitly disclose the limitations of Claim 3. However, in an analogous field of endeavor, Chao teaches “The method of claim 1, further comprising: defining the geographic area, wherein the geographic area comprises an area to be imaged. (Chao; Fig. 7 (See image below)). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan with the teachings of Chao to further define a geographic area to capture. One of ordinary skill in the art would be motivated to combine Pan and Chao to define the geographic region of interest for image stitching to crop the combine image to reduce noise and focus on the area of interest for more efficient further analysis of the output image. Accordingly, the combination of Pan and Chao discloses the invention of Claim 3.
PNG
media_image5.png
457
794
media_image5.png
Greyscale
Consider Claim 4, the combination of Pan and Chao teaches “The method of claim 1, wherein each of the set of images includes geographical coordinates of the portion of the geographic area,” (Chao; Section 3.2; “There are also feature-based stitching methods that georeference the photos. These take features on each photo and compare them to photos which already have GIS information. This method performs the stitching task better because each photo is georeferenced and has some absolute coordinates to tie to.”) “and wherein the geographical coordinates of at least two of the set of images are configured to overlap within the geographic area.” (Chao; Fig. 3 (See image below)). The proposed combination as well as the motivation for combining the Pan and Chao references presented in the rejection of Claim 3, apply to Claim 4 and are incorporated herein by reference. Thus, the method recited in Claim 4 is met by Pan and Chao.
PNG
media_image6.png
310
636
media_image6.png
Greyscale
Consider Claim 5, the combination of Pan and Chao teaches “The method of claim 4, wherein generating the combined image is configured to be further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.” (Chao; Fig. 3 (See image above)). The proposed combination as well as the motivation for combining the Pan and Chao references presented in the rejection of Claim 3, apply to Claim 5 and are incorporated herein by reference. Thus, the method recited in Claim 5 is met by Pan and Chao.
Consider Claim 7, the combination of Pan and Chao teaches “The method of claim 1, further comprising: capturing the set of images by an aerial vehicle.” (Chao; Abstract; “This paper presents an overview of ongoing research on small unmanned autonomous vehicles (UAVs) for cooperative remote sensing for real-time water management and irrigation control…. Keyworks: Unmanned aerial vehicle…”). The proposed combination as well as the motivation for combining the Pan and Chao references presented in the rejection of Claim 3, apply to Claim 7 and are incorporated herein by reference. Thus, the method recited in Claim 7 is met by Pan and Chao.
Consider Claim 10, the combination of Pan and Chao teaches “A system comprising: (Pan; Section 4; “A desktop computer with an Intel Pentium 4 CPU at 2.4 GH... and a hard disk with a capacity of 320 GB…”) “wherein the processor is configured to: determine a set of images capturing of a geographic area, wherein the set of images are configured to capture an overlapping geographic area to be used;” (Pan; Fig. 6 (See image above)) “determine a plurality of image points, wherein each image point includes a selected geographic location of a field of view of each image of the determined set of images;” (Pan; Section 2; “Let A={A1,A2,…,An} be a finite set of distinct areas in the 2-D Cartesian space R2. In A, the arbitrary area is not belonging to another area, but two arbitrary areas may overlap each other. Let Ai and Aj be the two arbitrary distinct areas in A. Let Ai∩Aj=Oj, Ai=A′i∪Oj, Aj=A′j∪Oj, p be an arbitrary point in Ai∪Aj, and da(p,Ai,Aj) be the Euclidean distance between point p and area Ai in the restriction of Aj. The da(p,Ai,Aj) is defined as the minimum Euclidean distance between point p and an arbitrary point in A′i, the nonoverlapping part of Ai between Ai and Aj.”) “partition the geographic area into a plurality of image regions based on the determined plurality of image points;” (Pan; Fig. 7 (See image above)) “and generate a combined image by stitching the partitioned geographic area of the determined set of images into the combined image.” (Pan; Fig. 8 (See image above)).
Pan does not explicitly disclose “A system comprising: an unmanned aerial vehicle (UAV)”. However, in an analogous field of endeavor, Chao teaches “A system comprising: an unmanned aerial vehicle (UAV)” (Chao; Abstract; “This paper presents an overview of ongoing research on small unmanned autonomous vehicles (UAVs) for cooperative remote sensing for real-time water management and irrigation control…. Keyworks: Unmanned aerial vehicle…”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan with the teachings of Chao to further implement an unmanned aerial vehicle to capture images of a geographical area. One of ordinary skill in the art would be motivated to combine Pan and Chao to use take aerial images to create an accurate image of a larger area. Accordingly, the combination of Pan and Chao discloses the invention of Claim 10.
Consider Claim 12, the combination of Pan and Chao teaches “The system of claim 10, further comprising: a sensor configured to be in communication with the processor, wherein the sensor is configured to capture the set of images.” (Pan; Section 3.A; “Most of the remote sensing images acquired by airborne and satellite borne sensors are captured along strips, and mosaicking these images to cover a large geographic region is one of the routine tasks at the mapping agencies or companies.”).
Consider Claim 19, the combination of Pan and Chao teaches “The system of claim 10, wherein the processor is further configured to: receive the geographic area, wherein the geographic area comprises an area to be imaged;” (Chao; Fig. 7 (See image above)) “wherein each image of the set of images includes geographical coordinates of a corresponding region of the geographic area;” (Chao; Fig. 7 (See image above); Examiner notes it would be obvious the geographical coordinates of the images captured during the preplanned flight path would correspond to the defined geographic area.) “and wherein generating the combined image is further based on the plurality of image regions and the geographical coordinates of each image of the determined set of images.” (Chao; Fig. 3 (See image above)). The proposed combination as well as the motivation for combining the Pan and Chao references presented in the rejection of Claim 10, apply to Claim 19 and are incorporated herein by reference. Thus, the method recited in Claim 19 is met by Pan and Chao.
Claims 11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in Chao et. al. (“Band-reconfigurable multi-UAV-based cooperative remote sensing for real-time water management and distributed irrigation control” published on July 6-11, 2008) and further view of in view of Simek et. al. (US 20180139431).
Consider Claim 11, the combination of Pan and Chao teaches “The system of claim 10, wherein the processor is further configured to: receive a plurality of images of the geographic area,” (Chao; Section 5.2; “Both the RGB and NIR videos are transmitted back to the ground station in real time and saved on the base laptop as a MPEG file for further processing.”) “wherein a portion of each received image is within the geographic area,” (Chao; Fig. 7 (See image above)) “
The combination of Pan and Chao does not explicitly disclose “wherein the set of images are configured to be determined by selecting at least two images capturing the overlapping geographic area from the received plurality of images, and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.”. However, in an analogous field of endeavor, Simek teaches “wherein the set of images are configured to be determined by selecting at least two images capturing the overlapping geographic area from the received plurality of images, and wherein selecting the at least two images is configured to be based on at least one of: whether the image falls within a set criteria and whether a desired overlap remains above a set amount.” (Simek; [0165]; “For example, in one or more exemplary embodiments, the depth-assisted stereo component 1302 can be configured to initially analyze the 2D images captured by the 2D/3D panoramic capture device to identify pairs of images depicting the same structure by counting keypoint matches that are visually similar and are geometrically consistent with the known poses of the cameras that captured the respective images….The depth-assisted stereo component 1302 can then filter out image pairs that have insufficient relative displacement relative to a threshold displacement, have insufficient overlap of viewing volumes relative to a threshold overlap volume, or whose difference in viewing angles are sufficiently large relative to a threshold angle.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan and Chao with the teachings of Simek to further determine pairs of images based on an overlap threshold. One of ordinary skill in the art would be motivated to combine Pan, Chao, and Simek to more efficiently determine which corresponding images to stitch together accurately and reduce computational time. Accordingly, the combination of Pan, Chao, and Simek discloses the invention of Claim 11.
Consider Claim 13, the combination of Pan, Chao and Simek teaches “The system of claim 10, further comprising: a global positioning system (GPS) configured to be in communication with the processor, (Chao; Section 3; “…the aerial images with both temporal and spatial information, for example the GPS coordinates and time at which the picture was taken.”) “wherein the processor is configured to use the GPS to determine the geographic location of each image point of the plurality of image points.” (Simek; [0117] In one or more embodiments, the alignment process can involve determining position (e.g., relative to a 3D coordinate space) and visual feature data for respective points in received 2D images and 2D panoramas associated with known camera capture positions and orientations relative to a global 3D coordinate space or volume. The 2D images and panoramas, 3D data respectively associated with the 2D images and 2D panoramas, feature data, and other sensor data (if available) can then be used as inputs to an algorithm that determines potential alignments between the different 2D images and 2D panoramas via coordinate transformations.”; Examiner notes Simek teaches a location component that “can determine a location of the 2D/3D panoramic capture device 400 using global positioning system (GPS) technology.” (Simek; [0094])). The proposed combination as well as the motivation for combining the Pan, Chao and Simek references presented in the rejection of Claim 11, apply to Claim 13 and are incorporated herein by reference. Thus, the method recited in Claim 13 is met by Pan, Chao and Simek.
Claims 14 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in Chao et. al. (“Band-reconfigurable multi-UAV-based cooperative remote sensing for real-time water management and distributed irrigation control” published on July 6-11, 2008) and further view of in view of Guissin et. al. (US 20140132804).
Consider Claim 14, the combination of Pan and Chao teaches “The system of claim 10, further comprising: a controller comprising: a processor having addressable memory,” (Pan; Section 4; “A desktop computer with an Intel Pentium 4 CPU at 2.4 GH... and a hard disk with a capacity of 320 GB…”) “wherein the processor is configured to: define the geographic area;” (Chao; Fig. 7 (See image below) “send the geographic area to the UAV;” (Chao; Fig. 7 (See image below; Examiner notes it would be obvious to someone in the art to send the planned flight plan to the UAV to capture images of the desired geographic area.) “(Pan; Section 4; “A desktop computer with an Intel Pentium 4 CPU at 2.4 GH... and a hard disk with a capacity of 320 GB…”) “
The combination of Pan and Chao does not explicitly disclose “…and receive the combined image from the UAV;” or “…wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.”. However, in an analogous field of endeavor, Guissin teaches “and receive the combined image from the UAV;” (Guissin; [0134]; “At 510, a multidirectional imaging device 100 according to the present invention is shown, along with images from four FOVs….The image 520 is the combined image from all the FOVs detected by the sensor of the device 100. The combined image 520 is fed into video utility…”) and “wherein the processor is configured to: receive the combined image from at least one of: the UAV and the controller; and analyze the combined image.” (Guissin; [0004]; “This image data indicative of the combined image data is processed and analyzed, and if existence of a certain condition or event is identified in the image data, a corresponding control signal is generated.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan and Chao with the teachings of Guissin to further analyze the combined output image. One of ordinary skill in the art would be motivated to combine Pan, Chao and Guissin to determine a condition or event in a geographic area from the combined image. Accordingly, the combination of Pan, Chao and Guissin discloses the invention of Claim 14.
Consider Claim 16, the combination of Pan, Chao, and Guissin teaches “The system of claim 14, wherein the processor of the computing device is further configured to: apply one or more image enhancements to one or more images of the set of images, wherein applying image enhancements to the one or more images comprises at least one of: brightening, darkening, color correcting, white balancing, sharpening, correcting lens distortion, and adjusting contrast.” (Pan; Section 3.C; “Moreover, radiometric or color differences between neighboring orthoimages may also exist. These differences cause additional problems in mosaicking. Therefore, further refinement of the seamline network is needed.”).
Consider Claim 17, the combination of Pan, Chao, and Guissin teaches “The system of claim 14, wherein the processor of the computing device is further configured to: receive the set of images;” (Chao; Section 5.2; “Both the RGB and NIR videos are transmitted back to the ground station in real time and saved on the base laptop as a MPEG file for further processing.”) “and stitch the set of images into a high resolution combined image” (Pan; Section 4; “The resolution of the orthoimage is 0.2 m, and the image size is approximately 3800 by 3800 pixels.”) “based on the plurality of image regions, wherein each pixel in the combined image is selected from its corresponding image region.” (Pan; Abstract; “…a seamline network formed by a novel area Voronoi diagrams with overlap and the use of effective mosaic polygons (EMPs) to define the pixels of each orthoimage for the final mosaic.” (emphasis added)). The proposed combination as well as the motivation for combining the Pan, Chao and Simek references presented in the rejection of Claim 14, apply to Claim 17 and are incorporated herein by reference. Thus, the method recited in Claim 17 is met by Pan, Chao and Simek.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in view of Chao et. al. (“Band-reconfigurable multi-UAV-based cooperative remote sensing for real-time water management and distributed irrigation control” published on July 6-11, 2008) and further view of in view of Guissin et. al. (US 20140132804) and in further view of Ezequiel et. al. ("UAV aerial imaging applications for post-disaster assessment, environmental management and infrastructure development” published on May 27-30, 2014).
Consider Claim 15, the combination of Pan, Chao and Guissin does not explicitly disclose the limitations of Claim 15. However, in an analogous field of endeavor, Ezequiel teaches “The system of claim 14, wherein the processor of the computing device is further configured to: filter one or more images of the set of images, wherein filtering the one or more images further comprises: removing the one or more images due to at least one of: overexposure, underexposure, distortion, blur, and an error with a camera taking the image.” (Ezequiel; Section 2.B; “The transformation of aerial images into orthomosaics involves the following steps. First, the dataset of aerial images are manually filtered to remove take-off/landing, blurry and oblique (i.e. non-level flight) images.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan, Chao and Guissin with the teachings of Ezequiel to further remove images due to blur. One of ordinary skill in the art would be motivated to combine Pan and Richman to remove blurry images in order to output a clear overall image of a geographic image. Accordingly, the combination of Pan, Chao, Guissin and Ezequiel discloses the invention of Claim 15.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Pan et. al. (“Automatic generation of seamline network using area Voronoi diagrams with overlap.” with publishing date of April 21, 2009) in Chao et. al. (“Band-reconfigurable multi-UAV-based cooperative remote sensing for real-time water management and distributed irrigation control” published on July 6-11, 2008) and in further view of in view of Guissin et. al. (US 20140132804) and in further view of Steedly et. al. (US 20070237420).
Consider Claim 18, the combination of Pan, Chao and Gussin does not explicitly disclose the limitations of Claim 18. However, in an analogous field of endeavor, Steedly teaches “The system of claim 14, wherein the processor of the computing device is further configured to smooth the combined image to account for at least one of: brightness, color, dead pixels, and lens distortion.” (Steedly; [0114]; “Therefore, after choosing the seams between images, the Oblique Stitcher interpolates a smooth mapping between the input and output images…. The Oblique Image Stitcher then interpolates a smoother distortion field between the seams. To generate the output mosaic, the oblique Image Stitcher then rewarps the input images using the composed homography warp and smoothed distortion field. Finally, the Oblique Image Stitcher blends the rewarped input images on the mosaic using conventional gradient-domain fusion techniques.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Pan, Chao and Guissin with the teachings of Steedly to further smooth the combined image. One of ordinary skill in the art would be motivated to combine Pan, Chao, Guissin, and Steedly to create a clear and detailed combined image. Accordingly, the combination of Pan, Chao, Guissin, and Steedly discloses the invention of Claim 18.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Annie Pham whose telephone number is (571)272-1673. The examiner can normally be reached Mon-Fri 9:00a – 5:00p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For
additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNIE H PHAM/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights