DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 4, 7, 10, 14, 17 and 20 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
With regards to claims 4 and 14, several of the features of these claims were known in the art as evidenced by the combination of Ruecker et al, “High Resolution Active Fire Monitoring for Global Change Analysis: The Upcoming FireBIRD Satellite Mission” in view of Warren (US Patent No. 12,384,561) and Nguyen et al (US PG Pub. No. 2021/0225011) which renders obvious the limitations of parent claims 1 and 11, respectively. In particular, Ruecker discloses detecting one or more features in the one or more captured images at p. 4, sec. 3 (“Both satellites will be equipped with on-board processing capabilities that will allow for: … Hotspot detection – based on the BIRD algorithms”); see, also, pp. 2-3 (“BIRD was able to detect and characterize extremely small as well as the very large and intense fires.”) However, Ruecker does not disclose receiving a feature detection method information structure that identifies a technique for identifying, detecting, or extracting relevant features or patterns from images.
With regards to claims 7 and 17, several of the features of these claims were known in the art as evidenced by the combination of Ruecker et al, “High Resolution Active Fire Monitoring for Global Change Analysis: The Upcoming FireBIRD Satellite Mission” in view of Warren (US Patent No. 12,384,561) and Nguyen et al (US PG Pub. No. 2021/0225011) which renders obvious the limitations of parent claims 1 and 11, respectively. In particular, Ruecker discloses transmitting the geolocation data and associated metadata at p. 4, sec. 3 (“On board processing allows for extremely rapid delivery to fire information systems…”). However, Ruecker does not disclose downscaling the images to generate thumbnails and sending the generated thumbnails over the low-bandwidth link.
With regards to claims 10 and 20, several of the features of these claims were known in the art as evidenced by the combination of Ruecker et al, “High Resolution Active Fire Monitoring for Global Change Analysis: The Upcoming FireBIRD Satellite Mission” in view of Warren (US Patent No. 12,384,561) and Nguyen et al (US PG Pub. No. 2021/0225011) which renders obvious the limitations of parent claims 1 and 11, respectively. In particular, Ruecker discloses transmitting the geolocation data and associated metadata at p. 4, sec. 3 (“On board processing allows for extremely rapid delivery to fire information systems…”). However, Ruecker does not disclose transmitting over the low-bandwidth link the geolocation data and associated metadata to a follow-on satellite deployed at the extreme network edge.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With respect to independent claims 1, 11 and 21, the term “low-bandwidth link” is a relative term which renders the claim indefinite. The term “low-bandwidth link” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. A threshold bandwidth such that one of ordinary skill in the art would understand what was meant by “low bandwidth” has not been set out in the application as filed.
With regards to claims 2-10 and 12-20, these claims depend from claim 1 and 10, respectively, and therefore incorporate the features of those claims that were found indefinite under 35 USC § 112(b). Furthermore, the additional limitations of these claims fail to rectify the shortcomings of their parent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 6, 8-9, 11, 13, 16 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Ruecker et al, “High Resolution Active Fire Monitoring for Global Change Analysis: The Upcoming FireBIRD Satellite Mission” in view of Warren (US Patent No. 12,384,561) and Nguyen et al (US PG Pub. No. 2021/0225011).
With regards to claim 1, the limitations of this claim are obvious over the teachings of the prior art, as evidenced by the following references:
The Ruecker reference
Ruecker discloses capturing one or more images at p. 6; see, also, Table 2 and Abstract.
Ruecker discloses detecting one or more features in the one or more captured images at p. 4, sec. 3 (“Both satellites will be equipped with on-board processing capabilities that will allow for: … Hotspot detection – based on the BIRD algorithms”); see, also, pp. 2-3 (“BIRD was able to detect and characterize extremely small as well as the very large and intense fires.”)
Ruecker discloses processing the detected features by clustering contiguous features p. 4, sec. 3 (“Both satellites will be equipped with on-board processing capabilities that will allow for: … derivation of hot cluster (groups of hot pixels)”); see, also, p. 7 (“Fire clusters were generated by grouping adjacent fire pixels into a cluster.”) Ruecker does not specify processing the detected features by filtering noise, however, this limitation was known in the art as evidenced by the Nguyen reference discussed below.
Ruecker discloses generating geolocation data for the detected features based on geographic coordinates at p. 4 (“Both satellites will be equipped with on-board processing capabilities that will allow for: … Extraction of fire attributes of the hot clusters, such as location…”) and p. 6, which discloses: “Results are output to a fire cluster table, which provides the brightness center coordinates for each cluster… Geolocation error was between one and one and a half pixels (370 m).”) Ruecker discloses georeferencing the satellite images at p. 6 (“Output images were georeferenced to their corresponding UTM zone using a set of ground control points and a third order polynom to a reference satellite image…”), but Ruecker does not specify that the geographic coordinates were derived by applying a transformation to the pixel coordinates of the processed detected features to determine geographic coordinates of the detected features. However, this limitation was known in the art as evidenced by the Warren reference discussed below.
Ruecker discloses transmitting the geolocation data and associated metadata at p. 4, sec. 3 (“On board processing allows for extremely rapid delivery to fire information systems…”)
The Warren reference
Warren discloses receiving from a mission operations center (e.g., “a consumer or user of the services and satellite imagery provided by the satellite”) an acquisition context and implicitly discloses storing the acquisition context in onboard memory at 17:49-18:8; to wit, “Here, the subscriber may specify the locations of sites to be monitored. In this way, the satellite is configured with locations of interest.” One of ordinary skill in the art would infer from “satellite is configured” the acquisition context was stored in some form of onboard memory.
Warren discloses capturing one or more images at 15:58-62.
Warren discloses detecting one or more features (e.g. “portion of the images that match”) in the one or more captured images and applying a transformation to the pixel coordinates of the detected features to determine geographic coordinates of the detected features at 16:7-65 (“While the satellite is able to determine where it is looking (e.g., within 100 meters), the onboard processing may be used to determine within centimeters where each pixel is. In some embodiments, the satellite geolocates captured pixels versus the onboard reference expected or predicted image… If the images are from the same location, at the same … [I]mage geo-registration is performed to align the two images for comparison… The combination of the reference image of the Earth, along with geo-contextual information of the collected image and/or reference image, such as the elevation of each 35 of the pixels, allows for a transformation on that reference image to be performed so that the transformed version of the reference image appears to be from the perspective of where the satellite is at (when capturing the collected image to which the reference image is compared). Transformation 40 may also be applied to the captured image in addition to, or instead of, the reference image… The alignment may be performed using a portion of the images that match.”)
Warren discloses generating geolocation data for the detected features based on the geographic coordinates at 16:66-17:3 (“After the transformation and alignment is performed, the offset that is determined may then be used to locally perform processing, such as local cloud detection or detecting objects in the ocean at the satellite, or determining changes between the two aligned collected and reference images.”) and 18:9-15 (“In this way, …any images at the locations of interest … are transmitted from the satellite.”) See, also, 17:34-48 (“In some embodiments, the onboard processing includes performing change detection to determine a difference between the collected raw image and the corresponding (aligned) reference image... Comparison of the two images may be used to determine a change or difference between the two images. Here, the presence of a change or difference may be used to determine that the collected image is of interest and should be retained and downlinked.”)
Warren discloses transmitting the geolocation data and associated metadata over a low-bandwidth link at 14:17-15:25; 17:34-48 (e.g., “Here, the presence of a change or difference may be used to determine that the collected image is of interest and should be retained and downlinked.”); see, also, 18:23-38 and 18:55-19:32 (“thereby conserving use of limited 5 bandwidth”).
Warren does not disclose whether the transformation for mapping pixel coordinates to geographic coordinates is determined onboard the satellite or received from a mission operations center. However, a genus may be so small that, when considered in light of the totality of the circumstances, it would anticipate the claimed species or subgenus. In re Petering, 301 F.2d 676, 681, 133 USPQ 275, 280 (CCPA 1962). In the present claim, there were only two options for where processing could have occurred. Based on the evidence as a whole (In re Bell, 991 F.2d 781,784, 26 USPQ2d 1529, 1531 (Fed. Cir. 1993); In re Kulling, 897 F.2d 1147, 1149, 14 USPQ2d 1056, 1057 (Fed. Cir. 1990)), the small genus would have motivated one of ordinary skill in the art to select the claimed species or subgenus. In making this obviousness determination, the number of variables which must be selected or modified, and the nature and significance of the differences between the prior art and the claimed invention were considered. See, e.g., In re Jones, 958 F.2d 347, 350, 21 USPQ2d 1941, 1943 (Fed. Cir. 1992).
At the time of filing of the present application, it would have been obvious to a person of ordinary skill in the art to determine geographic coordinates of detected features by detecting one or more features (e.g. “portion of the images that match”) in the one or more captured images and applying a transformation to the pixel coordinates of the detected features to determine geographic coordinates of the detected features, as taught by Warren, as a substitute for the method of determining geographic coordinates taught by Ruecker. This combination is a simple substitution of one known element for another to obtain predictable results. The prior art contained a method, taught by Ruecker, which differed from the claimed method by the substitution of the step for determining georeferenced coordinates in satellite imagery. Determining geographic coordinates of detected features in satellite images by applying a transformation to the pixel coordinates of the detected features, and its functions were known in the art as evidenced by the Warren reference. One of ordinary skill in the art could have substituted the step of applying a transformation to the pixel coordinates of the detected features into the method taught by Ruecker and the results would have been predictable; to wit, the corresponding geographic coordinates of pixels in the satellite would be ascertained regardless.
The Nguyen reference
Nguyen discloses processing satellite images, which would include any detected features, by filtering noise at ¶¶ [0019]-[0020] and ¶ [0032]. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to filter noise from image data, as taught by Nguyen, after detecting features (e.g., “Hotspots” or fire) in satellite images, as taught by Ruecker. The motivation for doing so comes from the prior art wherein the benefits of noise filtering were well known and include improving the quality of downstream image analysis. Therefore, it would have been obvious to combine Nguyen with Ruecker to obtain the invention specified in this claim.
With regards to claim 3, Warren discloses a function that computes geographic coordinates based on the pixel coordinates, the satellite's orientation, and environmental factors (e.g., clouds) at 16:7-65 (“In some embodiments, the geolocation includes determining where the telescope is pointing (e.g., latitude and longitude determined using a GPS sensor and telescope pointing angles). In some embodiments, the satellite has onboard an image that is expected to be observed at that location. For example, the satellite has a catalog of reference imagery. While the satellite is able to determine where it is looking (e.g., within 100 meters), the onboard processing may be used to determine within centimeters where each pixel is… The alignment may be performed using a portion of the images that match. For example, if an entire collected image is all clouds, then the system may be unable to find a reference at all.”) The motivation for the combination is the same as previously presented.
With regards to claim 6, Ruecker implicitly discloses transmitting the geolocation data and type data (e.g., “classification of background into non fire clear land, water, clouds, sun glints”) at p. 4 when it discloses performing its processing onboard the satellite; one of ordinary skill in the art would infer that the processing was performed for the benefit of human users on Earth, to whom the geolocation data and associated metadata would be transmitted.
With regards to claim 8, Ruecker discloses transmitting the geolocation data and associated metadata to the MOC (e.g., “fire information systems”) over the low-bandwidth link at p. 4, sec. 3 (“On board processing allows for extremely rapid delivery to fire information systems…”)
With regards to claim 9, Ruecker discloses transmitting the geolocation data and associated metadata to ground-based user equipment over the low-bandwidth link at p. 4, sec. 3 (“On board processing allows for extremely rapid delivery to fire information systems…”)
With regards to claim 11, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren and Nguyen for the same reasons as were provided in the discussion of claim 1, which recites a method performing these same steps.
With regards to claim 13, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren and Nguyen for the same reasons as were provided in the discussion of claim 3, which recites a method performing these same steps.
With regards to claim 16, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren and Nguyen for the same reasons as were provided in the discussion of claim 6, which recites a method performing these same steps.
With regards to claim 18, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren and Nguyen for the same reasons as were provided in the discussion of claim 8, which recites a method performing these same steps.
With regards to claim 19, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren and Nguyen for the same reasons as were provided in the discussion of claim 9, which recites a method performing these same steps.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Ruecker et al, “High Resolution Active Fire Monitoring for Global Change Analysis: The Upcoming FireBIRD Satellite Mission” in view of Warren (US Patent No. 12,384,561) and Nguyen et al (US PG Pub. No. 2021/0225011), in further view of Neser (US PG Pub. No. 2020/0293816).
With regards to claim 2, Warren discloses applying a transformation to the pixel coordinates of the detected features to determine geographic coordinates of the detected features at 16:7-65, but does not specify using a matrix that defines the relationship between pixel coordinates in the captured images and corresponding geographic coordinates, wherein the matrix is applied to each pixel in the detected features to compute the precise geographic location of the features. However, this limitation was known in the art:
Neser discloses applying a transformation to the pixel coordinates of the detected features to determine geographic coordinates using a matrix that defines the relationship between pixel coordinates in the captured images and corresponding geographic coordinates, wherein the matrix is applied to each pixel in the detected features to compute the precise geographic location of the features at ¶¶ [0033]-[0040]. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to use a matrix, as taught by Neser, when transforming image pixels into geographic coordinates, as taught by Warren. The motivation for the combination comes from the prior art wherein the benefits of matrix algebra in the context of image analysis were well known and include increasing the efficiency of calculations. Therefore, it would have been obvious to combine Neser with Warren and Ruecker to obtain the invention specified in this claim.
With regards to claim 12, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren, Nguyen and Neser for the same reasons as were provided in the discussion of claim 2, which recites a method performing these same steps.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Ruecker et al, “High Resolution Active Fire Monitoring for Global Change Analysis: The Upcoming FireBIRD Satellite Mission” in view of Warren (US Patent No. 12,384,561) and Nguyen et al (US PG Pub. No. 2021/0225011), in further view of Scharfenberger et al (US PG Pub. No. 2024/0029444).
With regards to claim 5, Ruecker discloses detecting one or more features in the one or more captured images at p. 4, sec. 3 (“Both satellites will be equipped with on-board processing capabilities that will allow for: … Hotspot detection – based on the BIRD algorithms”); see, also, pp. 2-3 (“BIRD was able to detect and characterize extremely small as well as the very large and intense fires.”) Ruecker does not specify using a neural network to detect the one or more features. However, this limitation was known in the art:
Scharfenberger discloses detecting one or more features in captured images comprises using a neural network to detect the one or more features in the captured images at ¶ [0063](“[T]he trained artificial neural network for correcting images is part of an onboard ADAS detection neural network , e.g., …, … or object detection…”) At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to use a neural network to detect the one or more features in the captured images, as taught by Scharfenberger, when detecting one or more features in the one or more captured images, as taught by Ruecker. The motivation for doing so comes from the prior art wherein the benefits of neural networks were well known and include automatically learning complex, non-linear relationships from data. Therefore, it would have been obvious to combine Scharfenberger with Ruecker to obtain the invention specified in this claim.
With regards to claim 15, the steps performed by the apparatus of this claim are obvious over the combination of Ruecker, Warren, Nguyen and Scharfenberger for the same reasons as were provided in the discussion of claim 15, which recites a method performing these same steps.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID F DUNPHY whose telephone number is (571)270-1230. The examiner can normally be reached 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID F DUNPHY/Primary Examiner, Art Unit 2673
Prosecution Insights