DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 13-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
With regards to claim 13, several of the features of this claim were known in the art as evidenced by the combination of Scheibenreif et al, “Toward Global Estimation of Ground-Level NO2 Pollution With Deep Learning and Remote Sensing” and Cowan et al (US PG Pub. No. 2023/0011668), which renders obvious the limitations of parent claim 1. In particular, Scheibenreif discloses reference sensors comprising a ground-based sensor (“air quality stations on the ground”) at pp. 2-4, secs. II-III. However, Scheibenreif does not disclose the reference sensors provide meteorological parameters including at least one of temperature, humidity, pressure, wind speed, or wind direction associated in time and location with the captured images.
With regards to claim 14, several of the features of this claim were known in the art as evidenced by the combination of Scheibenreif et al, “Toward Global Estimation of Ground-Level NO2 Pollution With Deep Learning and Remote Sensing” and Cowan et al (US PG Pub. No. 2023/0011668), which renders obvious the limitations of parent claim 1. In particular, Scheibenreif discloses a processing unit(e.g., “fully connected neural network (head)”) receives captured images (“Sentinel-2 and Sentinel-5P images”), and determines, based on the captured images and using the trained AI model, a pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) of the field of view at pp. 4-5, sec. IV(B) and FIGS. 2-3 and FIG. 10. However, Scheibenreif does not disclose the processing unit tracks temporal changes across successive pollutant concentration distribution maps to detect an increasing pollutant concentration trend.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 9-12, 15-17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Scheibenreif et al, “Toward Global Estimation of Ground-Level NO2 Pollution With Deep Learning and Remote Sensing” in view of Cowan et al (US PG Pub. No. 2023/0011668).
With regards to claim 1, the limitations of this claim are obvious over the teachings of the prior art, as evidenced by the following references:
The Scheibenreif reference
Scheibenreif discloses an image capturing unit (e.g., “Sentinel-2 and Sentinel-5P satellites”), wherein the image capturing unit captures one or more images (“Sentinel-2 and Sentinel-5P images”) of a field of view in one or more spectral bands (“13 spectral bands”) at pp. 2-3, sec. III(C); see, also, sec. IV(B).
Scheibenreif discloses a processing unit (e.g., “fully connected neural network (head)”) at pp. 4-5, sec. IV(B) and FIG. 2 (reproduced below). One of ordinary skill in the art would infer from this disclosure that the disclosed trained neural network processing unit (e.g.,”head”) comprises one or more processors and a memory storing the trained artificial intelligence (Al) model.
PNG
media_image1.png
280
820
media_image1.png
Greyscale
Scheibenreif does not specify that its processing unit is communicatively coupled to the image capturing unit (satellites). However, this limitation was known in the art as evidenced by the Cowan reference discussed below.
Scheibenreif discloses the processing unit(e.g., “fully connected neural network (head)”) receives the captured one or more images (“Sentinel-2 and Sentinel-5P images”), and determines, based on the captured one or more images and using the trained AI model, a pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) of the field of view at pp. 4-5, sec. IV(B) and FIGS. 2-3 and FIG. 10 (excerpted below):
PNG
media_image2.png
404
826
media_image2.png
Greyscale
See, also, pp. 6-7, sec. V(C); pp. 9-10, sec. VI.
The Cowan reference
Cowan discloses processing unit communicatively coupled to an image capturing unit comprising a satellite at ¶¶ [0038]-[0052]; in particular, see ¶ [0052]. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to acquire satellite imagery by communicatively coupling the processing unit to an image capturing unit comprising a satellite, as taught by Cowan, when acquiring satellite imagery for processing according to the method taught by Scheibenreif. The motivation for doing so comes from the prior art, wherein as a matter of common sense, one of ordinary skill in the art would infer the benefit of communicatively coupling the satellite with the processing unit as obviating the need to manually receive and load the satellite data into the server. Therefore, it would have been obvious to combine Cowan with Scheibenreif to obtain the invention specified in this claim.
With regards to claim 2, Scheibenreif discloses the AI model is trained using training data generated by synchronizing the captured one or more images (“Sentinel-2 and Sentinel-5P images”) with reference measurements (e.g., “ground-truth surface level NO2 concentration data”) obtained from one or more reference sensors (e.g., “air quality stations”) at pp. 2-4, sec. III; to wit: “The ground-truth surface level NO2 concentration data used in this work were extracted from the EEA’s network of air quality stations… To enable supervised model training, all three datasets are cropped to locations of EEA air quality stations… The satellite dataset for model training and evaluation is assembled by aligning Sentinel-2 and Sentinel-5P observations for the locations of EEA air quality stations with the temporally corresponding NO2 measurements from those stations.” Scheibenreif further discloses, after the training, that the pollutant concentration distribution map is determined based solely on the captured one or more images at pp. 4-5, sec. IV(B) and FIG. 2.
With regards to claim 3, Scheibenreif discloses the reference sensors comprises ground-based sensor (“air quality stations on the ground”) at pp. 2-4, secs. II-III.
With regards to claim 4, Scheibenreif discloses the image capturing unit (e.g., “Sentinel-2 and Sentinel-5P satellites”) captures multispectral images in the ultraviolet, visible, and near-infrared spectral ranges at pp. 3-4, sec. III(C); to wit: “The sensor covers eight spectral bands in the ultraviolet, visible, near-infrared, and short-wave infrared range to detect absorption from molecules such as NO2, O3, CO, or CH4.”
With regards to claim 5, Scheibenreif discloses the image capturing unit (e.g., “Sentinel-2 and Sentinel-5P satellites”) comprises a movable camera at pp. 3-4, sec. III(C).
With regards to claim 9, Scheibenreif discloses the pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) comprises concentration values of at least one atmospheric contaminant, wherein the at least one atmospheric contaminant comprises nitrogen oxides (NOx) at pp. 4-5, sec. IV(B) and FIGS. 2-3 and FIG. 10.
With regards to claim 10, this claim recites the AI model is further trainable to determine concentrations of additional pollutants by retraining using training data formed by synchronizing captured images to corresponding reference measurements of the additional pollutants. It is not clear from applicant’s disclosure what makes their system retrainable any more than any AI model can be retrained. These are functional limitations. A claim term is functional when it recites a feature "by what it does rather than by what it is" (e.g., as evidenced by its specific structure or specific ingredients). In re Swinehart, 439 F.2d 210, 212, 169 USPQ 226, 229 (CCPA 1971). The Scheibenreif reference does not specify that it can be retrained. However, these functional limitations appear to be inherent in the Scheibenreif reference because of the structural similarities between applicant’s disclosed invention and the Scheibenreif reference, which were discussed above in the rejection of claim 1 (i.e., neural network trained to generate a pollutant concentration map based on satellite images). Based upon the above structural similarities, it is found that the prior art structure inherently possesses the functionally defined limitations of the claimed apparatus. When a prior art structure is found to inherently possess the functionally defined limitations of a claimed apparatus, the burden then shifts to applicant to establish that the prior art does not possess the characteristic relied on. In re Schreiber, 128 F.3d at 1478, 44 USPQ2d at 1432; In re Swinehart, 439 F.2d 210, 213, 169 USPQ 226, 228 (CCPA 1971) ("where the Patent Office has reason to believe that a functional limitation asserted to be critical for establishing novelty in the claimed subject matter may, in fact, be an inherent characteristic of the prior art, it possesses the authority to require the applicant to prove that the subject matter shown to be in the prior art does not possess the characteristic relied on"). MPEP § 2112(V); MPEP § 2114(I).
With regards to claim 11, Scheibenreif discloses the processing unit(e.g., “fully connected neural network (head)”) receives the captured one or more images (“Sentinel-2 and Sentinel-5P images”), and determines, based on the captured one or more images and using the trained AI model, a pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) of the field of view at pp. 4-5, sec. IV(B) and FIGS. 2-3 and FIG. 10
Cowan discloses the output of the processing unit is continuously updated in real time as new images are captured at ¶ [0100]. The motivation for the combination is the same as previously presented.
With regards to claim 12, Scheibenreif discloses the processing unit integrates image data from a plurality of spectral bands to determine the pollutant concentration distribution map at pp. 3-4, sec. III(C); to wit: “The sensor covers eight spectral bands in the ultraviolet, visible, near-infrared, and short-wave infrared range to detect absorption from molecules such as NO2, O3, CO, or CH4.” See, also, pp. 4-5, sec. IV(B) and FIG. 2 (“The model performs data fusion at the feature level by concatenating the outputs of both backbones and then passes the resulting feature vector to a two-layer fully connected neural network (called head), which outputs an NO2 estimate…”)
With regards to claim 15, Scheibenreif discloses integrating images from a plurality of cameras (e.g., “Sentinel-2 and Sentinel-5P satellites”), providing different viewpoints prior to determining the pollutant concentration distribution map at pp. 3-4, sec. III(C); to wit: “The satellite dataset for model training and evaluation is assembled by aligning Sentinel-2 and Sentinel-5P observations for the locations of EEA air quality stations with the temporally corresponding NO2 measurements from those stations.” See, also, pp. 4-5, sec. IV(B) and FIG. 2 (“The model performs data fusion at the feature level by concatenating the outputs of both backbones and then passes the resulting feature vector to a two-layer fully connected neural network (called head), which outputs an NO2 estimate…”)
With regards to claim 16, the steps performed by the method of this claim are obvious over the combination of Scheibenreif and Cowan for the same reasons as were provided in the discussion of claim 1, which recites an apparatus configured to perform these same steps.
With regards to claim 17, Scheibenreif discloses obtaining reference measurements from one or more reference sensors corresponding in time (“2018-2020 time span”) and location to the captured one or more images and training the AI model using training data formed by synchronizing the captured images to the corresponding reference measurements at pp. 2-4, sec. III; to wit: “The ground-truth surface level NO2 concentration data used in this work were extracted from the EEA’s network of air quality stations… To enable supervised model training, all three datasets are cropped to locations of EEA air quality stations… The satellite dataset for model training and evaluation is assembled by aligning Sentinel-2 and Sentinel-5P observations for the locations of EEA air quality stations with the temporally corresponding NO2 measurements from those stations.” Scheibenreif further discloses, after the training, that the pollutant concentration distribution map is determined based solely on the captured one or more images at pp. 4-5, sec. IV(B) and FIG. 2.
With regards to claim 19, the steps stored in the computer readable medium of this claim are obvious over the combination of Scheibenreif and Cowan for the same reasons as were provided in the discussion of claim 1, which recites an apparatus configured to perform these same steps.
Claims 6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Scheibenreif et al, “Toward Global Estimation of Ground-Level NO2 Pollution With Deep Learning and Remote Sensing” in view of Cowan et al (US PG Pub. No. 2023/0011668), in further view of Zhang et al, “Air Pollution Hotspot Detection and Source Feature Analysis using Cross-Domain Urban Data.”
With regards to claim 6, Scheibenreif discloses the processing unit generates a heatmap representation of the pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) and identifies an emission source at pp. 4-5, sec. IV(B) and FIG. 2, FIG. 3 (“[M]aking it possible to map air pollution on the ground level to identify sources of air pollution…”) See, also, pp. 6-7, sec. V(C); pp. 9-10, sec. VI. Scheibenreif does not specify identifying an emission source by detecting a local maximum of concentration in the heatmap representation. However, this limitation was known in the art as evidenced by the Zhang reference:
Zhang discloses the processing unit generates a heatmap representation of the pollutant concentration distribution map and identifies an emission source by detecting a local maximum of concentration in the heatmap representation at p. 2, sec. 3.2. At the time of filing of the present application, it would have been obvious to a person of ordinary skill in the art to identify an emission source by detecting a local maximum, as taught by Zhang, as a substitute for the unspecified manner of identifying a pollution source in the heatmap representation of the pollutant concentration distribution map taught by Scheibenreif. This combination is a simple substitution of one known element for another to obtain predictable results. The prior art contained a method, taught by Scheibenreif, which differed from the claimed method by the substitution of the manner of locating pollution sources in a pollutant concentration distribution map. Using a local maximum to identify an emission source by detecting a local maximum of concentration in the heatmap representation was known in the art, as evidenced by Zhang. One of ordinary skill in the art could have substituted this manner of locating the pollution source into the method taught by Scheibenreif and the results would have been predictable; to wit, possible pollution sources would be identified.
With regards to claim 18, Zhang discloses identifying an emission source by locating a maximum concentration region in the pollutant concentration distribution map at p. 2, sec. 3.2. The motivation for the combination is the same as previously presented.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Scheibenreif et al, “Toward Global Estimation of Ground-Level NO2 Pollution With Deep Learning and Remote Sensing” in view of Cowan et al (US PG Pub. No. 2023/0011668), in further view of Qian et al (US PG Pub. No. 2017/0200309).
With regards to claim 7, Scheibenreif discloses preprocessing the satellite imagery at p. 2, sec. III, but does not specify motion compensation. However, this limitation was known in the art:
Qian discloses a processing unit that performs image preprocessing comprising motion compensation on a satellite image at ¶ [0043]. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to preprocess satellite imagery with motion compensation, as taught by Qian, when preprocessing satellite imagery as taught by Scheibenreif. The motivation for doing so comes from Qian, which discloses motion comensation obtains, “the set of parameters that most accurately correspond to the locations of the SIFT features (or other identified and matched points) in the set of satellite images”. (¶ [0043]). Therefore, it would have been obvious to combine Qian with Scheibenreif to obtain the invention specified in this claim.
Claims 8 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Scheibenreif et al, “Toward Global Estimation of Ground-Level NO2 Pollution With Deep Learning and Remote Sensing” in view of Cowan et al (US PG Pub. No. 2023/0011668), in further view of OFFICIAL NOTICE.
With regards to claim 8, Scheibenreif discloses the processing unit generates a heatmap representation of the pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) and identifies an emission source at pp. 4-5, sec. IV(B) and FIG. 2, FIG. 3 (“[M]aking it possible to map air pollution on the ground level to identify sources of air pollution…”) See, also, pp. 6-7, sec. V(C); pp. 9-10, sec. VI and FIG. 10. Scheibenreif does not specify a communication interface to transmit the pollutant concentration distribution map to a remote service for storage or visualization. However, this limitation was known in the art as evidenced by OFFICIAL NOTICE:
OFFICIAL NOTICE IS HEREBY TAKEN that, at the effective filing date of the present application, communication interfaces to transmit data to a remote service for storage or visualization were known in the art. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to use a communication interface to transmit data to storage, as was known in the art, after generating data comprising a heatmap representation of the pollutant concentration distribution map (e.g., “heatmaps in μg/m3”). The motivation for doing so comes from the prior art wherein one of ordinary skill in the art would understand, as a matter of common sense, that a user might want to store the pollution heatmaps that they have generated. Therefore, it would have been obvious to combine OFFICIAL NOTICE with Scheibenreif to obtain the invention specified in this claim.
With regards to claim 20, Scheibenreif discloses the processing unit generates a heatmap representation of the pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) and identifies an emission source at pp. 4-5, sec. IV(B) and FIG. 2, FIG. 3 (“[M]aking it possible to map air pollution on the ground level to identify sources of air pollution…”) See, also, pp. 6-7, sec. V(C); pp. 9-10, sec. VI and FIG. 10. Scheibenreif does not specify a user interface, wherein the user interface displays the pollutant concentration distribution map. However, this limitation was known in the art as evidenced by OFFICIAL NOTICE:
OFFICIAL NOTICE IS HEREBY TAKEN that, at the effective filing date of the present application, user interfaces, wherein the user interface displays data of interest were known in the art. At the time of the filing of the present application, it would have been obvious to a person of ordinary skill in the art to use a user interface, as was known in the art, after generating a heatmap representation of the pollutant concentration distribution map (e.g., “heatmaps in μg/m3”) in order to view the heatmap. The motivation for doing so comes from the prior art wherein one of ordinary skill in the art would understand, as a matter of common sense, that users of heatmaps want to view the heatmaps. Therefore, it would have been obvious to combine OFFICIAL NOTICE with Scheibenreif to obtain the invention specified in this claim.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID F DUNPHY whose telephone number is (571)270-1230. The examiner can normally be reached 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID F DUNPHY/Primary Examiner, Art Unit 2673