DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of copending Application No. 18/417,278 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because claims 1-20 are respectively broader than and fully encompassed by claims 1-20 of Application No. 18/417,278. Per MPEP 2111.03 “The transitional term "comprising", which is synonymous with "including," "containing," or "characterized by," is inclusive or open-ended and does not exclude additional, unrecited elements or method steps.” MPEP 2111.03 also states “The transitional phrase "consisting of" excludes any element, step, or ingredient not specified in the claim.” The instant application 18/417,234 uses the terminology “including” and the relied upon copending application 18/417,278 uses the terminology “consisting of.” This shows that claims 1-20 of the instant application are broader than and fully encompassed by claims 1-20 of application 18/417,278.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
Application 18/417,234
Application 18/417,278
Claim 1:
A computer-implemented method of estimating evapotranspiration (ET) using thermal images and optical images, comprising:
acquiring a color image that captures a target of interest, wherein the target of interest is associated with an agricultural crop; acquiring a thermal image that captures the target of interest at substantially the same time as the color image;
extracting one or more features from the color image, wherein extracting one or more features from the color image includes extracting color features and/or texture features;
segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components include one or more of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow, and other scene elements;
co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using an energy balance model or other ET model.
Claim 1:
A computer-implemented method of estimating evapotranspiration (ET) using thermal images and optical images, comprising:
acquiring a color image that captures a target of interest, wherein the target of interest is associated with an agricultural crop; acquiring a thermal image that captures the target of interest at substantially the same time as the color image;
extracting one or more features from the color image, wherein extracting one or more features from the color image includes extracting color features and/or texture features;
segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components are selected from the group consisting of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow;
co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using an energy balance model or other ET model.
Claim 2:
The method of claim 1,
wherein the color image and the thermal image are acquired every about 0.5 - 1 h to provide a color image time series and a thermal image time series capturing the target of interest over time.
Claim 2:
The method of claim 1,
wherein the color image and the thermal image are acquired every about 0.5-1 h to provide a color image time series and a thermal image time series capturing the target of interest over time.
Claim 3:
The method of claim 2,
wherein assigning component temperatures includes constructing a time series TS-Component for each of the surface temperature components.
Claim 3:
The method of claim 2,
wherein assigning component temperatures includes constructing a time series TS-Component for each of the surface temperature components.
Claim 4:
The method of claim 1,
wherein extracting features includes extracting, for each pixel, features including an individual pixel value, Gaussian blur at two scales, standard deviation at three scales, linear binary pattern at three scales, and Laplacian at three scales in each of nine bands, wherein the nine bands include red, green, blue, hue, saturation, value, and L*a*b* colorspace components.
Claim 4:
The method of claim 1,
wherein extracting features includes extracting, for each pixel, features including an individual pixel value, Gaussian blur at two scales, standard deviation at three scales, linear binary pattern at three scales, and Laplacian at three scales in each of nine bands, wherein the nine bands include red, green, blue, hue, saturation, value, and L*a*b* colorspace components.
Claim 5:
The method of claim 1,
wherein segmenting the color image includes performing classification by inputting a reduced feature set into a multi-layer perceptron.
Claim 5:
The method of claim 1,
wherein segmenting the color image includes performing classification by inputting a reduced feature set into a multi-layer perceptron
Claim 6:
The method of claim 5,
further comprising training and testing the multi-layer perceptron using images of a cropping system representative of the agricultural crop.
Claim 6:
The method of claim 5,
further comprising training and testing the multi-layer perceptron using images of a cropping system representative of the agricultural crop.
Claim 7:
The method of claim 1,
wherein co-registering the color image and the thermal image includes using a thermally reflective object that appears in the color image and the thermal image, matching a plurality of features of the thermally reflective object between the color image and the thermal image, and using the plurality of features of the thermally reflective object to calculate an affine transform to warp the thermal image to match the color image.
Claim 7:
The method of claim 1,
wherein co-registering the color image and the thermal image includes using a thermally reflective object that appears in the color image and the thermal image, matching a plurality of features of the thermally reflective object between the color image and the thermal image, and using the plurality of features of the thermally reflective object to calculate an affine transform to warp the thermal image to match the color image.
Claim 8:
The method of claim 1,
wherein the energy balance model is a two-source energy balance (TSEB) model.
Claim 8:
The method of claim 1,
wherein the energy balance model is a two-source energy balance (TSEB) model.
Claim 9:
The method of claim 1,
wherein the energy balance model is an extension of a two-source energy balance (TSEB) model extended to include at least one additional source.
Claim 9:
The method of claim 1,
wherein the energy balance model is an extension of a two-source energy balance (TSEB) model extended to include at least one additional source.
Claim 10:
The method of claim 1,
wherein the other ET model utilizes a recurrent neural network (RNN) that estimates ET based on the component temperatures and local meteorological time series.
Claim 10:
The method of claim 1,
wherein the other ET model utilizes a recurrent neural network (RNN) that estimates ET based on the component temperatures and local meteorological time series.
Claim 11:
A system for estimating evapotranspiration (ET) using thermal and optical images, comprising:
an optical camera for acquiring a color image that captures a target of interest, wherein the target of interest is associated with an agricultural crop;
a thermal camera for acquiring a thermal image that captures the target of interest at substantially the same time as the color image;
a computing device operatively connected to the optical camera and the thermal camera to receive the color image and the thermal image and to perform a method comprising:
extracting one or more features from the color image, wherein extracting one or more features from the color image includes extracting color features and/or texture features;
segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components include one or more of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow, and other scene elements; co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using an energy balance model or other ET model.
Claim 11:
A system for estimating evapotranspiration (ET) using thermal and optical images, comprising:
an optical camera for acquiring a color image that captures a target of interest, wherein the target of interest is associated with an agricultural crop;
a thermal camera for acquiring a thermal image that captures the target of interest at substantially the same time as the color image;
a computing device operatively connected to the optical camera and the thermal camera to receive the color image and the thermal image and to perform a method comprising:
extracting one or more features from the color image, wherein extracting one or more features from the color image includes extracting color features and/or texture features;
segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components are selected from the group consisting of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow; co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using an energy balance model or other ET model.
Claim 12:
The system of claim 11,
wherein the color image and the thermal image are acquired every abut 0.5 - 1 h to provide a color image time series and a thermal image time series capturing the target of interest over time.
Claim 12:
The system of claim 11,
wherein the color image and the thermal image are acquired every abut 0.5-1 h to provide a color image time series and a thermal image time series capturing the target of interest over time.
Claim 13:
The system of claim 12,
wherein assigning component temperatures includes constructing a time series TS-Component for each of the surface temperature components.
Claim 13:
The system of claim 12,
wherein assigning component temperatures includes constructing a time series TS-Component for each of the surface temperature components.
Claim 14:
The system of claim 11,
wherein extracting features includes extracting, for each pixel, features including an individual pixel value, Gaussian blur at two scales, standard deviation at three scales, linear binary pattern at three scales, and Laplacian at three scales in each of nine bands, wherein the nine bands include red, green, blue, hue, saturation, value, and L*a*b* colorspace components.
Claim 14:
The system of claim 11,
wherein extracting features includes extracting, for each pixel, features including an individual pixel value, Gaussian blur at two scales, standard deviation at three scales, linear binary pattern at three scales, and Laplacian at three scales in each of nine bands, wherein the nine bands include red, green, blue, hue, saturation, value, and L*a*b* colorspace components.
Claim 15:
The system of claim 11,
wherein segmenting the color image includes performing classification by inputting a reduced feature set into a multi-layer perceptron trained on images of a cropping system representative of the agricultural crop.
Claim 15:
The system of claim 11,
wherein segmenting the color image includes performing classification by inputting a reduced feature set into a multi-layer perceptron trained on images of a cropping system representative of the agricultural crop.
Claim 16:
The system of claim 11,
wherein co-registering the color image and the thermal image includes using a thermally reflective object that appears in the color image and the thermal image, matching a plurality of features of the thermally reflective object between the color image and the thermal image, and using the plurality of features of the thermally reflective object to calculate an affine transform to warp the thermal image to match the color image.
Claim 16:
The system of claim 11,
wherein co-registering the color image and the thermal image includes using a thermally reflective object that appears in the color image and the thermal image, matching a plurality of features of the thermally reflective object between the color image and the thermal image, and using the plurality of features of the thermally reflective object to calculate an affine transform to warp the thermal image to match the color image.
Claim 17:
The system of claim 11,
wherein the optical camera, the thermal camera, and the computing device are packaged together in a single housing.
Claim 17:
The system of claim 11,
wherein the optical camera, the thermal camera, and the computing device are packaged together in a single housing.
Claim 18:
The system of claim 11,
wherein the energy balance model is a two-source energy balance (TSEB) model.
Claim 18:
The system of claim 11,
wherein the energy balance model is a two-source energy balance (TSEB) model.
Claim 19:
The system of claim 11,
wherein the energy balance model is an extension of a two-source energy balance (TSEB) model that is extended to include at least one additional source.
Claim 19:
The system of claim 11,
wherein the energy balance model is an extension of a two-source energy balance (TSEB) model that is extended to include at least one additional source.
Claim 20:
A computer program product for estimating evapotranspiration (ET) using thermal images and optical images, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by one or more processors, to perform a method comprising:
acquiring a color image that captures a target of interest, wherein the target of interest is associated with an agricultural crop;
acquiring a thermal image that captures the target of interest at substantially the same time as the color image;
extracting one or more features from the color image, wherein extracting one or more features from the color image includes extracting color features and/or texture features; segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components include one or more of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow, and other scene elements;
co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using a two-source energy balance (TSEB) model or other ET model.
Claim 20:
A computer program product for estimating evapotranspiration (ET) using thermal images and optical images, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by one or more processors, to perform a method comprising:
acquiring a color image that captures a target of interest, wherein the target of interest is associated with an agricultural crop;
acquiring a thermal image that captures the target of interest at substantially the same time as the color image;
extracting one or more features from the color image, wherein extracting one or more features from the color image includes extracting color features and/or texture features; segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components are selected from the group consisting of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow;
co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using a two-source energy balance (TSEB) model or other ET model.
Allowable Subject Matter
Claims 1-20 are allowable over prior art. However claims 1-20 are still rejected under double patenting and 35 USC 101. As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Regarding claims 1, 11, and 20,
Zheng (2025/0078500) teaches, see Fig. 3, a decision making process for water and fertilizer stress of crops. A visible light image and thermal image. The visible light image is used to determine a crop coefficient, and the thermal image along with a feature extracted from the visible light image are used to extract a canopy temperature.
Iravantchi (2024/0111898) ¶84 teaches creating a mask based on body temperature of humans to segment images.
Moshelion (2017/0042098) teaches a system for characterizing a plant. The system uses sensors to acquire a transpiration rate from the plant, and an evaporation rate from a wick. The system uses a thermal image to estimate a cooling rate of a plant which is used as a proxy to the transpiration rate of the plant ¶83.
No prior art explicitly discloses segmenting the color image into surface temperature components based on the one or more features extracted from the color image, wherein the surface temperature components are selected from the group consisting of sunlit soil, sunlit residue, sunlit vegetation, sunlit snow, shaded soil, shaded residue, shaded vegetation, and shaded snow;
co-registering the color image and the thermal image to provide a registered thermal image;
assigning component temperatures by applying component masks to the registered thermal image to assign a component temperature to each of the surface temperature components;
estimating ET based on the component temperatures using an energy balance model or other ET model.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights