Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 8-10, 15-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kester (US 20200124525 A1), in view of Asano (US 20200082536 A1) and further in view of Germouni (US 7358860 B2).
Regarding claims 1, 8 and 15, Kester discloses one or more computer-readable storage media configured to store instructions ([0024] and a non-transitory memory with instructions configured to cause the at least one processor to perform a method); and
one or more processors communicatively coupled to the one or more computer-readable storage media and configured to, in response to execution of the instructions, cause the system to perform operations, the operations comprising ([0024] at least one processor):
receiving video data comprising a plurality of frames representative of infrared radiation (IR) within a scene, each frame of the plurality of frames comprising a plurality of pixels ([0409] detect the presence of methane in a gas cloud comprises optical components such that one or more of the plurality of optical channels is configured to collect IR radiation to provide spectral data corresponding to a discrete spectral band located in the wavelength range between about 7.9 μm and about 8.4 μm corresponding to an absorption peak of methane.
[0412] It is worth noting that the use of optically-filtered FPAs in various embodiments of the system described herein can provide a system with higher number of pixels. For example, embodiments including a single large format microbolometer FPA array can provide a system with large number of pixels.
[0414] To demonstrate the operation and gas detection capability of the imaging systems described herein, a prototype was constructed in accordance with the embodiment 300 of FIG. 3A and used to detect a hydrocarbon gas cloud of propylene at a distance of approximately 10 feet. FIG. 7 illustrates video frames 1 through 12 representing gas-cloud-detection output 710 (seen as a streak of light) in a sequence from t=1 to t=12. The images 1 through 12 are selected frames taken from a video-data sequence captured at a video-rate of 15 frames/sec. The detected propylene gas is shown as a streak of light 710 (highlighted in red) near the center of each image.);
identifying pixels within the plurality of frames that correspond to a gas plume released by a gas source within the scene based on the IR ([0381] These readings (pixel measurements) g.sub.i are estimates of the spectral intensities f.sub.i. The estimates g.sub.i are not equal to the corresponding f.sub.i values because of the measurement errors n.sub.i. However, if the measurement noise distribution has zero mean, then the ensemble mean of each individual measurement can be considered to be equal to the true value, i.e. (g.sub.icustom-character=f.sub.i. Here, the angle brackets indicate the operation of calculating the ensemble mean of a stochastic variable.); and
determining a size of the gas plume within each frame based on the identified pixels ([405] Once the distance z between the cloud and the imaging system is calculated, the size of the gas cloud can be determined based on the magnification, m=f/z, where each image pixel on the gas cloud, Δx′, corresponds to a physical size in object space Δx=Δx′/m. To estimate the volume of the gas cloud, a particular symmetry in the thickness of the cloud based on the physical size of the cloud can be assumed. For example, the cloud image can be rotated about a central axis running through the cloud image to create a three dimensional volume estimate of the gas cloud size.). Kester implicitly teaches wherein the pixels that correspond to the gas plume correspond to high-frequency changes in temperature that occur at a rate that is equal to or greater than a fraction of a frame rate associated with the video data ([0063] Two shutters at two different temperatures may be employed to provide more information for calibration; for example, the absolute value of the difference between FPAs at one temperature as well as the change in that difference with temperature change can be recorded. As discussed above, only one of the two shutters can be temperature controlled in various implementations while the other is not. In various implementations, multiple shutters can be employed to create a known reference temperature difference perceived by the FPA. This reference temperature difference is provided by the IR radiation emitted by the multiple shutters when they are positioned to block the radiation from the object 110.).
Kester does not explicitly disclose but Asano teaches wherein the pixels that correspond to the gas plume correspond to high-frequency changes in temperature that occur at a rate that is equal to or greater than a fraction of a frame rate associated with the video data (fig. 4a and 6 (temperature change from 20 c to 18.5 c in under 30 fpr)
[0061] FIG. 4A is a graph illustrating a temperature change at the point SP1 of the test site, and FIG. 4B is a graph illustrating a temperature change at the point SP2 of the test site. A vertical axis in each of the graphs represents the temperature. A horizontal axis in each of the graphs represents the frame order. For example, 45 stands for a 45th frame. A frame rate is 30 fps. Accordingly, a time from a first frame to the 450th frame is 15 seconds.
[0074] the high frequency component data D3 extracted from the time-series pixel data D1. A vertical axis and a horizontal axis of the graph are the same as the vertical axis and the horizontal axis of the graph of FIG. 4A. A temperature represented by the time-series pixel data D1 is changed relatively rapidly (a cycle of the change is relatively short), and a temperature indicated by the low frequency component data D2 is changed relatively slowly (a cycle of the change is relatively long). The high frequency component data D3 seems to substantially overlap with the time-series pixel data D1.).
It would have been obvious to one of ordinary skill in the art to combine the known system of plume identification as disclosed by Kester, with the known methods of temperature change detection, as taught by Asano, in order to yield the predictable results of finding a gas plume in an area more accurately by detecting the change of temperature in that same area in with a fast detection camera.
Even so, in a similar field of endeavor of gas plume detection, Germouni teaches, in better detail, wherein the pixels that correspond to the gas plume correspond to high-frequency changes in temperature that occur at a rate that is equal to or greater than a fraction of a frame rate associated with the video data (Col 5, lines 4-18: the determination may also be made based on a rate of temperature change of one or more array components over time (e.g., a cryogenic leak is detected where the rate of temperature change of a array component is greater than a threshold, e.g., where the temperature indicated by the array component is dropping faster than 5 degrees Celsius per second). Also, in some cases, the determination may be made using a combination of an absolute temperature threshold and a rate of change (e.g., a cryogenic leak is detected where either the temperature is below a threshold temperature value or where the rate of temperature change is greater than a threshold value, e.g., a cryogenic leak is detected if either the temperature of a array component is below 0 degrees Celsius or if the temperature of the array component is dropping faster than 5 degrees per second).
Col 9, lines 45-55: In one embodiment of the invention, the I/R camera 130 may provide a frame rate which is sufficient to detect a sudden change in temperature indicative of a cryogenic leak 702. For example, if the camera 130 is used to monitor an environment in which the cryogenic material subject to leakage dissipates quickly (e.g., in a warm environment in which the leaking material quickly evaporates), an I/R camera 130 with a sampling rate above a threshold frame rate may be selected. For example, an I/R camera 130 may be selected which provides a capture rate of 7 frames/second to 120 frames/second.).
It would have been obvious to one of ordinary skill in the art to combine the known system of temperature change detection for plume identification as disclosed by Kester and Asano, with the known methods of rapid temperature change detection, as taught by Germouni, in order to yield the predictable results of by isolating fast, transient thermal fluctuations of a moving gas cloud from static background noise.
Regarding claim 2, 9 and 16, Kester discloses determining a number of pixels within each frame that correspond to the gas plume, wherein the size of the gas plume within each frame is determined based on the number of pixels within each frame that correspond to the gas plume ([0082] Summing over all pixels in the scene therefore gives an estimate of the total gas quantity (in kg) within the cloud at a given time.); and
determining a probability of detection of the gas plume within the scene by a user based on the size of the gas plume within each frame ([0093] As shown in block 520, from the absorption spectrum at a pixel, we can perform spectral matching to estimate the probability that the absorption indicates presence of a gas. If the gas detection algorithm indicates that there is a high probability of there being a gas cloud at a given pixel, then we can go to the next step and calculate the gas column density at the pixel.).
Regarding claims 3, 10 and 17, Kester implicitly teaches filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames ([0063] Two shutters at two different temperatures may be employed to provide more information for calibration; for example, the absolute value of the difference between FPAs at one temperature as well as the change in that difference with temperature change can be recorded. As discussed above, only one of the two shutters can be temperature controlled in various implementations while the other is not. In various implementations, multiple shutters can be employed to create a known reference temperature difference perceived by the FPA. This reference temperature difference is provided by the IR radiation emitted by the multiple shutters when they are positioned to block the radiation from the object 110.). Kester does not explicitly disclose but Asano teaches filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames ([0067] The frequency component data, which has a frequency higher than the frequency of the frequency component data indicating the temperature change caused by the leaked gas and indicates high frequency noise, is defined as high frequency component data D3. The image processor 9 applies, to each of the plurality of pieces of time-series pixel data D1 constituting the moving image data MD, processing to remove the high frequency component data D3 in addition to the processing to remove the low frequency component data D2.).
It would have been obvious to one of ordinary skill in the art to combine the known system of plume identification as disclosed by Kester, with the known methods of temperature change detection, as taught by Asano, in order to yield the predictable results of finding a gas plume in an area more accurately by detecting the change of temperature in that same area.
Allowable Subject Matter
Claims 4-7, 11-14, and 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The examiner also suggests filing a terminal disclaimer.
The following is a statement of reasons for the indication of allowable subject matter: Prior art do not disclose or teach the unique combinations of filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames, each filtered frame comprising a plurality of filtered values;
calculating a signal strength value of each pixel based on the plurality of filtered frames according to:
σ(Fi,j,{k}){k}=k…k+L
in which, a represents a standard deviation operator, Fz,j,{k} represents the filtered values corresponding to current pixels of a subset of the plurality of filtered frames, i represents a horizontal axis index of the current pixels, j represents a vertical axis index of the current pixels, k represents a current filtered frame index, and L represents a number of the plurality of filtered frames to be included in the subset of the filtered frames; and
generating a plurality of strength frames, each strength frame corresponding to a different frame of the plurality of frames and each strength frame comprising the signal strength values of corresponding pixels, wherein the pixels identified within the plurality of frames that correspond to the gas plume are identified based on the corresponding signal strength value.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 rejected on the ground of nonstatutory type double patenting as being unpatentable over claims 1-16 of U.S. Patent No. 12,469,250 B2.
This rejection may be overcome by filing a terminal disclaimer in accordance with 35 U.S.C. § 253.
Emphasis has been added in bold to the elements which are identical; and elements which are similar but not identical have been italicized.
Instant Application: 18666744
Reference Application: 18559779 (Patented US12469250B2)
Claims 1, 8, and 15: receiving video data comprising a plurality of frames representative of infrared radiation (IR) within a scene, each frame of the plurality of frames comprising a plurality of pixels;
identifying pixels within the plurality of frames that correspond to a gas plume released by a gas source within the scene based on the IR; and
determining a size of the gas plume within each frame based on the identified pixels.
Claims 1, 7, and 12: receiving video data comprising a plurality of frames representative of infrared radiation (IR) within a scene, each frame of the plurality of frames comprising a plurality of pixels;
identifying high-frequency changes in temperature within the scene due to motion of a gas plume released by a gas source within the scene, wherein identifying the high-frequency changes in temperature includes identifying pixels within the plurality of frames that correspond to the gas plume released by the gas source within the scene based on the IR;
and
determining a size of the gas plume within each frame based on the identified pixels;
wherein the method further comprises:
filtering out, from each frame, pixels that correspond to low-frequency changes in temperature to generate a plurality of filtered frames, each filtered frame comprising a plurality of filtered values;
calculating a signal strength value of each pixel based on the plurality of filtered frames according to:
σ(Fi,j,{k}){k}=k…k+L
in which, σ represents a standard deviation operator, Fi,j,(k) represents the filtered values corresponding to current pixels of a subset of the plurality of filtered frames, i represents a horizontal axis index of the current pixels, j represents a vertical axis index of the current pixels, k represents a current filtered frame index, and L represents a number of the plurality of filtered frames to be included in the subset of the filtered frames; and
generating a plurality of strength frames, each strength frame corresponding to a different frame of the plurality of frames and each strength frame comprising the signal strength values of corresponding pixels, wherein the pixels identified within the plurality of frames that correspond to the gas plume are identified based on the corresponding signal strength value.
Claims 1 (similar to claims 8 and 15) of the instant application is rejected because it is broader than claim 1 (similar to claims 7 and 12) of the patented application.
Claims 2 (similar to claims 9 and 16) of the instant application is rejected because it is identical to claim 2 (similar to claims 8 and 13) of the patented application.
Claims 3 (similar to claims 10 and 17) of the instant application is rejected because it is identical to claim 3 (similar to claim 14) of the patented application.
Claim 4 (similar to claims 11 and 18) of the instant application is rejected because it is broader than claim 1 (similar to claims 7 and 12) of the patented application.
Claim 5 (similar to claims 12 and 19) of the instant application is rejected because it is identical to claim 4 (similar to claims 9 and 15) of the patented application.
Claim 6 (similar to claims 13 and 20) of the instant application is rejected because it is identical to claim 5 (similar to claims 10 and 16) of the patented application.
Claim 7 (similar to claim 14) of the instant application is rejected because it is identical to claim 6 (similar to claims 11) of the patented application.
Response to Arguments
Applicant's arguments filed 07/22/2026 have been fully considered but they are not persuasive.
Regarding the argument that the prior art Asano does not disclose wherein the pixels that correspond to the gas plume correspond to high-frequency changes in temperature that occur at a rate that is equal to or greater than a fraction of a frame rate associated with the video data, the examiner most respectfully disagrees.
Asano teaches:
[0061] FIG. 4A is a graph illustrating a temperature change at the point SP1 of the test site, and FIG. 4B is a graph illustrating a temperature change at the point SP2 of the test site. A vertical axis in each of the graphs represents the temperature. A horizontal axis in each of the graphs represents the frame order. For example, 45 stands for a 45th frame. A frame rate is 30 fps. Accordingly, a time from a first frame to the 450th frame is 15 seconds.
[0074] the high frequency component data D3 extracted from the time-series pixel data D1. A vertical axis and a horizontal axis of the graph are the same as the vertical axis and the horizontal axis of the graph of FIG. 4A. A temperature represented by the time-series pixel data D1 is changed relatively rapidly (a cycle of the change is relatively short), and a temperature indicated by the low frequency component data D2 is changed relatively slowly (a cycle of the change is relatively long). The high frequency component data D3 seems to substantially overlap with the time-series pixel data D1.).
The instant application discloses
[0022] The computing device may detect the gas plume by identifying high-frequency IR changes between the frames of various pixels (e.g., high-frequency changes in temperature due to motion of the gas plume during a period of time) as corresponding to the gas plume. In some embodiments, pixels that correspond to the high-frequency IR changes may include pixels that correspond to temperature changes at a rate equal to or greater than quarter of the frame rate of the camera (e.g., 0.25*(the frame rate of the camera)). For example, if the camera includes a frame rate of twelve frames per second, the pixels that correspond to the high-frequency IR changes may include pixels that correspond to temperature changes at a rate equal to or greater than three frames per second. In other embodiments, pixels that correspond to the high-frequency IR changes may include pixels that correspond to temperature changes at a rate equal to or greater than a tenth of the frame rate of the camera (e.g., 0.1*(the frame rate of the camera)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to associate the high frequency component data D3 extracted from the time-series pixel data D1, taught by Asano can be measured in under 30 frames pers second, which would occur at a rate that is equal to or greater than a fraction of a frame rate associated with the video data, as taught by the instant application, to be the same.
Even so, upon further consideration, a new ground(s) of rejection is made in view of Germouni (US 7358860 B2).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AHMED A NASHER whose telephone number is (571)272-1885. The examiner can normally be reached Mon - Fri 0800 - 1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AHMED A NASHER/Examiner, Art Unit 2675
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights