DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to various rejections of claims 11 and 18 under 102 and 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record, alone, for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 11 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over US 11,519,602 B2 to Krause et al (hereinafter ‘Krause’) in view of CN 113096103 A to Xie et al, (hereinafter ‘Xie’) (please refer to the attached USPTO translation version).
Regarding claim 11, Krause discloses a method (column 2, lines 13-15, one or more methods for monitoring a flare burner with a video camera have been invented which can be used in a system for observing and controlling a flare) comprising: identify, using a computer vision model, an emission indicator captured by an image (column 11, lines 21-23, and Fig. 2, step 104, wherein the present invention also contemplates using a machine learning/neural network system, as the model, to determine smoke visibility, as identifying); determine, using the computer vision model, one or more parameters of the emission indicator (column 10, lines 51-56, wherein one of the ROIs or a defined subset of the ROIs will then be chosen to perform the step 108 of comparing those index numbers, as the parameter, with a threshold value smoke index. The threshold value smoke index is threshold smoke index is the minimum index a trained observer would indicate that smoke was present and can be a fluid value); determining, in response to at least one of the one or more parameters exceeding an associated threshold of one or more thresholds, that the emission indicator is representative of the emission event; and determining, in response to the at least one of the one or more parameters being less than or equivalent to the associated threshold of the one or more thresholds, that the emission indicator is representative of another event (column 10, lines 53-61, wherein the threshold value smoke index is threshold smoke index is the minimum index a trained observer would indicate that smoke was present and can be a fluid value. For example, the database can be consulted for determining the threshold value smoke index. Based on the comparison, the process includes the step 110 of providing an indication of the presence, as emission event, or absence, inherently as normal event, of smoke based on the comparison of the index numbers with the threshold values.). However, Krause does not specifically disclose the model being a DCNN model to identify an emission indicator; to determine, using the DCNN, one or more parameters of the emission indicator. Xie discloses identifying, using a computer vision model deep convolutional neural network (DCNN) model, an emission indicator captured by an image; and determining, using the computer vision model the DCNN model, one or more parameters of the emission indicator (Page 3, Para 5, and Figs. 1 and 2, wherein the IDCNN network in step 2 is a deep double-channel neural network end to respectively, which is composed of deep layer subnet ISBNN which is good for extracting smoke detail features, inherently as the emission indicator, and deep layer sub-network ISCNN which is good for extracting smoke basic information, inherently as the parameter of emission indicator). Krause and Xie are combinable because they both disclose emission detection. Therefore, before the effective filing data of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the identifying of emission indicator and the parameter of the indicator, using deep convolutional neural network (DCNN) model, of Xi’s device with Krause’s so that the trimming of the total parameter of the IDCNN provides for searching the best parameter (page 6, Para 4).
Regarding claim 15, in the combination of Krause and Xie, Krause discloses the method comprising generating an instruction in response to the indication indicating that the emission indicator is representative of the emission event, wherein the instruction is based on the at least one of the one or more parameters (column 12, lines 11-15, wherein using this library of information, the system is able to identify normal, abnormal, and alternate flare situations, as event indication, and provide visual indications and the best recommendations and action, as instructions, for an operator of the flare system).
Claims 18, 19 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Krause in view of Xie and further in view of US 8,138,927 B2 to Diepenbroek et al (hereinafter ‘Diepenbroek’).
Regarding claim 18, Krause discloses a system (Fig. 1, system 10), comprising: an image sensor to capture an image of a flare stack (Column 6, lines 41-43 and Fig. 1, wherein an ultraviolet (UV) camera 38 that is configured to provide UV images of the flare burner 14); one or more non-transitory, computer-readable mediums storing a computer vision engine (column 13, lines 15-19, wherein computing devices or systems may include at least one processor and memory storing computer-readable instructions that, when executed by the at least one processor, cause the one or more computing devices to perform a process) configured to determine whether an emission indicator captured by the image indicates an emission event (column 10, lines 58-61, wherein based on the comparison, the process includes the step 110 of providing an indication of the presence or absence of smoke based on the comparison of the index numbers with the threshold values), a server engine configured to transmit a notification indicating whether the emission indicator indicates the emission event to a control system of the flare stack (column 5, lines 45-50, wherein the controller 24 is further configured to obtain, receive, and/or send information over a communication network (e.g., local communication network, the internet, an intranet). However, Krause does not specifically disclose the model being a DCNN model to identify an emission indicator; to determine, using the DCNN, one or more parameters of the emission indicator. Xie discloses identifying, using a computer vision model deep convolutional neural network (DCNN) model, an emission indicator captured by an image; and determining, using the computer vision model the DCNN model, one or more parameters of the emission indicator (Page 3, Para 5, and Figs. 1 and 2, wherein the IDCNN network in step 2 is a deep double-channel neural network end to respectively, which is composed of deep layer subnet ISBNN which is good for extracting smoke detail features, inherently as the emission indicator, and deep layer sub-network ISCNN which is good for extracting smoke basic information, inherently as the parameter of emission indicator). Additionally, in as much as applicant may disagree with examiner’s assessment of the server engine above, Diepenbroek discloses a server engine configured to enable communication between the electronic device and a control system of a flare stack, and wherein the processor is operable to transmit, via the server engine, the indication to the control system (column 4, lines 29-33, wherein the server may also enable alarms or events in the process control system 12 to initiate high resolution recordings, report any camera failures or recording failures to the process control system as an alarm 42, and provide a full audit log of all system status (camera and server availability) and operator actions). Krause, Xie and Diepenbroek are combinable because they all disclose emission detection. Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the server engine configured to enable communication between the electronic device and a control system of a flare stack, of Diepenbroek’s system with Krause’s and Xie’s because the server may provide a full audit log of all system status (camera and server availability) and operator actions (column 4, lines 31-33).
Regarding claim 19, in the combination of Krause, Xie and Diepenbroek, Krause discloses wherein the server engine is configured to receive one or more settings from the control system, and wherein the computer vision engine is configured to update one or more model settings, one or more thresholds, or a combination thereof, based on the one or more settings (column 12, lines 22-31, wherein operating instructions are provided with clear indicators of flare system performance. This will include color coded graphics for the operator to immediately understand the impact of changes being considered, as updating, to improve the performance of their flare system. This will enable the operator to make the best decision when they are faced with adjusting the operation of the flare system. As is usual for this type of neural network, the system will be trained, and the results of the training shall be encoded into the production system, as inherently include updating).
Regarding claim 25, in the combination of Krause, Xie and Diepenbroek, Krause discloses wherein the image sensor comprises a first image sensor having a first field of view of the flare stack and a second image sensor having a second field of view of the flare stack different from the first field of view, and wherein the computer vision engine is configured to determine a volume of the emission event based on the emission indicator captured by the first image sensor and the emission indicator captured by the second image sensor (column 1, lines 63 through column 2, line 1, wherein the main role of a flare monitoring and control device or system is to monitor and measure certain parameters of the flare such as amount/volume of the smoke, size of the flare, and noise level (typically in dB), and take certain countermeasures to control the flare so as to ensure compliance with EPA smoke and noise level regulations).
Allowable Subject Matter
Claims 1-7, 21 and 22 are allowed. . The following is a statement of reasons for the indication of allowable subject matter: the prior art or the prior art of record specifically, Krause and Xie, does not disclose:
. . . . generate an indication indicating whether the emission indicator is representative of an emission event based on the one or more parameters; wherein the processor is operable to generate a first stage of the computer vision model to segment the image using a deep convolution and deconvolutional neural network, a genetic algorithm, batch normalization, filtering, and dropout layers trained using a training set of data; wherein an output of the first stage is a binary matrix; and wherein the processor is operable to generate a second stage of the computer vision model to determine the one or more parameters of the emission event based on the binary matrix, in claim 1 combined with other features and elements of the claim;
Claims 2-7, 21 and 22 depend from an allowable base claim and are thus allowable themselves.
Claims 12-13, 16-17 and 24 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior art or the prior art of record specifically, Krause and Xie, does not disclose:
. . . . wherein the DCNN model includes a deep learning neural network that segments the image to identify the emission indicator, of claim 12 combined with other features and elements of the claim;
Claim 13 depend from an allowable base claim and is thus allowable itself;
identifying, using the DCNN model, the emission indicator in a second image associated with a second time stamp; and determining a period of the emission event based on the first and the second time stamps, and wherein a notification includes the period of the emission event, of claim 16 combined with other features and elements of the claim;
Claim 17 depend from an allowable base claim and is thus allowable itself;
. . . . wherein the computer vision engine is configured to determine a brightness index of the image based on an average brightness of pixels in the image and a sharpness index of the image based on a variance of a Laplacian of the image, of claim 24 combined with other features and elements of the claim.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHERVIN K NAKHJAVAN whose telephone number is (571)272-5731. The examiner can normally be reached Monday-Friday 9:00-05:00 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sue Lefkowitz can be reached at (571)272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHERVIN K NAKHJAVAN/Primary Examiner, Art Unit 2672
Prosecution Insights