DETAILED ACTION
Notice of AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
2. Applicant’s remarks received on 8/11/2025 with respect to the amended independent claims have been acknowledged and are moot in view of a new ground of rejection. Currently claims 1-20 are rejected.
Response to Amendment
Claim Rejections - 35 USC § 101
3. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Regarding claims 1-20, under the broadest reasonable interpretation, the terms of the claims are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skilled in the art. See MPEP 2111.
The claimed invention identifies and verifies an object presented in video frames based on color and grayscale features associated with the frames using neural networks.
The claims are directed to mathematical concepts or mental processing and does/do not fall within at least one of the four categories of patent eligible subject matter because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea).
The claims do not have any limitations that are indicative of integration of the judicial exception into a practical application such as improvements to functioning of a computer or a technical field, using any particular machine, effect a transformation of a particular article to a different state or thing, or apply the judicial exception in any meaningful way beyond generally linking the use to a particular technological environment. Therefore, the claims as a whole do not amount to significantly more than the judicial exception.
A patent may be obtained for “any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof," 35 U.S.C. § 101, but “laws of nature, natural phenomena, and abstract ideas are not patentable.”
Step 1
Claims 1-20 are directed to one of the four statutory categories of eligible subject matter (process): thus, the claim pass Step 1 of the Subject Matter Eligibility Test.
Step 2A, prong 1 analysis
Claims 1-20 recite a judicial exception in terms of a mental process. The claimed determining and verifying steps can be performed by a human being observing video frames for object detection by identifying and comparing color and grayscale characteristics of the frames. The transforming step in claims 4 and 20 can be achieved and observed by a human being with a pair of color evaluator glasses. Notice, a human brain can be trained just like the claimed neural network for object and feature identification and classification based on various criteria.
Step 2A, prong 2 analysis
The judicial exception is not integrated into a practical application because additional elements in claims 3, 6, 10, 11, 13, 18, and 19 are only for data collection or definition which are considered insignificant extra-solution activities to the judicial exception.
Step 2B
Finally, the claims do not include other additional elements that are beyond what is well-understood, routine, conventional activities in the field and sufficient to amount to significantly more than the judicial exception.
Conclusion:
The claims do not include additional elements amount to significantly more in terms of improving functionalities of a computer/device itself, improving another technology or technical field, effecting a transformation or reduction of a particular article to a different state or thing, adding unconventional steps that confine claim to a particular useful application or by use of a particular machine that is unconventional. In conclusion, the claims 1-20 do not comply with the current standards for patent eligible subject matter under 35 USC § 101.
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
51066.. Claims 1-4, 7, 15-17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Fan (CN Pub: 111666901) and in further view of Rizzotti et al (US Patent: 6,937,743).
Regarding claim 1 (currently amended), Fan teaches: A method comprising: determining, by a first classification model and based on a plurality of color features associated with a frame of video, an object in the frame [page 9: p04, p06, page 11: p02, p03]; determining, by a second classification model and based on a plurality of greyscale features associated with the frame, the object in the frame [page 9: p09, page 10: p01]; and based on the determination of the object in the frame by the first classification model and the second classification model, verifying the object is present in the frame [page 2: p09].
Fan does not perform the second prediction associated with a neighboring frame. In the same field of endeavor, Rizzotti et al determines an object in a frame based on a plurality of color features associated with a frame [col 5: lines 56-62]; and perform a second determination/prediction of the object based on greyscale features [col 5: lines 12-26, lines 43-49] and at least one neighboring frame [col 5: lines 29-36]; and verifies the object based on the first and the second prediction [col 6: lines 53-56].
Therefore, given Rizzotti et al’s teaching on using multiple algorithms involving color features and greyscale features of an image and comparison with a neighboring frame for prediction and using combined prediction to confirm the result, it would have been obvious for an ordinary skilled in the art before the effective filing date of the claimed invention to combine the teaching of the two to implement Rizzotti et al’s algorithm to Fan’s models to make ML based classifier for quick identification of objects such as fire with improved accuracy.
Regarding claim 2 (Original), the rationale applied to the rejection of claim 1 has been incorporated herein. Fan and Rizzotti et al further teaches: The method of claim 1, wherein determining the object in the frame comprises determining, based on the first classification model and the plurality of color features, a prediction that the frame comprises the object [Fan: page 9: p06, page 11: p02, p03; Rizzotti: col 5: lines 37-62].
Regarding claim 3 (Original), the rationale applied to the rejection of claim 1 has been incorporated herein. Fan further teaches: The method of claim 1, wherein the video is associated with at least one of: a content provider, a user device, or a security camera [page 8: p06, page 9: p04].
Regarding claim 4 (Original), the rationale applied to the rejection of claim 1 has been incorporated herein. Fan further teaches: The method of claim 1, further comprising: transforming the plurality of color features into the plurality of grayscale features [page 4: p04, page 5: p01 (Transformation takes place when fusion characteristic is obtained from fusing depth characteristic extracted from color image and undetermined degree characteristic obtained from grayscale/infrared image.)].
Regarding claim 7 (Original), the rationale applied to the rejection of claim 1 has been incorporated herein. Fan and Rizzotti et al further teaches: The method of claim 1, wherein verifying the object is present in the frame comprises at least one of: determining that the plurality of grayscale features are indicative of the frame comprising the object; or determining that the plurality of grayscale features are indicative of at least one neighboring frame comprising the object [Fan: page 2: p09, page 3: p01; Rizzotti: col 5: lines 31-50].
Claims 15 (Currently Amended) has been analyzed and rejected with regard to claim 1.
Regarding claims 16 and 17 (Original), the rationale applied to the rejection of claim 15 has been incorporated herein. Fan and Rizzotti et al further teach: The method of claim 15, wherein determining the prediction comprises :determining, based on a deep-learning model and the plurality of color/grayscale features, the prediction, wherein the deep-learning model is configured to detect the object in frames of video based on color/grayscale features [Fan: page 7; Rizzotti: col 5: lines 30-62].
Regarding claims 19 and 20 (Original), the rationale applied to the rejection of claim 15 has been incorporated herein. Claims 19 and 20 have been analyzed and rejected with regard to claims 3 and 4 respectively.
61066.. Claims 5 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Fan (CN Pub: 111666901) and Rizzotti et al (US Patent: 6,937,743); and in further view of Danielsson et al (EP 3321844).
Regarding claim 5 (Currently Amended), the rationale applied to the rejection of claim 1 has been incorporated herein. Fan in view of Rizzotti et al does not verify prediction. In the same field of endeavor, Danielsson et al teaches: the method of claim 1, wherein verifying the object is present in the frame comprises, based on the second prediction satisfying a confidence threshold verifying the first prediction [claim 1]. Therefore, given Danielsson et al’s prescription on verifying recognition result from the first action recognition algorithm verification using second action recognition algorithm and Fan in view of Rizzotti et al’s teaching on applying combined models for an object recognition, it would have been obvious for an ordinary skilled in the art before the effective filing date of the claimed invention to combine the teaching of all to compare the first and the second prediction for recognition confirmation.
Regarding claim 18 (Original) the rationale applied to the rejection of claim 17 has been incorporated herein. Claim 18 has been analyzed and rejected with regard to claim 5.
71066.. Claims 6, 8-10, and 12-14 are rejected under 35 U.S.C. 103 as being unpatentable over Fan (CN Pub: 111666901) and Rizzotti et al (US Patent: 6,937,743); and in further view of Rush et al (US Patent: 10,489,661).
Regarding claim 6 (Currently Amended), the rationale applied to the rejection of claim 1 has been incorporated herein. Rizzotti et al performs temporal analysis over successive images and use object’s color change across frame to signal flames: The method of claim 1, wherein the at least one neighboring frame comprises a first neighboring frame that precedes the frame and a second neighboring frame that follows the frame, and wherein at least one color feature of the first neighboring frame partially differs from at least one color feature of the second neighboring frame [col 5: lines 37-40, 55-63].
For a redundant teaching in the same field of endeavor, Rush et al further teaches: wherein the at least one neighboring frame comprises a first neighboring frame that precedes the frame and a second neighboring frame that follows the frame, and wherein at least one color feature of the first neighboring frame partially differs from at least one color feature of the second neighboring frame [col 2: lines 22-58, col 8: lines 39-67, col 9: lines 1-3, col 16: lines 57-65, col 26: lines 35-45 (A patient’s various positions in bed at night or day time can be identified through change of pixels grayscale features such as depth characteristics or color features from infrared or color image frames throughout a time period.)]. Therefore, given Fan in view of Rizzotti et al and Rush et al’s prescription on object identification based on grayscale or color characteristics of multiple consecutive frames, it would have been obvious for an ordinary skilled in the art before the effective filing date of the claimed invention to combine the teaching of all to identify a fire using neural network based on grayscale or color features of frames obtained in a time period for continuous monitoring and improved accuracy.
Claim 8 (Currently Amended) has been analyzed and rejected with regard to claims 1 and 6 and in accordance with Rizzotti et al’s further teaching on a second plurality of grayscale features associated with at least one neighboring frame of the first frame [Rizzotti: col 5: lines 31-50].
Regarding claim 9 (Original), the rationale applied to the rejection of claim 8 has been incorporated herein. Claim 9 has been further analyzed and rejected with regard to claim 6.
Regarding claim 10 (Original), the rationale applied to the rejection of claim 8 has been incorporated herein. Claim 10 has been further analyzed and rejected with regard to claim 3.
Regarding claim 12 (Original), the rationale applied to the rejection of claim 8 has been incorporated herein. Fan and Rush et al further teaches: The method of claim 8, wherein determining the first plurality of grayscale features comprises: transforming the plurality of color features into the first plurality of grayscale features, and wherein determining the second plurality of grayscale features comprises transforming at least one plurality of color features associated with the at least one neighboring frame into the second plurality of grayscale features [Fan: page 4: p04, page 5: p01 (Transformation takes place when fusion characteristic is obtained from fusing depth characteristic extracted from color image and undetermined degree characteristic obtained from grayscale/infrared image.) And Rush’s processing on neighboring frames: col 8: lines 61-67].
Regarding claim 13 (Original), the rationale applied to the rejection of claim 8 has been incorporated herein. Rush et al further teaches: The method of claim 8, wherein the at least one neighboring frame comprises a first neighboring frame that precedes the first frame and a second neighboring frame that follows the first frame, and wherein the first neighboring frame is associated with a plurality of color features that at least partially differs from a plurality of color features associated with the second neighboring frame [col 2: lines 22-58, col 8: lines 39-67, col 26: lines 35-45 (A patient’s positions relative to a bed can be identified through change of pixels color features from neighboring color image frames during a time period.)].
Regarding claim 14 (Original), the rationale applied to the rejection of claim 8 has been incorporated herein. Claim 14 has been analyzed and rejected with regard to claim 7.
81066.. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Fan (CN Pub: 111666901), Rizzotti et al (US Patent: 6,937,743), and Rush et al (US Patent: 10,489,661); and in further view of Wang (CN Pub: 112347942).
Regarding claim 11 (Original), the rationale applied to the rejection of claim 8 has been incorporated herein. Fan in view of Rizzotti et al and Rush et al does not specify the object comprising a flame or sort. In the same field of endeavor, Wang teaches: The method of claim 8, wherein the object comprises an explosion, a flame, or smoke [abstract]. Therefore, it would have been obvious for an ordinary skilled in the art before the effective filing date of the claimed invention to combine the teaching of all to detect fire using neural network based on color/grayscale characteristics for more accurate result.
Conclusion
9. There is a new ground of rejection necessitated by the corresponding amendment presented in this Office Action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a).
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Contact
10. Any inquiry concerning this communication or earlier communications from the examiner should be directed to FAN ZHANG whose telephone number is (571)270-3751. The examiner can normally be reached on Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benny Tieu can be reached on 571-272-7490. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Fan Zhang/
Patent Examiner, Art Unit 2682
Prosecution Insights