DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record on file.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-4 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claims are generally narrative and indefinite, failing to conform with current U.S. practice. They appear to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors.
The claims have antecedent basis issues in several places, such as
“information from step 2” in claim 1, line 13, wherein there is no clear indication of which information is being used in step 2 since there is no explicit first instantiation of “information” in the recite step 2 to be used as antecedent basis for step 3, and other antecedent places of same/similar issues in claim 1;
“the short clips cut out” in claim 2, line 10, wherein there is no first instantiation of “short clips cut out” prior for this antecedent basis and other antecedent places of same/similar issues in claim 2;
“the proposed model used in this invention” in claim 3, step 3, wherein there is no first instantiation of “a proposed model used in this invention” prior to this antecedent and other antecedent places of same/similar issues in claim 3;
“the probability of an anomaly” or “the probability of an anomalous behaviors” in claim 4 and other places, there are no first instantiations of these references to have proper antecedent basis and other antecedent places of same/similar issues in claim 4;
Appropriate corrections are required.
Regarding claim 2, the phrases "(a.k.a….)" and “(i.e…..),” specially, the terms a.k.a and i.e. and the “()” paratheses render the claim indefinite because it is unclear whether the limitation(s) following the phrases and within the parathesis are part of the claimed invention. See MPEP § 2173.05(d). For the interest of continued examination, the examiner interprets these phrases in the claim such as “raw data is video information continuously transmitted from surveillance cameras (a.k.a. Real Time Streaming Protocol (RTSP))” to be read as “raw data is video information continuously transmitted from surveillance cameras, which is known as Real Time Streaming Protocol (RTSP),” and “with a sliding window stride on the original video of 1 second ( i.e., every 1 second in a row, extract a short clip of length 5 seconds)” to be read as “with a sliding window stride on the original video of 1 second, that is every 1 second in a row, extract a short clip of length 5 seconds,”.
Appropriate corrections are required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-4 are rejected under 35 U.S.C. 101
Regarding Independent Claim 1 and its dependent claims 2-4:
Step 1 Analysis: Claim 1 is directed to a method/process, which falls within one of the four statutory categories.
Step 2A Prong 1 Analysis: Claim 1 recites, in part:
“calculates a probability of having anomalous behaviors in each pre-processed short clip and forwards them to step 3;
accurately predict whether or not abnormal behavior will occur, and issues a warning if any abnormal behavior is predicted to occur”
The limitations as drafted, are processes that, under broadest reasonable interpretation, covers the performance of the limitation in the mind which falls within the “Mathematical Calculation” and “Mental Processes” grouping of abstract ideas. The limitations of:
Such as the human mind can perform mental processes, based on BRI (broadest reasonable interpretation), of processes such as observation and evaluation to predict abnormalities and issue a warning and the step of calculation in the claim is an explicit mathematical operation.
Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 Analysis: This judicial exception is not integrated into a practical application. particular, the claim recites the following additional element(s) –
Step 1: data pre-processing; raw data streamed from cameras is cut to short clips, the short clips are brought back to a same playback rate, sampled, resized and normalized to produce pre-processed short clips; Step 2: feature extraction and abnormal prediction; a three-dimensional convolution neural network (3D CNN) is used to extract spatial-temporal features from the pre-processed short clips from step 1, the 3D-CNN;
Step 3: post-processing and synthesizing information to issue warning; this step integrates information from step 2 and removes noise.
The additional elements include insignificant extra-solution activities of data gathering such as streaming raw data to then gather further data inform of short clips and being brought back and playbacked (a form of data gathering) and being sampled, resized and normalized to produce further data, the feature extraction integrating information are all forms of data gathering which are insignificant; the additional elements of data pre-processing being an insignificant pre-solution activity, and the post-processing being an insignificant post-solution activity; moreover, the 3D CNN is recited at high level of generality to be generic neural network, recited as a mere attempt to implement or apply the judicial exceptions using generic neural network; moreover, the additional element of “cameras” is being recited at high level of generality performing generic function hence, not indicative of an integration of the judicial exceptions into a practical application. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea. Accordingly, these additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Please see MPEP §2106.04.(d).III.C.
Step 2B Analysis: there are no additional elements that amount to significantly more than the judicial exception. Please see MPEP §2106.05. The claim is directed to an abstract idea.
The additional elements include insignificant extra-solution activities of data gathering such as streaming raw data to then gather further data inform of short clips and being brought back and playbacked (a form of data gathering) and being sampled, resized and normalized to produce further data, the feature extraction integrating information are all forms of data gathering which are insignificant; the additional elements of data pre-processing being an insignificant pre-solution activity, and the post-processing being an insignificant post-solution activity; moreover, the 3D CNN is recited at high level of generality to be generic neural network, recited as a mere attempt to implement or apply the judicial exceptions using generic neural network; moreover, the additional element of “cameras” is being recited at high level of generality performing generic function hence, not indicative of an integration of the judicial exceptions into a practical application. Accordingly, these additional elements do not indicate a consideration of the claims features and the additional elements being significantly more because they do not impose any meaningful limits on practicing the abstract idea. The claim as a whole is directed to an abstract idea. Please see MPEP §2106.04.(d).III.C.
For all of the foregoing reasons, claim 1 does not comply with the requirements of 35 USC 101.
Accordingly, the dependent claims 2-4 do not provide elements that overcome the deficiencies of the independent claim 1. Moreover, claim 2 recites, in part, a series of limitations that are additional elements, under Step 2A, to be insignificant extra-solution activities of data gathering such as the steps of processing raw data, transmitting raw data, inputting data and cutting video into short clips based on certain condition criteria, using data as in video segments according to certain condition/criteria, sampling and resizing of frames, there are still merely steps of data gathering, transmitting data, converting changing data/information, hence being insignificant extra-solution activities which are not indicative of an integration of the judicial exceptions into a practical application under step 2A, nor considered significantly more under step 2B. Claim 3 recites, in part, also a series of significant extra-solution activities of data gathering for the steps of inputting data/information into a model and outputting of data/information from the model, and combining data/information, resulting in further output/outcome data/information, converting output data/information and further specification of what the data/information are hence, still merely data gathering being insignificant, and the steps of taking a dot product, is a mathematical operation abstract idea under step 2A prong 1, and the different layers, activation function, ReLU, and the SlowFASt model and the Slow and FAST branch are generic well-known components, routines and neural network components to perform generic well known functions in the art, being recited at high level of generality without further limiting of how these components and the neural network works, in details to arrive at such output, therefore, the claim is not an indication of an integration of the judicial exceptions into a practical application, not being considered significantly more; even though the claim recites “helping to increase the…efficiency” however, “help” is a relative action that the action after help does not always come at full effect or have no such consequence at some instances, moreover, the increase of the efficiency is recited without referring to any of the results of output of the previous steps hence, not indicative of integrating them into a practical application not bringing them to be significantly more. Claim 4 recites, in part, also a series of insignificant extra-solution activities of data gathering under Step 2A Prong 2, and mathematical operation abstract ideas under Step 2A Prong 1, and additional elements of generic components, routines and neural network components, and further specification of what data/information being used, hence, still merely insignificant.
Accordingly, the dependent claims 2-4 are not patent eligible under 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Yepeng Guan e.t al. (“Abnormal behavior recognition using 3D-CNN combined with LSTM, Feb. 2021, Multimedia Tools and Applications, Vol. 80, pp. 18787-18801” hereinafter as “Guan”) in view of Yuan Yao et. al. “(Video Playback Rate Perception for Self-Supervised Spatio-Temporal Representation Learning, 2020, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 6548-6557” hereinafter as “Yao”) and further in view of Duarte Duque et. al. (“Prediction of Abnormal Behaviors for Intelligent Video Surveillance Systems, 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining” hereinafter as “Duque”)
Regarding claim 1, Guan discloses method of crowd abnormal behavior detection from video using artificial intelligence (title and abstract disclose a crowd abnormal behavior detection from video using artificial intelligence), comprising three steps: Step 1: data pre-processing; raw data streamed from cameras is cut to short clips (section 1, last 2pars., section 2.2 discloses the video is cut into clips for the processing which is analogous to the recited data pre-processing as claimed which includes raw data [the video] being cut into short clips ); Step 2: feature extraction and abnormal prediction; a three-dimensional convolution neural network (3D CNN) is used to extract spatial-temporal features from the pre-processed short clips from step 1 (section 3.1, 1st 2 pars., discloses using the 3D CNN to extract spatial-temporal features from the short clips previously obtained), the 3D-CNN then calculates a probability of having anomalous behaviors in each pre-processed short clip and forwards them to step 3 (section 3.1, 1st 3 pars., discloses the 3D-CNN is then to calculate the importance of each region contributes to the abnormal behavior detection, which is based on a weighted matrix, therefore, the importance calculated to contribute to the recognition based on a weight matrix is analogous to a probability of having anomalous behavior, since each region has an importance calculated specifically to determine its importance to contribute to the final abnormal behavior detection which is an indication of probability of the region having anomalous behavior in the image, different regions would have different abnormal behavior importances); Step 3: post-processing and synthesizing information (FIG. 4 shows the network to output information which is analogous to synthesize information as claimed, based on BRI); this step integrates information from step 2 and removes noise to accurately predict whether or not abnormal behavior will occur (and based on the output of the network is to predict the abnormal behavior [abstract], moreover, section 2.2, discloses a step of removing some samples that are not needed which can be understood to be removes noise to accurately predict the abnormal behavior better, based on BRI).
However, Guan does not explicitly disclose the short clips are brought back to a same playback rate, sampled, resized and normalized to produce pre-processed short clips; to issue warning and issues a warning if any abnormal behavior is predicted to occur.
In the same field of short clip processing (title and abstract, Yao) Yao discloses the short clips are brought back to a same playback rate (section 3, 1st par., discloses browsing the video short clips with playback to process the video clips which is analogous to processing video clips of Guan, moreover, Yao teaches that the video clips are playback at the same rate for each clip [section 3, 1st 3 pars.] which is analogous to the claimed feature, by BRI), sampled, resized and normalized to produce pre-processed short clips (section 3.1 and 3.2, discloses the video clips are sampled and resized and normalized to produce the output for obtaining spatio-temporal features which is analogous to the obtaining of the features of Guan, by BRI,; the resizing is further taught in page 6653, 1st col., last par., wherein the video frames are resized for the processing, which, by BRI, is analogous to being resized as claimed; moreover, the normalizing is disclosed in page 6551, 1st par., wherein the batches of the video clips are normalized, by BRI, is analogous to the recited feature; the output of Yao after these processing is analogous to the recited pre-processed short clips as claimed, by BRI).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Guan to perform processing of a video into short clips and have the short clips being playback at the same rate and being resized and normalized to produce pre-processed short clips as taught by Yao to arrive at the claimed invention discussed above. Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to process video in video clips for further post-processing more efficiently (abstract, Yao).
However, Guan in view of Yao does not explicitly disclose to issue warning and issues a warning if any abnormal behavior is predicted to occur.
In the same field of abnormal behavior detection (title and abstract, Duque), Duque discloses to issue warning and issues a warning if any abnormal behavior is predicted to occur (page 362, 1st col., last par., discloses when an abnormal behavior is detected in the crow, the processing will issue an alarm, which, by BRI, is analogous to the claimed limitation).
Thus, it would have been obvious for a person of ordinary skill in the art before the effective filing date to modify Guan in view of Tao to perform processing of a video into short clips and have the short clips being playback at the same rate and being resized and normalized to produce pre-processed short clips and use the short clips to detect abnormal behavior, when any abnormal behavior is predicted, issue a warning as taught by Duque to arrive at the claimed invention discussed above. Such a modification is the result of combing prior art elements according to known methods to yield predictable results. The motivation for the proposed modification would have been to detect abnormal behavior more efficiently (abstract, Duque) and issue a warning to warn about abnormal behavior in an improved approach (page 362, 1st col., last par., Duque).
Pertinent Prior Art(s)
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“G. Screenu et. al., Intelligent video surveillance: a review through deep learning techniques for crowd analysis, 2019, Sreenu and Saleem Durai J Big Data (2019) 6:48” discloses crowd analysis for abnormal behavior detection (abstract) using deep learning method (page 10, last par.) using feature extraction (page 12, 1st par.) further shown in FIG. 2.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUONG HAU CAI whose telephone number is (571)272-9424. The examiner can normally be reached M-F 8:30 am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUONG HAU CAI/Examiner, Art Unit 2673
/MICHAEL HORABIK/Supervisory Patent Examiner, Art Unit 2675