DETAILED ACTION
This action is in response to the application filed on October 20th, 2023. Claims 1-10 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1 and 7 are objected to because of the following informalities:
Claim 1: “extracting the video frame image corresponding to the cropped images.” The cropped images which are referred to in this portion actually refer to the n selected cropped videos which have passed a confidence level selection. Furthermore, “the video frame image” should be plural. Therefore, this should read “extracting the video frame images corresponding to the n selected cropped images”
Claim 1: “performing similarity matching on the feature extracted from each target image and features of the video frame images in the retrieved monitoring video data” In this claim, there is no reference to features which are extracted from retrieved monitoring video data. This seems to be referring to the features extracted in step 2; however these features are extracted from cropped images. Therefore, this limitation should read “performing similarity matching on the feature extracted from each target image and features of the video frame images in the cropped image”
Claim 7: “in step 3-2, there is a fixation time of 5s before each trial starts; after the trial starts, each stimulus image is presented for 500ms” The reference to “the trial” lacks antecedent basis, this should probably read “after each trial starts”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation “framing and clipping a target in the video frame images to obtain cropped images”, which is performed in step 2. The specification describes the corresponding step 2 in paragraphs [0069]-[0071]. As can best be understood from this description, a plurality of images are extracted from monitoring videos. These images are then processed by the target detection model to identify any potential target which is within the video frame. This could include a plurality of targets on a single video frames, as implied by the naming convention of the .jpg file described in paragraph [0070] where each frame number in a video has a target number associated with it. However, the corresponding claim language does not support identifying multiple targets, and instead is directed towards detecting a target. This rejection could be overcome by amending this limitation to recite a plurality of potential targets or just a plurality of targets which are framed and clipped in the video frame images. Then, each reference to “target” in the remaining claim limitations would have to specify whether it is done for each target, or only the target which is currently of interest. So, instead of “extracting a feature of the target in the cropped images” this would read “extracting a feature of each target of the plurality of targets in the cropped images”. Step 3 would be changed from “determining… whether a user from which an EEG signal is collected observes the target” to “determining… whether a user from which an EEG signal is collected observes a target of interest”, or another wording of applicant’s choice. Then, step 4-1 would be changed from “… according to a coarse-grained feature of the target” to “… according to a coarse-grained feature of the target of interest.” Step 4-2 would be changed from “when the EEG classification model determines that the user observes the target” to “when the EEG classification model determines that the user observes the target of interest.” Finally, step 4-4 would be changed from “…as an image in which the target is present” to “… as an image in which the target of interest is present.” Claims 6 and 7 also refers to “stimulus images comprising the target”, it should be clarified whether this target is the target of interest identified in claim 1, or if this is any target of the plurality of targets.
Claim 1 refers to “mass videos” in line 1 and “monitoring videos” in step 2 line 1. However, in step 4, target retrieval is performed on “the videos.” It is unclear if this target retrieval is performed on the mass videos, or on the monitoring videos.
Claims 2-10 are also rejected due to their dependency on claim 1.
Allowable Subject Matter
The examiner found neither prior art cited in its entirety, nor based on the prior art, found any motivation to combine any of the said prior art that teaches the following combination in the context of the claim as a whole: “extracting video frame images from retrieved monitoring video data; performing detection analysis on the video frame images using the target detection model, and framing and clipping a target in the video frame images to obtain cropped images; and extracting a feature of the target in the cropped images using the reidentification model;… prescreening, by using the reidentification model, the cropped images identified by the target detection model according to a coarse-grained feature of the target; selecting n cropped images according to a confidence level by the reidentification model; and extracting the video frame image corresponding to the cropped images; providing the video frame images obtained after the prescreening in step 4-1 for viewing by the user; recording an EEG signal and eye movement information while the user views the video frame images; preprocessing the obtained BEG signal and then inputting the preprocessed BEG signal to the BEG classification model; when the BEG classification model determines that the user observes the target, processing eye movement data of the user, extracting regions of interest for the user from the video frame images; and extracting a candidate target set from the regions of interest; … and extracting a feature from each target image obtained in step 4-3 using the reidentification model, and performing similarity matching on the feature extracted from each target image and features of the video frame images in the retrieved monitoring video data; and taking each video frame image corresponding to the feature having a similarity exceeding a threshold as an image in which the target is present.” This combination of limitations requires 3 machine learning models, one for target identification, one for reidentification and feature extraction, and a final model to process the EEG data to identify if the target of interest is present.
The prior art of record (US20180089531) does teach using a ML model to interpret EEG data to identify if a target of interest is present. Furthermore, it is well known in the art to perform feature extraction and target detection with, for example, bounding boxes. However, the claim language specifically requires that, in step 2, the reidentification model extract a feature of each target, which is then used later in step 4-4 to compare to a feature of each target image identified by the EEG model. Furthermore, the claim requires that the reidentification model perform the prescreening step of comparing a coarse grained feature of the target of interest to each cropped image. There is currently no evidence to suggest that any one model (which would subsequently be interpreted as the reidentification model) would perform both of these functions. The current prior art of record (“A coarse-to-fine deep learning for person re-identification”) teaches that two networks can compare their output to ensure there is a similarity between two targets (See figure 1a Step 3, and Section 2.4). However, the two networks described in this specification are “Siamese” networks, referring to the fact that they perform the same function, and merely compare their output data to ensure they received the same output. This is functionally different from step 4-4 of the current disclosure, as the features for comparison were obtained from two different methods, one from the EEG model, and the other from the reidentification model. Furthermore, the current prior art of record (“Faster Person Re-Identification: One-Shot-Filter and Coarse-to-Fine Search”) teaches a method of performing coarse person identification (for example, ensuring that only images of women with white tops are considered when searching to identify a woman with a white top as a target) and refining the identification by using more fine attributes. This is similar to step 4-1 of the current disclosure, as the coarse feature used would be analogous to the prescreening step. However, the combination of this reference along with the previous two reference would not be obvious, as the combination would require that the reidentification model be interpreted as multiple different machine learning models which are functionally independent. Furthermore, there is no teaching, motivation, or suggestion within the secondary (non-EEG) references that would suggest to combine these references with an EEG model. For at least these reasons, claim 1 is not taught by the prior art. Claims 2-10 are not taught by the prior art due to their dependency on claim 1.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US20220051039 teaches a method of using EEG data to determine if a user has observed a target.
“Person Re-Identification Using Hybrid Representation Reinforced by Metric Learning” teaches person reidentification, where after a person representation phase, person matching is performed using two separate analysis methods.
“Coarse-to-Fine Person Re-Identification with Auxiliary-Domain Classification and Second-Order Information Bottleneck” teaches a method of person identification, where images which are determined to not contain people are discarded, and only image which contain humans are further processed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CALEB LOGAN ESQUINO whose telephone number is (703)756-1462. The examiner can normally be reached M-Fr 8:00AM-4:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CALEB L ESQUINO/ Examiner, Art Unit 2677
/ANDREW W BEE/ Supervisory Patent Examiner, Art Unit 2677