Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1 – 11, 13 – 21 are pending.
This action is in response to the communication filed on 2/13/26.
All objections and rejections not set forth below have been withdrawn.
Claim Objections
Claim 1 is objected for failing to accurately reflect the changes made by amendment. Specifically, it is noted that the recitation “including a feature array having a plurality” was not part of the previously recited claim language and it is improperly shown to be removed by amendment.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the features of:
“…a grayscale image including a feature array …” (e.g. claim 1, 14, 19) and
“…wherein each location comprises a grayscale value … (e.g. claim 20)
must be shown or the feature(s) canceled from the claim(s). No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required:
Regarding claims 1, 14, and 19, the applicant’s original disclosure fails to teach “…a grayscale image including a feature array …”. Specifically, the examiner notes that the original disclosure teaches a grayscale image comprising a “pixel array” of a plurality of shaded pixels (e.g. Fig. 2). While, the applicant does teach that the grayscale image is generated based upon a feature array, the applicant fails to disclose that the “grayscale image” comprises the “feature array”.
Regarding claim 20, the applicant’s specification fails to disclose a “feature array” (e.g. of claim 1) “…wherein each location comprises a grayscale value …”. Rather, the examiner notes that the applicant’s specification teaches a “pixel array”, i.e. a grayscale image, wherein each location of the pixel array comprises a grayscale value (e.g. Specification, par. 47).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 – 11 and 13 – 21 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claims 1, 14, and 19, the applicant’s original disclosure fails to teach “…a grayscale image including a feature array …”. Specifically, the examiner notes that the original disclosure teaches a grayscale image comprising a “pixel array” of a plurality of shaded pixels (e.g. Fig. 2). Specifically, the shaded pixels within the image have grayscale values created by encoding a feature array, i.e. converting feature information into RGB values representing different shades of gray. While, the applicant does teach that the grayscale image is generated based upon a feature array, the applicant fails to disclose that the “grayscale image” comprises the “feature array”.
Regarding claim 20, the applicant’s specification fails to disclose a “feature array” of claim 1, “…wherein each location comprises a grayscale value … “. Rather, the examiner notes that the applicant’s specification teaches a “pixel array”, i.e. a grayscale image, wherein each location of the pixel array comprises a grayscale value (e.g. Specification, par. 47).
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1- 11 and 13 – 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1, 14, and 19, the recitation “…a grayscale image including a feature array …” renders the scope of the claims indefinite. Specifically, the examiner notes that one of ordinary skill in the art would understand (as is also taught by applicant’s original disclosure; fig. 2; par. 47) that a grayscale image would comprise an array of pixels – thus a “pixel array” instead of the claimed “feature array”. Thus, it is not clear how the grayscale image is said to “include” a “feature array”.
Regarding claim 20, the recitation “…wherein each location comprises a grayscale value … “ renders the scope of the claims indefinite. Specifically, the examiner notes that the recited “location” references the “feature array” of claim 1. However, one of ordinary skill in the art would recognize that a “feature array” is an array of features – such as activities representing behaviors (e.g. Specification, par. 46, 45, 46). Thus, a “feature array” would not comprise ‘locations’ comprising a “grayscale value”, such as claimed. Rather, one of ordinary skill in the art would readily understand that a “pixel array” (such as disclosed by applicant, Specification, par. 47) may be said to have ‘locations’ comprising a “grayscale value”. Thus, the scope of the claims are indefinite.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 11, 13 – 21 are rejected under 35 U.S.C. 103 as being unpatentable over Khanna, “ReportIT: Improving Insider Threat Detection Model Explainability Through Report Retrieval”, Custom Project in view of Gayathri et al. (Gayathri), “Image-Based Feature Representation for Insider Threat Classification”.
Regarding claim 1, as best understood in view of the above noted deficiencies of clarity, Khanna discloses:
A system comprising: at least one physical memory device to store report generation logic; and one or more processors coupled with the at least one physical memory device to execute the report generation logic to (e.g. Khanna, sect. 3; fig. 1; fig. 7 – herein disclosed is a report generation method implemented using a computer processing device including modules such as encoders and logic for generating a report):
generate a grayscale image including a feature array … (e.g. Khanna, fig. 7 – “Feature Extraction” array which is encoded into a greyscale image).
While Khanna teaches a “feature array” derived from a feature extraction process, Khanna does not appear to explicitly teach that a derived “array” of features may comprise “…a plurality of locations, each location including behavioral features of a feature type defined for the location…”.
However, like Khanna, Gayathri also discloses the generation of grayscale images from an array of features, i.e. a “feature array”, derived from a feature extraction process (e.g. Gayathri, fig. 2). Furthermore, Gayathri teaches that the array of extracted features should be comprised within a feature vector having “…a plurality of locations, each location including behavioral features of a feature type defined for the location” (e.g. Gayathri, sect. 3.2; Table 1; Fig. 3; feature vector – L1…L9, E1…E5,H1…H3,D1…D3,F1).
It would have been obvious to one of ordinary skill in the art to recognize the defined feature vector teachings of Gayathri within the system of Khanna because one of ordinary skill in the art would have been motivated by the teachings that feature vectors are a useful technique for organizing relevant log data (e.g. Gayathri, sect. 3.1).
Thus, the combination enables:
receive text data comprising a plurality of reports, each indicating a type of behavior (e.g. Khanna, fig. 7; fig. 1, sets of textual labels – Khanna, fig. 1 – a plurality of potential insider threat reports are received);
generate a plurality of image-report encodings based on the behavioral features and the text data (e.g. Khanna, fig. 1: I*T encodings – a plurality of threat reports and the image are encoded by a text encoder and image encoder respectively so as to generate a plurality of image-report encodings);
and generate a report based on the image-report encodings, including selecting a first candidate report from the plurality of candidate reports that is associated with a stored image-text pair that matches a first of the image-report encodings (e.g. Khanna, sect. 3; fig. 1 – I * T4 selection; sect. 3 – a matching insider threat report is generated).
Regarding claim 2, the combination enables:
wherein the type of behavior comprises at least one of: text data indicating a user login, user connection to a drive, and uploaded to a remote site (e.g. Khanna, fig. 1 – potential insider text reports).
Regarding claim 3, the combination enables:
wherein the candidate report matching the image-text pair comprises an image-text pair having a highest cosine similarity with the image-report encoding (e.g. Khanna, sect. 3, par. 2).
Regarding claim 4, the combination enables:
wherein the selected candidate report indicates whether malicious activity has been detected from the behavioral information in the encoded image (e.g. Khanna, fig. 1 – the selected report indicates whether there was “malicious activity”).
Regarding claim 5, the combination enables:
wherein the selected candidate report indicates whether the malicious activity has been detected in the behavioral features (e.g. Khanna, Table 7; sect. 7.1, par. 9, 10; fig. 21 – the image classification may indicate that the picture comprises crime or offensive content).
Regarding claim 6, the combination enables:
wherein the selected candidate report indicates a type of malicious activity upon a determination that the malicious activity has occurred (e.g. Khanna, pg. 4, “Issue with Traditional Contrastive Learning”, “Baselines; pg. 8, “Report Generation”; herein sentences are generated for image reports to explain the malicious activity).
Regarding claim 7, the combination enables:
wherein the report generation logic comprises a transferable visual model to generate the report (e.g. Khanna, fig. 2; pg. 2, “Contrastive Learning”).
Regarding claim 8, the combination enables:
wherein the report generation logic further to train the transferable visual model (e.g. Khanna, pg. 3, “Training”, pg. 4-5, “Proposed Solutions”).
Regarding claim 9, the combination enables:
wherein training the transferable visual model comprises: generating a batch of image-text pairs based on a plurality of images and a plurality of text reports (e.g. Khanna, fig. 2 – report batches; pg. 7-8, “Linear Evaluation and Finetuning”);
and modifying the batch of image-text pairs (e.g. Khanna, fig. 2 – modifying normal batch to produce prune batches and class batches).
Regarding claim 10, the combination enables:
wherein modifying the batch of image-text pairs comprises removing image-text pairs in instances in which there is an identical report already in the batch in order to reduce false negative image-text pairs within a batch (e.g. Khanna, fig. 2; pg. 4, “Proposed Solutions” – removing identical reports).
Regarding claim 11, the combination enables:
wherein modifying the batch of image-text pairs comprises classifying text related to each of the plurality of images (e.g. Khanna, fig. 2; pg. 4, “Proposed Solutions” – ClassBatch, i.e. classifying text associated with the images of batch reports).
Regarding claim 13, the combination enables:
wherein classifying the text comprises performing a contrastive loss operation (e.g. Khanna, pg. 3, “Training”; pg. 4, “Proposed Solutions” – using a modified contrastive loss function to account for class weights).
Regarding claims 14 – 19, they are method and medium claims, essentially corresponding to the claims above, and they are rejected, at least, for the same reasons.
Regarding claim 20, the combination enables:
wherein each location comprises a grayscale value associated with a behavioral feature (e.g. Khanna, fig. 7; fig. 8).
Regarding claim 21, the combination enables:
wherein the grayscale image represents a timeframe for a target (e.g. Khanna, fig. 7b – the image is developed as a channel representing a day of detected behavior for a target).
Claims 1 – 11, 13 – 21 are rejected under 35 U.S.C. 103 as being unpatentable over Gayathri et al. (Gayathri), “Image-Based Feature Representation for Insider Threat Classification” in view of Radford et al. (Radford), “Learning Transferrable Visual Models From Natural Language Supervision” ArXiv.org.
Regarding claim 1, as best understood in view of the above noted deficiencies of clarity, Gayathri discloses:
A system … to:
generate a grayscale image including a feature array comprising a plurality of locations, each location including behavioral features of a feature type defined for the location (e.g. Gayathri, Fig. 2; Fig. 3; Fig. 4; feature vector – L1…L9, E1…E5,H1…H3,D1…D3,F1 – of user behaviors used to generate a grayscale image).
receive text data comprising a plurality of candidate reports, each indicating a type of behavior (e.g. Gayathri, Fig. 2; table 1 – text based log files of user actions, i.e. “reports”);
Gayathri teaches using computer vision and transfer learning to classify image data. However, Gayathri does not appear to explicitly teach the generation of image-report encodings.
Like Gayathri, Radford also teaches a system for using computer vision and transfer learning using image data (e.g. Radford, Abstract; sect. 3.1.1). Furthermore, Gayathri teaches the generation of image-report encodings (e.g. Radford, fig. 1: I*T encodings).
It would have been obvious to one of ordinary skill in the art to recognize the image-report encoding teachings of Radford within the system of Gayathri. This would have been obvious because one of ordinary skill in the art would have been motivated by the teachings that training using raw text has greatly improved within the art, and the combination or joint training a computer vision system upon both the image data and the raw text report associated with the image data would greatly improves the effectiveness of a computer vision system (e.g. Radford, Abstract; sect. 1; Gayathri, sect. 1, par. 6).
Thus, the combination enables:
generate a plurality of image-report encodings based on the behavioral features and the text data (e.g. Radford, fig. 1: I*T encodings);
and generate a report based on the image-report encodings, including selecting a first candidate report from the plurality of candidate reports that is associated with a stored image-text pair that matches a first of the image-report encodings (e.g. Radford, fig. 1: output prediction or image caption of the selected I*T encoding; Gayathri, sect. 3.3 - classification).
Further, it is noted that Gayathri does not appear to explicitly disclose the use of processors, memory, and computing code to implement the system. However, Radford does disclose … at least one physical memory device to store report generation logic; and one or more processors coupled with the at least one physical memory device to execute the report generation logic … (e.g. Radford, fig. 1; sect. 2.5; sect. 9 – system comprising encoders with GPUs of a hardware and software infrastructure).
It would have been obvious to one of ordinary skill in the art to recognize the computing hardware and software teachings of Radford within the system of Gayathri because one of ordinary skill in the art would have motivated by the need to practically implement, using known computing technology, the computing system of Gayathri.
Regarding claim 2, the combination enables:
wherein the type of behavior comprises at least one of: text data indicating a user login, user connection to a drive, and uploaded to a remote site (e.g. Gayathri, Table 1; fig. 2).
Regarding claim 3, the combination enables:
wherein the candidate report matching the image-text pair comprises an image-text pair having a highest cosine similarity with the image-report encoding (e.g. Radford, sect. 2.3, par. 4; sect. 3.1.2, par. 1; fig. 3).
Regarding claim 4, the combination enables:
wherein the selected candidate report indicates whether malicious activity has been detected from the behavioral information in the encoded image (e.g. Gayathri, sect. 3.3; fig. 5 – the image classification may be indicated as “malicious”).
Regarding claim 5, the combination enables:
wherein the selected candidate report indicates whether the malicious activity has been detected in the behavioral features (e.g. Gayathri, sect. 3.3; fig. 5 – the image classification may be indicated as “malicious”).
Regarding claim 6, the combination enables:
wherein the selected candidate report indicates a type of malicious activity upon a determination that the malicious activity has occurred (e.g. Radford, Table 7; sect. 7.1, par. 9, 10; fig. 21 – the image classification may indicate crime or offensive content).
Regarding claim 7, the combination enables:
wherein the report generation logic comprises a transferable visual model to generate the report (e.g. Radford, Title, Abstract; fig. 1).
Regarding claim 8, the combination enables:
wherein the report generation logic further to train the transferable visual model (e.g. Gayathri, sect. 3 – Transfer Learning; Radford, Title, Abstract; fig. 1).
Regarding claim 9, the combination enables:
wherein training the transferable visual model comprises: generating a batch of image-text pairs based on a plurality of images and a plurality of text reports (e.g. Radford, fig. 1: sets of textual labels and sets of images generate a plurality of I*T encodings; sect. 3.1.2);
and modifying the batch of image-text pairs (e.g. Radford, sect. 5, par. 3 – removal of duplicates; pg. 44, sect. “C” – duplicate removal; see also e.g. Radford, sect. 2.3; sect. 2.4; fig. 3 – normalization and loss function; sect. 3.1.2 – regression classification with normalized inputs; see also Radford, sect. 2.3, par. 5 – data augmentation and sect. 3.1.4 – prompt engineering and the ensembling of a plurality of image labels; pg. 44, appendix “C” – augmentation pipeline comprising image manipulations and transformations).
Regarding claim 10, the combination enables:
wherein modifying the batch of image-text pairs comprises removing image-text pairs in instances in which there is an identical report already in the batch in order to reduce false negative image-text pairs within a batch (e.g. Radford, sect. 5, par. 3 – removal of duplicates; pg. 44, appendix “C” – duplicate removal).
Regarding claim 11, the combination enables:
wherein modifying the batch of image-text pairs comprises classifying text related to each of the plurality of images (e.g. Radford, sect. 2.3, par. 5 – data augmentation and sect. 3.1.4 – prompt engineering and the ensembling of a plurality of image labels).
Regarding claim 13, the combination enables:
wherein classifying the text comprises performing a contrastive loss operation (e.g. Radford, sect. 2.3, par. 4; fig. 3).
Regarding claims 14 – 19, they are method and medium claims, essentially corresponding to the claims above, and they are rejected, at least, for the same reasons.
Regarding claim 20, the combination enables:
wherein each location comprises a grayscale value associated with a behavioral feature (e.g. Gayathri, sect. 3.2 – each pixel is a normalized behavioral feature in the range from 0 – 255).
Regarding claim 21, the combination enables:
wherein the grayscale image represents a timeframe for a target (e.g. Gayathri, sect. 3.2 – the grayscale image is generated based upon a user-per-day feature vector).
Response to Arguments
Applicant's arguments filed 2/13/26 have been fully considered but they are not persuasive.
Applicant argues or alleges essentially that:
…
… Moreover, applicant submits that Khanna fails to disclose, or reasonably suggest, that the behavior image encodings includes a feature array having a plurality of locations each including behavioral features of a feature type defined for the location. …
…
As shown above, Gayathri discloses using grayscale images for anomaly detection to represent malicious behavior. However, there is no disclosure or suggestion that the grayscale images include a feature array having a plurality of locations each including behavioral features of a feature type defined for the location.
…
(Remarks, pg. 7, 8)
Examiner respectfully responds:
The examiner respectfully disagrees. Specifically, the examiner notes that the office action relies upon Gayathri for teaching the argued feature of a “feature array” having a plurality of locations each including behavioral features of a feature type.
Specifically, Gayathri teaches that user behavior information is organized according to a feature vector, i.e. “feature array”, having a plurality of locations, e.g. L1…L9, E1…E5,H1…H3,D1…D3,F1, which define a particular type of user behavior (e.g. Gayathri, Fig. 2; Fig. 3; Fig. 4; feature vector – L1…L9, E1…E5, H1…H3, D1…D3, F1).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY L WILLIAMS whose telephone number is (571)272-7965. The examiner can normally be reached on 7:30 am - 4:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Farid Homayounmehr can be reached on 571-272-3739. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEFFERY L WILLIAMS/Primary Examiner, Art Unit 2495