Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1-20 are pending. Claims 1, 7 and 14 are independent. Only Claim 5 is amended to overcome an informality.
This Application was published as U.S. 2024/0062570.
Apparent priority: 19 August 2022.
Applicant’s amendments and arguments are considered but are either unpersuasive or moot in view of the new grounds of rejection that, if presented, were necessitated by the amendments to the Claims.
This action is Final.
Response to Amendments and Arguments
Objection to Claim 5 is withdrawn.
Applicant’s arguments are not persuasive.
Claim 1:
Claim 1 provides:
1. A computer-implemented method for detecting Unicode injection in text, the method comprising:
training a language model to determine if text data conforms with human writing habits; and
receiving text data by said language model to determine if said text data is suspect for containing Unicode characters based on whether said text data conforms with human writing habits.
This Claim is broad.
Claim begins by referring to a “method of detecting Unicode injection in text” and continues by training a model to determine if “text data conforms with human writing habits” as the method of detecting Unicode injection/attack. In other words: “determine if said text data is suspect for containing Unicode characters based on whether said text data conforms with human writing habits.”
There is no definition provided for “human writing habits” and the only way the “human writing habits” are circumscribed is that they are connected with “Unicode injection” and can be used to detect Unicode injection.
While a definition must be present in the independent Claim, the dependent Claims were searched in the hope of finding a Claim that would elaborate upon what these “human writing habits” may be. Only Claim 2 mentions them and not to define but to say that negative and positive samples are used for the training of a model which is a well-known process of training.
The Claim needs to define the phrase with particularity and right now they only particularity expressed by the Claim is that “text data is suspect for containing Unicode characters based on whether said text data conforms with human writing habits.”
All that the current articulation says is this: Conforming with human writing habits is somehow correlated with containing Unicode characters.
Does not say what “human writing habits” are.
Does not say how “human writing habits” are determined.
Does not provide any examples of “human writing habits.”
Does not even say text that conforms to human habits does or does not have Unicode injection.
Specification:
While the Specification may not be read into the Claim, we refer to the Specification to see if a definition may be found.
The Specification too, like the Claims, does not include any definition or examples of “human writing habits.”
The closest the Specification comes is as follows:
[0035] In one embodiment, Unicode injection detection mechanism 101 is configured to detect Unicode characters injected in text 104 (e.g., text fragment), which is received by Unicode injection detection mechanism 101, by training a language model to detect text (e.g., text fragment) that does not conform with human writing habits. It has been discovered that text injected with Unicode characters does not conform to human writing habits. For example, the phrase “% co % af,” which includes Unicode characters to encode certain characters in the text fragment, does not conform to human writing habits. As a result, in one embodiment, a language model is trained to detect text (e.g., text fragment) that does not conform with human writing habits so as to detect text (e.g., text fragment) that is suspect for containing Unicode characters. A more detailed description of these and other features is provided below.
This is one single sole example and a definition or a rule cannot be deduced from this one example.
Considering that different humans have different “human writing habits,” one example does not create a definition.
The only definition provided by the Specification (and also the Claim) is that “human writing habits” and Unicode text are correlated.
“homographs” as Two references, Brown and Lee, were cited in a 103 combination.
However, each reference alone would have formed a strong single reference 103 against the Claim:
Brown lacks an express “training of language model” and instead included “training a machine learning model.”
Lee did not include “detecting Unicode injection” and was instead directed to detecting phishing scams. However, on way Phishing is by Unicode injection. Thus, detected Phishing may include detecting Unicode injections.
See the following as evidence that Unicode Injection is a type of Phishing:
Stokes (U.S. 20220279014):
PNG
media_image1.png
130
432
media_image1.png
Greyscale
Proux (U.S. 20070283000):
“[0004] Another approach (this is referring to another approach for Phishing) is utilization of a technique which consists of replacing standard ASCII characters by the Unicode of some other visually similar characters used by non-roman character based languages. For example, a recent case concerned the Paypal website. In this case the forged URL was http://www.pа:ypal.com/, which appeared on the address bar of the Internet browser as a valid and official Paypal URL (http://www.paypal.com). However, the first "a" was not the standard English form but rather a character from another character set (e.g., Cyrillic). This forgery appeared visually identical to the official address (as displayed in the address field of the browser due to its Unicode interpretation capabilities), but which actually corresponded to a different URL.”
“4. The method for detection of phishing attempts in received electronic mail messages according to claim 2, wherein said visual homonym is derived through transformation of any Unicode character similar to an ASCII character into its ASCII counterpart.”
Brown
The Specification defines the Unicode Injection of the Claim as:
[0005] Unicode injection may also occur by converting commonly used English letters in text, such as a text training set used by natural language processing tasks, into letters used in a different language with a similar appearance, such as Cyrillic letters. Such a type of Unicode injection is referred to as “indirect Unicode injection,” which may also cause failures of a natural language processing task.
Brown detects “Unicode injection” by detecting “homographs” and the ”homographs” of Brown are defined the same way that the “Unicode injection” is defined by the instant Application as provided above:
“In computer typography, it is not uncommon for different characters to resemble each other, or be “homographs.” For example, in a sans serif font, a lower case letter “L” may look identical to an upper case letter “I.” However, in moving beyond the relatively small American Standard Code for Information Interchange (ASCII) character set to the internationalized Unicode character set with thousands of characters, the problem of character homographs is compounded.” Brown: 1:5-15.
See Figure 1 of Brown as an example:
PNG
media_image2.png
248
400
media_image2.png
Greyscale
Thus “Unicode injection” of the Claim and “homograph attacks” of Brown are the same thing.
Thus:
In Brown: detecting homographs [Wingdings font/0xE8] there was a Unicode injection attack.
In Claim: detecting (presumably lack of) “conform to human writing habits” [Wingdings font/0xE8] there was a Unicode injection attack.
Homographs do not conform to human writing habits. A person writing “americabank” does not use the Cyrillic letters “ Б ” or “ Ъ” (be or ve) in place of the English “b” in bank.
Thus, absent some definition to the contrary, Brown is very much on point and teaches the “human writing habit” of the Claim.
Lee:
Lee is directed to detecting Phishing attacks. However, as set forth above, see Stokes and Proux as evidence that Unicode Injection is a common type of Phishing attacks.
Applicant’s Arguments:
Applicant’s arguments against the cited references are focused on the references not teaching methods that are based on “human writing habits.”
PNG
media_image3.png
186
642
media_image3.png
Greyscale
Response 7.
In Reply: As provided above, “human writing habits” are not defined by either the Claim or the Specification. The phrase by itself is very broad and may even be considered “indefinite.” The detection of Unicode injection by way of homographs of Brown and detection of Phishing scams of Lee, which is based on detection of Unicode injection by way of homographs/homologs, both fit the description provided by the Specification of the instant Application for the correlation of Unicode injection and human writing habits.
In short: the current definitions of both Brown and Lee nicely fit and conform to the single example of the Specification. Anything else intended must be claimed with particularity that finds support in the disclosure as filed, before it may be argued.
Applicant provides a description of Brown and Lee (Response 7-8) and concludes that neither teaches the first limitation:
PNG
media_image4.png
90
626
media_image4.png
Greyscale
PNG
media_image5.png
94
624
media_image5.png
Greyscale
PNG
media_image6.png
244
620
media_image6.png
Greyscale
Response 8.
In Reply: the discussion of “human writing habits” above provides a reply to all of the points raised by the Applicant:
Claim does not include a definition for “human writing habits,” and what can be gleaned from the scant supporting Specification conforms to both Brown and Lee.
The machine learning of Brown is trained to detect homologs that may be Unicode injections which is another way of saying detecting writing that is wrong in some way, i.e. does not conform to correct human writing.
Applicant admits that Lee’s model is train to detect phishing attacks and as provided above Unicode injection is a type of phishing attack which uses letters that do not conform to correct writing of the words.
Applicant next moves to the second limitation and argues regarding Brown:
PNG
media_image7.png
118
630
media_image7.png
Greyscale
PNG
media_image8.png
174
620
media_image8.png
Greyscale
PNG
media_image9.png
126
630
media_image9.png
Greyscale
Response 8-9.
In Reply, as provided above, the Claim does not specify what conforms and what does not conform to “human writing habits” or what “human writing habits” are. Someone raised with a Slavic language as their mother tongue and writing English as their second language is actually very likely to use the Cyrillic alphabet when writing by hand, if not when using a keyboard. This error would be completely based on his writing habits and he is human. Further, as also provided above, the manner of operation of Brown by detecting homographs as indicators of Unicode injection is quite similar to the description provided by the Specification of the instant Application.
Applicant argues with respect to Lee;
PNG
media_image10.png
332
622
media_image10.png
Greyscale
PNG
media_image11.png
126
622
media_image11.png
Greyscale
Response 9.
In Reply: while Applicant is setting forth the details of training a Language Model, the end result and bottom line is a Language Model that is trained to detect Phishing Attacks and Unicode Injection is a type of Phishing Attack. While Lee does not mention Unicode injection, the combination of Brown and Lee can substitute the Unicode Injection of Brown into the model training of Lee and get a model that is trained to detect Unicode Injections.
Both arguments hinge on the “human writing habits” and the Applicant returns to this argument:
PNG
media_image12.png
232
634
media_image12.png
Greyscale
Response 10.
In Reply: as provided above: what are “human writing habits”? This phrase is key to the Applicant’s arguments and is undefined in both the Claim and the Specification. Provide a definition. Provide examples. Note that a Slavic native is very likely to have a habit of injecting Cyrillic characters when writing by hand.
Further, the manner is which the Applicant is characterizing the references leaves out that: 1) Brown detects Unicode injection where Unicode injection is defined as injecting homologs that look like an English character but are not such as the example shown in Figure 1 of this reference and reproduced above and 2) Lee is directed to detecting Phishing scams one type of which is Unicode injection that is the subject of the Claim.
Claims 7 and 14:
Applicant has presented arguments regarding the pair of parallel independent Claims 7 and 14 that are addressed below. Claim 7 has done away with the illusive “human writing habit” and is based on more concrete determinations.
7. A computer-implemented method for detecting Unicode injection in text, the method comprising:
recording image data from a copy of original data;
recording a first set of text data from said original data;
performing optical character recognition on said recorded image data to generate a second set of text data;
generating a first feature vector for said first set of text data;
generating a second feature vector for said second set of text data; and
comparing said first and second feature vectors to determine if said first set of text data is suspect for containing Unicode characters.
The heart and main feature of this Claim is taught by Brown as shown by the mapping. Brown like this Claim knows what the text string (URL of Bank of America, for example) should be and compares it against a text string obtained from performing OCR on an image of a received text string (the fake URL with the Unicode Injection of a Cyrillic character that looks like “b” but is not “b”) and compares the two text strings to detect the fake (the one with Unicode Injection).
Brown leaves out a very well-known implementation detail which is comparison of two text strings by forming feature vectors of each. For that Malkiel was combined. Any number of references teach comparing text strings by obtaining a vector distance.
Applicant argues:
PNG
media_image13.png
158
634
media_image13.png
Greyscale
Response 11.
Brown was cited for teaching the comparison of two text strings in order to determine their similarity or differences.
Maikel was only cited for teaching the comparison of two text strings using feature vectors corresponding to each.
Applicant characterizes Maikel as:
PNG
media_image14.png
150
626
media_image14.png
Greyscale
Response 11.
This is correct. Maikel generates feature vectors representative of search queries.
Then, the Applicant concludes:
PNG
media_image15.png
90
636
media_image15.png
Greyscale
PNG
media_image16.png
92
636
media_image16.png
Greyscale
…
PNG
media_image17.png
98
616
media_image17.png
Greyscale
PNG
media_image18.png
162
624
media_image18.png
Greyscale
…
PNG
media_image19.png
230
640
media_image19.png
Greyscale
Response 11-14.
In Reply:
First: the “recordation” argument is a red Heron. The recordation occurs for the image and in the context of performing OCR on an image to get text. “performing optical character recognition on said recorded image data to generate a second set of text data …” Further, image capture is recordation and is taught by the cited portions of Brown and shown in its Figures: “…According to the present disclosure, an image 109 of the address bar 106b may be captured, and an optical character recognition (OCR) process performed….” Brown, 3:20-23.
Second: a search query is a text string or a first set of text data. Therefore, generating a feature vector from a search query is generating a feature vector of a text string. Brown compares text strings. Malkiel teaches that one method of comparing text strings is by comparing feature vectors generated from the text strings. Thus, the combination teaches a particular method of comparing text strings, i.e. by the vector distance of Malkiel: “… A first feature vector representative of the search query is obtained. A respective semantic similarity score is determined between the first feature vector and each of a plurality of second feature vectors. …” Abstract. Additionally, finding the similarity between two pieces of text by finding the vector distance between the vectorized forms of text is quite common. “…The similarities between the sentence vectors are then calculated and stored in a similarity matrix. The similarity is scored between the vectors via a standard distance function….” Malkiel, 12: 21-25.
Third: Brown is dealing with comparison of two strings of text and Malkiel teaches that comparison of two strings of text can be done by vector distance. Malkiel is comparing “sentence vectors” and obtaining “similarity” based on a “distance function.” All of these point to at least two vectors to be compared: first and second.
Fourth: Brown is comparing two text strings in search of homographs that may be injected by Unicode Injection. This is discussed for rejection of Claims 1 and 7 both at length. Further, Figure 1 of Brown that is provided above and below again is a clear and concise teaching of the argued: “comparing said first and second {feature vectors =text strings} to determine if said first set of text data is suspect for containing Unicode characters.”
PNG
media_image20.png
746
484
media_image20.png
Greyscale
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 is rejected under 35 U.S.C. 103 as being unpatentable over Brown (U.S. 10,943,067) in view of Lee (U.S. 20220094713).
Regarding Claim 1, Brown teaches:
1. A computer-implemented method for detecting Unicode injection in text, the method comprising: [Brown detects and removes or replaces “homographs” which are generated by inserting Unicode look-alikes into text. “In computer typography, it is not uncommon for different characters to resemble each other, or be “homographs.” For example, in a sans serif font, a lower case letter “L” may look identical to an upper case letter “I.” However, in moving beyond the relatively small American Standard Code for Information Interchange (ASCII) character set to the internationalized Unicode character set with thousands of characters, the problem of character homographs is compounded.” 1:5-15.]
training a language model to determine if text data conforms with human writing habits; and [Brown, Figure 2, “data processing applications 221” trains a machine learning model that identifies homograph attacks which are a type of Unicode injection. Figure 5, 515. “In box 515, the data processing application 221 may process the result string. As an example, the data processing application 221 may use the result string to train a machine learning model, thus defeating homograph attacks meant to poison machine learning training data sets….” 9:47-50. “The data processing applications 221 are executed to perform a data processing function with respect to a data corpus. For example, a data processing application 221 may be training a machine learning model, indexing network pages for a search engine, performing plagiarism detection, sanitizing source code repositories, or performing other functions. The homograph recognition engine 218 may be used to sanitize the data corpus to remove homographic strings or to replace them with the strings that they appear to be before the data corpus is processed.” 4:60 to 5:2. Homographic strings do not conform with human writing habits because they, e.g., put small L where the letter should be I: they could write “lSLAND” instead of “ISLAND” where the first letter of the first occurrence is L and not i.]
receiving text data by said language model to determine if said text data is suspect for containing Unicode characters based on whether said text data conforms with human writing habits. [Brown, Figure 5, 512: “Obtain result string from homograph recognition engine.” “… In some cases, if such steganographic data is detected (e.g., an abnormal quantity or pattern of non-rendering characters or zero-width spaces), the occurrence may be flagged for further review by a system administrator. In box 512, the data processing application 221 obtains a result string from the homograph recognition engine 218….” 9:40-46. Figure 3, 315.]
Brown teaches all of the limitations of this broad Claim either expressly or impliedly. However, the training is implicit in Brown and a second more express reference is added to expedite prosecution. Additionally, Brown does not expressly use the phrase “language model” although the content implies that the machine learned models are language models.
Lee teaches:
1. A computer-implemented method for detecting Unicode injection in text, the method comprising: [Lee is directed to detecting phishing attacks.]
training a language model to determine if text data conforms with human writing habits; and [Lee, Figure 1, “machine learning model 114.” Figure 2, “Machine learning training engine 200” which takes the “training data 206” and trains a “detection model 202.” See Figure 7 for the process of training. “[0052] In some implementations, the first machine learning model 310 may be based on a neural network construct referred to as a transformer block, also referred to herein as a transformer layer. … Transformer blocks may be used in self-attention language models such as BERT (see Devlin et al., Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805, 2018, which is incorporated by reference in its entirety herein) and OpenAI's GPT, which uses a differing transformer block-based approach (see Radford, Alec et al., Language models are unsupervised multitask learners. (2019)), which is incorporated by reference in its entirety herein)….”]
receiving text data by said language model to determine if said text data is suspect for containing Unicode characters based on whether said text data conforms with human writing habits. [Lee, Figure 4, “[0069] FIG. 4 illustrates a model for classification of malicious messages. An implementation of a second model 400 (such as the second model 320 of FIG. 3) may use context information such as the context input 402 shown in FIG. 4. … Content features may be fed into the embedding layer and/or fed directly into the classification layer. …The classifier (Classifier2) head of the second model 400 may then return sigmoid outputs which indicate the maliciousness of input emails.”]
Brown and Lee pertain to natural language processing and detection of malicious content and it would have been obvious to combine the express language model training of Lee with the system of Brown which indirectly indicates model training by teaching that a trained model is used and by content implies that the machine learned model is a language model. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Claims 2-5 are rejected under 35 U.S.C. 103 as being unpatentable over Brown and Lee in view of Prasad (U.S. 20140297252) and Kang (U.S. 20220129644).
Regarding Claim 2, Brown teaches “avoiding training a machine learning model with bad data and having to retrain the model, and so on.” 2: 61-63. Lee has an express training of a language model. Neither discuss and the use of positive and negative samples/examples/data-points which is well-known in the training arts.
Prasad teaches and the teachings suggest:
2. The method as recited in claim 1 further comprising:
receiving text data conforming to human writing habits as negative samples; and [Prasad pertains to detecting errors which include homophones. See Abstract.]
randomly inserting Unicode characters into said negative samples to form positive samples. [Prasad: “[0076] … We trained a binary MaxEnt classifier that predicts whether any input n-gram is an idiom. We used 3.2 k gold standard canonical idioms as positive samples and all 15M non-idiom n-grams in our data as negative samples. …” Here the correct idiom n-grams are called positive whereas in the Claim the incorrect examples (the ones that have Unicode injected) are called positive sample. In Prasad the incorrect idioms (Positive samples of the Claim) far exceed the correct n-grams (Negative samples of the Claim). “[0077] … Training data for incomplete utterances were automatically generated using an error simulator that randomly removed words from the beginning and/or end of a clean, fully-formed sentence. A number of lexical and syntactic features were used to train and evaluate the incomplete utterance classifier.” The teachings in [0076] and [0077] suggest that random insertion of characters can be used to generate negative or positive examples. This is not expressly taught.]
Brown/Lee and Prasad pertain to natural language processing and training of models to detect anomalous, erroneous, or malicious content and it would have been obvious to use the positive/negative samples of Prasad that are routinely used in the training of models with the training of models of the combination as one known method of training. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Kang expressly teaches random insertion/deletion of characters to generate new training samples:
randomly inserting Unicode characters into said negative samples to form positive samples. [Kang: “An apparatus for augmenting textual data according to an embodiment includes a data augmenter configured to generate augmented data by augmenting input textual data according to a data augmentation scheme decided based on a type of natural language processing task of the input textual data and a data classifier configured to classify the augmented data into a positive sample or a negative sample by determining whether or not the augmented data maintains label information of the input textual data based on one or more data classification criteria.” Abstract. “[0028] The deciding may comprise deciding the type of the dominant language of the input textual data based on Unicode for each language.” “[0040] According to an example, the data augmentation scheme may be at least one of a paraphrasing scheme for augmenting data by performing machine translation twice or more, a sentence negation scheme for augmenting textual data by replacing textual data with a negative form of a verb obtained as a result of morpheme analysis using a POS (Part-Of-Speech) tagger, a pronoun swap scheme for replacing a pronoun obtained as a result of morphological analysis using the POS tagger with another pronoun, or performing the pronoun replacement by utilizing anaphora resolution, an entity swap scheme for replacing an entity obtained by performing entity recognition with another entity of the same type, a number swap scheme for converting an entity obtained by performing entity recognition is obtained by replacing it with another entity of the same type, or an element of a numeric type obtained as a result of morphological analysis into another value, a noise injection scheme for adding the same value for each element obtained as a result of morphological analysis n times or adding values of stop words, a synonym replacement scheme for performing replacement of a thesaurus dictionary for each element of the same tag obtained as a result of morpheme analysis, substitution of a synonym dictionary for each entity obtained by performing word embeddings based substitution, or entity recognition, word embeddings based substitution, a random insertion scheme for adding random values between elements and elements obtained as a result of morphological analysis, a random swap scheme for changing the order of elements and elements obtained as a result of morphological analysis, a random deletion scheme for removing some of the elements obtained as a result of morphological analysis, a spacing scheme for arbitrarily removing or adding spaces between elements of pre-trained model or language model output result or result of morphological analysis are, and a summarization scheme for performing summarization.”]
Brown/Lee/Prasad and Kang pertain to natural language processing and training of models to detect anomalous, erroneous, or malicious content and it would have been obvious to use the various methods listed by Kang for data augmentation that include random insertion of characters to generate error with the system of combination for better performance. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Regarding Claim 3, Brown teaches:
3. The method as recited in claim 2 further comprising:
training said language model to recognize normal text and text containing Unicode characters based on said negative and positive samples. [Brown, Figure 2, 221 and Figure 5, 515 mention the training of a machine learned model for recognizing homographs which are generated by Unicode injection. They do not discuss the particulars of training.]
Lee does not discuss positive and negative training data.
Prasad teaches:
training said language model to recognize normal text and text containing Unicode characters based on said negative and positive samples. [ Prasad: “[0076] … We trained a binary MaxEnt classifier that predicts whether any input n-gram is an idiom. We used 3.2 k gold standard canonical idioms as positive samples and all 15M non-idiom n-grams in our data as negative samples. …” Prasad as applied to claim 2 teaches the use of negative and positive examples for training of machine learned models. The erroneous non-idiom n-grams are equivalent to the “text containing Unicode characters” of the Claim.]
Rationale for combination as provided for Claim 2.
Regarding Claim 4, Brown did not address the particulars of training. Lee does not discuss positive and negative training data.
Prasad teaches:
4. The method as recited in claim 3,
wherein a number of said positive samples used to train said language model exceeds a number of said negative samples used to train said language model. [ Prasad: “[0076] … We trained a binary MaxEnt classifier that predicts whether any input n-gram is an idiom. We used 3.2 k gold standard canonical idioms as positive samples and all 15M non-idiom n-grams in our data as negative samples. …” In this example, the number of one type of data is in the thousands and the other in the millions. Thus, one number clearly exceeds the other. The choice of which you call positive and which you call negative is arbitrary. Here the correct ones are called positive whereas in the Claim the incorrect examples (the ones that have Unicode injected) are called positive sample. In Prasad the incorrect idioms (Positive samples of the Claim) far exceed the correct n-grams (Negative samples of the Claim).]
Rationale as provided for Claim 3. Positive and negative samples were introduced from Prasad.
Regarding Claim 5, Brown teaches and the teaching suggests:
5. The method as recited in claim 3 further comprising:
training said language model to recognize one or more regions of text with Unicode characters by an entity recognition method. [Brown in Figure 1 shows recognizing an address bar region 106a/106b of text where the URL is located and recognizing the “domain name” / “entity” “americabank.” See Figure 2, “Domain Name List 233” which includes a list of named entities. “12. … determining … a particular region, wherein the OCR process is based at least in part on the particular region.” “Many users rely upon the address shown in the address bar 106a of the browser in order to determine whether the network site is legitimate. In this case, the attackers have employed a homograph attack in which the “b” in the domain name “americabank.com” has been replaced with a look-alike Cyrillic character. According to the present disclosure, an image 109 of the address bar 106b may be captured, and an optical character recognition (OCR) process performed….” 3:15-25. “… Where the untrusted text corresponds to a domain name, the homograph recognition engine 218 may compare the recognized text with domain names in a domain name list 233 (FIG. 2), and indicate a possible homograph attack only if a match occurs. This is because attackers may be more likely to stage a homograph attack against a popular or frequently visited domain name than one that sees little network traffic….” 7:60 to 8:10. Figure 4, “submit domain name to homograph recognition engine for verification 409.” Brown does not expressly teach that its machine learned model is trained to recognized the regions of text with Unicode characters by an entity recognition method but Figure 1 shows that its model is doing exactly that and therefore suggests that it has been trained to do so.]
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Brown in view of Lee, Prasad, and Kang and further in view of Steedman Henderson (U.S. 11132988).
Regarding Claim 6, Brown does not teach the use of a BERT.
Lee teaches:
6. The method as recited in claim 5,
wherein said entity recognition method uses a bidirectional encoder representations from transformers model and a conditional random fields model. [Lee’s language model is BERT: “In a natural language processing model such as a Bidirectional Encoder Representations from Transformers (BERT) model, transformer layers can be replaced with simplified adapters without significant loss of predictive ability. This compressed model may in turn be trained to perform security classification tasks such as detection of new phishing attacks in electronic mail communications.” Abstract.]
Lee was combined for express teaching of a language model and BERT is a language model such that the rationale for combination remains as provided for Claim 1.
The use of a CRF model in addition to a BERT is not taught by Lee/Prasad/Kang.
Steedman Henderson teaches:
wherein said entity recognition method uses a bidirectional encoder representations from transformers model and a conditional random fields model. [Steedman Henderson uses CRF and BERT together for tokenization and labeling of text that may be presented in Unicode characters. “The tokenisation algorithm 501 shown in FIG. 5(b) is configured to perform the following steps: Initial word-level tokenization: divide the entire text into a set of “words”, where a “word” is a unicode string of either all alphanumeric or all non-alphanumeric characters….” 30: 1-5. “An example SLU process S202 will be described in more detail in relation to FIG. 3 below. In this process, SLU is treated as a sequence labelling problem. A labelling model based on Conditional Random Fields is used….” 10:50-53. “Step S501 comprises a subword tokenisation algorithm that splits arbitrary input into subword units. The subword units into which the arbitrary input is split into is what is learned in S501. A number of subword tokenization methods are available for learning a vocabulary of units including subwords, including: …and WordPiece … or a neural variant of this model such as is used in several natural language processing representation architectures such as BERT…” 33:55-34:26.]
Brown/Lee/Prasad/Kang and Steedman Henderson pertain to natural language processing and it would have been obvious to use the combination of CRF and BERT from Steedman Henderson instead of the method of the combination. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Claims 7-20 are rejected under 35 U.S.C. 103 as being unpatentable over Brown in view of Malkiel (U.S. 11836175).
Regarding Claim 7, Brown teaches:
7. A computer-implemented method for detecting Unicode injection in text, the method comprising: [Brown is directed to detecting homographs which are created by injecting Unicode into text. “Disclosed are various embodiments for detecting homograph attacks using text recognition. A first string of untrusted text is received. A second string is determined corresponding to what the first string of untrusted text appears to be in a particular language. The second string is determined to differ from the first string of untrusted text….” Abstract. “Although typical Latin character fonts may include homographs, the potential for homographs is greatly magnified in considering extended Unicode, with thousands of different characters for over a hundred different writing systems. For example, the lower case Greek letter omega may resemble the lower case Latin “w,” and the Latin character “a” may look the same as the Cyrillic character “a.”…” 2:5-11. “… Even without external dependencies, homographs can be used to hide malicious behavior in plain sight. For instance, in a random on-call selector, a Unicode space may be utilized to generate a parsing error on a particular name, ensuring that the name will never be randomly selected.” 10:1-12.]
recording image data from a copy of original data; [Brown in Figure 1 shows capture/recording of an image of the URL region 106a. “According to the present disclosure, an image 109 of the address bar 106b may be captured, and an optical character recognition (OCR) process performed.” 3:20-22. Figure 2, 218: “Although the homograph recognition engine 218 may include an image generator 224, it is understood that in some embodiments, the homograph recognition engine 218 may receive previously generated images, which may be screen captures or portions of screen captures.” 4:33-38.]
recording a first set of text data from said original data; [Brown, Figure 1, receiving the URL at 106a which is the “original data” of the Claim. Figure 3, “receive first string of untrusted text 303.” “Beginning with box 303, the homograph recognition engine 218 receives a first string of untrusted text. For example, the homograph recognition engine 218 may execute as a service, and the client computing device 206 (FIG. 2) may have submitted the untrusted text for homograph detection as a service. Alternatively, the homograph recognition engine 218 may execute on the client computing device 206 and intercept the untrusted text before or as it is being rendered for display by a client application 266 (FIG. 2).” 6:61-7:3.]
performing optical character recognition on said recorded image data to generate a second set of text data; [Brown, Figure 1, “OCR” being performed on the captured image of 109. “According to the present disclosure, an image 109 of the address bar 106b may be captured, and an optical character recognition (OCR) process performed.” 3:20-23. Figure 2, 227: “The text recognition application 227 is executed to perform an optical character recognition (OCR) process on an input image. Specifically, the text recognition application 227 examines the image and determines which characters or glyphs are present in the image….” 4:39-44.]
generating a first feature vector for said first set of text data;
generating a second feature vector for said second set of text data; and
comparing said first and second feature vectors to determine if said first set of text data is suspect for containing Unicode characters. [Brown, Figure 3, 315. “In box 315, the homograph recognition engine 218 compares the first string of untrusted text that was received with the second string determined from the text recognition application 227. In box 318, the homograph recognition engine 218 determines whether the strings differ. …” 7:48-59.]
Brown does not include the details of determining the similarity and thus does not teach the comparison of feature vectors which is a well-known method in the ar.
Malkiel teaches:
generating a first feature vector for said first set of text data; [Malkiel, Figure 2, “obtain a first feature vector representative of the search query 204.”]
generating a second feature vector for said second set of text data; and [Malkiel, Figure 2, “… each of a plurality of second feature vectors … representative of a … respective first text-based content item … 206.” Figure 1, “vector repository 112.” “Vector repository 112 is configured to store a plurality of feature vectors 120, each being representative of a summary of a respective content item stored in content item repository 120….” 5:42-45.]
comparing said first and second feature vectors to determine if said first set of text data is suspect for containing Unicode characters. [Malkiel, Figure 2, 206, “At step 206, a respective semantic similarity score between the first feature vector and each of a plurality of second feature vectors generated by a transformer-based machine learning model is determined….” 7: 11-15. Figure 1, “similarity determiner 114.” “Semantic search techniques via focused summarizations are described. For example, a search query is received for a text-based content item in a data set comprising a plurality of text-based content items. A first feature vector representative of the search query is obtained. A respective semantic similarity score is determined between the first feature vector and each of a plurality of second feature vectors. Each of the second feature vectors is representative of a machine-generated summarization of a respective text-based content item….” Abstract.]
Brown and Malkiel pertain processing text including Unicode characters and it would have been obvious to combine the feature vector comparison of Malkiel which is very commonly used method of comparison of pieces of text with the system of Brown which leaves out the details of implementation of the comparison process. This combination falls under simple substitution of one known element for another to obtain predictable results or use of known technique to improve similar devices (methods, or products) in the same way. See MPEP 2141, KSR, 550 U.S. at 418, 82 USPQ2d at 1396.
Regarding Claim 8, Brown teaches:
8. The method as recited in claim 7 further comprising:
copying said original data that is to be processed by a natural language processing task; and [Brown, Figure 1, the “capture” is a form of copying.]
recording said image data from said copy of original data that is to be processed by said natural language processing task. [Brown, Figure 1, the “capture” copies and records for a subsequent OCR.]
Regarding Claim 9, Brown does not teach generating feature vectors.
Malkiel teaches:
9. The method as recited in claim 7 further comprising:
generating said first and second feature vectors by a natural language processing task. [Malkiel: Figure 1, the “feature vectors 118” are the product of the “transformer based machine learning model 106 which extracts words and keywords as features and is thus performing an NLP: “To generate a feature vector 118, a representation of each of the plurality of multi-word fragments may be provided to transformer-based machine learning model 106. The representation may be a feature vector representative of the multi-word fragment. For instance, features (e.g., words) may be extracted from the multi-word fragment and such features may be utilized to generate the feature vector. The feature vectors may take any form, such as a numerical, visual and/or textual representation, or may comprise any other suitable form and may be generated using various techniques, such as, but not limited to, keyword featurization, semantic-based featurization, bag-of-words featurization, and/or n-gram-TF-IDF (term frequency-inverse document frequency) featurization.” 4:64 to 5:10.]
Rationale for combination as provided for Claim 7 considering that Malkiel was cited for teaching the generation of the feature vectors.
Regarding Claim 10, Brown teaches:
10. The method as recited in claim 9, wherein said natural language processing task comprises one of the following:
text classification, entity recognition, machine reading comprehension, semantic matching and machine translation. [Brown performs entity recognition by its “domain name list 233” as shown in Figure 4 and also teaches text classification as a homograph or not in Figure 3.]
Regarding Claim 11, Brown teaches:
11. The method as recited in claim 7 further comprising:
identifying said first set of text data as being suspect for containing Unicode characters in response to a difference between measurements of said first and second features exceeding a threshold value. [Brown in Figure 1 and Figure 3, 315 teaches that the difference between two strings indicates if there has been a homograph attack or the text is normal. “Strings differ? 318” to Yes. “However, if the strings differ, a homograph attack may be occurring, and the homograph recognition engine 218 implements one or more actions in box 321. For example, the homograph recognition engine 218 may cause an alert to be generated in a user interface 269 rendered by a client computing device 206. …” 7:60 to 8:10.]
Brown does not teach comparison of vector features and a threshold.
Malkiel teaches:
identifying said first set of text data as being suspect for containing Unicode characters in response to a difference between measurements of said first and second features exceeding a threshold value. [Malkiel, Figure 1, “similarity determiner 114”: “… For example, similarity determiner 114 may determine whether a respective semantic similarity score determined between feature vectors 122 and feature vector 124 has a predetermined relationship with a predetermined threshold value. For instance, similarity determiner 114 may determine whether a respective semantic similarity score reaches and/or exceeds a predetermined threshold value. Content item(s) corresponding to feature vector(s) 122 being associated with semantic similarity scores having the predetermined relationship with the predetermined threshold may be returned to application 128 via one or more search results 126….” 6:10-21.]
Rationale for combination as provided for Claim 7 considering that Malkiel was cited for teaching the generation of the feature vectors and comparison of the vectors to determine similarity. The use of thresholds is common in the art.
Regarding Claim 12, Brown teaches:
12. The method as recited in claim 7 further comprising:
identifying said first set of text data as being normal in response to a difference between measurements of said first and second features not exceeding a threshold value. [Brown in Figure 1 and Figure 3, 315 teaches that the difference between two strings indicates if there has been a homograph attack or the text is normal. 318: “Strings differ?” to NO. “In box 315, the homograph recognition engine 218 compares the first string of untrusted text that was received with the second string determined from the text recognition application 227. In box 318, the homograph recognition engine 218 determines whether the strings differ. … If the strings are the same, no homograph attack is detected, and the operation of the homograph recognition engine 218 ends.” 7:48-59.]
Brown does not teach comparison of vector features and a threshold.
Malkiel as applied to Claim 11 above, teaches the use of thresholds in the comparison and obviously if exceeding the threshold indicates similarity then not reaching the threshold would indicate absence of similarity. Rationale as provided for Claim 11.
Regarding Claim 13, Brown teaches:
13. The method as recited in claim 7, wherein said original data corresponds to data to be processed by a natural language processing task. [Brown teaches that is directed to “using text recognition to defeat homograph attacks.” Text recognition is a natural language processing task.]
Claim 14 is a computer program product system claim with limitations corresponding to the limitations of method Claim 7 and is rejected under similar rationale.
Claim 15 is a computer program product system claim with limitations corresponding to the limitations of method Claim 8 and is rejected under similar rationale.
Claim 16 is a computer program product system claim with limitations corresponding to the limitations of method Claim 9 and is rejected under similar rationale.
Claim 17 is a computer program product system claim with limitations corresponding to the limitations of method Claim 10 and is rejected under similar rationale.
Claim 18 is a computer program product system claim with limitations corresponding to the limitations of method Claim 11 and is rejected under similar rationale.
Claim 19 is a computer program product system claim with limitations corresponding to the limitations of method Claim 12 and is rejected under similar rationale.
Claim 20 is a computer program product system claim with limitations corresponding to the limitations of method Claim 13 and is rejected under similar rationale.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Starbuck (U.S. 20040260776) Figure 8. “Train filter using machine learning algorithm 850” which identifies spam from non-spam. Non-spam conforms with human writing habits.
Chhichhia (U.S. 20160137891): Figure 5, “feature extraction module 510.” “[0083] The feature extraction module 510 extracts features from documents in the content catalog database 402….” “[0092] After extracting the feature vectors from the content entity samples ….” “classification module 550.” The extracted text is in Unicode and the feature vectors will contain Unicode characters. “[0058] Text is extracted from the pages of the original document tagged as having text. The text extraction may be done at the individual character level, together with markers separating words, lines, and paragraphs. The extracted text characters and glyphs are represented by the Unicode character mapping determined for each….” “[0060] The output of text extraction 302, therefore, a dataset referenced by the page number, comprising the characters and glyphs in a Unicode character mapping with associated location information and embedded fonts used in the original document.” “[0084] … In one embodiment, the entity relationship analysis module 520 determines a similarity between the content entities based on the feature vectors received from the feature extraction module 510. The entity relationship analysis module uses the document features to determine a similarity between each content entity and each of the other content entities. For example, the entity relationship analysis module 520 builds a classifier to classify documents of one content entity into each of the other content entities based on the documents' features….”
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FARIBA SIRJANI whose telephone number is (571)270-1499. The examiner can normally be reached 9 to 5, M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Fariba Sirjani/
Primary Examiner, Art Unit 2659