DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment/Status of Claims
Claims 1, 3, 8, 10, 12, 17, 19, and 20 were amended.
Claims 4 and 13 were cancelled.
Claims 21 and 22 are new.
Claims 1-3, 5-12, and 14-22 are pending and examined herein.
Claims 2 and 11 are objected to.
Claims 1-3, 5-12, and 14-22 are rejected under 35 U.S.C. 103.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 06/02/2026 has been entered.
Claim Objections
Claim 2 is objected to because of the following informalities:
In claims 2 and 11, “comprises” should be “comprise.”
Appropriate correction is required.
Response to Arguments
Applicant’s arguments, see pages 8-19, filed 06/02/2026, with respect to the rejection(s) of claim(s) 1-20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Marco Lippi et al., “CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Service”, February 18, 2019, arXiv, James Mullenbach et al., "CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes," August 1, 2021, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 1365-1378, and US 2008/0059425 A1.
Applicant argues, see pages 17-18, that "Even accepting for purposes of argument that Brock teaches some form of document fingerprint generation, matching, and access deactivation, Brock is entirely unrelated to the NLP machine learning framework recited in the remaining limitations of claims 1 and 10."
Examiner respectfully disagrees. MPEP 2141.01(a) states "A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). Note that "same field of endeavor" and "reasonably pertinent" are two separate tests for establishing analogous art; it is not necessary for a reference to fulfill both tests in order to qualify as analogous art."
Both Brock and the current application are in the field of endeavor of analyzing non-compliant data. Therefore, the art is analogous.
Applicant further argues, "The Office Action's motivation for adding Brock-alleging that improved automated content monitoring methods are needed-is a generic statement of need that does not provide the specific, articulated motivation required under KSR to combine a DMCA copyright enforcement system with the NLP compliance prediction pipeline allegedly taught by the combination of Lippi, Xie, and Nguyen," and "The addition of Brock as a fourth reference lacks motivation because Brock is a DMCA copyright enforcement system with no NLP machine learning components and a fingerprinting workflow that operates in the opposite direction from the claims."
MPEP 2145 states "Applicants may argue that the examiner’s conclusion of obviousness is based on improper hindsight reasoning. However, "[a]ny judgment on obviousness is in a sense necessarily a reconstruction based on hindsight reasoning, but so long as it takes into account only knowledge which was within the level of ordinary skill in the art at the time the claimed invention was made and does not include knowledge gleaned only from applicant’s disclosure, such a reconstruction is proper." In re McLaughlin, 443 F.2d 1392, 1395, 170 USPQ 209, 212 (CCPA 1971)."
One of ordinary skill in the art would recognize that, that the fingerprinting system of Brock could be used to retrieve similar documents as used in the instant application. Although the system of Brock discovers the non-compliant documents using fingerprinting, which as applicant states works in an opposite direction, using the fingerprinting system to find copies of the non-compliant documents would be obvious to one of ordinary skill in the art. Examiner disagrees that Brock teaches the opposite direction of the instant application. [0089] states "In some embodiments, a common pool of objectionable content is maintained based on input from multiple content hosts. For example, the common pool may include content that has been identified by various content hosts as containing pornography, child pornography, profanity, or racial content. Depending on the compliance rules specified in their host policies, an OSP may have an interest in contributing to, sharing, and using the common pool to identify objectionable content and remove or reject it." Therefore, one of ordinary skill in the art would be motivated to perform this identification of objectionable content automatically, such as using the compliance prediction of Lippi, as automatic identification as taught by Lippi states on page 2, "Such a tool could be useful both for consumer protection organizations and agencies, GS: by making their work more effective and efficient, and for consumers, by increasing their understanding of what they agree upon." Therefore, the identified non-compliant documents in the objectionable content pool of Brock are checked against the incoming monitored content, which is in the monitored content store.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 5-12, and 14-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Marco Lippi et al., “CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Service”, February 18, 2019, arXiv, hereinafter “Lippi,” James Mullenbach et al., "CLIP: A Dataset for Extracting Action Items for Physicians from Hospital Discharge Notes," August 1, 2021, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 1365-1378, hereinafter “Mullenbach,” and US 2008/0059425 A1, hereinafter “Brock.”
Regarding claim 1, Lippi teaches
A computer system for predicting compliance of text documents with a ruleset using self-supervised machine learning, the system comprising a server computing device having a memory for storing computer-executable instructions and a processor that executes the computer-executable instructions to: (Page 2 states "To address this problem, we propose a machine learning-based method and tool for partially automating the detection of potentially unfair clauses." Page 2 states "According to art. 3 of the Directive 93/13 on Unfair Terms in Consumer Contracts, a contractual term is unfair if: 1) it has not been individually negotiated; and 2) contrary to the requirement of good faith, it causes a significant imbalance in the parties rights and obligations, to the detriment of the consumer. This general definition is GS: further specified in the Annex to the Directive, containing an indicative and non-exhaustive list of the terms which may be regarded as unfair, as well in a few dozen judgments of the Court of Justice of the EU (Micklitz and Reich 2014). Examples of unfair clauses encompass taking jurisdiction away from the consumer, limiting liability for damages on health and/or gross negligence, imposing obligatory arbitration in a country different from consumers residence etc." Therefore, detecting potentially unfair clauses is interpreted as predicting compliance of documents. Page 5 states "The corpus consists of 50 relevant on-line consumer contracts, i.e. the Terms of Service of on-line platforms." Therefore, the documents are text documents. Page 10 states "We address two different tasks: a detection task, aimed at predicting whether a given sentence contains a (potentially) unfair clause, and a classification task, aimed at predicting the category an unfair clause belongs to, which indeed could be a valuable piece of information to a potential user." One of ordinary skill in the art would realize that machine learning, as is present in Lippi, would be executed on a computer. Modern computers, as would be used for executing machine learning, are capable of running a server, and therefore, the computer used to execute the machine learning model is interpreted as the server computing device. A computer, as one of ordinary skill in the art would understand, in order to execute the method taught by Lippi, must have a memory storing computer-readable instructions and a processor to execute those instructions.)
train a NLP [model] using a corpus of text documents in which each sentence is associated with a compliance label indicating whether the sentence is in compliance with one or more rulesets; (Page 10 states "In sentence-wide classification the problem can be formalized as follows. Given a sentence, the goal is to classify it as positive if it contains a potentially unfair clause, or negative otherwise. Within this setting, a machine learning classifier is trained with a data set
D
=
x
i
,
,
y
i
i
=
1
N
, which consists of a collection of
N
pairs, where
x
i
encodes some representation of a sentence, and
y
i
is its corresponding (positive or negative) class. In collective classification, the data set consists of a collection of M documents, represented as sequences of sentences:
D
=
d
j
=
x
1
j
,
y
1
j
,
.
.
.
,
x
k
j
j
,
y
k
j
j
j
=
1
M
, where the
j
-th document contains
k
j
sentences.” The label for if the sentence is a potentially unfair clause is interpreted as the compliance label.)
[training] the NLP [model that is] using as input a first plurality of … sentences from each of a first plurality of text documents (Page 10 states “We address the problem of detecting potentially unfair contract clauses as a sentence classification task. Such a task could be tackled by treating sentences independently of one another (sentence-wide classification).” Page 10 further states "In sentence-wide classification the problem can be formalized as follows. Given a sentence, the goal is to classify it as positive if it contains a potentially unfair clause, or negative otherwise. Within this setting, a machine learning classifier is trained with a data set
D
=
x
i
,
y
i
i
=
1
N
, which consists of a collection of
N
pairs, where
x
i
encodes some representation of a sentence, and
y
i
is its corresponding (positive or negative) class." The machine learning classifier is a natural language processing model as page 5 states ""The corpus consists of 50 relevant on-line consumer contracts, i.e. the Terms of Service of on-line platforms." The contracts are natural language, and as the classifier processes the contracts, the model is a natural language processing model. Page 12 states "We obtained a total of 9,414 sentences, 1,032 of which (11.0%) were labeled as positive, thus containing a potentially unfair clause. We run experiments following the leave-one-document-out (LOO) procedure, in which each document in the corpus, in turn, is used as test set, leaving the remaining documents for training set (4/5) and validation set (1/5) for model selection." The training set is interpreted as the first plurality of text documents, and the sentences of the training set of documents is interpreted as the first plurality of sentences. One of ordinary skill in the art would realize that the training set is used to train the NLP model.)
execute the trained NLP … model, using as input a second plurality of … sentences from each of a second plurality of text documents, to generate a second compliance [label] for each unlabeled sentence in the second plurality of unlabeled sentences; and (Page 12 states "We obtained a total of 9,414 sentences, 1,032 of which (11.0%) were labeled as positive, thus containing a potentially unfair clause. We run experiments following the leave-one-document-out (LOO) procedure, in which each document in the corpus, in turn, is used as test set, leaving the remaining documents for training set (4/5) and validation set (1/5) for model selection." The test set is interpreted as the second plurality of text documents and the sentences in the second plurality of text documents is interpreted as the second plurality of unlabeled sentences. One of ordinary skill in the art would realize that a test set is entered into the model without a label in order to compare the output label to the ground-truth label, and thus, is unlabeled. Page 12 states "For the first task (potentially unfair clause detection) we compared several systems. The problem is formulated as a binary classification task, where the positive class is either the union of all potentially unfair sentences, or the set of potentially unfair clauses of a single category, as described below." The output of classification, as one of ordinary skill in the art would understand, is a label, which are interpreted as pseudo-labels as they are not the ground-truth label. As the output is all of the unfair sentences, each sentence is essentially labeled as compliant or non-compliant (not included in the unfair sentences). This is supported by page 10, which states "We address two different tasks: a detection task, aimed at predicting whether a given sentence contains a (potentially) unfair clause, and a classification task, aimed at predicting the category an unfair clause belongs to, which indeed could be a valuable piece of information to a potential user.")
determine whether each text document in the second plurality of text documents is in compliance with one or more rulesets using the second compliance pseudo-labels generated for the text document; and (Page 12 states "We run experiments following the leave-one-document-out (LOO) procedure, in which each document in the corpus, in turn, is used as test set, leaving the remaining documents for training set (4/5) and validation set (1/5) for model selection." Page 12 further states "For the first task (potentially unfair clause detection) we compared several systems. The problem is formulated as a binary classification task, where the positive class is either the union of all potentially unfair sentences, or the set of potentially unfair clauses of a single category, as described below." Therefore, as each test document is input into the model, the binary classification of the negative class (no sentences labeled as unfair) is determining that the text document is in compliance with one or more rulesets. The categories of clause unfairness in Table 1, described in pages 5-9 are interpreted as the rulesets.)
Lippi does not appear to explicitly teach
[train a] teacher model
execute the NLP teacher model . . . to generate a first . . . pseudo-label for each unlabeled [input data];
[train a NLP] student [model] using the first plurality of unlabeled [data] and associated first … pseudo-labels, including injecting input noise during the training process by aggregating each unlabeled sentence with one or more sentences adjacent to each unlabeled sentence within the same text document as the unlabeled sentence into a sentence block and providing the aggregated sentence blocks as input to train the NLP student model;
[executing the trained] student model
for each text document in the second plurality of text documents that is determined to be non-compliant:
generate a document fingerprint based upon digital attributes of the text document, wherein the digital attributes comprise metadata of the text document,
match the document fingerprint to one or more text documents in document storage to identify copies of the non-compliant text document, and
deactivate access to the one or more text documents in document storage.
However, Mullenbach—directed to analogous art—teaches
[train a] teacher model (Page 1370 states "To create a task-targeted dataset, we first fine-tune a vanilla BERT model on our task, and then we use the learned model to classify all unlabeled sentences.” The vanilla BERT model is interpreted as the teacher model.
execute the NLP teacher model . . . to generate a first . . . pseudo-label for each unlabeled sentence in the first plurality of unlabeled sentences; (Page 1370 states "To create a task-targeted dataset, we first fine-tune a vanilla BERT model on our task, and then we use the learned model to classify all unlabeled sentences. We select all sentences that the model predicts as having action items, using a fixed threshold." The classifications of the unlabeled sentences are interpreted as pseudo-labels.
train a NLP student model using the first plurality of unlabeled [data] and associated first … pseudo-labels, including injecting input noise during the training process by aggregating each unlabeled sentence with one or more sentences adjacent to each unlabeled sentence within the same text document as the unlabeled sentence into a sentence block and providing the aggregated sentence blocks as input to train the NLP student model; (Page 1370 states "After selecting data, we pre-train a BERT Context model on the targeted dataset, pulling in neighboring sentences of the targeted sentences." Page 1370 states "Surrounding contexts are critical for the task, for two reasons: 1) an individual sentence may not have the full picture on the type of the action; 2) neighboring sentences tend to share the same label (occurs for 27% of sentences). So, we incorporate context beyond an individual sentence into our BERT-based sentence representations, by concatenating the two sentences each that immediately precede and follow the sentence to the input." The neighboring sentences are interpreted as the input noise. Page 1367 states "We annotated 718 randomly sampled discharge notes from the set of patients that were discharged from the ICU (i.e., survived) and thus brought back to the care of their primary care physician or relevant specialists." Therefore, the training set includes documents (notes). One of ordinary skill in the art would reason that the neighboring sentences would be pulled from the same text document, as they are neighboring sentences.)
[executing the trained] student model . . . to generate a second . . . pseudo-label (Page 1370 states "This process can be repeated, by using the final resulting model to then select a new set of sentences for pre-training." Therefore, the labels for the new sentences will be pseudo-labels.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Lippi with the teachings of Mullenbach because, as Mullenbach states on page 1370, Page 1370 states "Surrounding contexts are critical for the task, for two reasons: 1) an individual sentence may not have the full picture on the type of the action; 2) neighboring sentences tend to share the same label (occurs for 27% of sentences). So, we incorporate context beyond an individual sentence into our BERT-based sentence representations, by concatenating the two sentences each that immediately precede and follow the sentence to the input."
The combination of Lippi and Mullenbach does not appear to explicitly teach
for each text document in the second plurality of text documents that is determined to be non-compliant:
generate a document fingerprint based upon digital attributes of the text document, wherein the digital attributes comprise metadata of the text document
match the document fingerprint to one or more text documents in document storage to identify copies of the non-compliant text document, and
deactivate access to the one or more text documents in document storage.
However, Brock—directed to analogous art—teaches
for each text document in the second plurality of text documents that is determined to be non-compliant: ([0104] states "At 264, the fingerprint (or another analysis result) is checked against a database of known non-compliant (or known compliant) content objects. In some embodiments, the database includes a common pool of content that has previously been identified either manually or automatically as non-compliant or compliant. The content can be looked up by fingerprint or any other appropriate index." The database of known non-compliant documents are interpreted as the second plurality of text documents determined to be non-compliant. [0025] states "As used herein, a unit of content may be referred to as a content object. Content objects can include any object type. Examples of content objects include a text document, an image, Video, audio, flash, animation, game, lyrics, code, or portions thereof (e.g., a phrase/sentence/paragraph, a subimage, or a video clip)." Therefore, content objects are interpreted as text documents.)
generate a document fingerprint based upon digital attributes of the text document, wherein the digital attributes comprise metadata of the text document ([0104] states "At 264, the fingerprint (or another analysis result) is checked against a database of known non-compliant (or known compliant) content objects. In some embodiments, the database includes a common pool of content that has previously been identified either manually or automatically as non-compliant or compliant. The content can be looked up by fingerprint or any other appropriate index." Therefore, as the content can be looked up by fingerprint, the non-compliant text documents must have had a document fingerprint generated. [0051] states "For example, in the case of text content, various techniques may be used, such as unique phrase extraction, word histograms, text fingerprinting, etc. An example is described in T. Hoad and J. Zobel, “Methods for identifying versioned and plagiarized documents, in Journal of the American Society for Information Science and Technology, Volume 54, Issue 3, 2003." As the fingerprint is based on the text of the text document, the fingerprint is based upon digital attributes including the metadata of the document (the actual text).)
match the document fingerprint to one or more text documents in document storage to identify copies of the non-compliant text document, and ([0104] states "At 264, the fingerprint (or another analysis result) is checked against a database of known non-compliant (or known compliant) content objects. In some embodiments, the database includes a common pool of content that has previously been identified either manually or automatically as non-compliant or compliant. The content can be looked up by fingerprint or any other appropriate index." Therefore, the new text documents are matched to the new documents by fingerprint. [0034] "A fingerprint includes a signature of an object that can be used to detect a copy of an object as a whole or in part." Therefore, when checking the fingerprint against the database, copies of the non-compliant text document will be identified. [0052] states “The entire web may be searched, or a subset of the web (e.g., websites that have been identified as sites that copy the most often or data in a content store Such as monitored content store 118). A database of sites that have been crawled and resulting data may be maintained that is updated at various times. Rather than searching the entire web, the database may be used instead. Searching may comprise a combination of searching the web and consulting a database of previously crawled websites. In some embodiments, monitored content store 118 in system 100 stores previously crawled websites. In some embodiments, 208 is performed by crawler 112 in system 100." Therefore, the monitored content is stored in document storage.)
deactivate access to the one or more text documents in document storage. ([0104] states "At 266, it is determined whether the content object is non-compliant according to the database. If it is non-compliant according to the database, the content object is removed at 272." Therefore, as the content object is removed, the access to the text document is deactivated.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi, Xie, and Nguyen with the teachings of Brock because, as Brock states in [0004] "From an OSPs perspective, monitoring for content that does not comply with the OSPs host policy is also typically a manual process. When OSPs monitor content as it is uploaded, typically a human views and approves content before (or after) it is displayed and non-compliant content is rejected (or removed). OSPs also must manually review and compare content when they receive DMCA notices, and often have little information to determine if content is out of compliance and no automated way to determine the identity or reputation of the complaining party. As the amount of content on the Internet grows, manual content monitoring and enforcement processes are becoming increasingly impractical. Therefore, improved methods for monitoring content and managing enforcement of non-compliant content are needed." Additonally, as Brock states in [0089] "In some embodiments, a common pool of objectionable content is maintained based on input from multiple content hosts. For example, the common pool may include content that has been identified by various content hosts as containing pornography, child pornography, profanity, or racial content. Depending on the compliance rules specified in their host policies, an OSP may have an interest in contributing to, sharing, and using the common pool to identify objectionable content and remove or reject it." Therefore, one of ordinary skill in the art would be motivated to perform this identification of objectionable content automatically, such as using the compliance prediction of Lippi, as automatic identification as taught by Lippi states on page 2, "Such a tool could be useful both for consumer protection organizations and agencies, GS: by making their work more effective and efficient, and for consumers, by increasing their understanding of what they agree upon." Therefore, the identified non-compliant documents in the objectionable content pool of Brock are checked against the incoming monitored content, which is in the monitored content store.
Additionally, [0089] states "In some embodiments, a common pool of objectionable content is maintained based on input from multiple content hosts. For example, the common pool may include content that has been identified by various content hosts as containing pornography, child pornography, profanity, or racial content. Depending on the compliance rules specified in their host policies, an OSP may have an interest in contributing to, sharing, and using the common pool to identify objectionable content and remove or reject it." Therefore, one of ordinary skill in the art would be motivated to perform this identification of objectionable content automatically, such as using the compliance prediction of Lippi, as automatic identification as taught by Lippi states on page 2, "Such a tool could be useful both for consumer protection organizations and agencies, GS: by making their work more effective and efficient, and for consumers, by increasing their understanding of what they agree upon."
Regarding claim 2, the rejection of claim 1 is incorporated herein. Lippi does not appear to explicitly teach
wherein the NLP teacher model and the NLP student model each comprises a deep learning NLP model architecture.
However, Mullenbach—directed to analogous art—teaches
wherein the NLP teacher model and the NLP student model each comprises a deep learning NLP model architecture. (Page 1370 states "To create a task-targeted dataset, we first fine-tune a vanilla BERT model on our task, and then we use the learned model to classify all unlabeled sentences. Therefore, the teacher model is an deep learning NLP model architecture (BERT). Page 1370 states "After selecting data, we pre-train a BERT Context model on the targeted dataset, pulling in neighboring sentences of the targeted sentences." Therefore, the student model is an deep learning NLP model architecture (BERT).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi and Xie for the reasons given above in regards to claim 1.
Regarding claim 3, the rejection of claim 1 is incorporated herein. Lippi teaches
wherein the corpus of text documents used to train the NLP . . . model comprises domain-specific documents from a regulated industry. ("In analyzing the Terms of Service of the selected on-line platforms, we identified eight different categories of unfair clauses. For each type of clause we defined a corresponding XML tag, as shown in Table 1. Notice that not necessarily all the documents contain all clause categories. For example, Twitter provides two different ToS, the first one for US and non-US residents and the second one for EU residents. The tagged version is the version applicable in the EU and it does not contain any choice of law, arbitration or jurisdiction clauses. We assumed that each type of clause could be classified as clearly fair, potentially unfair, or clearly unfair. In order to mark the different degrees of (un)fairness we appended a numeric value to each XML tag, with 1 meaning clearly fair, 2 potentially unfair, and 3 clearly unfair. Nested tags were used to annotate text segments relevant to more than one type of clause. If one clause covers more then one paragraphs. we chose to tag each paragraph separately, possibly with different degrees of (un)fairness." Therefore, the terms of service, which are a domain-specific document from a regulated industry, are used to train the NLP model.)
Lippi does not appear to explicitly teach
[training the] teacher [model with labeled data]
However, Mullenbach—directed to analogous art—teaches
[training the] teacher [model with labeled data] (Page 1370 states "To create a task-targeted dataset, we first fine-tune a vanilla BERT model on our task, and then we use the learned model to classify all unlabeled sentences. Fine-tuning is training, and training on the task means training with the labeled data.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi and Mullenbach for the reasons given above in regards to claim 1.
Regarding claim 5, the rejection of claim 1 is incorporated herein. Lippi teaches
wherein the ... compliance [output label] is a prediction of whether the corresponding sentence is in compliance with one or more rulesets. (Page 12 states "For the first task (potentially unfair clause detection) we compared several systems. The problem is formulated as a binary classification task, where the positive class is either the union of all potentially unfair sentences, or the set of potentially unfair clauses of a single category, as described below." As explained in regards to claim 1, the clauses are interpreted as the ruleset. As the output is the unfair sentences, the output label is a prediction of whether the corresponding sentences are in compliance with one or more rulesets.)
Lippi does not appear to explicitly teach
first … pseudo-label
However, Mullenbach—directed to analogous art—teaches
first … pseudo-label (Page 1370 states "To create a task-targeted dataset, we first fine-tune a vanilla BERT model on our task, and then we use the learned model to classify all unlabeled sentences. The classification of the unlabeled sentences results in pseudo-labels.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi and Mullenbach for the reasons given above in regards to claim 1.
Regarding claim 6, the rejection of claim 1 is incorporated herein. Lippi teaches
wherein the second compliance pseudo-label is a prediction of whether the corresponding sentence is in compliance with one or more rulesets. (Page 12 states "For the first task (potentially unfair clause detection) we compared several systems. The problem is formulated as a binary classification task, where the positive class is either the union of all potentially unfair sentences, or the set of potentially unfair clauses of a single category, as described below." As explained in regards to claim 1, the clauses are interpreted as the ruleset. As the output is the unfair sentences, the output label is a prediction of whether the corresponding sentences are in compliance with one or more rulesets. As explained in regards to claim 1, the output label is interpreted as the second compliance pseudo-label.)
Regarding claim 7, the rejection of claim 1 is incorporated herein. Lippi teaches
determining that the text document in the second plurality of text documents is not in compliance with the one or more rulesets when at least one sentence in the text document is labeled as being non-compliant. (Page 12 states "We run experiments following the leave-one-document-out (LOO) procedure, in which each document in the corpus, in turn, is used as test set, leaving the remaining documents for training set (4/5) and validation set (1/5) for model selection." Page 12 further states "For the first task (potentially unfair clause detection) we compared several systems. The problem is formulated as a binary classification task, where the positive class is either the union of all potentially unfair sentences, or the set of potentially unfair clauses of a single category, as described below." Therefore, as each test document is input into the model, the binary classification of the negative class (no sentences labeled as unfair) is determining that the text document is in compliance with one or more rulesets. The categories of clause unfairness in Table 1, described in pages 5-9 are interpreted as the rulesets. When the binary class is negative, the text document is found compliant. If one sentence is labeled non-compliant, it will be included in the union of all potentially unfair sentences, making the binary classification positive, meaning that the document is non-compliant.)
Regarding claim 8, the rejection of claim 1 is incorporated herein. Lippi teaches
sentences and associated second compliance pseudo-labels (As explained in regards to claim 1, the output of the model is interpreted as the second compliance pseudo-labels. The corpus is composed of sentences, as explained in regards to claim 1.)
[a] NLP [model] (As the model processes text documents that are written in natural language, the model is a NLP model.
compliance [pseudo-labels] (As explained in regards to claim 1, the output of the model is interpreted as the compliance pseudo-labels)
and determines whether each text document in the … plurality of text documents is in compliance with one or more rulesets using the … compliance pseudo-labels generated for the text document. (Page 12 states "We run experiments following the leave-one-document-out (LOO) procedure, in which each document in the corpus, in turn, is used as test set, leaving the remaining documents for training set (4/5) and validation set (1/5) for model selection." Page 12 further states "For the first task (potentially unfair clause detection) we compared several systems. The problem is formulated as a binary classification task, where the positive class is either the union of all potentially unfair sentences, or the set of potentially unfair clauses of a single category, as described below." Therefore, as each test document is input into the model, the binary classification of the negative class (no sentences labeled as unfair) is determining that the text document is in compliance with one or more rulesets. The categories of clause unfairness in Table 1, described in pages 5-9 are interpreted as the rulesets.)
Lippi does not appear to explicitly teach
trains a second … student model using the second plurality of unlabeled [data], including injecting input noise during the training process by aggregating each unlabeled sentence with one or more sentences adjacent to each unlabeled sentence into a sentence block within the same text document as the unlabeled sentence and providing the aggregated sentence blocks as input to train the second … student model;
executes the trained second … student model, using as input a third plurality of [data], to generate a third … pseudo-label for each unlabeled sentence in the third plurality of [data];
[use the] third [data using the] third pseudo-labels
However, Mullenbach—directed to analogous art—teaches
trains a second … student model using the second plurality of unlabeled [data], including injecting input noise during the training process by aggregating each unlabeled sentence with one or more sentences adjacent to each unlabeled sentence into a sentence block within the same text document as the unlabeled sentence and providing the aggregated sentence blocks as input to train the second … student model; (Page 1370 states "This process can be repeated, by using the final resulting model to then select a new set of sentences for pre-training." Therefore, the following results in training a second student model using a second plurality of data. Page 1370 states "After selecting data, we pre-train a BERT Context model on the targeted dataset, pulling in neighboring sentences of the targeted sentences." Page 1370 states "Surrounding contexts are critical for the task, for two reasons: 1) an individual sentence may not have the full picture on the type of the action; 2) neighboring sentences tend to share the same label (occurs for 27% of sentences). So, we incorporate context beyond an individual sentence into our BERT-based sentence representations, by concatenating the two sentences each that immediately precede and follow the sentence to the input." The neighboring sentences are interpreted as the input noise. Page 1367 states "We annotated 718 randomly sampled discharge notes from the set of patients that were discharged from the ICU (i.e., survived) and thus brought back to the care of their primary care physician or relevant specialists." Therefore, the training set includes documents (notes). One of ordinary skill in the art would reason that the neighboring sentences would be pulled from the same text document, as they are neighboring sentences.)
[use the] third [data using the] third pseudo-labels (Page 1370 states "This process can be repeated, by using the final resulting model to then select a new set of sentences for pre-training." Therefore, the next iteration would use the third-pseudo labels.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi and Mullenbach for the reasons given above in regards to claim 1.
Regarding claim 9, the rejection of claim 1 is incorporated herein. Lippi does not appear to explicitly teach
wherein the second plurality of text documents comprises a larger number of sentences than the first plurality of documents.
However, Mullenbach—directed to analogous art—teaches
wherein the second plurality of text documents comprises a larger number of sentences than the first plurality of sentences. (Page 1369, caption of Fig. 1 states "(b) Our pre-training method. First, we train a supervised model with the labeled data (blue)." Therefore, the blue data represent the sentences/documents used to train the teacher. Page 1369, caption of Fig. 1 states "Then, we apply it to unlabeled data (gray) to surface a fraction of the data to pre-train the model (a) with (red). After pre-training, we fine-tune on the labeled data, which leads to similar results as pre-training with all unlabeled data." Therefore, the student model is trained with both the data used to train the teacher model (blue data) and the red data. Therefore, the second plurality of text documents will comprise a larger number of sentences than the first plurality of sentences.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi and Mullenbach for the reasons given above in regards to claim 1.
Regarding claim 10, Lippi teaches
A computerized method of predicting compliance of text documents with a ruleset using self-supervised machine learning, the method comprising: (Page 2 states "To address this problem, we propose a machine learning-based method and tool for partially automating the detection of potentially unfair clauses." Page 2 states "According to art. 3 of the Directive 93/13 on Unfair Terms in Consumer Contracts, a contractual term is unfair if: 1) it has not been individually negotiated; and 2) contrary to the requirement of good faith, it causes a significant imbalance in the parties rights and obligations, to the detriment of the consumer. This general definition is GS: further specified in the Annex to the Directive, containing an indicative and non-exhaustive list of the terms which may be regarded as unfair, as well in a few dozen judgments of the Court of Justice of the EU (Micklitz and Reich 2014). Examples of unfair clauses encompass taking jurisdiction away from the consumer, limiting liability for damages on health and/or gross negligence, imposing obligatory arbitration in a country different from consumers residence etc." Therefore, detecting potentially unfair clauses is interpreted as predicting compliance of documents. Page 5 states "The corpus consists of 50 relevant on-line consumer contracts, i.e. the Terms of Service of on-line platforms." Therefore, the documents are text documents. Page 10 states "We address two different tasks: a detection task, aimed at predicting whether a given sentence contains a (potentially) unfair clause, and a classification task, aimed at predicting the category an unfair clause belongs to, which indeed could be a valuable piece of information to a potential user." One of ordinary skill in the art would realize that machine learning, as is present in Lippi, would be executed on a computer.)
The remainder of claim 10 recites substantially similar subject matter to claim 1 and is rejected with the same rationale, mutatis mutandis.
Claims 11-12 and 14-18 recite substantially similar subject matter to claims 2-3 and 5-9 respectively and are rejected with the same rationale, mutatis mutandis.
Regarding claim 19, the rejection of claim 8 is incorporated herein. The combination of Lippi and Mullenbach does not appear to explicitly teach
for each text document in the third plurality of text documents that is determined to be non-compliant:
generate a document fingerprint based upon digital attributes of the text document, wherein the digital attributes comprise metadata of the text document
match the document fingerprint to one or more text documents in document storage to identify copies of the non-compliant text document, and
deactivate access to the one or more text documents in document storage.
However, Brock—directed to analogous art—teaches
for each text document in the third plurality of text documents that is determined to be non-compliant: ([0104] states "At 264, the fingerprint (or another analysis result) is checked against a database of known non-compliant (or known compliant) content objects. In some embodiments, the database includes a common pool of content that has previously been identified either manually or automatically as non-compliant or compliant. The content can be looked up by fingerprint or any other appropriate index." The database of known non-compliant documents are interpreted as the third plurality of text documents determined to be non-compliant. [0025] states "As used herein, a unit of content may be referred to as a content object. Content objects can include any object type. Examples of content objects include a text document, an image, Video, audio, flash, animation, game, lyrics, code, or portions thereof (e.g., a phrase/sentence/paragraph, a subimage, or a video clip)." Therefore, content objects are interpreted as text documents.)
generate a document fingerprint based upon digital attributes of the text document, wherein the digital attributes comprise metadata of the text document ([0104] states "At 264, the fingerprint (or another analysis result) is checked against a database of known non-compliant (or known compliant) content objects. In some embodiments, the database includes a common pool of content that has previously been identified either manually or automatically as non-compliant or compliant. The content can be looked up by fingerprint or any other appropriate index." Therefore, as the content can be looked up by fingerprint, the non-compliant text documents must have had a document fingerprint generated. [0051] states "For example, in the case of text content, various techniques may be used, such as unique phrase extraction, word histograms, text fingerprinting, etc. An example is described in T. Hoad and J. Zobel, “Methods for identifying versioned and plagiarized documents, in Journal of the American Society for Information Science and Technology, Volume 54, Issue 3, 2003." As the fingerprint is based on the text of the text document, the fingerprint is based upon digital attributes including the metadata of the document (the actual text).)
match the document fingerprint to one or more text documents in document storage to identify copies of the non-compliant text document, and ([0104] states "At 264, the fingerprint (or another analysis result) is checked against a database of known non-compliant (or known compliant) content objects. In some embodiments, the database includes a common pool of content that has previously been identified either manually or automatically as non-compliant or compliant. The content can be looked up by fingerprint or any other appropriate index." Therefore, the new text documents are matched to the new documents by fingerprint. [0034] "A fingerprint includes a signature of an object that can be used to detect a copy of an object as a whole or in part." Therefore, when checking the fingerprint against the database, copies of the non-compliant text document will be identified. [0034] states "Controlled content store 116 includes controlled content. In some embodiments, controlled content store 116 includes the following information: a copy of the content, an index of fingerprints associated with the content, and meta data about the content (e.g., filename, URL, fetch date, etc.). In some embodiments, the copy of the content is stored in a separate cache." Therefore, the document is stored in a document storage.)
deactivate access to the one or more text documents in document storage. ([0104] states "At 266, it is determined whether the content object is non-compliant according to the database. If it is non-compliant according to the database, the content object is removed at 272." Therefore, as the content object is removed, the access to the text document is deactivated.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine the teachings of Lippi, Xie, and Nguyen with the teachings of Brock for the reasons given above in regards to claim 1.
Claim 20 recites substantially similar subject matter to claim 19 and is rejected with the same rationale, mutatis mutandis.
Regarding claim 21, the rejection of claim 1 is incorporated herein. Lippi does not appear to explicitly teach
wherein the one or more sentences adjacent to each unlabeled sentence are selected from a predefined contextual window surrounding the unlabeled sentence within the same text document.
However, Mullenbach—directed to analogous art—teaches
wherein the one or more sentences adjacent to each unlabeled sentence are selected from a predefined contextual window surrounding the unlabeled sentence within the same text document. (Page 1370 states "After selecting data, we pre-train a BERT Context model on the targeted dataset, pulling in neighboring sentences of the targeted sentences." Page 1370 states "Surrounding contexts are critical for the task, for two reasons: 1) an individual sentence may not have the full picture on the type of the action; 2) neighboring sentences tend to share the same label (occurs for 27% of sentences). So, we incorporate context beyond an individual sentence into our BERT-based sentence representations, by concatenating the two sentences each that immediately precede and follow the sentence to the input." Therefore, the contextual window is the immediately preceding sentence and the immediately following sentence. As they immediately precede and follow the sentence, they are within the same text document.)
It would have been obvious to a person having ordinary skill in the art before the effective filing date of this application to combine the teachings of Lippi with the teachings of Mullenbach for the reasons given above in regards to claim 1.
Claim 22 recites substantially similar subject matter to claim 21 and is rejected with the same rationales, mutatis mutandis.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSICA THUY PHAM whose telephone number is (571)272-2605. The examiner can normally be reached Monday - Friday, 9 A.M. - 5:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.P./ Examiner, Art Unit 2121
/Li B. Zhen/ Supervisory Patent Examiner, Art Unit 2121
