DETAILED ACTION
This action is in response to the amendments and remarks filed 05/04/2026. Claims 1-20 are pending and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/04/2026 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Claims 15-20 refer to a “computer program product comprising a computer readable storage medium”. Paragraph [0066] of the instant Specification states, “A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media”. Accordingly, the computer readable storage media is not interpreted to include transitory signals per se.
The following is a quotation of MPEP 2111.04 II:
The broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met. For example, assume a method claim requires step A if a first condition happens and step B if a second condition happens. If the claimed invention may be practiced without either the first or second condition happening, then neither step A or B is required by the broadest reasonable interpretation of the claim. If the claimed invention requires the first condition to occur, then the broadest reasonable interpretation of the claim requires step A. If the claimed invention requires both the first and second conditions to occur, then the broadest reasonable interpretation of the claim requires both steps A and B.
The broadest reasonable interpretation of a system (or apparatus or product) claim having structure that performs a function, which only needs to occur if a condition precedent is met, requires structure for performing the function should the condition occur. The system claim interpretation differs from a method claim interpretation because the claimed structure must be present in the system regardless of whether the condition is met and the function is actually performed.
Limitation 6 of Claim 8 recites a step of “identifying the document as abnormal”, only performed if “the anomaly score exceeds a threshold”. Thus, this limitation is found to be contingent, and is consequently interpreted as not being a required component of the claimed method under broadest reasonable interpretation. To become a required limitation of the method, it must be rewritten as a positively recited element.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 5 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 5 recites “the similarity model compares the representative vectors using a learned similarity function”. While the instant specification discloses using a similarity judgment model to compare vectors ([0029-0031]), it fails to disclose a similarity function that’s specifically learned. Thus, claim 5 contains new matter not described in the instant specification, and fails to comply with the written description requirement.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites “wherein the neural network comprises a graph neural network configured to perform node representation updates based on neighboring nodes”. It’s unclear whether “the neural network” refers to the neural network of ancestral claim 2 (used for object detection) or the network of ancestral claim 1 (used for re-encoding a knowledge graph), or both. Thus, the scope of the claim is rendered indefinite. This deficiency is inherited by dependent claim 7. “the neural network” is interpreted as referring to either of the ancestral networks.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed inventions are directed to non-statutory subject matter without significantly more.
Claim 1
Step 1: The claim recites “A system”, and is therefore directed to the statutory category of machine
Step 2A Prong 1: The claim recites the following judicial exception(s)
(a) perform layout analysis on a document using an object detection model to segment a page of the document into a plurality of page elements: This can be performed as a mental process. One can merely identify a set of page elements in a document.
(b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information: This can be performed as a mental process. One can merely imagine a vector for each page element, containing type information, semantic information, visual information, and position information about it.
(c) construct a knowledge graph comprising: a plurality of nodes corresponding to the page elements, and edges selectively established between pairs of nodes: This can be performed as a mental process. One can merely imagine a graph representing the document, with nodes corresponding to page elements and edges to relationships.
(d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold: This can be performed as a mental process. One can merely imagine edges between similar nodes.
(e) re-encode the knowledge graph using a neural network configured to propagate information between connected nodes to generate updated node representations: This can be performed as a mental process. One can merely imagine information about connected nodes within the graph.
(f) compare the re-encoded knowledge graph with one or more additional knowledge graphs corresponding to other documents to determine an anomaly score based on differences in nodes and edges of the knowledge graphs, and determine that the document is abnormal when the anomaly score satisfies the threshold: This can be performed as a mental process. One can mentally compare knowledge graphs and determine an anomaly score based on the similarities of their nodes and edges.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the following additional element(s)
a memory storing instructions; and a processor operably coupled to the memory and configured to execute the instructions: This is mere instruction to execute the recited judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
(a) perform layout analysis on a document using an object detection model to segment a page of the document into a plurality of page elements: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(e) re-encode the knowledge graph using a neural network configured to propagate information between connected nodes to generate updated node representations: This is mere instruction to execute a judicial exception with a neural network in a generic manner (MPEP 2106.05(f)).
Step 2B: The following additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
a memory storing instructions; and a processor operably coupled to the memory and configured to execute the instructions: This is mere instruction to execute the recited judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
(a) perform layout analysis on a document using an object detection model to segment a page of the document into a plurality of page elements: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(e) re-encode the knowledge graph using a neural network configured to propagate information between connected nodes to generate updated node representations: This is mere instruction to execute a judicial exception with a neural network in a generic manner (MPEP 2106.05(f)).
Claim 2
Step 1: The claim recites a machine, as in claim 1
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the object detection model comprises a trained object detection neural network configured to identify page elements based on layout structure: This can be performed as a mental process. One can merely identify page elements and their layout(s).
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the object detection model comprises a trained object detection neural network configured to identify page elements based on layout structure: This is mere instruction to train a neural network to execute a judicial exception in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the object detection model comprises a trained object detection neural network configured to identify page elements based on layout structure: This is mere instruction to train a neural network to execute a judicial exception in a generic manner (MPEP 2106.05(f)).
Claim 3
Step 1: The claim recites a machine, as in claim 2
Step 2A Prong 1: The claim recites no further judicial exception(s)
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the representative vector is generated by concatenating outputs of: a semantic embedding model, a visual feature extraction model, and a position encoding model: This is mere instruction to execute a judicial exception by combining the outputs of generic models (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the representative vector is generated by concatenating outputs of: a semantic embedding model, a visual feature extraction model, and a position encoding model: This is mere instruction to execute a judicial exception by combining the outputs of generic models (MPEP 2106.05(f)).
Claim 4
Step 1: The claim recites a machine, as in claim 3
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the similarity model determines the similarity score based on the representative vectors: This can be performed as a mental process. One can mentally decide a similarity score for pairs of representative vectors.
the threshold comprises a predefined similarity threshold: Determining that the document is abnormal can still be performed as a mental process.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the similarity model determines the similarity score based on the representative vectors: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the similarity model determines the similarity score based on the representative vectors: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
Claim 5
Step 1: The claim recites a machine, as in claim 4
Step 2A Prong 1: The claim recites the following further judicial exception(s)
similarity model compares the representative vectors using a learned similarity function: This can be performed as a mental process. One can mentally compare the vectors using some learned similarity criteria.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
similarity model compares the representative vectors using a learned similarity function: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
similarity model compares the representative vectors using a learned similarity function: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
Claim 6
Step 1: The claim recites a machine, as in claim 4
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the neural network comprises a graph neural network configured to perform node representation updates based on neighboring nodes: This can be performed as a mental process. One can merely incorporate neighboring node information into the imagined node representations.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
wherein the neural network comprises a graph neural network configured to perform node representation updates based on neighboring nodes: This is mere instruction to execute a judicial exception with a graph neural network in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
wherein the neural network comprises a graph neural network configured to perform node representation updates based on neighboring nodes: This is mere instruction to execute a judicial exception with a graph neural network in a generic manner (MPEP 2106.05(f)).
Claim 7
Step 1: The claim recites a machine, as in claim 6
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein the anomaly score is based on a difference between the re-encoded knowledge graph and the one or more additional knowledge graphs: Determining an anomaly score can still be performed as a mental process.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 8
Step 1: The claim recites “A computer implemented method”, and is therefore directed to the statutory category of process
Step 2A Prong 1: The claim recites the following judicial exception(s)
(a) performing layout analysis on a document to segment a page into a plurality of page elements: This can be performed as a mental process. One can merely identify a set of page elements in a document.
(b) generating representative vectors for the page elements using one or more embedding models, each representative vector comprising a concatenation of outputs of: type, semantic, visual, and position information: This can be performed as a mental process. One can merely imagine a vector for each page element, containing type information, semantic information, visual information, and position information about it.
(c) constructing a knowledge graph comprising nodes corresponding to the page elements: This can be performed as a mental process. One can merely imagine a graph representing the document, with nodes corresponding to page elements.
(d) determining edges between nodes by: applying a similarity model to representative vectors, and establishing edges based on a similarity threshold: This can be performed as a mental process. One can merely imagine edges between similar nodes.
(e) re-encoding the knowledge graph using a neural network to update node representations: This can be performed as a mental process. One can merely imagine information about connected nodes within the graph.
(f) comparing the re-encoded knowledge graph with one or more additional knowledge graphs to determine an anomaly score and identifying the document as abnormal when the anomaly score exceeds a threshold: This can be performed as a mental process. One can mentally compare knowledge graphs and determine an anomaly score based on the similarities of their nodes and edges.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the following additional element(s)
(d) determining edges between nodes by: applying a similarity model to representative vectors, and establishing edges based on a similarity threshold: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(e) re-encoding the knowledge graph using a neural network to update node representations: This is mere instruction to execute a judicial exception with a neural network in a generic manner (MPEP 2106.05(f)).
Step 2B: The following additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
(d) determining edges between nodes by: applying a similarity model to representative vectors, and establishing edges based on a similarity threshold: This is mere instruction to execute a judicial exception with a generic computing component (MPEP 2106.05(f)).
(e) re-encoding the knowledge graph using a neural network to update node representations: This is mere instruction to execute a judicial exception with a neural network in a generic manner (MPEP 2106.05(f)).
Claim 9
Step 1: The claim recites a process, as in claim 8
Step 2A Prong 1: The claim recites the following further judicial exception(s)
employing layout analysis based on object detection to segment one or more pages of the document into page elements: Identifying page elements can still be performed as a mental process.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 10
Step 1: The claim recites a process, as in claim 9
Step 2A Prong 1: The claim recites the following further judicial exception(s)
employing multimodal embedding such that the page elements of the document are represented by the nodes in the knowledge graph: Generating a knowledge graph can still be performed as a mental process. One merely has to assign nodes to a document containing multiple forms of media (images, text).
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 11
Step 1: The claim recites a process, as in claim 10
Step 2A Prong 1: The claim recites the following further judicial exception(s)
determining whether the edge is present between the vectors by comparing a similarity between the vectors to a predefined threshold: Identifying present edges can still be performed as a mental process. One can merely imagine a threshold value, estimate the difference between one dimension of every pair of vectors, and assign edges for all pair differences over the threshold value.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 12
Step 1: The claim recites a machine, as in claim 10
Step 2A Prong 1: The claim recites the following further judicial exception(s)
determining whether two of the vectors include the edge by comparing the multimodal embedding manner of the page elements: Identifying present edges can still be performed as a mental process. If the vectors correspond to document elements of different mediums, one merely needs to mentally compare the differences as per the mental process described for claim 11.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 13
Step 1: The claim recites a process, as in claim 11
Step 2A Prong 1: The claim recites the following further judicial exception(s)
employing pairwise comparisons based on a graph attention algorithm for the re-encoded knowledge graph and the one or more other knowledge graphs: This can be performed as a mental process. One can merely compare the knowledge graphs.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the further additional element(s)
employing pairwise comparisons based on a graph attention algorithm for the re-encoded knowledge graph and the one or more other knowledge graphs: This is mere instruction to execute a judicial exception with a graph attention algorithm in a generic manner (MPEP 2106.05(f)).
Step 2B: The further additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
employing pairwise comparisons based on a graph attention algorithm for the re-encoded knowledge graph and the one or more other knowledge graphs: This is mere instruction to execute a judicial exception with a graph attention algorithm in a generic manner (MPEP 2106.05(f)).
Claim 14
Step 1: The claim recites a process, as in claim 12
Step 2A Prong 1: The claim recites the following further judicial exception(s)
wherein if an abnormal score of a node of the document is significantly higher than one or more other nodes of the one or more other documents, the document is abnormal: This can be performed as a mental process. One need only mentally assign documents with nodes that have abnormal scores higher than other documents as “abnormal”.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the additional element(s)
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
Claim 15
Step 1: The claim recites “A computer program product”, and is therefore directed to the statutory category of article of manufacture
Step 2A Prong 1: The claim recites the following judicial exception(s)
(a) segment a document into page elements using layout analysis: This can be performed as a mental process. One can merely identify a set of page elements in a document.
(b) generate representative vectors for the page elements using one or more embedding models, each representative vector comprising type, semantic, visual, and position information: This can be performed as a mental process. One can merely imagine a vector for each page element, containing type information, semantic information, visual information, and position information about it.
(c) construct a knowledge graph comprising nodes corresponding to the page elements: This can be performed as a mental process. One can merely imagine a graph representing the document, with nodes corresponding to page elements.
(d) determining edges between nodes based on similarity between representative vectors: This can be performed as a mental process. One can merely imagine edges between similar nodes.
(e) re-encode the knowledge graph using a neural network: This can be performed as a mental process. One can merely imagine information about connected nodes within the graph.
(f) compare the knowledge graph with one or more additional knowledge graphs to determine an anomaly score and identify abnormal documents based on the anomaly score: This can be performed as a mental process. One can mentally compare knowledge graphs and determine an anomaly score based on the similarities of their nodes and edges.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through the following additional element(s)
A computer program product comprising a computer-readable storage medium storing instructions that, when executed, cause a processor to: This is mere instruction to execute the recited judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
(e) re-encode the knowledge graph using a neural network: This is mere instruction to execute a judicial exception with a neural network in a generic manner (MPEP 2106.05(f)).
Step 2B: The following additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s)
A computer program product comprising a computer-readable storage medium storing instructions that, when executed, cause a processor to: This is mere instruction to execute the recited judicial exceptions with generic computer hardware (MPEP 2106.05(f)).
(e) re-encode the knowledge graph using a neural network: This is mere instruction to execute a judicial exception with a neural network in a generic manner (MPEP 2106.05(f)).
Claims 16-20
Step 1: Claims 16-20 recite an article of manufacture, as in claim 15.
Step 2A Prong 1: Claims 16-20 recite the same judicial exception(s) as claims 9-13, respectively.
Step 2A Prong 2: The judicial exception(s) are not integrated into a practical application through any additional elements. The analysis of claims 16-20 at this step mirrors that of claims 9-13, respectively, with the exception that claims 16-20 are directed to “A computer program product abnormal document self-discovery, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to”, performing operations mirroring those of claims 9-13. This is a mere instruction to apply the exceptions using generic computer equipment (MPEP 2106.05(f)).
Step 2B: The additional element(s) of the claim, taken alone or in combination, do not amount to significantly more than the recited judicial exception(s). The analysis of claims 16-20 at this step mirrors that of claims 9-13, with the exception that claims 16-20 are directed to “A computer program product abnormal document self-discovery, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to”, performing operations mirroring those of claims 9-13. This is a mere instruction to apply the exceptions using generic computer equipment (MPEP 2106.05(f)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Biswas (AUTONOMOUS MONITORING OF APPLICATIONS IN A CLOUD ENVIRONMENT, published 4/23/2020, US 2020/0128047 A1), in view of Fan (ANOMALYDAE: DUAL AUTOENCODER FOR ANOMALY DETECTION ON ATTRIBUTED NETWORKS, published 2020, arXiv:2002.03665v2), and further in view of Liang (Logical Labeling of Document Images Using Layout Graph Matching with Adaptive Learning, published 2002, DAS 2002, LNCS 2423, pp. 224–235), and Luo (IDENTIFYING DUPLICATE USER ACCOUNTS IN AN IDENTIFICATION DOCUMENT PROCESSING SYSTEM, published 10/1/2020, US 2020/0311844 A1).
Regarding claim 1, Biswas discloses [a] system comprising: a memory storing instructions; and a processor operably coupled to the memory and configured to execute the instructions: “The processes depicted herein, such as those described with reference to the figures in this disclosure, may be implemented in software (e.g., code, instructions, program) executed by one or more processing units (e.g., processors cores), hardware, or combinations thereof. The software may be stored in a memory (e.g., on a memory device, on a non-transitory computer-readable storage medium)” (Biswas, [0022])
Said instructions to:
(b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information: “Each node in the directed graph may represent an action performed by the particular user” (Biswas, [0006]); “Weights assigned to each node may indicate a number of times the corresponding actions represented by the nodes were performed” (Biswas, [0009])
(c) construct a knowledge graph comprising: a plurality of nodes corresponding to the page elements, and edges selectively established between pairs of nodes:
“The method may additionally include generating, using the actions, a directed graph. Each node in the directed graph may represent an action performed by the particular user. Each connection (edge) between two nodes may represent a sequence in performance of actions represented by the two nodes” (Biswas, [0006])
“the number of times one action is followed by another action can be represented as an edge weight
w
e
” (Biswas, [0187])
PNG
media_image1.png
520
447
media_image1.png
Greyscale
(Biswas, Figure 4)
“FIG. 4 illustrates a weighted directed graph representing the series of actions with assigned weights, according to some embodiments.” (Biswas, [0188])
(f) compare the re-encoded knowledge graph with one or more additional knowledge graphs corresponding to other documents to determine an anomaly score based on differences in nodes and edges of the knowledge graphs, and determine that the document is abnormal when the anomaly score satisfies the threshold:
“The system can, for example, generate a graph profile (knowledge graph) for each user for a time window (e.g., one day, one week, etc.) for all the actions performed by the user, as well as associated parameters such as time stamps, resources affected, IP addresses used, geolocations, etc … The system can then superimpose a graph of more recent data over a historical graph profile of the same user. Using the cumulative superimposed graph, the system can detect anomalies if the current single graph profile of the user deviates from the historical cumulative graph (additional knowledge graph). Any such deviation can be considered as anomalous” (Biswas, [0204])
“The method may further include detecting anomalies in the user actions by comparing the single graph profile to the cumulative graph profile (709)” (Biswas, [0212])
“After identifying all or a portion of the cumulative graph profile (additional knowledge graph) that matches the single graph profile (knowledge graph), a comparison may be made between the individual vertices and/or edges to detect substantial differences. In some embodiments, the system may look for individual differences in vertices and/or edge weights to detect anomalies. For example, if a single vertex in the single graph profile has a weight that is different from a corresponding weight of the vertex in the cumulative graph profile by more than a threshold amount, an anomaly may be detected” (Biswas, [0213])
Biswas relates to constructing and comparing knowledge graphs for a graph anomaly detection and is analogous to the claimed invention.
While Biswas fails to disclose the further limitations of the claim, Fan discloses instructions to: (e) re-encode the knowledge graph using a neural network configured to propagate information between connected nodes to generate updated node representations:
“we propose a deep joint representation learning framework for anomaly detection through a dual autoencoder (AnomalyDAE) (neural network), which captures the complex interactions between the network structure and node attribute for highquality embeddings” (Fan, page 1, right column, paragraph 2)
“In order to obtain sufficient representative high-level node features, structure encoder firstly transforms the original observed node attribute X into the low-dimentional [sic] latent representation
Z
~
V
” (Fan, page 2, right column, paragraph 2)
“Given the transformed node embedding
Z
~
V
, a graph attention layer [16] is then employed to aggregate the representation from neighbor (connected) nodes, by performing a shared attentional mechanism on the nodes:
PNG
media_image2.png
40
532
media_image2.png
Greyscale
… its final embedding
Z
i
V
can be obtained by weighted sum based on the learned importance weights” (Fan, page 2, right column, paragraph 3).
“Finally, structure decoder takes the final node embeddings
Z
V
as inputs to decode them for reconstruction of the original network structure (re-encoded knowledge graph)” (Fan, page 2, right column, paragraph 3)
Fan relates to graph anomaly detection on attention-encoded graphs and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the primary reference to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; & page 4, right column, paragraph 3.
While Fan fails to disclose the further limitations of the claim, Liang discloses instructions to:
(a) perform layout analysis on a document using an object detection model to segment a page of the document into a plurality of page elements; (b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information; (c) construct a knowledge graph comprising: a plurality of nodes corresponding to the page elements, and edges selectively established between pairs of nodes:
PNG
media_image3.png
400
772
media_image3.png
Greyscale
”System overview” (Liang, page 225, Fig. 1)
“Fig. 1 shows an overview of our document analysis system. First, document images are processed by a segmentation-and-OCR engine (object detection model)” (Liang, page 225, paragraph 1).
PNG
media_image4.png
200
400
media_image4.png
Greyscale
PNG
media_image5.png
200
400
media_image5.png
Greyscale
“Original document page and converted HTML result” (Liang, page 227, Fig. 3)
PNG
media_image6.png
198
708
media_image6.png
Greyscale
”Example layout and layout graphs” (Liang, page 227, Fig. 4)
“A layout graph (knowledge graph) is a fully connected attributed relational graph. Each node corresponds to a segmented block on a page. The attributes (representative vector[s]) of a node are the position (position information) and size of the bounding box (visual information), and the normalized font size (type information) (small, middle, or large as compared to the average font size over the whole page). An edge between a pair of nodes reflects the spatial relationship between two corresponding blocks in the image.” (Liang, page 227, paragraph 1)
(d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold:
“An edge between a pair of nodes reflects the spatial relationship between two corresponding blocks in the image” (Liang, page 227, paragraph 1)
“The attributes of edge AB in the left graph are shown in Fig. 5” (Liang, page 228, paragraph 1)
PNG
media_image7.png
373
767
media_image7.png
Greyscale
”Example edge attributes” (Liang, page 228, Fig. 5). To store these attributes, the block corresponding to node A inherently must be compared to the block corresponding to node B.
(e) re-encode the knowledge graph using a neural network configured to propagate information between connected nodes to generate updated node representations:
“Our approach is a two-step approximate solution that aims at sub-optimal N-1 match. First, we search for the best 1-1 match from U to M, where U is the candidate graph, and M is the model graph” (Liang, page 229, paragraph 7). The candidate knowledge graph is re-encoded into the model knowledge graph.
“We need a metric to measure which mapping is the best. For a given mapping, an intermediate layout graph, T, is first constructed based on U such that the mapping between T and M is 1-1. Then a cost is computed for the 1-1 mapping and defined as the quality measurement of the mapping between U and M. The best match is the one with minimal cost.” (Liang, page 228, paragraph 4)
“For a pair of mapped nodes, the cost is defined as the sum of differences between corresponding attributes, weighted by the weight factors in model node.” (Liang, page 228, paragraph 5); “A cost is similarly defined for a pair of edges” (Liang, page 229, paragraph 2); “The graph match cost is the sum of all node pair costs and edge pair costs” (Liang, page 229, paragraph 3).
Liang relates to generating relational knowledge graphs based on document layout analysis and is analogous to the claimed invention. The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Liang fails to disclose the further limitations of the claim, Luo discloses instructions to: (d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold:
“The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
“The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (representative vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Claims 2-7 are rejected under 35 U.S.C. 103 as being unpatentable over Biswas (AUTONOMOUS MONITORING OF APPLICATIONS IN A CLOUD ENVIRONMENT, published 4/23/2020, US 2020/0128047 A1), in view of Fan (ANOMALYDAE: DUAL AUTOENCODER FOR ANOMALY DETECTION ON ATTRIBUTED NETWORKS, published 2020, arXiv:2002.03665v2), and further in view of Liang (Logical Labeling of Document Images Using Layout Graph Matching with Adaptive Learning, published 2002, DAS 2002, LNCS 2423, pp. 224–235), Luo (IDENTIFYING DUPLICATE USER ACCOUNTS IN AN IDENTIFICATION DOCUMENT PROCESSING SYSTEM, published 10/1/2020, US 2020/0311844 A1), and Nguyen (2D DOCUMENT EXTRACTOR, published 4/29/2021, US 2021/0125034 A1).
Regarding claim 2, the rejection of claim 1 is incorporated. While the aforementioned references fail to disclose the further limitations of the claim, Nguyen discloses a system, wherein the object detection model comprises a trained object detection neural network configured to identify page elements based on layout structure:
“Generally speaking, the 2D document extractor 300 is trained as a standard neural network. In one embodiment, the 2D document extractor 300 is trained using a variant of stochastic gradient descent and backpropagation” (Nguyen, [0146])
“In accordance with a broad aspect of the present technology, there is provided a server for extracting textual entities from a structured document, the server having access to a plurality of machine learning algorithms (MLAs) comprising a first MLA, a second MLA, and a third MLA, the server comprising: a processor, a computer-readable storage medium connected to the processor, the computer-readable storage medium comprising instructions, the processor, upon executing the instructions, being configured for: receiving a plurality of text sequences having been extracted from the structured document by an optical character recognition (OCR) model having processed the image to generate the structured document, receiving a plurality of structural elements, each structural element being indicative of a location of a respective text sequence in the document, encoding, by the first MLA, the plurality of text sequences and the plurality of structural elements to obtain a 3D encoded image, the 3D encoded image being indicative of semantic characteristics of the plurality of text sequences, the 3D encoded image having a spatial structure of the structured document, compressing, by the second MLA, the 3D encoded image to obtain an aggregated feature vector, the aggregated feature vector being indicative of a combination of spatial characteristics and semantic characteristics of the 3D encoded image, and decoding, by the third MLA, the aggregated feature vector to extract an associated set of textual entities, a given textual entity being associated with at least one text sequence in the plurality of text sequences” (Nguyen, [0031])
Nguyen relates to using neural networks to extract document elements and their attributes and is analogous to the claimed invention. Liang teaches an OCR engine that segments documents. The claimed invention differs from this method by using a neural network for segmentation and layout analysis. Nguyen teaches a method of using a neural network for document segmentation via layout analysis. Because both Liang and Nguyen teach the use of document analysis and segmentation techniques, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute Liang’s OCR engine for Nguyen’s neural network to achieve the predictable result of segmenting documents using a supervised machine learning method (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding claim 3, the rejection of claim 2 is incorporated. Liang further discloses a system, wherein the representative vector is generated by concatenating outputs of: a semantic embedding model, a visual feature extraction model, and a position encoding model: “The attributes (representative vector[s]) of a node are the position (position encoding) and size of the bounding box (visual feature), and the normalized font size (small, middle, or large as compared to the average font size over the whole page)” (Liang, page 227, paragraph 1)
Liang relates to generating relational knowledge graphs based on document layout analysis and is analogous to the claimed invention. The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Liang fails to disclose the further limitations of the claim, Nguyen discloses a system, wherein the representative vector is generated by concatenating outputs of: a semantic embedding model, a visual feature extraction model, and a position encoding model: “In accordance with a broad aspect of the present technology, there is provided a server for extracting textual entities from a structured document, the server having access to a plurality of machine learning algorithms (MLAs) comprising a first MLA, a second MLA, and a third MLA, the server comprising: a processor, a computer-readable storage medium connected to the processor, the computer-readable storage medium comprising instructions, the processor, upon executing the instructions, being configured for: receiving a plurality of text sequences having been extracted from the structured document by an optical character recognition (OCR) model having processed the image to generate the structured document, receiving a plurality of structural elements, each structural element being indicative of a location of a respective text sequence in the document, encoding, by the first MLA, the plurality of text sequences and the plurality of structural elements to obtain a 3D encoded image, the 3D encoded image being indicative of semantic characteristics of the plurality of text sequences, the 3D encoded image having a spatial structure (visual feature / position encoding) of the structured document, compressing, by the second MLA, the 3D encoded image to obtain an aggregated feature vector, the aggregated feature vector being indicative of a combination of spatial characteristics (visual feature / position encoding) and semantic characteristics of the 3D encoded image, and decoding, by the third MLA, the aggregated feature vector to extract an associated set of textual entities, a given textual entity being associated with at least one text sequence in the plurality of text sequences” (Nguyen, [0031])
Liang teaches a method of extracting type, spatial, and position information from elements of a document. The claimed invention improves upon this method by additionally extracting semantic element information and collating it with the other information. Nguyen teaches a method of extracting semantic information from document elements and combining it with other forms of page element information, applicable to Liang. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that including semantic information in the element extraction process would lead to the predictable result of including semantic information as a node similarity criteria, and would improve the known device by enabling it to detect anomalies indicated by divergence of semantic relationships in compared graphs (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 4, the rejection of claim 3 is incorporated. Luo further discloses a system, wherein the similarity model determines the similarity score based on the representative vectors and the threshold comprises a predefined similarity threshold
“The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
“The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (representative vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 5, the rejection of claim 4 is incorporated. Luo further discloses a system, wherein the similarity model compares the representative vectors using a learned similarity function: “a neural network may be used to determine similarity between user accounts. For example, the neural network may be trained on labelled sets of known duplicate user accounts to determine a degree (or percentage) of similarity, represented as a similarity score, between user accounts based on embeddings describing the users' faces” (Luo, [0039]).
The existing combination teaches a method of calculating similarities between representative vectors. Luo teaches a method of calculating such similarity metrics using trained neural networks. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the existing combination and Luo by using a neural network to calculate similarity metrics. This would achieve the predictable result of developing similarity criteria based on patterns observed in a training dataset, with the existing combination’s similarity metrics and Luo’s similarity metric calculation method performing the same together as they did separately. (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Regarding claim 6, the rejection of claim 4 is incorporated. Fan further discloses a system, wherein the neural network comprises a graph neural network configured to perform node representation updates based on neighboring nodes:
“we propose a deep joint representation learning framework for anomaly detection through a dual autoencoder (AnomalyDAE) (neural network), which captures the complex interactions between the network structure and node attribute for highquality embeddings” (Fan, page 1, right column, paragraph 2)
“Given the transformed node embedding
Z
~
V
, a graph attention layer [16] is then employed to aggregate the representation from neighbor nodes, by performing a shared attentional mechanism on the nodes:
PNG
media_image2.png
40
532
media_image2.png
Greyscale
… its final embedding
Z
i
V
can be obtained by weighted sum based on the learned importance weights” (Fan, page 2, right column, paragraph 3).
PNG
media_image8.png
318
843
media_image8.png
Greyscale
”The framework of the proposed AnomalyDAE” (Fan, page 3, Fig. 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the primary reference to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; & page 4, right column, paragraph 3.
Claims 8-14 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Biswas (AUTONOMOUS MONITORING OF APPLICATIONS IN A CLOUD ENVIRONMENT, published 4/23/2020, US 2020/0128047 A1), in view of Fan (ANOMALYDAE: DUAL AUTOENCODER FOR ANOMALY DETECTION ON ATTRIBUTED NETWORKS, published 2020, arXiv:2002.03665v2), and further in view of Liang (Logical Labeling of Document Images Using Layout Graph Matching with Adaptive Learning, published 2002, DAS 2002, LNCS 2423, pp. 224–235), Nguyen (2D DOCUMENT EXTRACTOR, published 4/29/2021, US 2021/0125034 A1), and Luo (IDENTIFYING DUPLICATE USER ACCOUNTS IN AN IDENTIFICATION DOCUMENT PROCESSING SYSTEM, published 10/1/2020, US 2020/0311844 A1).
Regarding claim 7, the rejection of claim 6 is incorporated. Biswas, in combination with Fan, further discloses a system, wherein the anomaly score is based on a difference between the re-encoded knowledge graph and the one or more additional knowledge graphs: (Biswas) “After identifying all or a portion of the cumulative graph profile (additional knowledge graph) that matches the single graph profile (knowledge graph), a comparison may be made between the individual vertices and/or edges to detect substantial differences. In some embodiments, the system may look for individual differences in vertices and/or edge weights to detect anomalies. For example, if a single vertex in the single graph profile has a weight that is different from a corresponding weight of the vertex in the cumulative graph profile by more than a threshold amount, an anomaly may be detected” (Biswas, [0213]). As noted in the rejection for claim 1, Fan discloses a method of re-encoding knowledge graphs.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the primary reference to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; & page 4, right column, paragraph 3.
While Liang fails to disclose the further limitations of the claim, Luo discloses instructions to: (d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold:
“The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
“The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (representative vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 8, Biswas discloses [a] computer implemented method comprising:
(b) generating representative vectors for the page elements using one or more embedding models, each representative vector comprising a concatenation of outputs of: type, semantic, visual, and position information: “Each node in the directed graph may represent an action performed by the particular user” (Biswas, [0006]); “Weights assigned to each node may indicate a number of times the corresponding actions represented by the nodes were performed” (Biswas, [0009])
(c) constructing a knowledge graph comprising nodes corresponding to the page elements:
“The method may additionally include generating, using the actions, a directed graph. Each node in the directed graph may represent an action performed by the particular user. Each connection (edge) between two nodes may represent a sequence in performance of actions represented by the two nodes” (Biswas, [0006])
PNG
media_image1.png
520
447
media_image1.png
Greyscale
(Biswas, Figure 4)
“FIG. 4 illustrates a weighted directed graph representing the series of actions with assigned weights, according to some embodiments.” (Biswas, [0188])
(f) comparing the re-encoded knowledge graph with one or more additional knowledge graphs to determine an anomaly score and identifying the document as abnormal when the anomaly score exceeds a threshold:
“The system can, for example, generate a graph profile (knowledge graph) for each user for a time window (e.g., one day, one week, etc.) for all the actions performed by the user, as well as associated parameters such as time stamps, resources affected, IP addresses used, geolocations, etc … The system can then superimpose a graph of more recent data over a historical graph profile of the same user. Using the cumulative superimposed graph, the system can detect anomalies if the current single graph profile of the user deviates from the historical cumulative graph (additional knowledge graph). Any such deviation can be considered as anomalous” (Biswas, [0204])
“The method may further include detecting anomalies in the user actions by comparing the single graph profile to the cumulative graph profile (709)” (Biswas, [0212])
“After identifying all or a portion of the cumulative graph profile (additional knowledge graph) that matches the single graph profile (knowledge graph), a comparison may be made between the individual vertices and/or edges to detect substantial differences. In some embodiments, the system may look for individual differences in vertices and/or edge weights to detect anomalies. For example, if a single vertex in the single graph profile has a weight that is different from a corresponding weight of the vertex in the cumulative graph profile by more than a threshold amount, an anomaly may be detected” (Biswas, [0213])
Biswas relates to constructing and comparing knowledge graphs for a graph anomaly detection and is analogous to the claimed invention.
While Biswas fails to disclose the further limitations of the claim, Fan discloses a method comprising: (e) re-encoding the knowledge graph using a neural network to update node representations:
“we propose a deep joint representation learning framework for anomaly detection through a dual autoencoder (AnomalyDAE) (neural network), which captures the complex interactions between the network structure and node attribute for highquality embeddings” (Fan, page 1, right column, paragraph 2)
“In order to obtain sufficient representative high-level node features, structure encoder firstly transforms the original observed node attribute X into the low-dimentional [sic] latent representation
Z
~
V
” (Fan, page 2, right column, paragraph 2)
“Given the transformed node embedding
Z
~
V
, a graph attention layer [16] is then employed to aggregate the representation from neighbor nodes, by performing a shared attentional mechanism on the nodes:
PNG
media_image2.png
40
532
media_image2.png
Greyscale
… its final embedding
Z
i
V
can be obtained by weighted sum based on the learned importance weights” (Fan, page 2, right column, paragraph 3).
“Finally, structure decoder takes the final node embeddings
Z
V
as inputs to decode them for reconstruction of the original network structure (re-encoded knowledge graph)” (Fan, page 2, right column, paragraph 3)
Fan relates to graph anomaly detection on attention-encoded graphs and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the primary reference to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; & page 4, right column, paragraph 3.
While Fan fails to disclose the further limitations of the claim, Liang discloses instructions to:
(a) performing layout analysis on a document to segment a page into a plurality of page elements; (b) generating representative vectors for the page elements using one or more embedding models, each representative vector comprising a concatenation of outputs of: type, semantic, visual, and position information; (c) constructing a knowledge graph comprising nodes corresponding to the page elements:
PNG
media_image3.png
400
772
media_image3.png
Greyscale
”System overview” (Liang, page 225, Fig. 1)
“Fig. 1 shows an overview of our document analysis system. First, document images are processed by a segmentation-and-OCR engine (object detection model)” (Liang, page 225, paragraph 1).
PNG
media_image4.png
200
400
media_image4.png
Greyscale
PNG
media_image5.png
200
400
media_image5.png
Greyscale
“Original document page and converted HTML result” (Liang, page 227, Fig. 3)
PNG
media_image6.png
198
708
media_image6.png
Greyscale
”Example layout and layout graphs” (Liang, page 227, Fig. 4)
“A layout graph (knowledge graph) is a fully connected attributed relational graph. Each node corresponds to a segmented block on a page. The attributes (vector) of a node are the position (position information) and size of the bounding box (visual information), and the normalized font size (type information) (small, middle, or large as compared to the average font size over the whole page). An edge between a pair of nodes reflects the spatial relationship between two corresponding blocks in the image.” (Liang, page 227, paragraph 1)
(d) determining edges between nodes by: applying a similarity model to representative vectors, and establishing edges based on a similarity threshold:
“An edge between a pair of nodes reflects the spatial relationship between two corresponding blocks in the image” (Liang, page 227, paragraph 1)
“The attributes of edge AB in the left graph are shown in Fig. 5” (Liang, page 228, paragraph 1)
PNG
media_image7.png
373
767
media_image7.png
Greyscale
”Example edge attributes” (Liang, page 228, Fig. 5). To store these attributes, the block corresponding to node A inherently must be compared to the block corresponding to node B.
(e) re-encoding the knowledge graph using a neural network to update node representations:
“Our approach is a two-step approximate solution that aims at sub-optimal N-1 match. First, we search for the best 1-1 match from U to M, where U is the candidate graph, and M is the model graph” (Liang, page 229, paragraph 7). The candidate knowledge graph is re-encoded into the model knowledge graph.
“We need a metric to measure which mapping is the best. For a given mapping, an intermediate layout graph, T, is first constructed based on U such that the mapping between T and M is 1-1. Then a cost is computed for the 1-1 mapping and defined as the quality measurement of the mapping between U and M. The best match is the one with minimal cost.” (Liang, page 228, paragraph 4)
“For a pair of mapped nodes, the cost is defined as the sum of differences between corresponding attributes, weighted by the weight factors in model node.” (Liang, page 228, paragraph 5); “A cost is similarly defined for a pair of edges” (Liang, page 229, paragraph 2); “The graph match cost is the sum of all node pair costs and edge pair costs” (Liang, page 229, paragraph 3).
Liang relates to generating relational knowledge graphs based on document layout analysis and is analogous to the claimed invention. The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Liang fails to disclose the further limitations of the claim, Nguyen discloses a method comprising: (b) generating representative vectors for the page elements using one or more embedding models, each representative vector comprising a concatenation of outputs of: type, semantic, visual, and position information: “In accordance with a broad aspect of the present technology, there is provided a server for extracting textual entities from a structured document, the server having access to a plurality of machine learning algorithms (MLAs) comprising a first MLA, a second MLA, and a third MLA, the server comprising: a processor, a computer-readable storage medium connected to the processor, the computer-readable storage medium comprising instructions, the processor, upon executing the instructions, being configured for: receiving a plurality of text sequences having been extracted from the structured document by an optical character recognition (OCR) model having processed the image to generate the structured document, receiving a plurality of structural elements, each structural element being indicative of a location of a respective text sequence in the document, encoding, by the first MLA, the plurality of text sequences and the plurality of structural elements to obtain a 3D encoded image, the 3D encoded image being indicative of semantic characteristics of the plurality of text sequences, the 3D encoded image having a spatial structure (visual / position information) of the structured document, compressing, by the second MLA, the 3D encoded image to obtain an aggregated feature vector, the aggregated feature vector being indicative of a combination of spatial characteristics (visual / position information) and semantic characteristics of the 3D encoded image, and decoding, by the third MLA, the aggregated feature vector to extract an associated set of textual entities, a given textual entity being associated with at least one text sequence in the plurality of text sequences” (Nguyen, [0031])
Nguyen relates to using neural networks to extract document elements and their attributes and is analogous to the claimed invention. Liang teaches a method of extracting type, spatial, and position information from elements of a document. The claimed invention improves upon this method by additionally extracting semantic element information and collating it with the other information. Nguyen teaches a method of extracting semantic information from document elements and combining it with other forms of page element information, applicable to Liang. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that including semantic information in the element extraction process would lead to the predictable result of including semantic information as a node similarity criteria, and would improve the known device by enabling it to detect anomalies indicated by divergence of semantic relationships in compared graphs (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
While Nguyen fails to disclose the further limitations of the claim, Luo discloses a method comprising: (d) determining edges between nodes by: applying a similarity model to representative vectors, and establishing edges based on a similarity threshold:
“The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
“The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (representative vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 9, the rejection of claim 8 is incorporated. Liang discloses a method, further comprising: employing layout analysis based on object detection to segment one or more pages of the document into page elements: “The task of logical labeling is to label segmented blocks (page elements) on a document image as title, author, header, text column, etc. The set of labels will depend on document classes and/or applications“ (Liang, page 224, paragraph 2); “Fig. 1 shows an overview of our document analysis system. First, document images are processed by a segmentation-and-OCR engine (object detection component). We assume that the results are reasonably good. In particular, mild over-segmentation is acceptable, while under-segmentation which crosses logical content is not welcome. The outcome XML file (PHY-XML in the figure) contains information about the physical layout and text content of the original document page. The LOG-XML in the figure stands for logical structure XML file, which contains information about document class and logical labels corresponding to the PHY-XML file” (Liang, page 225, paragraph 1).
The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding claim 10, the rejection of claim 9 is incorporated. Liang discloses a method, further comprising: employing multimodal embedding such that the page elements of the document are represented by the nodes in the knowledge graph:
PNG
media_image9.png
1475
1070
media_image9.png
Greyscale
“Example image model and labeling result (II)” (Liang, page 234, Fig. 11). A multimodal input (a document with text and images) is segmented into blocks for a knowledge graph.
The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding claim 11, the rejection of claim 10 is incorporated. Luo further discloses a method, comprising: determining whether the edge is present between the vectors by comparing a similarity between the vectors to a predefined threshold:
“The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
“The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 12, the rejection of claim 10 is incorporated. Luo, in combination with Liang discloses a method, further comprising: determining whether two of the vectors include the edge by comparing the multimodal embedding manner of the page elements:
(Luo) “The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
(Luo) “The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Examiner’s note: In this expression, the information vectors of two nodes, i and j, are compared to enforce edge connections between similar nodes. In combination with Liang, node information comprises embeddings of multi-modal page elements.
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 13, the rejection of claim 11 is incorporated. Fan discloses a method, further comprising: employing pairwise comparisons based on a graph attention algorithm for the re-encoded knowledge graph and the one or more other knowledge graphs:
“In order to obtain sufficient representative high-level node features, structure encoder firstly transforms the original observed node attribute X into the low-dimentional [sic] latent representation
Z
~
V
” (Fan, page 2, right column, paragraph 2)
“Given the transformed node embedding
Z
~
V
, a graph attention layer [16] is then employed to aggregate the representation from neighbor nodes, by performing a shared (pairwise) attentional mechanism on the nodes:
PNG
media_image2.png
40
532
media_image2.png
Greyscale
… its final embedding
Z
i
V
can be obtained by weighted sum based on the learned importance weights” (Fan, page 2, right column, paragraph 3).
“Finally, structure decoder takes the final node embeddings
Z
V
as inputs to decode them for reconstruction of the original network structure (re-encoded knowledge graph)” (Fan, page 2, right column, paragraph 3)
Fan relates to graph anomaly detection and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; and page 4, right column, paragraph 3.
Regarding claim 14, the rejection of claim 12 is incorporated. Biswas, in combination with Liang, further discloses a method, wherein if an abnormal score of a node of the document is significantly higher than one or more other nodes of the one or more other documents, the document is abnormal: (Biswas) “After identifying all or a portion of the cumulative graph profile that matches the single graph profile, a comparison may be made between the individual vertices and/or edges to detect substantial differences. In some embodiments, the system may look for individual differences in vertices and/or edge weights to detect anomalies. For example, if a single vertex in the single graph profile has a weight that is different from a corresponding weight of the vertex in the cumulative graph profile by more than a threshold amount, an anomaly may be detected” (Biswas, [0213]). While Biswas doesn’t disclose applying this method to document knowledge graphs, this deficiency is remedied by Liang, as discussed regarding claim 1.
Liang relates to generating relational knowledge graphs based on document layout analysis and is analogous to the claimed invention. The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding claim 18, the rejection of claim 17 is incorporated. While the aforementioned references fail to disclose the further limitations of the claim, Luo discloses a computer program, comprising: determine, using the processor, whether the edge is present between the vectors by a similarity between two of the vectors is greater than a pre-determined edge threshold:
“The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
“The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 19, the rejection of claim 17 is incorporated. While the aforementioned referenced fail to fully disclose the claim, Luo, in combination with Liang discloses a method, further comprising: determine, using the processor, whether the two of the vectors include the edge by comparing the multimodal embedding manner of the page elements:
(Luo) “The steps include determining one or more connected components of a graph of nodes and edges where each node represents a user account and a pair of nodes has an edge if the similarity score of the pair of nodes indicates a greater degree of similarity than indicated by the threshold similarity score” (Luo, [0005])
(Luo) “The edge determination module 350 determines the edges between sets of nodes. The edge determination module 350 compares the user accounts associated with a set of nodes to determine a similarity score indicating a degree of similarity between the user accounts. In some embodiments, the edge determination module 350 compares the information (vectors) of the user accounts to determine duplicate information. The information of the user accounts may be entered by a user or may be gathered from the text detected from the user's associated identification document. Further, in some embodiments, the edge determination module 350 compares the images from identification documents associated with user accounts for similarity by comparing the location of pixels within the images” (Luo, [0039])
Examiner’s note: In this expression, the information vectors of two nodes, i and j, are compared to enforce edge connections between similar nodes. In combination with Liang, node information comprises embeddings of multi-modal page elements.
Luo relates to forming knowledge graphs where edges represent node similarities over a threshold and is analogous to the claimed invention. The existing combination teaches a method of performing anomaly detection on document knowledge graphs. The claimed invention improves upon this method by connecting knowledge graph nodes based on similarity. Luo teaches a method of connecting knowledge graph nodes based on node similarity, applicable to the existing combination. A person of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that connecting segment block nodes generated by Liang’s document analysis method based on Luo’s node similarity threshold would lead to the predictable result of producing knowledge graphs representative of document element similarity, and would improve the known device by producing an anomaly detection method that identifies, in-part, discrepancies of relationships between different documents (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Claims 15-17 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Biswas (AUTONOMOUS MONITORING OF APPLICATIONS IN A CLOUD ENVIRONMENT, published 4/23/2020, US 2020/0128047 A1), in view of Fan (ANOMALYDAE: DUAL AUTOENCODER FOR ANOMALY DETECTION ON ATTRIBUTED NETWORKS, published 2020, arXiv:2002.03665v2), and further in view of Liang (Logical Labeling of Document Images Using Layout Graph Matching with Adaptive Learning, published 2002, DAS 2002, LNCS 2423, pp. 224–235), and Nguyen (2D DOCUMENT EXTRACTOR, published 4/29/2021, US 2021/0125034 A1).
Regarding claim 15, Biswas discloses [a] computer program product comprising a computer-readable storage medium storing instructions that, when executed, cause a processor to: “The processes depicted herein, such as those described with reference to the figures in this disclosure, may be implemented in software (e.g., code, instructions, program) executed by one or more processing units (e.g., processors cores), hardware, or combinations thereof. The software may be stored in a memory (e.g., on a memory device, on a non-transitory computer-readable storage medium)” (Biswas, [0022])
Said instructions to:
(b) generate representative vectors for the page elements using one or more embedding models, each representative vector comprising type, semantic, visual, and position information: “Each node in the directed graph may represent an action performed by the particular user” (Biswas, [0006]); “Weights assigned to each node may indicate a number of times the corresponding actions represented by the nodes were performed” (Biswas, [0009])
(c) construct a knowledge graph comprising nodes corresponding to the page elements:
“The method may additionally include generating, using the actions, a directed graph. Each node in the directed graph may represent an action performed by the particular user. Each connection (edge) between two nodes may represent a sequence in performance of actions represented by the two nodes” (Biswas, [0006])
PNG
media_image1.png
520
447
media_image1.png
Greyscale
(Biswas, Figure 4)
“FIG. 4 illustrates a weighted directed graph representing the series of actions with assigned weights, according to some embodiments.” (Biswas, [0188])
(f) compare the knowledge graph with one or more additional knowledge graphs to determine an anomaly score and identify abnormal documents based on the anomaly score:
“The system can, for example, generate a graph profile (knowledge graph) for each user for a time window (e.g., one day, one week, etc.) for all the actions performed by the user, as well as associated parameters such as time stamps, resources affected, IP addresses used, geolocations, etc … The system can then superimpose a graph of more recent data over a historical graph profile of the same user. Using the cumulative superimposed graph, the system can detect anomalies if the current single graph profile of the user deviates from the historical cumulative graph (additional knowledge graph). Any such deviation can be considered as anomalous” (Biswas, [0204])
“The method may further include detecting anomalies in the user actions by comparing the single graph profile to the cumulative graph profile (709)” (Biswas, [0212])
“After identifying all or a portion of the cumulative graph profile (additional knowledge graph) that matches the single graph profile (knowledge graph), a comparison may be made between the individual vertices and/or edges to detect substantial differences. In some embodiments, the system may look for individual differences in vertices and/or edge weights to detect anomalies. For example, if a single vertex in the single graph profile has a weight that is different from a corresponding weight of the vertex in the cumulative graph profile by more than a threshold amount, an anomaly may be detected” (Biswas, [0213])
Biswas relates to constructing and comparing knowledge graphs for a graph anomaly detection and is analogous to the claimed invention.
While Biswas fails to disclose the further limitations of the claim, Fan discloses instructions to: (e) re-encode the knowledge graph using a neural network:
“we propose a deep joint representation learning framework for anomaly detection through a dual autoencoder (AnomalyDAE) (neural network), which captures the complex interactions between the network structure and node attribute for highquality embeddings” (Fan, page 1, right column, paragraph 2)
“In order to obtain sufficient representative high-level node features, structure encoder firstly transforms the original observed node attribute X into the low-dimentional [sic] latent representation
Z
~
V
” (Fan, page 2, right column, paragraph 2)
“Given the transformed node embedding
Z
~
V
, a graph attention layer [16] is then employed to aggregate the representation from neighbor nodes, by performing a shared attentional mechanism on the nodes:
PNG
media_image2.png
40
532
media_image2.png
Greyscale
… its final embedding
Z
i
V
can be obtained by weighted sum based on the learned importance weights” (Fan, page 2, right column, paragraph 3).
“Finally, structure decoder takes the final node embeddings
Z
V
as inputs to decode them for reconstruction of the original network structure (re-encoded knowledge graph)” (Fan, page 2, right column, paragraph 3)
Fan relates to graph anomaly detection on attention-encoded graphs and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the primary reference to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; & page 4, right column, paragraph 3.
While Fan fails to disclose the further limitations of the claim, Liang discloses instructions to:
(a) segment a document into page elements using layout analysis; (b) generate representative vectors for the page elements using one or more embedding models, each representative vector comprising type, semantic, visual, and position information; (c) construct a knowledge graph comprising nodes corresponding to the page elements:
PNG
media_image3.png
400
772
media_image3.png
Greyscale
”System overview” (Liang, page 225, Fig. 1)
“Fig. 1 shows an overview of our document analysis system. First, document images are processed by a segmentation-and-OCR engine (object detection model)” (Liang, page 225, paragraph 1).
PNG
media_image4.png
200
400
media_image4.png
Greyscale
PNG
media_image5.png
200
400
media_image5.png
Greyscale
“Original document page and converted HTML result” (Liang, page 227, Fig. 3)
PNG
media_image6.png
198
708
media_image6.png
Greyscale
”Example layout and layout graphs” (Liang, page 227, Fig. 4)
“A layout graph (knowledge graph) is a fully connected attributed relational graph. Each node corresponds to a segmented block on a page. The attributes (vector) of a node are the position (position information) and size of the bounding box (visual information), and the normalized font size (type information) (small, middle, or large as compared to the average font size over the whole page). An edge between a pair of nodes reflects the spatial relationship between two corresponding blocks in the image.” (Liang, page 227, paragraph 1)
(d) determine edges between nodes based on a similarity between representative vectors:
“An edge between a pair of nodes reflects the spatial relationship between two corresponding blocks in the image” (Liang, page 227, paragraph 1)
“The attributes of edge AB in the left graph are shown in Fig. 5” (Liang, page 228, paragraph 1)
PNG
media_image7.png
373
767
media_image7.png
Greyscale
”Example edge attributes” (Liang, page 228, Fig. 5). To store these attributes, the block corresponding to node A inherently must be compared to the block corresponding to node B.
(e) re-encode the knowledge graph using a neural network:
“Our approach is a two-step approximate solution that aims at sub-optimal N-1 match. First, we search for the best 1-1 match from U to M, where U is the candidate graph, and M is the model graph” (Liang, page 229, paragraph 7). The candidate knowledge graph is re-encoded into the model knowledge graph.
“We need a metric to measure which mapping is the best. For a given mapping, an intermediate layout graph, T, is first constructed based on U such that the mapping between T and M is 1-1. Then a cost is computed for the 1-1 mapping and defined as the quality measurement of the mapping between U and M. The best match is the one with minimal cost.” (Liang, page 228, paragraph 4)
“For a pair of mapped nodes, the cost is defined as the sum of differences between corresponding attributes, weighted by the weight factors in model node.” (Liang, page 228, paragraph 5); “A cost is similarly defined for a pair of edges” (Liang, page 229, paragraph 2); “The graph match cost is the sum of all node pair costs and edge pair costs” (Liang, page 229, paragraph 3).
Liang relates to generating relational knowledge graphs based on document layout analysis and is analogous to the claimed invention. The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
While Liang fails to disclose the further limitations of the claim, Nguyen instructions to: (b) generate representative vectors for the page elements using one or more embedding models, each representative vector comprising type, semantic, visual, and position information: “In accordance with a broad aspect of the present technology, there is provided a server for extracting textual entities from a structured document, the server having access to a plurality of machine learning algorithms (MLAs) comprising a first MLA, a second MLA, and a third MLA, the server comprising: a processor, a computer-readable storage medium connected to the processor, the computer-readable storage medium comprising instructions, the processor, upon executing the instructions, being configured for: receiving a plurality of text sequences having been extracted from the structured document by an optical character recognition (OCR) model having processed the image to generate the structured document, receiving a plurality of structural elements, each structural element being indicative of a location of a respective text sequence in the document, encoding, by the first MLA, the plurality of text sequences and the plurality of structural elements to obtain a 3D encoded image, the 3D encoded image being indicative of semantic characteristics of the plurality of text sequences, the 3D encoded image having a spatial structure (visual / position information) of the structured document, compressing, by the second MLA, the 3D encoded image to obtain an aggregated feature vector, the aggregated feature vector being indicative of a combination of spatial characteristics (visual / position information) and semantic characteristics of the 3D encoded image, and decoding, by the third MLA, the aggregated feature vector to extract an associated set of textual entities, a given textual entity being associated with at least one text sequence in the plurality of text sequences” (Nguyen, [0031])
Nguyen relates to using neural networks to extract document elements and their attributes and is analogous to the claimed invention. Liang teaches a method of extracting type, spatial, and position information from elements of a document. The claimed invention improves upon this method by additionally extracting semantic element information and collating it with the other information. Nguyen teaches a method of extracting semantic information from document elements and combining it with other forms of page element information, applicable to Liang. A person of ordinary skill in the art would have recognized that including semantic information in the element extraction process would lead to the predictable result of including semantic information as a node similarity criteria, and would improve the known device by enabling it to detect anomalies indicated by divergence of semantic relationships in compared graphs (MPEP 2143 I. (D) Applying a known technique to a known device (method, or product) ready for improvement to yield predictable results).
Regarding claim 16, the rejection of claim 15 is incorporated. Liang discloses a program, further comprising: employ, using the processor, layout analysis based on object detection to segment one or more pages of the document into page elements: “The task of logical labeling is to label segmented blocks (page elements) on a document image as title, author, header, text column, etc. The set of labels will depend on document classes and/or applications“ (Liang, page 224, paragraph 2); “Fig. 1 shows an overview of our document analysis system. First, document images are processed by a segmentation-and-OCR engine (object detection component). We assume that the results are reasonably good. In particular, mild over-segmentation is acceptable, while under-segmentation which crosses logical content is not welcome. The outcome XML file (PHY-XML in the figure) contains information about the physical layout and text content of the original document page. The LOG-XML in the figure stands for logical structure XML file, which contains information about document class and logical labels corresponding to the PHY-XML file” (Liang, page 225, paragraph 1).
The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding claim 17, the rejection of claim 16 is incorporated. Liang discloses a program, further comprising: employ, using the processor, multimodal embeddings such that the page elements of the document are represented by the nodes in the knowledge graph:
PNG
media_image9.png
1475
1070
media_image9.png
Greyscale
“Example image model and labeling result (II)” (Liang, page 234, Fig. 11). A multimodal input (a document with text and images) is segmented into blocks for a knowledge graph.
The existing combination teaches a method of detecting anomalies by comparing knowledge graphs. The claimed invention differs from this method by performing anomaly detection on document knowledge graphs. Liang teaches a method of generating knowledge graphs from documents. Because both the existing combination and Liang teach the use of knowledge graph construction, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to substitute the knowledge graphs of the existing combination for Liang’s document knowledge graphs to achieve the predictable result of detecting anomalies in documents based on analysis of their graphs (MPEP 2143 I. (B) Substituting one known element for another for predictable results).
Regarding claim 20, the rejection of claim 19 is incorporated. Fan discloses a program, further comprising: employ, using the processor, pairwise comparisons based on a graph attention algorithm for the re-encoded knowledge graph and the one or more other knowledge graphs:
“In order to obtain sufficient representative high-level node features, structure encoder firstly transforms the original observed node attribute X into the low-dimentional [sic] latent representation
Z
~
V
” (Fan, page 2, right column, paragraph 2)
“Given the transformed node embedding
Z
~
V
, a graph attention layer [16] is then employed to aggregate the representation from neighbor nodes, by performing a shared (pairwise) attentional mechanism on the nodes:
PNG
media_image2.png
40
532
media_image2.png
Greyscale
… its final embedding
Z
i
V
can be obtained by weighted sum based on the learned importance weights” (Fan, page 2, right column, paragraph 3).
“Finally, structure decoder takes the final node embeddings
Z
V
as inputs to decode them for reconstruction of the original network structure (re-encoded knowledge graph)” (Fan, page 2, right column, paragraph 3)
Fan relates to graph anomaly detection and is analogous to the claimed invention. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the existing combination to re-encode graphs with an attention mechanism, as disclosed by Fan. Attention mechanisms can capture complex interactions between nodes, a feature important for anomaly detection, as well as reduce computational overhead caused by shallow learning mechanisms. Fan’s method additionally outperforms contemporary state-of-the-art methods. See Fan, page 1, Abstract; page 1, right column, paragraph 2; and page 4, right column, paragraph 3.
Response to Arguments
The following responses address arguments and remarks made in the instant remarks dated 05/04/2026.
Claim Interpretation
In light of the instant amendments, claims are no longer interpreted under 35 U.S.C. 112(f).
The Examiner notes that claim 8, as amended, is found to recite contingent limitations.
112 Rejections
Previous rejections under 35 U.S.C. 112(a) and 112(b) are withdrawn in light of the instant amendments. However, new rejections under 35 U.S.C. 112(a) and 35 U.S.C. 112(b) have been found.
101 Rejections
On pages 15-16 of the instant remarks, the Applicant argues that the claimed invention does not recite judicial exceptions:
“At Step 2A, Prong One, the claims are not directed to an abstract idea. Rather,
independent claim 1 recites a specific, structured computer-implemented technique for abnormal
document self-discovery
These limitations define a specific computational architecture and data transformation
pipeline in which document layout data is converted into a graph-based representation and then
processed using neural network techniques to detect structural anomalies. The claims therefore
are not directed to a mental process or other abstract idea, as the recited operations require
machine learning models, graph data structures, and neural network processing that cannot be
practically performed in the human mind. See Enfish, LLC v. Microsoft Corp., 822 F.3d 1327,
1336-37 (Fed. Cir. 2016); McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1313-
15 (Fed. Cir. 2016).”
In regards to the Applicant’s arguments above, the Examiner respectfully disagrees that claim 1, as amended, recites no mental processes. As stated in MPEP 2106.04(a)(2)(III), The courts do not distinguish between mental processes that are performed entirely in the human mind and mental processes that require a human to use a physical aid (e.g., pen and paper or a slide rule) to perform the claim limitation. See, e.g., Benson, 409 U.S. at 67, 65, 175 USPQ at 674-75, 674 … Nor do the courts distinguish between claims that recite mental processes performed by humans and claims that recite mental processes performed on a computer. As the Federal Circuit has explained, "[c]ourts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind." Versata Dev. Group v. SAP Am., Inc., 793 F.3d 1306, 1335, 115 USPQ2d 1681, 1702 (Fed. Cir. 2015). See also Intellectual Ventures I LLC v. Symantec Corp., 838 F.3d 1307, 1318, 120 USPQ2d 1353, 1360 (Fed. Cir. 2016) (‘‘[W]ith the exception of generic computer-implemented steps, there is nothing in the claims themselves that foreclose them from being performed by a human, mentally or with pen and paper.’’); Mortgage Grader, Inc. v. First Choice Loan Servs. Inc., 811 F.3d 1314, 1324, 117 USPQ2d 1693, 1699 (Fed. Cir. 2016) (holding that computer- implemented method for "anonymous loan shopping" was an abstract idea because it could be "performed by humans without a computer").
Claim 1, as amended, recites limitations amounting to mental processes performed on generic computer hardware, insufficient to render a mentally performable task non-abstract. For example, claim 1 recites the limitation “(b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information”, reciting a mental process performed by “a memory storing instructions; and a processor operably coupled to the memory and configured to execute the instructions”, generic computer hardware insufficient to render the limitation non-abstract.
The Examiner asserts that claim 1, as amended, recites mental processes. Claims 2-20 are found to recite mental processes under similar rationales (see the 101 rejections section for more detail). Thus, no rejections are withdrawn on these grounds.
On page 16 of the instant remarks, the Applicant argues that recited judicial exceptions are integrated through improvements to technology:
“Moreover, the claims are directed to a specific improvement in computer functionality. In
particular, the claims improve how a computer system represents and analyzes document layout
by transforming unstructured document data into a structured knowledge graph and performing
graph-based analysis to detect anomalies that are not identifiable using conventional similarity or
classification techniques. This type of data structure-driven improvement is analogous to the
improvements found patent-eligible in Enfish.
Any characterization of the claims at a high level as merely "analyzing document data" is
improper. The Federal Circuit has cautioned against describing claims at a high level of
abstraction untethered from the claim language. Enfish, 822 F.3d at 1337. Here, the claims recite
a particular implementation involving multimodal embedding, similarity-based edge
determination, graph construction, neural re-encoding, and cross-document structural
comparison. These are concrete technological operations that define a specific solution to a
technological problem.
Even assuming, arguendo, that the claims involve an abstract idea, they satisfy Step 2A,
Prong Two because any such idea is integrated into a practical application. The claims do not
merely apply an abstract idea using generic computing elements, but instead recite a specific
technique for transforming document layout into a graph-based representation and performing
structured analysis on that representation using neural processing. This constitutes a practical
application that improves the functioning of a computer system. See DDR Holdings, LLC v.
Hotels.com, L.P., 773 F.3d 1245, 1257-59 (Fed. Cir. 2014).”
In response to the Applicant’s argument that the claimed invention represents an improvement to existing technology or technical field, the Examiner notes that improvements cannot be made through a recited judicial exception. As noted by MPEP 2106.05(a), It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)) in subsection II, below. In addition, the improvement can be provided by the additional element(s) in combination with the recited judicial exception. See MPEP § 2106.04(d) (discussing Finjan, Inc. v. Blue Coat Sys., Inc., 879 F.3d 1299, 1303-04, 125 USPQ2d 1282, 1285-87 (Fed. Cir. 2018)).
The Applicant is arguing improvement through graph-based analysis of document layouts in abnormal document detection, recited in claim 1 as, in part, “(c) construct a knowledge graph comprising: a plurality of nodes corresponding to the page elements, and edges selectively established between pairs of nodes”, which can be performed as a mental process. Mere instruction to execute this mental process with a generic data structure (“an object detection model”) or generic computer hardware (“a memory storing instructions” is not representative of an improvement to technology or a technical field. As detailed further in the 101 rejections section, none of the additional elements of the claimed invention are sufficiently representative of the alleged improvement to technology argued by the Applicant.
Thus, no rejections are withdrawn on these grounds. See the 101 rejections section for more detail.
On pages 16-17 of the instant remarks, the Applicant argues that the claimed invention amounts to significantly more than its recited judicial exceptions due to an unconventional arrangement of operations:
“Under Step 2B, the claims recite significantly more than any alleged abstract idea. The
claimed invention requires a non-conventional and ordered combination of operations, including:
generating multimodal representative vectors that include position information; determining
graph connectivity through similarity-based edge formation; re-encoding the resulting
The Federal Circuit has held that such ordered combinations of elements can provide an
inventive concept even where individual elements are known. See Bascom Global Internet
Servs., Inc. v. AT&T Mobility LLC, 827 F.3d 1341, 1350 (Fed. Cir. 2016). Here, the claimed
combination defines a specific architecture for abnormal document detection that departs from
conventional approaches and provides improved analytical capability.
The claims also do not raise preemption concerns. They are narrowly directed to a
particular technique for abnormal document detection using layout-derived graph representations
and do not preempt all approaches to document analysis or anomaly detection.”
In response to the Applicant’s argument that the claimed invention amounts to significantly more by virtue of providing a specific, technology-based system, the Examiner notes that for the sake of Alice / Mayo analysis, claims can only amount to significantly more than their recited judicial exceptions through inventive concepts, and an inventive concept cannot be made through a recited judicial exception. As noted by MPEP 2106.05(I), An inventive concept "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself." Genetic Techs. Ltd. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016). See also Alice Corp., 573 U.S. at 21-18, 110 USPQ2d at 1981 (citing Mayo, 566 U.S. at 78, 101 USPQ2d at 1968 (after determining that a claim is directed to a judicial exception, "we then ask, ‘[w]hat else is there in the claims before us?") (emphasis added)); RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1327, 122 USPQ2d 1377 (Fed. Cir. 2017) ("Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract"). Instead, an "inventive concept" is furnished by an element or combination of elements that is recited in the claim in addition to (beyond) the judicial exception, and is sufficient to ensure that the claim as a whole amounts to significantly more than the judicial exception itself. Alice Corp., 573 U.S. at 27-18, 110 USPQ2d at 1981 (citing Mayo, 566 U.S. at 72-73, 101 USPQ2d at 1966).
The Applicant is arguing inventiveness through abstract mental processes. For example, as recited in claim 1, “(f) compare the re-encoded knowledge graph with one or more additional knowledge graphs corresponding to other documents to determine an anomaly score based on differences in nodes and edges of the knowledge graphs, and determine that the document is abnormal when the anomaly score satisfies the threshold” can be performed as a mental process.
While the claimed invention contains several additional elements, they are insufficient to furnish an inventive concept and make the claim as a whole amount to significantly more than its recited judicial exceptions.
Thus, no rejections are withdrawn on these grounds. See the 101 rejections section for more detail.
103 Rejections
On pages 17-19 of the instant remarks, the Applicant argues that the relied upon references fail to disclose the amended claims:
“These limitations define a data transformation pipeline in which document layout is
converted into a graph representation through learned multimodal embeddings and similarity-based
edge inference, followed by neural processing and cross-document structural comparison.
This architecture is not taught or suggested by the cited references
…
Liang does not disclose generating representative vectors using embedding models, combining
multimodal features including semantic, visual, and positional information, or determining edge
connectivity based on similarity scores produced by a learned similarity model. Instead, Liang
relies on template-based matching over a preconstructed graph, and therefore fundamentally
differs from the claimed invention, which derives both node representations and graph topology
from learned models
Peng and Fan likewise fail to remedy these deficiencies. Both references operate on preexisting
attributed networks in which node attributes and graph connectivity are assumed as
input … Neither reference discloses or suggests
constructing nodes from document page elements, generating node representations via
multimodal embedding of document features, or inferring graph edges based on similarity
relationships between learned representations. Instead, both references assume that graph
topology is known and fixed, which is fundamentally different from the claimed approach, in
which the graph itself is constructed from document data
…
Moreover, even if the references were combined, they would not yield the claimed
invention. The claims require a specific ordered sequence of operations in which document
layout is transformed into representative vectors, used to construct a graph through similarity-based
edge inference, re-encoded using neural network processing, and compared across
multiple documents to detect structural anomalies. None of the references teaches or suggests
this ordered transformation pipeline or its resulting functionality”
Regarding the Applicant’s arguments above, the Examiner respectfully disagrees. Regarding “(a) perform layout analysis on a document using an object detection model to segment a page of the document into a plurality of page elements”, Liang discloses a method of segmenting a document page into a plurality of page elements (Liang, page 255, paragraph 1 & page 227, paragraph 1). Combining Liang with the existing combination of anomaly detection references would achieve the predicable result of detecting anomalies in documents (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Regarding “(b) generate, using one or more embedding models, a respective representative vector for each of the page elements, each representative vector comprising a concatenation of outputs of at least two of: type information, semantic information, visual information, or position information”, Biswas discloses a method of generating representative vectors in the form of weighted user action nodes (Biswas, [0009]). While Biswas fails to disclose forming representative vectors for page elements, this deficiency is remedied by Liang, which discloses a method of forming representative vectors for page elements, each comprising type, visual, and position information in the form of position, bounding box, and font size information (Liang, page 227, paragraph 1). Liang extracts and embeds page element information into nodes in a graph, and can thus be considered an embedding model.
Regarding “(c) construct a knowledge graph comprising: a plurality of nodes corresponding to the page elements, and edges selectively established between pairs of nodes”, Biswas discloses constructing a knowledge graph with nodes representing user actions and edges representing sequences of actions (Biswas, [0006] & [0187-0188]). While Biswas fails to disclose constructing knowledge graphs of page elements, this deficiency is remedied by Liang, which discloses forming knowledge graphs with selective edges between nodes from document page elements (Liang, page 227, paragraph 1).
Regarding “(d) determine whether to establish an edge between a pair of nodes by: applying a similarity model to corresponding representative vectors, and establishing the edge when a similarity score satisfies a threshold”, Liang further discloses selectively connecting page element nodes with edges corresponding to spatial relationships (Liang, page 227, paragraph 1). While Liang fails to disclose determining edges based on a similarity model score satisfying a threshold, this deficiency is remedied by Luo, which discloses assigning edges to pairs of nodes only when their similarity score is over a threshold (Luo, [0005] & [0039]). Combining Luo’s edge-formation method with Liang’s document node identification would yield the predictable result of generating graphs that model similarity between page elements in a document (MPEP 2143 I. (A) Combining prior art elements according to known methods to yield predictable results).
Regarding “(e) re-encode the knowledge graph using a neural network configured to propagate information between connected nodes to generate updated node representations”, Fan discloses a method of re-encoding a knowledge graph using an attention-based neural network that propagates information from neighboring nodes to update the node representations (Fan, page 1, right column, paragraph 2, & page 2, right column, paragraphs 2-3). Fan makes explicit numerous benefits of its method for anomaly detection, and would have been obvious to combine with Biswas (Fan, page 1, Abstract; page 1, right column, paragraph 2; & page 4, right column, paragraph 3)
Regarding “(f) compare the re-encoded knowledge graph with one or more additional knowledge graphs corresponding to other documents to determine an anomaly score based on differences in nodes and edges of the knowledge graphs, and determine that the document is abnormal when the anomaly score satisfies the threshold”, Biswas discloses a method of determining graph anomaly scores over a threshold by comparing nodes and edges in different graphs. While Biswas fails to re-encode graphs, this deficiency is remedied by Fan, as discussed above.
As further detailed in the 103 rejections section, all amended claims are found to be obvious over the prior art. Thus, no rejections are withdrawn on this basis.
On pages 18-19 of the instant remarks, the Applicant argues that the relied upon references are incompatible:
“Peng and Fan likewise fail to remedy these deficiencies … Neither reference discloses or suggests
constructing nodes from document page elements, generating node representations via
multimodal embedding of document features, or inferring graph edges based on similarity
relationships between learned representations. Instead, both references assume that graph
topology is known and fixed, which is fundamentally different from the claimed approach, in
which the graph itself is constructed from document data
Belligundu also fails to cure these deficiencies. Although Belligundu relates to anomaly
detection using multimodal knowledge graphs, it operates on pre-existing knowledge graphs in
which entities and relationships are already defined. Belligundu does not disclose deriving graph
nodes from document layout elements, generating multimodal embeddings from page elements, or constructing graph topology through similarity-based edge inference.
Importantly, the cited references rely on fundamentally incompatible data generation
paradigms. Liang uses predefined spatial relationship graphs, Peng and Fan assume fixed
network topology, and Belligundu operates on pre-built knowledge graphs. In contrast, the
claimed invention dynamically constructs graph topology from document layout data using
learned multimodal representations and similarity-based edge formation. The claimed invention
further requires neural re-encoding of the constructed graph and cross-document structural
comparison, which are not taught or suggested by the cited art
The Office's rejection appears to rely on a generalized assertion that it would have been
obvious to combine Liang with Peng, Fan, and Belligundu. However, such a combination would
require fundamental redesign of the cited systems, including replacing Liang's predefined spatial
relationships with learned similarity-based edge inference, eliminating the fixed adjacency
assumptions in Peng and Fan, and modifying Belligundu's reliance on pre-existing knowledge
graphs to instead construct graphs from raw document layout. These changes are not routine
substitutions, but would alter the core operating principles of the references. There is no
teaching, suggestion, or motivation in the cited art to make such modifications. Any such
reconstruction would require impermissible hindsight using Applicant's disclosure as a blueprint,
contrary to KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398,421 (2007)”
In response to applicant's arguments that the formats and encoding of the graphs in each reference differ, the Examiner notes that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
As discussed above and in further detail in the 103 rejections section, sufficient motivation existed for one of ordinary skill in the art to combine these references. Regardless of their mechanical compatibility (and it’s not made clear that any of the references possesses a graph incompatible with the other references), the ideas are clearly compatible. One of ordinary skill in the art, with a graph anomaly detection reference such as Biswas in hand, would understand that such a system would be broadly compatible with a variety of different types of knowledge graphs, such as document knowledge graphs.
In response to applicant's argument that the Examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
The Examiner asserts that the claimed invention, as amended, is obvious over Biswas in view of Fan, Liang, Luo, and Nguyen, as detailed further in the 103 rejections section. Thus, no rejections are withdrawn on these grounds.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Ma et al. (A Comprehensive Survey on Graph Anomaly Detection with Deep Learning, published April 2022, arXiv:2106.07178v5) discloses a multitude of state-of-the-art methods for graph anomaly detection from near the effective filing date of the instant application
Ni et al. (Semantic Documents Relatedness using Concept Graph Representation, published 2016, WSDM '16: Proceedings of the Ninth ACM International Conference on Web Search and Data Mining Pages 635 – 644) discloses a method of constructing a concept graph with similarity edges from a document
Hu et al. (An Embedding Approach to Anomaly Detection, published 2016, ICDE 2016) discloses a method of embedding and connecting nodes in a graph such that each cluster of nodes is related to some particular community.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aaron P Gormley whose telephone number is (571)272-1372. The examiner can normally be reached Monday - Friday 12:00 PM - 8:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle T Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AG/Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights