DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
Claim(s) 1-6, 10-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Vuang M. Ngo et al (hereinafter Ngo) (“A Semantic Search Engine for Historical Handwritten Document Images”, 2021) in view of Assadollahi, Ramin et al (Hereinafter Ramin) (EP 2 088 536 A1, 2009), further in view of Wei-guang Wen (hereinafter Wen) (CN 101101706 A, 2008).
Regarding claim 1, Ngo teaches;
a concept building circuitry (the methods of Ngo are computer implemented, which includes circuitry, see pg. 1) configured to:
determine one or more conceptual terms ([pg. 4] William_sutton/person, occu_clerk, etc.) from one or more ([pg. 4] William Sutton, clerk, etc.)
wherein the one or more conceptual terms include at least one of: terms corresponding to the recognized text in one or more languages, one or more synonym terms corresponding to the recognized text, one or more abbreviation terms corresponding to the recognized text, or one or more internally defined terms corresponding to the recognized text
[pg. 3]
PNG
media_image1.png
202
879
media_image1.png
Greyscale
and determine a multi-level relation between one or more
[pg. 3]
PNG
media_image2.png
200
400
media_image2.png
Greyscale
NOTE: The above image teaches determining a multi-level relation (a knowledge graph with multiple levels) between one or more recognized terms, and the handwritten document (the terms in this graph include the recognized terms from the handwritten document).
and a knowledge graph building circuitry (the methods of Ngo are computer implemented, which includes circuitry, see pg. 1) configured to build a knowledge graph, wherein the knowledge graph is built based at least on one of: the plurality of potential terms, the one or more conceptual terms, the determined multi-level relation between the one or more potential recognized terms and the handwritten document,
[pg. 3]
PNG
media_image2.png
200
400
media_image2.png
Greyscale
wherein the knowledge graph is used to enable at least a semantic searching of one or more handwritten documents. ([pg. 4] We proposed a novel semantic full-text search system for images of historical handwritten manuscripts. Unlike the existing approach only using KW extracted from images, we exploited NE, KW and KG of increase search performance.)
Ngo fails to teach but Ramin teaches;
a receiver circuitry ([0010] An embodiment of the invention comprises a computer with a touch-screen device and implements three software components: … Handwriting recognition component; User interface component allowing to write letters and select candidate words)
configured to receive a handwritten document ([0026] The character handwriting recognition component may comprise handwriting input data processing means for receiving a sequence of 2-dimensional input data from the touch sensitive input field (i.e. the dots activated by the user when writing on the input field))
along with dynamic handwritten data ([0035] A character handwriting recognition component 330 receives handwriting input data from a touch sensitive input field 340, e.g. a touch screen.) from an electronic device;
a recognition circuitry ([0010] An embodiment of the invention comprises a computer with a touch-screen device and implements three software components: … Handwriting recognition component; User interface component allowing to write letters and select candidate words)
configured to recognize a plurality of potential terms (candidate words) for one or more objects in the handwritten document by employing a handwriting recognition technique, ([Abstract] The present invention relates to a text input system and method involving finger-_based handwriting recognition and word prediction. A text input device (300) comprises: a text prediction component (310) for predicting a plurality of follow-up words based on a text context, the text prediction component (310) outputting a set of candidate words; a character handwriting recognition component (330) for recognizing a handwritten character candidate, the handwritten character candidate being determined based upon handwriting input received from a touch sensitive input field (340); a candidate word filtering component (350) for filtering the set of candidate words received from the text prediction component (310) based on the recognized handwritten character candidate;)
wherein the plurality of potential terms for each of the one or more objects includes a closest recognized term and at least one alternative recognized term;
([0021] Preferably, the text prediction component outputs a ranked list of input candidate words according to respective scores that indicate the likelihood or probability of a candidate word to follow the text context.)
NOTE: Teaches wherein the plurality of potential terms for each of the one or more objects (aforementioned candidate words) includes a closest recognized term (word with the highest score) and at least one alternative recognized term (outputs a list of ranked words, indicating multiple alternative recognized terms having lower scores)
OBVIOUSNESS TO COMBINE RAMIN WITH NGO:
Ngo and Ramin are analogous art to each other and the present disclosure as they all pertain to methods for parsing and processing handwritten data. Specifically, Ngo pertains to a method of semantic searching for handwritten documents using a knowledge graph while Ramin pertains to methods of handwriting recognition.
[pg. 2]
PNG
media_image3.png
520
896
media_image3.png
Greyscale
Additionally, Ngo recites the need for a handwriting text recognition circuitry to be utilized in the system of their disclosure to digitize the handwritten document for further processing.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the receiver and recognition circuitry taught by Ramin to digitize and parse the handwritten document to allow the recognized terms of the plurality of potential terms to be processed as taught by Ngo.
Ngo and Ramin fail to teach but Wen teaches;
wherein the dynamic handwritten data comprises data on x-axis, data on v-axis, and pressure data;
([pg. 4] in which each data tuple (Xt, Yt, Pt) corresponding to the coordinate position of the point T in the handwriting pen is writing at the point T, Xt, Yt, Pt is the point T the sensed pressure value.)
OBVIOUSNESS TO COMBINE WEN:
Wen is analogous art to the present disclosure as it pertains to recording data pertaining to handwriting.
Ramin already teaches recording handwriting data tuples comprising data on an x and y axis, while Wen teaches recording handwriting data tuples comprising data on an x and y axis, as well as pressure data p.
Additionally, Wen indicates that erroneous points which indicate an absence of sensed writing pressure (P = 0, and therefore do not represent actual pen-contact portions of the written stroke) are disregarded, predictably providing a more accurate data record of what the user actually wrote;
([pg. 5] Note that if the data tuple Pt (Xt, Yt, Pt) is 0, then not considering the point in the algorithm.)
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the handwriting data of Ramin to include pressure data using the process of Wen, predictably providing a more accurate data record of what the user actually wrote.
Regarding claim 2, Ngo in view of Ramin teaches;
The system of claim 1,
(Using the same reasoning as claim 1)
Ngo fails to teach but Ramin teaches;
wherein the dynamic handwritten data is received in the form of one or more tuples…
([0026] The character handwriting recognition component may comprise handwriting input data processing means for receiving a sequence of 2-dimensional input data from the touch sensitive input field (i.e. the dots activated by the user when writing on the input field). The input data may be arranged in a one-dimensional input layer of the neural network.)
NOTE: Teaches the dynamic handwritten data is received (handwriting input data) in the form of one or more tuples (received in the form of an ordered list of x, y coordinates)
… data on x-axis, data on y-axis.
([0052] The values of the re-sampled dots are arranged in a one-dimensional array such that the two position values of a dot are put side by side and dots are put side by side. This results in a data representation as: x1, y1, x2, y2, x3, y3, ..., where x1 is the first (offset compensated, normalized, re-sampled) sample value in x-direction, y1 is the first sample value in y-direction, x2 the second sample value in x-direction, etc.)
NOTE: Teaches the tuples (the input data is resampled into the recited one-dim array) having at least data on x-axis and y-axis (x1, y1, x2, y2, etc.)
OBVIOUSNESS:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the receiver and recognition circuitry taught by Ramin to digitize and parse the handwritten document to allow the recognized terms of the plurality of potential terms to be processed as taught by Ngo.
Ngo and Ramin fail to teach but Wen teaches;
wherein the dynamic handwritten data is received in the form of one or more tuples having data on x-axis, data on y-axis, and pressure data
([pg. 4] in which each data tuple (Xt, Yt, Pt) corresponding to the coordinate position of the point T in the handwriting pen is writing at the point T, Xt, Yt, Pt is the point T the sensed pressure value.)
OBVIOUSNESS:
Using the same reasoning from claim 1, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the handwriting data of Ramin to include pressure data using the process of Wen, predictably providing a more accurate data record of what the user actually wrote.
Regarding claim 3, Ngo in view of Ramin teaches;
The system of claim 2,
(Using the same reasoning as the claim 2 rejection)
Ngo fails to teach but Ramin teaches;
wherein the handwriting recognition techniques analyze each of the received one or more tuples to identify the closest recognized term along with one or more alternative recognized terms that each of the received one or more tuples potentially represents.
([0021] Preferably, the text prediction component outputs a ranked list of input candidate words according to respective scores that indicate the likelihood or probability of a candidate word to follow the text context. This allows the word presentation component to present, e.g. on the display device, at least one candidate word from the filtered list of candidate words in accordance with its respective score. For instance, the candidate words are presented in descending order of probabilities so that the most likely candidate is presented in a first position of a list, the second most likely candidate in a second position, and the least likely candidate that matches the input character_(s) is arranged in last position of the list. Preferably, the list of candidate words is presented so that the user can easily select the most likely words, e.g. without scrolling. Less likely words may initially not be presented on the display, but may be displayed when the user scrolls down the list. This helps to select likely words and reduces the average number of interactions required by the user to select the intended word.)
NOTE: Teaches wherein the handwriting recognition techniques analyze each of the received one or more tuples to identify the closest recognized term (determine scores indicating the most likely [closest] term) along with one or more alternative recognized terms (includes second most likely as an alternative for the user to select) that each of the received one or more tuples potentially represents (the candidate words represent what the input data [which has already been determined to be tuples] potentially represents).
Regarding claim 4, Ngo teaches;
The system of claim 1,
wherein the concept building circuitry is configured to perform a named entity linking on the
([pg. 2] Transkribus (Kahle et al. 2017) is used for training and deploying handwritten Text Recognition (HTR) models to derive text transcription from image scans. Given the rate at which transcriptions can be generated, NE Recognition (NER) and Entity Linking (EL) are required to automated annotate all instances of entities occurring in the transcription text. We used SpaCy (Honnibal et al. 2020) for NER and had highly results on 18th century English text. To provide flexibility, an NLP pipeline has been implemented as a thin layer over a number of standard NLP tools. The output of the pipeline is a NLP Interchange Format (Hellmann et al. 2013) in which a NER tool has annotated classes of entities and, where possible, an EL tool has connected the recognized entities to KG.)
NOTE: Teaches the concept building circuitry configured to perform named entity linking on the terms to determine the one or more conceptual terms (uses named entity recognition to recognize the terms, and entity linking and entity linking to link entities to related conceptual terms in the knowledge graph).
Using the same reasoning to combine Ngo and Ramin in claim 1, is would be obvious to perform named entity linking as taught by Ngo on the plurality of potential terms as taught by Ramin.
Regarding claim 6, Ngo teaches;
The system of claim 1,
wherein each of the
[pg. 3]
PNG
media_image4.png
605
880
media_image4.png
Greyscale
NOTE: Teaches each of the terms (William, Sutton, etc.) along with the one or more corresponding conceptual terms (Surname, Male, etc.) are placed as a node in the built knowledge graph, such that one node is connected to another node (nodes have connections) based on the determined multi-level relation (the ‘surname’ and ‘name’ nodes are connected to Sutton_William by a degree of 2, therefore the relation is multi-level) between the corresponding recognized terms and the handwritten document (the terms in the pictured knowledge graph are the same recognized terms from the handwritten document).
Ngo fails to teach but Ramin teaches;
plurality of potential terms
(Using the same teaching and combination reasoning from claim 1)
Regarding claim 10;
Claim 10 is a method claim directly corresponding to claim 1 and is rejected for the same reasons.
Regarding claim 11;
Claim 11 is a method claim directly corresponding to claim 3 and is rejected for the same reasons.
Regarding claim 12;
Claim 12 is a method claim directly corresponding to claim 4 and is rejected for the same reasons.
Regarding claim 13;
Claim 13 is a method claim directly corresponding to claim 6 and is rejected for the same reasons.
Claim(s) 7-9, 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ngo (“A Semantic Search Engine for Historical Handwritten Document Images”, 2021) in view of Ramin (EP 2 088 536 A1, 12/08/2009) further in view of Wen (CN 101101706 A, 2008) further in view of Corodescu A et al, (hereinafter Corodescu) (US 20220398331 A1, 12/15/2022).
Regarding claim 7, Ngo in view of Ramin teaches;
The system of claim 1,
(Using the same reasoning from claim 1)
Ngo, Ramin, and Wen fail to teach but Corodescu teaches;
wherein the knowledge graph building circuitry is further configured to facilitate the user to set visibility of newly added nodes and their relationships in the knowledge graph.
([0019] The visibility policy may be enforced against properties of knowledge-graph objects, which include nodes and edges.)
NOTE: Teaches that the nodes and relationships (edges) of a knowledge graph have a set visibility.
([0027] The visibility-policy collection system provides an interface through which users may specify their visibility preferences for the properties of an object. The preferences may be used to form a visibility policy for the object. The ability to create or edit a visibility policy may be governed at different levels in the system. For example, a user with full access to a node (e.g., document, file) may be able to edit the property visibility profile. A visibility record may comprise properties governed and a user or group of users with visibility to the property.)
NOTE: Teaches facilitating the user to set the visibility of newly added nodes and their relationships (users can edit visibility properties for knowledge graph objects, this could be any object in the KG, which includes newly added nodes and relationships)
OBVIOUSNESS TO COMBINE CORODESCU WITH NGO AND RAMIN:
Corodescu is analogous art to the present disclosure and Ngo as it pertains to knowledge graphs, and is analogous to Ramin as it pertains to data processing. Specifically, Corodescu pertains to a knowledge graph representation with visibility properties for knowledge graph objects.
Additionally, Corodescu states;
([Abstract] The technology described herein protects the privacy and security of data stored in a knowledge graph (“graph”) by enforcing visibility policies when returning property information in response to a query or other attempt to extract property information from the graph and/or about the graph.)
NOTE: This discloses that by enforcing visibility properties on knowledge graph objects, it allows protecting certain portions of the data from un-authorized data.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the visibility properties taught by Corodescu to the objects of the generated knowledge graph of claim 1 to protect sensitive information contained in the handwritten data of the present disclosure.
Regarding claim 8, Ngo in view of Ramin teaches;
The system of claim 1,
(Using the same reasoning from claim 1)
Ngo, Ramin, and Wen fail to teach but Corodescu teaches;
wherein the knowledge graph building circuitry is further configured to automatically set visibility of newly added nodes and their relationships in the knowledge graph based on historical visibilities of nodes and their relationships.
([0020] A visibility policy may govern read-access to a property of a knowledge-graph object. A visibility policy with a default-restricted visibility may restrict access to all users, except those designated in the policy as having access. Alternatively, a visibility policy with a default-unrestricted visibility may grant access to all users, except those designated as not having access. The technology described herein may work with both default statuses.)
NOTE: Teaches automatically setting visibility of newly added nodes and their relationships (default visibility will automatically be applied to new objects, which includes nodes and their relationships, as previously mentioned) in the KG based on historical visibility of nodes and relationships (the visibility of new nodes is based on the rule which defines the visibility of historical nodes and their relationships, and is therefore itself based on the historical visibilities of nodes and their relationships).
Using the same reasoning from claim 7, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the visibility properties taught by Corodescu to the objects of the generated knowledge graph of claim 1 to protect sensitive information contained in the handwritten data.
Regarding claim 9, Ngo in view of Ramin further in view of Corodescu teach;
The system of claim 8,
Ngo, Ramin, and Wen fail to teach but Corodescu teaches;
wherein based on the set visibility, the one or more nodes and their relationships are divided into at least one of:
one or more public nodes and relationships corresponding to the documents publicly available to each user, one or more shared nodes and relationships corresponding to the documents on a subject to which the user is invited, and one or more private nodes and relationships corresponding to the documents that are specific to one user.
([0020] A visibility policy may govern read-access to a property of a knowledge-graph object. A visibility policy with a default-restricted visibility may restrict access to all users, except those designated in the policy as having access. Alternatively, a visibility policy with a default-unrestricted visibility may grant access to all users, except those designated as not having access. The technology described herein may work with both default statuses.)
NOTE: Teaches dividing the one or more nodes and their relationships [knowledge graph objects] into at least one or more shared nodes and relationships corresponding to the documents on a subject to which the user is invited (a group of objects pertaining to a subject can have the same visibility policy to allow only authorized users)
Regarding claim 14;
Claim 14 is a method claim directly corresponding to claim 7, and is therefore rejected for the same reasons.
Regarding claim 15;
Claim 15 is a method claim directly corresponding to claim 8, and is therefore rejected for the same reasons.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ngo (“A Semantic Search Engine for Historical Handwritten Document Images”, 2021) in view of Yang C et al. (Hereinafter Yang) (CN 112148851 A, 12/29/2020), further in view of Wen (CN 101101706 A, 2008).
Regarding claim 16, Ngo teaches;
A semantic searching system for searching handwritten documents using a knowledge graph, the semantic searching system comprising: ([pg. 1] This paper proposes a semantic search engine for full-text retrieval of historical handwritten document images based on named entity (NE), keyword (KW) and knowledge graph (KG).)
a receiver circuitry configured to receive, from an electronic device, text data having one or more terms associated with a user’s intended search;
[pg. 4]
PNG
media_image5.png
313
1085
media_image5.png
Greyscale
NOTE: Teaches a receiver circuitry (user interface) configured to receive, from an electronic device, text data having one or more terms associated with the user’s intended search (in the search text box, we can see the terms Meath, silver, and shilling)
an entity recognition circuitry configured to perform entity recognition from the text data to determine one or more entities present in the text data;
[pg. 3]
PNG
media_image6.png
186
767
media_image6.png
Greyscale
([pg. 3] Figure 3 {above} presents an image of a handwritten medieval historical manuscript, its transcription and its concept set d, applied in the model. In the transcription, there are three kinds of words determined by our NER tool: (1) stop-words being the, to, of, we and you; (2) NEs being sheriff, Meath, clerk and William Sutton; and (3) KWs being king, &c, greeting, direct, pay, shilling and silver. The stop-words are not added into the concept set d.)
NOTE: Teaches an entity recognition circuitry configured to perform entity recognition (NER here means ‘Named Entity Recognition’) from the text data to determine one or more entities present in the text data (some of the entities they determine include king, occu-sherrif, etc.).
a concept determination circuitry configured to determine one or more conceptual terms for each of the determined one or more entities via a named entity linking;
[pg. 3]
PNG
media_image7.png
335
854
media_image7.png
Greyscale
([pg. 2] The output of the pipeline is a NLP Interchange Format (Hellmann et al. 2013) in which a NER tool has annotated classes of entities and, where possible, an EL tool has connected the recognized entities to KG.)
NOTE: Teaches a concept determination circuitry configured to determine one or more conceptual terms (male, surname, etc.) for each of the determined one or more entities (William_Sutton) via named entity linking (in Ngo, named entity recognition [NER] determines the entities in the text and entity linking [EL] is used to determine the conceptual terms connected to each entity in the knowledge graph [KG], as depicted in the above image).
and a rendering circuitry configured to render ranked and selected search results to the user, ([pg. 2] Finally, the KW-NE-Based IR Model circuitry compares the annotated query and the annotated documents to return the ranked transcriptions and images.)
NOTE: Teaches a circuitry configured to render (images are considered renderings) ranked and selected search results.
([Abstract] In the next steps, we apply the named entity recognition and historical knowledge graph to build a semantic search model, which can understand the user's intent in the query and the contextual meaning of concepts in documents, to return correctly the transcriptions and their corresponding images for users.)
NOTE: Teaches returning the ranked (as taught above) renderings of search results to users.
wherein the one or more ranked and selected search results include at least one of: shortcuts to open a handwritten document associated with the search results, or online links associated with the search results.
([Abstract] In the next steps, we apply the named entity recognition and historical knowledge graph to build a semantic search model, which can understand the user's intent in the query and the contextual meaning of concepts in documents, to return correctly the transcriptions and their corresponding images for users.)
NOTE: Returning transcripts and corresponding images to users is considered a shortcut to the handwritten document.
Ngo fails to teach but Yang teaches;
an activation graph creation circuitry configured to create an activation graph based on the determined one or more conceptual terms by adding nodes and their relationships corresponding to at least of:
terms corresponding to the recognized entity in one or more languages, one or more synonym terms corresponding to the recognized entity, one or more abbreviation terms corresponding to the recognized entity, or one or more internally defined terms corresponding to the recognized entity;
([pg. 3] Responding to the medical knowledge question answering system on the server, firstly dividing the question sentence input by the user through the jieba word, identifying the naming entity and the entity relation in the question sentence, further combining the syntax dependent tree to convert the natural language question sentence into the semantic query graph;)
NOTE: Teaches creating the activation graph (the query graph taught by Yang is considered to be an activation graph as it activates additional conceptually relevant nodes, further explained later) by adding nodes and their relationships corresponding to at least one or more internally defined terms corresponding to the recognized entity (adds recognized entities from the query sentence with their internally defined corresponding entities [other recognized entities from the query sentence connected to the recognized entity via internally defined relations] ).
an associative searching circuitry configured to perform an associated searching for obtaining one or more search results based on matching of the one or more nodes of the activation graph with one or more nodes of the knowledge graph;
([pg. 3]; using the sub-graph matching mode for answer retrieval;)
NOTE: Teaches associated searching circuitry for obtaining one or more search results (the sub-graph matching mode is used for answer retrieval, thereby obtaining at least one search result)
([pg. 4] for each node in the semantic query graph, constructing the node candidate set matched with the semantic query graph in the existing medical knowledge graph; starting from the node candidate set; using dynamic programming method to traverse the medical knowledge map; finding the sub-graph most likely to match.)
NOTE: The subgraph matching mode explained here matches nodes of the activation graph (query graph) with the knowledge graph to obtain relevant nodes corresponding to the activation graph (query graph), which then spread to other conceptually related nodes (here, activation of other nodes is taught based on relation to other nodes, which therefore makes the query graph an activation graph) to obtain search results (the closest matching sub-graph).
This therefore teaches performing associative searching (via the sub-graph matching mode) for obtaining one or more search results (closest sub-graph) based on matching of the one or more nodes of the activation graph (query graph) with one or more nodes of the knowledge graph.
OBVIOUSNESS TO COMBINE YANG WITH NGO:
Yang and Ngo are analogous to each other and to the present disclosure as they all pertain to semantic searching utilizing a knowledge graph. Specifically, Yang pertains to answering a query utilizing a query graph and a knowledge graph.
Additionally, Ngo further states;
([Abstract] This paper proposes a semantic search engine for full-text retrieval of historical handwritten document images based on named entity (NE), keyword (KW) and knowledge graph (KG). This would help not only in processing, storing and indexing automatically, but also would allow users to access quickly and retrieve efficiently manuscripts.)
NOTE: Ngo discloses that semantic searching of manuscripts using their system which includes NE, KW, and KG is effective in terms of efficient processing, storing, and accessibility.
The disclosure of Yang already utilizes named entities, keywords, and knowledge graphs, so it would be simple to substitute the searching method utilized in the system disclosed by Ngo with the searching method disclosed by Yang.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of knowledge graph searching disclosed by Yang to enable the implementation of the efficient semantic searching system disclosed by Ngo.
Ngo and Yang fail to teach but Wen teaches;
the handwritten documents associated with dynamic handwritten data comprising tuples having data on x-axis, data on v-axis, and pressure data
([pg. 4] each data tuple (Xt, Yt, Pt) corresponding to the coordinate position of the point T in the handwriting pen is writing at the point T, Xt, Yt, Pt is the point T the sensed pressure value)
OBVIOUSNESS:
Ngo teaches identifying terms in handwritten documents, while Wen teaches recording handwriting as structured data tuples comprising data on an x-axis, y-axis, and pressure data.
Additionally, Wen indicates that erroneous points which indicate an absence of sensed writing pressure (P=0, and therefore do not represent actual pen-contact portions of the written stroke) are disregarded, providing a more accurate data record of the written data;
([pg. 5] Note that if the data tuple Pt (Xt, Yt, Pt) is 0, then not considering the point in the algorithm.)
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to associate the handwritten documents of Ngo with the data tuples of Wen, necessary context of the handwritten documents to the natural language processing operations of Ngo, predictably improving the accuracy of recognized terms.
Claim(s) 17-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ngo (“A Semantic Search Engine for Historical Handwritten Document Images”, 2021) in view of Yang (CN 112148851 A, 12/29/2020), further in view of Wen (CN 101101706 A, 2008), further in view of Corodescu (US 20220398331 A1, 12/15/2022).
Regarding claim 17, Ngo in view of Yang teach;
The semantic searching system of claim 16,
(Using the same reasoning from claim 16)
Ngo, Yang, and Wen fail to teach but Corodescu teaches;
wherein the associative searching circuitry selects the search results based on an accessibility level of the user and visibility level of the one or more nodes and their relationships.
([0005] The technology described herein protects the privacy and security of data stored in a knowledge graph (“graph”) by enforcing visibility policies when returning property information in response to a query or other attempt to extract property information from the graph and/or about the graph. The visibility policy may be enforced against properties of knowledge-graph objects, which include nodes and edges.)
NOTE: The visibility policy of the disclosure of Corodescu is enforced against nodes and edges (relationships) of the knowledge graph.
([0028] The visibility-policy enforcement system compares an information request to applicable visibility policies. In an aspect, a query may be submitted with a security token that may identify a requestor of the query. Depending on the result of the comparison, all property information responsive the query or a portion thereof may be provided. If a portion of the responsive property information is protected by a visibility policy, then that portion may be omitted from the response, and the portion of information that is not protected by the visibility policy may be output to the requesting entity.)
NOTE: Teaches selecting the search results based on an accessibility level of the user and visibility level of the one or more nodes and their relationships (as previously mentioned, the visibility policy of the disclosure of Corodescu is enforced against nodes and relationships of the knowledge graph).
OBVIOUSNESS TO COMBINE CORODESCU WITH NGO AND YANG:
Corodescu is analogous art to the present disclosure, Ngo, and Yang as they all pertain to systems utilizing knowledge graphs. Specifically, Corodescu pertains to a knowledge graph representation with visibility properties for knowledge graph objects.
Additionally, Corodescu states;
([Abstract] The technology described herein protects the privacy and security of data stored in a knowledge graph (“graph”) by enforcing visibility policies when returning property information in response to a query or other attempt to extract property information from the graph and/or about the graph.)
NOTE: This discloses that by enforcing visibility properties on knowledge graph objects, it allows protecting certain portions of the data from un-authorized users.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the visibility properties taught by Corodescu to the objects of the semantic searching system of claim 16 to protect sensitive information contained in the handwritten data.
Regarding claim 18, Ngo in view of Yang teach;
The semantic searching system of claim 16,
(Using the same reasoning from claim 16)
Ngo and Yang fail to teach but Corodescu teaches;
wherein the visibility level of the one or more nodes and their relationships is at least one of: automatically defined based on historical visibilities of nodes and their relationships, or manually defined based on user inputs in a documents database.
([0019] The visibility policy may be enforced against properties of knowledge-graph objects, which include nodes and edges.)
NOTE: Teaches that the nodes and relationships (edges) of a knowledge graph have a set visibility.
([0027] The visibility-policy collection system provides an interface through which users may specify their visibility preferences for the properties of an object. The preferences may be used to form a visibility policy for the object. The ability to create or edit a visibility policy may be governed at different levels in the system. For example, a user with full access to a node (e.g., document, file) may be able to edit the property visibility profile. A visibility record may comprise properties governed and a user or group of users with visibility to the property.)
NOTE: Teaches facilitating the user to manually define the visibility of nodes and their relationships in the document data base (knowledge graph).
OBVIOUSNESS:
Using the same reasoning from claim 17, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the visibility properties taught by Corodescu to the objects of the semantic searching system of claim 16 to protect sensitive information contained in the handwritten data.
Regarding claim 19, Ngo in view of Yang further in view of Corodescu teach;
The semantic searching system of claim 18,
(Using the same reasoning from claim 18)
Ngo and Yang fail to teach but Corodescu teaches;
wherein based on the pre-defined visibility, the documents database includes at least: one or more public documents corresponding to the documents publicly available to each user, one or more shared documents corresponding to the documents on a subject to which the user is invited, and one or more private documents corresponding to the documents that are specific to one user.
([0020] A visibility policy may govern read-access to a property of a knowledge-graph object. A visibility policy with a default-restricted visibility may restrict access to all users, except those designated in the policy as having access. Alternatively, a visibility policy with a default-unrestricted visibility may grant access to all users, except those designated as not having access. The technology described herein may work with both default statuses.)
NOTE: Teaches the documents database (knowledge graph) including at least one or more shared documents (nodes and relationships) corresponding to the documents on a subject to which the user is invited (a group of objects pertaining to a subject can have the same visibility policy to allow only authorized users).
OBVIOUSNESS:
Using the same reasoning from claim 17, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to implement the visibility properties taught by Corodescu to the objects of the semantic searching system of claim 16 to protect sensitive information contained in the handwritten data.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ngo (“A Semantic Search Engine for Historical Handwritten Document Images”, 2021) in view of Yang (CN 112148851 A, 12/29/2020) further in view of Cheng Chen et al. (Hereinafter Cheng) (CN 115757738 A, 03/07/2023).
Regarding claim 20, Ngo in view of Yang teaches;
The semantic searching system of claim 16,
(Using the same reasoning from claim 16)
Ngo teaches;
which further comprises a direct searching circuitry configured to: pre-process the received text data by performing at least one of: tokenization, removal of stop words, removal of punctuation marks, or removal of spaces;
[pg. 4]
PNG
media_image8.png
568
1091
media_image8.png
Greyscale
NOTE: Teaches preprocessing received text data by tokenization of search terms.
Ngo, Yang, and Wen fail to teach but Cheng teaches;
and perform a direct searching by matching [taught above by Ngo] data with one or more nodes of the comprehensive knowledge graph for obtaining the one or more search results.
([pg. 5] The above-mentioned file search method based on a knowledge graph, wherein the knowledge graph described in step 1 is established by importing text data into a graph database created using neo4j)
NOTE: The disclosure of Cheng uses a knowledge graph for direct searching.
([pg. 5] in step 3, the two keywords in the keyword and its type data are recorded as word1 and word2, and the query statement is generated according to the following rules, and the The query statement is queried in the graph database to obtain the query results as result1 and result2 or result1, and the search word set list is generated according to the query result processing, and the rules are as follows: If word1 and word2 are both instance data types, then result1 is a collection of all node names that can generate a one-degree relationship between the node whose node name is word1 and its alias and the node whose name is word2, and result2 is the node whose node name is word2 and A collection of all node names whose alias and node name word1 can generate a one-degree relationship;)
NOTE: Teaches performing direct searching by matching the input words to nodes of the knowledge graph for obtaining one or more search results (word1 from the query is matched with a node from the knowledge graph having the name word1, to return result1 containing search results).
OBVIOUSNESS TO COMBINE CHENG WITH NGO AND YANG:
Cheng, Ngo, and Yang are all analogous to each other and the present disclosure as they all pertain to searching using knowledge graph. Specifically, Cheng pertains to a file searching method and system based on knowledge graph.
Additionally, Cheng states;
([pg. 4] Knowledge graph is a data storage method based on graph database, which builds a knowledge network through the node-relationship-node model. With the development of artificial intelligence technology, it has become a trend to use knowledge graph search to answer or solve problems; in actual work, files are often an important way to carry content, so it is very important to use knowledge graphs to quickly retrieve files)
NOTE: This discloses that knowledge graph-based searching is an efficient means of quickly retrieving files. The present disclosure pertains to retrieving handwritten files by a user query, so a graph-based searching method would be well suited for this task.
Also, using the pre-processed text data taught by Ngo in the searching method taught by Cheng would be a simple substitution of input data.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use the method of knowledge graph-based searching method taught by Cheng using the preprocessed input taught by Ngo in order to provide an efficient means of direct searching of the handwritten documents of the present disclosure.
Response to Arguments
In review of Applicant’s amendments, filed 06/08/2026, the rejections from the previous office action made under 35 U.S.C. 112 have been withdrawn.
In review of Applicant’s amendments, filed 06/08/2026, the abstract idea rejections from the previous office action made under 35 U.S.C. 101 have been withdrawn.
Applicant's arguments filed 06/08/2026 regarding 35 U.S.C. 103 have been fully considered but they are not persuasive.
Starting on page 3, the applicant states;
“Ramin, however, does not teach or suggest that the dynamic handwritten data comprises "data on x-axis, data on y-axis, and pressure data" as explicitly recited in claims 1, 10 and 16, as amended.
Based on the foregoing, applicant respectfully submits that claims 1, 10 and 16 are now allowable over Ngo and Ramin.
The rest of the prior art does not cure the deficiencies of Ngo and Ramin and, therefore, claims 1, 10 and 16 are allowable over the prior art of record in any combination.
Claims 2-4, 6-9, 11-15 and 17-20 are allowable for their dependency from allowable base claims and for additional subject matter respectively recited therein.”
However, as cited in the current rejection of claims 1, 10, and 16, Wen explicitly teaches dynamic handwritten data comprising data on an x-axis, data on a y-axis, and pressure data.
Thus, with the addition of the reference Wen teaching the subject matter introduced in the amendments, the rejections under 35 U.S.C. 103 stand.
CONCLUSION
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Matthew Alan Cady whose telephone number is (571) 272-7229. The examiner can normally be reached Monday - Friday, 7:30 am - 5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached on (571)272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC)
at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW ALAN CADY/ Examiner, Art Unit 2145
/CESAR B PAULA/ Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights