DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims as a whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea. The claims are directed to the abstract idea (i.e. obtaining/ processing/identifying a set of nearest neighbor candidate documents/ finding the nearest neighbor document of the set of nearest neighbor candidate documents/ extracting entities), which is a method of mental process because each limitation can be performed by a human being and (i. e., extracting entities from the query document based on the labeled regions of interest), which is nothing more than a mathematical concept. The additional elements or combination of elements in the claims other than the abstract idea per se amounts to no more than: recitation of generic computer structure that serves to perform generic computer functions that are well-understood, routine, and conventional activities previously known to the pertinent industry. Viewed as a whole, these additional claim element(s) do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Therefore, the claims are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim 11 is rejected for the same reason as claim 1.
None of the dependent claims has a limitation that amounts to a significantly more than abstract idea. Therefore, claims 2-10 and 12-20 are also rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-9, 11-13 and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (U.S. 2020/0167558) in view of Fujimoto et al. (U.S. 2023/0010202).
With regard to claim 1, Yang teaches a method for document entity extraction ([abstract] categorize text regions of an electronic document into document object types based on a combination of semantic information and appearance information from the electronic document), the method comprising:
obtaining a query document ([0028] the structural information allows users to sort tables based on different columns or to query the data contained in the tables; [0107] specific queries are used to increase the diversity of downloaded tables within training data 116);
processing the query document using Optical Character Recognition (OCR) ([0023] Text contained within an electronic document might be obtained using, for example, optical character recognition (“OCR”) techniques);
identifying a set of nearest neighbor candidate documents for the query document from a document gallery of candidate documents using text-embedding distance ([0067] The text mapping generator 190 might use different models to create the textual feature representation 240 for the words within a sentence…The skip-gram model determines a vector for each word and uses a prediction model to predict the next word in the sentence to create the vector representation…the training objective is to find a N-dimensional (N<<V) vector representation for each word that is useful for predicting the neighboring words);
finding the nearest neighbor document of the set of nearest neighbor candidate documents, the nearest neighbor document having labeled regions of interest ([0067] The text mapping generator 190 might use different models to create the textual feature representation 240 for the words within a sentence…The skip-gram model determines a vector for each word and uses a prediction model to predict the next word in the sentence to create the vector representation…the training objective is to find a N-dimensional (N<<V) vector representation for each word that is useful for predicting the neighboring words); and
extracting entities from the query document based on the labeled regions of interest (Fig. 4; Fig. 6; [0061] The document 400, which can be a vector graphics document, is an example of an electronic document 201 that is segmented by the page segmentation application 106; [0074] The encoder 506 performs one or more feature-extraction operations that generate visual feature representations 270 from an input image; [0083] The page segmentation application 106 might also perform a post-processing algorithm. The post-processing algorithm helps to improve the results of the page segmentation application 106 by improving the classification). However, Yang does not specifically teach:
- using RANSAC
Fujimoto teaches recognizing handwritten information in a genealogical record such as an image of a physical form having a structured layout, fields, and handwritten information [abstract]. Fujimoto also teaches using an algorithm such as a Random Sample Consensus (“RANAC”) or Universal RANAC (“USAC”), which may be utilized as the robust estimator to randomly sample the filtered keypoints and generate a transform as a consensus between the sampled keypoints [0152]. Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to have modified the system and method of categorizing text regions of an electronic document into document object types taught by Yang, with the handwritten information using a Random Sample Consensus (“RANAC”) as taught by Fujimoto, to have achieved a system and method of deriving the page segmentation from both the visual appearance of the document and the text of the document.
With regard to claim 2, the limitations are addressed above and Yang teaches wherein finding the nearest neighbor document ignores unique OCR words in the processed query document ([0067] the text mapping generator 190 uses a skip-gram model to learn the textual feature representation 240. The skip-gram model determines a vector for each word and uses a prediction model to predict the next word in the sentence to create the vector representation).
With regard to claim 3, the limitations are addressed above and Yang teaches wherein extracting the entities from the query document ignores words that are in both the query document and the nearest neighbor document related to the labeled regions of interest ([0067] the text mapping generator 190 uses a skip-gram model to learn the textual feature representation 240. The skip-gram model determines a vector for each word and uses a prediction model to predict the next word in the sentence to create the vector representation. If V is the number of words in a vocabulary, and w is a V-dimensional, one-hot vector representing a word, the training objective is to find a N-dimensional (N<<V) vector representation for each word that is useful for predicting the neighboring words).
With regard to claim 5, the limitations are addressed above and Yang teaches further comprising for document entity extraction ([abstract] categorize text regions of an electronic document into document object types based on a combination of semantic information and appearance information from the electronic document) by:
uploading a set of representative document samples ([0108] In particular, because the page segmentation model relies on the textual information to classify objects, the content of text regions (paragraph, section heading, list, caption) must be carefully selected. For example, for paragraphs, sample sentences are randomly selected from a dump of Wikipedia data. Section headings are generated by sampling sentences and phrases that are section or subsection headings in the “Contents” block in a Wikipedia page);
running an Optical Character Recognition (OCR) application ([0023] Continuing with the example, the page segmentation application also uses the text from the electronic document to provide additional context and therefore improve the page segmentation. Text contained within an electronic document might be obtained using, for example, optical character recognition (“OCR”) techniques) on those representative document samples to generate the text and associating bounding boxes (Fig. 4; Fig. 6; [0072] The visual feature representation 270 includes an identification of the features within an image necessary for the identification of the visual objects within the document. For example, an image of a document might depict text from the document that is arranged in a certain manner, such as text boxes, captions, etc.); and
storing the labeled documents in the document gallery ([0042] The workspace, the projects or the assets are stored as application program data 122 in the data storage unit 112 by a synchronization engine 146; [0049] the user installs various applications supported by the creative apparatus 104 via an application download management engine 144. Application installers or application programs 108 (which might include the content manipulation application 110 or other software usable to perform operations described herein) are present in the data storage unit 112 and are fetched by the application download management engine 144). However, Yang does not specifically teach:
- preparing template documents
- de-skewing the documents with estimated transformation using the bounding boxes;
- labeling entity regions of interest (ROIs) in the de-skewed documents;
Fujimoto teaches recognizing handwritten information in a genealogical record such as an image of a physical form having a structured layout, fields, and handwritten information [abstract]. Fujimoto teaches preparing template documents ([0011] comparing the first set of keypoints to a second set of keypoints in a template record having template area coordinates; generating a transform based on the first and second set of keypoints; and generating divided area coordinates by applying the transform to the template area coordinates in the template record), de-skewing the documents with estimated transformation using the bounding boxes ([0018] the handwriting recognition model is configured to rotate the second collection of handwriting written in the second direction; [0050] the image may be distorted, rotated and otherwise changed from a standard template form so that the areas in the physical form do not completely align with the template form), as well as labeling entity regions of interest (ROIs) in the de-skewed documents ([0148] The generated transform may then be applied to the areas for the query image 704, e.g. to generate query image region of interest coordinates 724, allowing the data in the query image 704 to be properly, automatedly, and efficiently extracted; [0162] The computing server 130 may generate 790 divided area or region-of-interest coordinates for the query image by applying the transform to the template image template area coordinates… downstream models, such as handwriting recognition models, may retrieve data from the query image using the generated region-of-interest coordinates and properly associate the extracted data with an information field). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to have modified the system and method of categorizing text regions of an electronic document into document object types taught by Yang, with the handwritten information and template documents as taught by Fujimoto, to have achieved a system and method of deriving the page segmentation from both the visual appearance of the document and the text of the document.
With regard to claim 6, the limitations are addressed above and Yang teaches wherein the ROIs are two-dimensional bounding boxes ([0066] a document image of size H×W pixels in which one or more pixels are mapped to N-dimensional vectors (i.e., textual feature representations), the text mapping generator 190 generates a textual mapping of size H×W×N) that will contain document content (e.g., text values, checkboxes, signatures, etc.) ([0066] The size of a text mapping with various textual feature representations 240 depends on the number of pixels in the electronic document 201 and the depth of the generated vectors (i.e., textual feature representations 240)).
With regard to claim 7, the limitations are addressed above and Yang teaches wherein the labeling is done using a human labeler ([claim 5] receiving a human-generated sentence; receiving a human-generated section heading; receiving a human-generated list comprising elements from a common source; receiving a human-generated caption; and creating a training document by inserting at least one of a paragraph, a sentence, a section heading, or a caption into a human-generated unstructured vector graphics document).
With regard to claim 8, the limitations are addressed above and Yang teaches wherein the labeling is done using a heuristic function ([0005] Existing solutions for performing page segmentation on electronic documents, such as unstructured vector graphics documents, typically use complex, heuristic rules to automatically identify and tag various structural objects within the document…For instance, some existing solutions use region-based classifications involving heuristic algorithms).
With regard to claim 9, the limitations are addressed above and Yang teaches wherein the labeling is done using AI/ML ([abstract] A page segmentation application executing on a computing device provides a textual feature representation and a visual feature representation to a neural network; [0002] this disclosure relates to semantic page segmentation of vector graphics documents (e.g., document files in Adobe® Portable Document Format, or PDF) or other electronic documents via machine learning techniques that derive the page segmentation from both the visual appearance of the document and the text of the document).
With regard to claim 11, the system claim corresponds to the method claim 1, respectively, and therefore is rejected with the same rationale.
With regard to claim 12, the system claim corresponds to the method claim 2, respectively, and therefore is rejected with the same rationale.
With regard to claim 13, the system claim corresponds to the method claim 3, respectively, and therefore is rejected with the same rationale.
With regard to claim 15, the system claim corresponds to the method claim 5, respectively, and therefore is rejected with the same rationale.
With regard to claim 16, the system claim corresponds to the method claim 6, respectively, and therefore is rejected with the same rationale.
With regard to claim 17, the system claim corresponds to the method claim 7, respectively, and therefore is rejected with the same rationale.
With regard to claim 18, the system claim corresponds to the method claim 8, respectively, and therefore is rejected with the same rationale.
With regard to claim 19, the system claim corresponds to the method claim 9, respectively, and therefore is rejected with the same rationale.
Claims 4, 10, 14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (U.S. 2020/0167558) in view of Fujimoto et al. (U.S. 2023/0010202) and further in view of Holub et al. (U.S. 2024/0412188).
With regard to claim 4, the limitations are addressed above and Yang teaches further comprising:
including labels associated with the labeled regions of interest ([0067] The text mapping generator 190 might use different models to create the textual feature representation 240 for the words within a sentence…The skip-gram model determines a vector for each word and uses a prediction model to predict the next word in the sentence to create the vector representation…the training objective is to find a N-dimensional (N<<V) vector representation for each word that is useful for predicting the neighboring words) and corresponding entities extracted from the query document (Fig. 4; Fig. 6; [0061] The document 400, which can be a vector graphics document, is an example of an electronic document 201 that is segmented by the page segmentation application 106; [0074] The encoder 506 performs one or more feature-extraction operations that generate visual feature representations 270 from an input image; [0083] The page segmentation application 106 might also perform a post-processing algorithm. The post-processing algorithm helps to improve the results of the page segmentation application 106 by improving the classification). However, Yang and Fujimoto do not specifically teach:
- generating a JSON output document
Holub teaches a composite image frame produced from a sequence of captured image frames and different regions in the composite image are derived from different ones of the image frames in the captured sequence [abstract]. Holub also teaches generating a JSON output document ([0132] expresses the information as JSON data for sending to a consumer of the data; [0135] The ScanScene function performs one pass through the depicted modules-using the most-current motion-aware averaged camera frame, and outputting a JSON with object mask information, and the watermark/barcode payloads (if any) associated with each mask; [0137] resulting JSON then conveys output information for four scenes). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to have modified the system and method of categorizing text regions of an electronic document into document object types taught by Yang and using a Random Sample Consensus (“RANAC”) as taught by Fujimoto, with a system for producing a composite image frame outputting a JSON with object mask information as taught by Holub, to have achieved a system and method of deriving the page segmentation from both the visual appearance of the document and the text of the document including a composite image frame outputting a JSON.
With regard to claim 10, the limitations are addressed above and Yang teaches wherein the labeled documents are stored in the document gallery ([0042] The workspace, the projects or the assets are stored as application program data 122 in the data storage unit 112 by a synchronization engine 146; [0049] the user installs various applications supported by the creative apparatus 104 via an application download management engine 144. Application installers or application programs 108 (which might include the content manipulation application 110 or other software usable to perform operations described herein) are present in the data storage unit 112 and are fetched by the application download management engine 144). However, Yang and Fujimoto do not specifically teach:
- using a JSON structure data format
Holub teaches a composite image frame produced from a sequence of captured image frames and different regions in the composite image are derived from different ones of the image frames in the captured sequence [abstract]. Holub also teaches generating a JSON output document ([0132] expresses the information as JSON data for sending to a consumer of the data; [0135] The ScanScene function performs one pass through the depicted modules-using the most-current motion-aware averaged camera frame, and outputting a JSON with object mask information, and the watermark/barcode payloads (if any) associated with each mask; [0137] resulting JSON then conveys output information for four scenes). Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which said subject matter pertains to have modified the system and method of categorizing text regions of an electronic document into document object types taught by Yang and using a Random Sample Consensus (“RANAC”) as taught by Fujimoto, with a system for producing a composite image frame outputting a JSON with object mask information as taught by Holub, to have achieved a system and method of deriving the page segmentation from both the visual appearance of the document and the text of the document including a composite image frame outputting a JSON.
With regard to claim 14, the system claim corresponds to the method claim 4, respectively, and therefore is rejected with the same rationale.
With regard to claim 20, the system claim corresponds to the method claim 10, respectively, and therefore is rejected with the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREA C. LEGGETT whose telephone number is (571)270-7700. The examiner can normally be reached M-F 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at 571-272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREA C LEGGETT/Primary Examiner, Art Unit 2171
Prosecution Insights