DETAILED ACTION
This action is in response to the application filed on March 29th, 2024. Claims 1-20 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on March 29th, 2024 is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 6-13, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over “Extracting social networks and contact information from email and the Web” (herein after referred to by its primary author, Culotta) in view of “Automated forms processing software and services” (herein after referred to by its primary author, Gopisetty), “Scene Text Recognition with Semantics” (herein after referred to by its primary author, Placidi), and US10834286 (herein after referred to by its primary author, Wai).
In regards to claim 1, Culotta teaches a computer-implemented method comprising: a) responsive to receipt of an input Culotta Section 2 “The system’s input is the set of email messages in a user’s inbox… 1. Person name extraction. Names are extracted from the headers of email messages by first locating the header of the email, and then using a set of patterns to find people’s names and email addresses.”); b) responsive to identification of one or more text and/or data fields containing desired Culotta Section 2 “1. Person name extraction. Names are extracted from the headers of email messages by first locating the header of the email, and then using a set of patterns to find people’s names and email addresses.”); d) using the constructed key to access a dictionary of company information (Culotta Section 2 “The output is an automatically-filled address book of people and their contact information, with keywords describing each person, and links between people defining the user’s social network” Examiner note: In Section 1, a person’s name (analogous to the constructed key) is extracted from an email. This constructed key is used to check within the address book.); and e) responsive to a match between at least one record in the dictionary and the constructed key, Culotta Section 2 “2. Name coreference. A set of string matching rules are used to resolve multiple mentions of the same person. For example, we create rules that will merge people with the names ‘Joseph Conrad’ and ‘J Conrad’.” Examiner note: When a name is found that already exists in the address book (in this example, J. Conrad is queried and Joseph Conrad is found to already exist), the matching information is stored in the address book alongside the original information).
Culotta fails to teach an input scanned financial document; identification of one or more text and/or data fields containing desired financial information; c) wherein at least one of the identified one or more text and/or data fields has at least some of its information obscured or otherwise illegible or undecipherable, and wherein the constructing comprises using the company and/or financial information that is not obscured or is otherwise legible or decipherable; and extracting and displaying the record.
Gopisetty teaches an input scanned financial document (Gopisetty Figure 1 “Input forms”); and identification of one or more text and/or data fields containing desired financial information (Gopisetty Figure 1 “Form dropout”; System overview section “The next task is to extract images of fields that are to be recognized. The input image is carefully registered (i.e., aligned) with its matching blank form (the template image), which is stored in the system during the forms training phase. The template image is then subtracted from the registered input image, leaving only the filled in data. This is done in the "form-dropout" stage.”).
Gopisetty is considered to be analogous to the claimed invention because they are both in the same field of form image processing. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Culotta to include the teachings of Gopisetty, to provide the advantage of image compression (Gopisetty System Overview “This process helps in compressing the image significantly, since the dropped-out image has far fewer black pixels than the original filled-in image. The dropped-out image is stored in an image database for display and archival purposes. Notice that an equivalent of the original image can be displayed by overlaying the dropped-out image with the template image.”)
Furthermore, Placidi teaches c) wherein at least one of the identified one or more text and/or data fields has at least some of its information obscured or otherwise illegible or undecipherable, and wherein the constructing comprises using the company and/or financial information that is not obscured or is otherwise legible or decipherable (Placidi Section 1 “In this work we are interested in more challenging scenarios where text contains significant distortion and obstruction to the point that some characters can be completely obscured from the image, we refer to this as hard text. Figure 1 shows examples of each text category.”)
Placidi is considered to be analogous to the claimed invention because they are both in the same field of image processing with blocked text. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Culotta in view of Gopisetty to include the teachings of Placidi, to provide the advantage of a system which considers the whole context of an image when performing OCR of blocked text (Placidi Section 1 “Previous STR works are focused on regular text and irregular text [1, 2, 18, 23, 24, 28, 30]. While high performance has been achieved on these tasks, past approaches struggle with imperfect and noisy input images (hard text). In this work we look at common failure cases in STR models and how models can be designed to be more robust to them. Conventional approaches to STR [1, 3, 19, 35] only consider the cropped text image when making predictions and discard the larger but useful scene, which implicitly provides information that can aid STR, particularly on hard text samples.”)
Lastly, Wai teaches extracting and displaying the record. (Wai Figure 3 316-328 Examiner note: In step 316, OCRed characters are matched against entries in an address book. The matches are then sent to a user who can select one or more, from the displayed list of matches.)
Wai is considered to be analogous to the claimed invention because they are both in the same field of OCR. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Culotta in view of Gopisetty and Placidi to include the teachings of Wai, to provide the advantage of user notification when an OCRed key does not match a key in the database (Wai Column 4 Lines 51-60 “If desired, entries of the list of potential recipients are suitably compared against existing address information, such as information in a stored address book. Such comparison may identify discrepancies or errors between extracted destination information and stored information. A user is suitably prompted relative to the discrepancy. Depending on the discrepancy, the user may correct extracted information to account for erroneously entered information or information that was incorrectly OCRed.”)
In regards to claim 2, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 1, further comprising: f) responsive to no match between at least one record in the dictionary and the constructed key, using the constructed key to correct or add one or more entries to the dictionary (Culotta Section 2 “The output is an automatically-filled address book of people and their contact information, with keywords describing each person, and links between people defining the user’s social network... 1. Person name extraction. Names are extracted from the headers of email messages by first locating the header of the email, and then using a set of patterns to find people’s names and email addresses.” Examiner note: The first time a name is found, it is added to the address book along with any identifying information such as an email address.); and g) repeating e) (Culotta Figure 1 “Name Coreference”; Section 2 “4. Contact information and person name extraction… Newly extracted people who are coreferent to people we have already discovered are resolved as in step 2.” Examiner note: The name coreference step is repeated when a new name is found that is coreferent to an existing name in the address book. This happens after the name has been found, as seen if figure 1, bottom arrow labeled ‘names’).
In regards to claim 3, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 2, further comprising: h) repeating f) and g) until e) produces a match (Culotta Figure 1 Examiner note: In this reference, step e (name coreference) is repeated when a name is found that is coreferent to a person already discovered. Therefore, when step e is repeated, a match will be found. So, this reference teaches repeating e once, after which a match will be found and processing can continue to the next name).
In regards to claim 6, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 1, further comprising correcting the scanned financial document before a), the correcting including orienting and/or deskewing one or more pages of the financial document (Placidi Section 3.1 “Normalisation Word images are first processed by an image normalisation module which is used to transform a cropped text bounding box image X into a normalised image X. We use the thin-plate spline (TPS) variation of a spatial transformation network originally proposed by [30] and used more recently by [1, 18]. This normalisation essentially allows for subsequent modules to focus less on distortion correction and instead treat inputs as more regular in shape.”).
In regards to claim 7, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 2, wherein correction is carried out manually (Gopisetty System Overview “If the form type cannot be identified automatically, the data must be keyed in manually.”).
In regards to claim 8, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 2, wherein correction is carried out using a machine learning system (Placidi Section 3.1 “Normalisation Word images are first processed by an image normalisation module which is used to transform a cropped text bounding box image X into a normalised image X. We use the thin-plate spline (TPS) variation of a spatial transformation network originally proposed by [30] and used more recently by [1, 18]. This normalisation essentially allows for subsequent modules to focus less on distortion correction and instead treat inputs as more regular in shape.”).
In regards to claim 9, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 1, wherein the information about the company comprises company name and address information, and contact information (Culotta Section 2 “1. Person name extraction. Names are extracted from the headers of email messages by first locating the header of the email, and then using a set of patterns to find people’s names and email addresses.”).
In regards to claim 10, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 9, wherein the contact information comprises a telephone and/or facsimile number, or one or more email addresses (Culotta Section 2 “1. Person name extraction. Names are extracted from the headers of email messages by first locating the header of the email, and then using a set of patterns to find people’s names and email addresses.”).
In regards to claim 11, Culotta in view of Gopisetty, Placidi, and Wei teaches a system comprising: a processor; and non-volatile memory connected to said processor, said non-volatile memory containing instructions which, when the processor executes them, perform the following method (Wai Column 2 Lines 9-19 “In accordance with an example embodiment, a multifunction peripheral includes a scanning engine configured to scan a document, a memory configured to store the scanned document, and a processor configured to use optical character recognition (OCR) on the scanned document to determine recipient contact information and generate a list of recipients, and a user interface configured to present the list to the user and prompt the user to select a recipient from the determined recipient contact information. The processor then transmits the scanned document to the selected recipients.”) and renders obvious the remaining claim language as in the consideration of claim 1.
In regards to claim 12, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 2.
In regards to claim 13, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 3.
In regards to claim 16, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 6.
In regards to claim 17, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 7.
In regards to claim 18, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 8.
In regards to claim 19, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 9.
In regards to claim 20, Culotta in view of Gopisetty, Placidi, and Wei renders obvious the claim language as in the consideration of claim 10.
Claims 4-5 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Culotta in view of Gopisetty, Placidi, and Wei, and further in view of US6243480 (herein after referred to by its primary author, Zhao).
In regards to claim 4, Culotta in view of Gopisetty, Placidi, and Wei teaches the method of claim 1, but fails to teach wherein the key comprises information about a bank with which the company has one or more accounts, the information including at least a bank account number.
However, Zhao teaches wherein the key comprises information about a bank with which the company has one or more accounts, the information including at least a bank account number (Zhao Column 11 lines 5-13 “The bank verifies the check by detecting the watermark from the digital check., decrypting the digital signature with the payer's public key, and comparing the bank account number and the amount from the image with the bank account number and the amount on the face of the check. A digital check can be used in either electronic form or paper form. In the latter case, a scanner (including OCR technology and watermark reader) is needed to read the watermark from the paper check.” Examiner note: This references teaches that bank account information can be OCRed and compared against other data. This reference, when considered in combination with Culotta in view of Gopisetty, Placidi, and Wei, suggests that the OCRed bank account number could be used as a key for a database query.).
Zhao is considered to be analogous to the claimed invention because they are both in the same field of OCR. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified the system of Culotta in view of Gopisetty, Placidi, and Wai to include the teachings of Zhao, to provide the advantage of an OCR system which accounts for what is expected in certain contexts (Zhao Column 9 Lines 16-22 “For example, many errors can be eliminated if what is being read is specific fields, for example in a check or identification card, and the OCR equipment is programmed to take the nature of the field's contents into account. For example, if a field contains only numeric characters, the OCR equipment can be programmed to treat the letters o and O as the number 0 and the letters l,i, or I as the number 1. Moreover, if a match fails and the semantic information contains a character that is easily confused by the OCR equipment, the character may be replaced by one of the characters with which it is confused, the digest may be recomputed, and the match may again be attempted with the recomputed digest.”)
In regards to claim 5, Culotta in view of Gopisetty, Placidi, Wei, and Zhao teaches the method of claim 4, wherein the dictionary of company information contains bank address information and/or bank branch address information with which to match up the bank account number (Culotta Section 2 “The output is an automatically-filled address book of people and their contact information, with keywords describing each person, and links between people defining the user’s social network.”; Zhao Column 11 lines 5-13 “The bank verifies the check by detecting the watermark from the digital check., decrypting the digital signature with the payer's public key, and comparing the bank account number and the amount from the image with the bank account number and the amount on the face of the check. A digital check can be used in either electronic form or paper form. In the latter case, a scanner (including OCR technology and watermark reader) is needed to read the watermark from the paper check.” Examiner note: The address book of Culotta could include bank account information, as this could be included in their contact information of links. Zhao then suggests that a database containing financial information could be searched using a bank account number).
In regards to claim 14, Culotta in view of Gopisetty, Placidi, Wei, and Zhao renders obvious the claim language as in the consideration of claim 4.
In regards to claim 15, Culotta in view of Gopisetty, Placidi, Wei, and Zhao renders obvious the claim language as in the consideration of claim 5.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“A survey of modern optical character recognition techniques” provides an overview of current and past OCR techniques and their applications.
“Web Pay - Payee Flowcharts and Wireframes” provides an example flow of a web payment system where payees are added and stored in a database, along with tier respective account numbers.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CALEB LOGAN ESQUINO whose telephone number is (703)756-1462. The examiner can normally be reached M-Fr 8:00AM-4:00PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CALEB L ESQUINO/ Examiner, Art Unit 2677
/ANDREW W BEE/ Supervisory Patent Examiner, Art Unit 2677
Prosecution Insights