Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 04/13/2026 has been entered.
Response to Amendment
This is in response to applicant’s amendment/response filed on 03/06/2026, which has been entered and made of record. Claims 1, 10, 11 have been amended. No claim has been cancelled. Claims 1-13 are pending in the application.
Response to Arguments
Applicant's arguments filed on 03/06/2026 regarding claims rejection under 35 U.S.C 103 have been fully considered but they are not persuasive.
Applicant submits “Wang similarly fails to disclose comparing past and present videos to detect an unfilled entry.” (Remarks, Page 11, third paragraph.)
The examiner disagrees with Applicant’s premises and conclusion. Wang teaches pattern matching during the filling of a form in ¶0073, ¶101, ¶104, ¶107. INOMATA also teaches during the filling of an entry on the document in abstract and page 10.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5-6, 9-11 are rejected under 35 U.S.C. 103 as being unpatentable over Daher et al. (US Pub 2019/0258656 A1) in view of Wang et al. (US Pub 2017/0220540 A1) and INOMATA KOSHIRO (JP 2021-114040 A1).
As to claim 1, Daher discloses an information processing apparatus comprising: a processor configured to:
acquire a (Daher, ¶0003, “documents and forms (e.g., W2 or other tax forms and documents, patent filing and examination related forms and documents, immigration related forms and documents, etc.) are acquired in digital form” “The digital documents can contain user inputs such as personal information, a signature, a photo, etc.” ¶0100, “The user device 700 may include camera 709, which includes a lens and shutter for capturing photographs of the surroundings around the user device 700. In an embodiment, the camera 709 captures surroundings of an opposite side of the user device 700 from the user. The images of the captured photographs can be displayed on the display panel 300.”);
compare a past (Daher, ¶0005, “comparing the stored forms with the digital document under consideration.” ¶0059, “The server 200 further compares the extracted features with the candidate master documents”);
display the unfilled entry of the document in the (Daher, ¶0006, “perform a text search using the extracted textual content on an indexed master document database to identify one or more master documents that are similar, within a pre-determined threshold, to the digital document, generate a candidate master document list using the one or more master documents identified based on the text search” “identify a master document of the one or more master documents that matches the digital document based on the comparison of the features.” ¶0028, “An unfilled form or document used to collect data or a document used as an exemplary for a kind of documents is referred as a master document.” “An unfilled W2 form can be referred as a master document of W2. Similarly, unfilled ADS can be referred to as a master document of ADS.” ¶0032, “display the identified master document on a display 300.”).
Daher does not explicitly disclose the digital format is a video and during filling of an entry on the document. However, a digital video is well known to one of ordinary skill in the art. The claim would have been obvious because a particular known technique was recognized as part of the ordinary capabilities of one skilled in the art.
Wang teaches a video input (Wang, ¶0051, “The input data may come from different sources such as text, scanned or photographed images, audio files or streams, video files or streams, etc. and may be recognized, extracted, and aggregated through data aggregation or fusion (e.g., by using the one or more data aggregation modules 110).”).
Wang also teaches during filling of an entry on the document (Wang, ¶0073, “The one or more data collection modules 104 may include character recognition module 204C to recognize characters in a computer file by performing a pattern matching process that compares an image to a stored glyph on a pixel-by-pixel basis.” ¶0101, “The predicted completion candidate populated into the field identified at 314B may be optionally presented to the user for confirmation at 322B before the user provides any input at all in some embodiments. A user's input may be received at 324B for the field via an input device. For example, the user may start typing the user's input for the field with a computer keyboard or start speaking the user's input to a microphone operatively connected to the user's computing device at 324B.” ¶0107, “the consistency checking process may include comparing recognized contents to determine whether there exist any discrepancies in the recognized contents. For example, a consistency checking process may compare two names extracted from two separate electronic files and determine whether these two names are identical or different.” Fig. 3C, ¶0104, “The list or the first list of one or more completion candidates may be updated at 334B when it is determined that the list or the first list is to be updated at 332B. When the list or the first list is updated, the updated list of one or more completion candidates for the field of interest may be presented at 336B in the user interface to the user. The process may then return to 338B to identify one or more next fields and repeat the processes for automatically filling or completing these one or more next fields.”)
Daher and Wang are considered to be analogous art because all pertain to document processing. It would have been obvious before the effective filing date of the claimed invention to have modified Daher with the features of “a video input” and comparing document during filling of an entry on the document as taught by Wang. The claim would have been obvious because a particular known technique was recognized as part of the ordinary capabilities of one skilled in the art.
Daher does not explicitly disclose detect an unfilled entry of the document as a region that remains unchanged between the past video and the present video.
INOMATA teaches detect an unfilled entry of the document as a region that remains unchanged between the past video and the present video and during filling of an entry on the document (INOMATA, abstract, “identifying an unfilled part in an accepted electronic document by comparison with an electronic document belonging to a determined group is provided.” Page 2, “identifies an unfilled portion by a difference between each image of an electronic document belonging to a determined group and an image of an received electronic document.” Page 3, “the identification unit 35 identifies the unfilled portion by the difference between each image of the electronic document belonging to the group determined by the determination unit 34 and the image of the received electronic document. That is, the difference between the image of the received electronic document and each image of the electronic document belonging to the determined group is created. This difference extracts regions that differ from each other in these images. Then, in the received electronic document, the area where the difference is negative is set as an unfilled part.” Page 10, “FIG. 5 (a) is an image of the received electronic document, and FIG. 5 (b) is an image of the electronic document belonging to the determined group. Further, FIG. 5 (c) shows an unfilled area extracted based on the image of FIG. 5 (a) and the image of FIG. 5 (b). Here, as shown in FIG. 5A, the received electronic document includes an unfilled portion M1 and an filled-in portion K1 filled in by the user. Further, in FIG. 5B, the unfilled portion M1 is entered as the entered portion K2. In addition, in FIG. 5 (b), the entry point K1 of FIG. 5 (a) is similarly filled in and is shown as the entry point K3. Then, by obtaining the difference between these images, as shown in FIG. 5 (c), the unfilled portion M1 in FIG. 5 (a) is extracted as a region where the difference becomes negative. The blank portion M1 can be presented to the user by, for example, being surrounded by a frame and being displayed in red or highlighted”). Video is taught by Wang.).
Daher, Wang and Inomata are considered to be analogous art because all pertain to document processing. It would have been obvious before the effective filing date of the claimed invention to have modified Daher with the features of “detect an unfilled entry of the document as a region that remains unchanged between the past video and the present video” and comparing document during filling of an entry on the document as taught by Inomata. The suggestion/motivation would have been in order to confirm the presence or absence of an unfilled part (Inomata, Page 5).
As to claim 2, claim 1 is incorporated and the combination of Daher, Wang and Inomata disclose
receive a format identifying the entry of the document (Daher, ¶0032, “The form recognition server 200 can implement a server-centric architecture. The form recognition server 200 can be configured to receive the digital document 120, extract a textual content (and/or graphical content) from the digital document 120, identify a master document corresponding to the digital document 120 based on the textual content (and/or graphical content), and display the identified master document on a display 300. The form recognition server 200 can be further configured to identify and extract various features from the digital document 120. The features can be characteristic properties of the master document or of the document 100 being considered that can be further supplemented with associations such as location on a page, section within which the feature was identified, type of feature, etc.” ¶0038-0040.); and
output the entry identified by the format of the document in the present video (Daher, ¶0054, “Once the digital document 120 is identified, optionally, the server 200 can display a master document with highest ranking within the candidate master document list on the display 300. The user can further confirm or request the server 200 to display a next document from the candidate list.”).
As to claim 3, claim 2 is incorporated and the combination of Daher, Wang and Inomata disclose
The format is received based on an identification notification representing information identifying the format, the identification notification having been photographed (Daher, ¶051, “the server 200 can compare one or more specific features such as a bar code, a form name or a location of a logo. The specific feature can be predetermined and is customizable based on the particular application and stored in the server 200. The extracted features can be converted to a format similar to that of the master documents stored within the indexed master document database 400 for faster and more accurate comparison. For example, the format can be a string of features, where each feature can be associated with a location, or the format can include a description for feature types, e.g., image, line, and barcode. Furthermore, each such feature can be separated by a separator such as a comma, a semicolon, slash or other appropriate symbols to form a string of features. For example, the features can be the size of an image at location L2, a barcode at location L3 acting as an identifier, etc.”); and
The processor is configured to apply to the document the format identified by the photographed identification notification (Daher, ¶0051, “the server 200 can compare one or more specific features such as a bar code, a form name or a location of a logo. The specific feature can be predetermined and is customizable based on the particular application and stored in the server 200. The extracted features can be converted to a format similar to that of the master documents stored within the indexed master document database 400 for faster and more accurate comparison. For example, the format can be a string of features, where each feature can be associated with a location, or the format can include a description for feature types, e.g., image, line, and barcode. Furthermore, each such feature can be separated by a separator such as a comma, a semicolon, slash or other appropriate symbols to form a string of features. For example, the features can be the size of an image at location L2, a barcode at location L3 acting as an identifier, etc.” ¶0052-0056.).
As to claim 5, claim 2 is incorporated and the combination of Daher, Wang and Inomata disclose
associate, in accordance with the present video, the format with information identifying the entry filled by a user (Daher, ¶0056, “the server 200 extracts data (e.g., user inputs) from the digital document 120. The data can be extracted using an OCR process or other image processing functions. The data can correspond to user input fields that can be filled (handwritten or typed) within the digital document 120. For example, the digital document 120 can be an ADS of a patent application and the data extracted can include the name “John Doe” and the address “Cool Tech, Inc., 100 S Base Street, Suite C, Charlotte, N.C. 28000 USA”.” ¶0057, “The server 200 can be further configured to use the extracted data to organize and store the information in the digital document 120 according to client details, document type, etc. The client can be a name of person (e.g., John Doe) or an organization (e.g., Cool Tech, Inc.) identified from the digital document 120. The extracted data can be stored, in the memory of the server 200, a client-specific folder or other folders based on a folder organization system. Alternatively or in addition, the extracted data can be used to identify a person in charge of handling the digital document 120 and automatically email/forward the digital document 120 to the person in charge. In another example, the digital document 120 can be annotated to hide sensitive information in the digital document 120 from none authorized people using digital marks or redaction annotations. In another example, the digital document 120 can be sorted/organized to have all pages in correct order, particularly when the digital document 120 is scanned while pages are not in order. In another example, a part of the extracted data can be used to identify location (e.g., one or more pages, i.e., not all pages) in the digital document and attach the identified pages to an application such as in a follow up procedure.” Wang, ¶0084, “A field to be completed or filled in a user interface of the application software or software service may be identified at 304.”).
As to claim 6, claim 1 is incorporated and the combination of Daher, Wang and Inomata disclose
display the region of the unfilled entry in superimposition on the present video (Daher, ¶0032, “display the identified master document on a display 300.” ¶0079, “the ADS master document 500C can be displayed on a display 300 of the server 200 or of the user device 700.”).
As to claim 9, claim 1 is incorporated and the combination of Daher, Wang and Inomata disclose
the processor is configured to, if all entries are filled, provide an instruction to perform scanning (Wang, ¶0003, “The computing platform further identifies a field to be filled or completed in the user interface of the application software or a software service. One or more automatic fill or completion modules that are stored at least partially in memory”).
As to claim 10, the combination of Daher, Wang and Inomata disclose
a non-transitory computer readable medium storing a program causing a computer to execute a process processing information, the process comprising: acquiring a present video captured, at a present time, by photographing a side of a document where an entry is filled; comparing a past video captured at a past time with the present video during filling of an entry on the document; detecting an unfilled entry of the document as a region that remains unchanged between the past video and the present video; and displaying the unfilled entry of the document in the present video (See claim 1 for detailed analysis.).
As to claim 11, the combination of Daher, Wang and Inomata discloses
an information processing method comprising: acquiring a present video captured, at a present time, by photographing a side of a document where an entry is filled; comparing a past video captured at a past time with the present video during filling of the entry on the document; detecting an unfilled entry of the document as a region that remains unchanged between the past video and the present video; and displaying the unfilled entry of the document in the present video. (See claim 1 for detailed analysis.).
As to claim 12, claim 1 is incorporated and the combination of Daher, Wang and Inomata disclose the processor is configured to display the region of the unfilled entry in superimposition on the present video in a highlighted form or a blinking form (Inomata, Page 8, “if there is an unfilled part, a notification or a warning is given to the terminal device 20b (1I).” Page 10, “FIG. 5 (a) is an image of the received electronic document, and FIG. 5 (b) is an image of the electronic document belonging to the determined group. Further, FIG. 5 (c) shows an unfilled area extracted based on the image of FIG. 5 (a) and the image of FIG. 5 (b). Here, as shown in FIG. 5A, the received electronic document includes an unfilled portion M1 and an filled-in portion K1 filled in by the user. Further, in FIG. 5B, the unfilled portion M1 is entered as the entered portion K2. In addition, in FIG. 5 (b), the entry point K1 of FIG. 5 (a) is similarly filled in and is shown as the entry point K3. Then, by obtaining the difference between these images, as shown in FIG. 5 (c), the unfilled portion M1 in FIG. 5 (a) is extracted as a region where the difference becomes negative. The blank portion M1 can be presented to the user by, for example, being surrounded by a frame and being displayed in red or highlighted”).
As to claim 13, claim 1 is incorporated and the combination of Daher, Wang and Inomata disclose the processor is configured to delete the region of the unfilled entry, which is displayed in superimposition on the present video, when the unfilled entry is filled by a user (Inomata, Page 12, “if there is an unfilled portion in step 104 (Yes in step 104), a warning is issued to the terminal device 20 (step 106). This warning notifies the user of the blank part. As a warning, for example, an electronic document may be displayed, and an unfilled portion may be displayed by coloring it in red or enclosing it in a red frame.” Page 15, “the user can be made to fill in the blank parts. Furthermore, the accuracy of grouping is further improved by replenishing the unfilled parts and eliminating the unfilled parts.”).
Claims 4, 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Daher et al. (US Pub 2019/0258656 A1) in view of Wang et al. (US Pub 2017/0220540 A1), INOMATA KOSHIRO (JP 2021-114040 A1) and Siegel (US Patent 6, 026, 187)
As to claim 4, claim 2 is incorporated and the combination of Daher and Wang does not disclose
the processor is configured to: receive a designation of the entry designated by a user; and associate the format with information identifying the designated entry.
Siegel teaches receive a designation of the entry designated by a user (Siegel, Col 4, lines 58-67, “an operator manually inputting required fill-in information into the second document D2, and specifically into areas of D2 that are overlaying the fill-in areas of the first document D1. With the image of the unfilled original form displayed as D1 directly underneath the blank, but aligned, second document D2 on the user's computer screen 26, the user can then begin typing the required fill-in information onto D2 using keyboard 28. To the user it would appear as if D1 the original form is being filled-in electronically. Actually, the user is typing onto the second electronic document D2 which is stored separately from the underlying document D1. The net effect is much like placing a transparency on top of the original form and filling-in the required information onto the transparency in areas thereof directly over the fill-in areas of the original form.”); and
associate the format with information identifying the designated entry (Siegel, Col 4, lines 58-67, “an operator manually inputting required fill-in information into the second document D2, and specifically into areas of D2 that are overlaying the fill-in areas of the first document D1. With the image of the unfilled original form displayed as D1 directly underneath the blank, but aligned, second document D2 on the user's computer screen 26, the user can then begin typing the required fill-in information onto D2 using keyboard 28. To the user it would appear as if D1 the original form is being filled-in electronically. Actually, the user is typing onto the second electronic document D2 which is stored separately from the underlying document D1. The net effect is much like placing a transparency on top of the original form and filling-in the required information onto the transparency in areas thereof directly over the fill-in areas of the original form.”).
Daher, Wang, Inomata and Siegel are considered to be analogous art because all pertain to document processing. It would have been obvious before the effective filing date of the claimed invention to have modified Daher with the features of “receive a designation of the entry designated by a user; and associate the format with information identifying the designated entry.” as taught by Siegel. The claim would have been obvious because the technique for improving a particular class of devices was part of the ordinary capabilities of a person of ordinary skill in the art, in view of the teaching of the technique for improvement in other situations.
As to claim 7, claim 6 is incorporated and the combination of Daher, Wang, Inomata and Siegel disclose the processor is configured to modify a display form of the region of the unfilled entry in accordance with a type of the unfilled entry (Daher, ¶0055, “The alignment can involve modifying the size, orientation or performing other image processing functions on the digital document 120 and/or the recognized master document.” ¶0057, “The server 200 can be further configured to use the extracted data to organize and store the information in the digital document 120 according to client details, document type, etc.” Siegel, Col 4, lines 58-67, “an operator manually inputting required fill-in information into the second document D2, and specifically into areas of D2 that are overlaying the fill-in areas of the first document D1. With the image of the unfilled original form displayed as D1 directly underneath the blank, but aligned, second document D2 on the user's computer screen 26, the user can then begin typing the required fill-in information onto D2 using keyboard 28. To the user it would appear as if D1 the original form is being filled-in electronically. Actually, the user is typing onto the second electronic document D2 which is stored separately from the underlying document D1. The net effect is much like placing a transparency on top of the original form and filling-in the required information onto the transparency in areas thereof directly over the fill-in areas of the original form.”).
As to claim 8, claim 6 is incorporated and the combination of Daher, Wang, Inomata and Siegel disclose the processor is configured to, if a shield is photographed overlapping the region of the unfilled entry, display the region of the unfilled entry in superimposition on the shield (Siegel, Col 4, lines 58-67, “an operator manually inputting required fill-in information into the second document D2, and specifically into areas of D2 that are overlaying the fill-in areas of the first document D1. With the image of the unfilled original form displayed as D1 directly underneath the blank, but aligned, second document D2 on the user's computer screen 26, the user can then begin typing the required fill-in information onto D2 using keyboard 28. To the user it would appear as if D1 the original form is being filled-in electronically. Actually, the user is typing onto the second electronic document D2 which is stored separately from the underlying document D1. The net effect is much like placing a transparency on top of the original form and filling-in the required information onto the transparency in areas thereof directly over the fill-in areas of the original form.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YU CHEN whose telephone number is (571)270-7951. The examiner can normally be reached on M-F 8-5 PST Mid-day flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YU CHEN/Primary Examiner, Art Unit 2613
Prosecution Insights