DETAILED ACTION
This FINAL action is in response to Application No. 18/540,694 filed 12/14/2023 which claims priority from JP2022-203058 filed 12/20/2022. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The amendment presented on 2/27/2026 which provides amendments to claims 1, 2, 8, and 9, and cancellation of claim 5, is hereby acknowledged. Claims 1-4 and 6-9 are currently pending.
Title
The replacement Title submitted on 2/27/2026 is accepted and entered.
Claim Rejections – Withdrawn
The previous 35 U.S.C § 112(b) rejection of claim 5 has been withdrawn as necessitated by cancelation.
Response to Arguments
Applicant’s arguments with respect to the 35 USC § 101 rejections have been considered; however, they are unpersuasive.
Applicant has not addressed the issues solely related to claim 9, and therefore, the Examiner maintains that claim 9 does not fall into one of the four statutory categories of invention.
As for all claims 1-4 and 6-9, the Examiner maintains the claims are directed toward non-statutory subject matter as abstract ideas without significantly more. As shown in the updated rejection below, the broadest reasonable interpretation of the amended claims includes abstract ideas as mental processes performable in the human mind or on paper. The new “generation” step referenced by applicant is yet another abstract idea (and not considered a practical application) because it is defined as “drawing the translated text to a layout similar to that of the input image.” A human can certainly (evaluate) translate text within an image in their mind, draw the translation in a layout similar to the image on paper, and choose characters in the translation text to assign a label.
Therefore, the Examiner maintains claims 1-4 and 6-9 recite non-statutory subject matter.
Applicant’s arguments with respect to Maloney and Klima have been considered; however, they are unpersuasive.
The rejection of the independent claims now relies on a combination of Maloney and Klima. Maloney is relied on to teach at least the acquiring of an input image, translating of text within the input image, drawing or displaying the translated text in a layout similar to the input image, and providing a text selection mechanism as shown in the rejections below. The “translation image” is defined by the claim to be “drawing the translation text to a layout similar to that of the input image.” At least Figure 78 depicts a document side-by-side with its translation. As can be readily seen from the figure, the layout of the translated text of the document in the right side is drawn almost identically to the layout of the text of the original document in the left side (e.g., the punctuation and line/paragraph spacing is substantially the same). Each side is construable as a displayed “image” of the document because the content displayed by a device is typically considered an image and the claim does not define any sort of translation image file type, thus, Maloney teaches a “translation image” as claimed. Therefore, Maloney teaches at least “acquire an input image; acquire a translation text by executing a translation process on an input text included in the input image; [and] generate a translation image by drawing the translation text to a layout similar to that of the input image.” See the rejections below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter.
Regarding claim 9, the specification provides “a storage medium (which may also be referred to more fully as a 'non-transitory computer-readable storage medium')” at [0118]. Phrased this way, the claimed “non-transitory computer-readable storage medium” has the same definition as “storage medium”. The specification is open-ended as far as what “storage medium” encompasses in the same paragraph. The ordinary meaning, of the “storage medium” includes non-statutory media (i.e. signals) which are not a Manufacture within the meaning of 101, on which the program is still unavailable for the data processing system (processor). Thus, it is the Office’s position at this time that storage mediums directed towards signals or carrier waves renders a claim deficient in view of 35 USC 101.
Claims 1-4 and 6-9 are also rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (abstract idea) without significantly more.
The independent claim(s) recite(s) at least “acquire a translation text by executing a translation process on an input text, included in the input image”, ”generate a translation image by drawing the translation text to a layout similar to that of the input image”, “select one or more consecutive characters in the translation text as a selection text, based on a user operation on the translation image”, and “assign a label selected based on a user operation, to a character string in the input text, the character string corresponding to the selection text.” The broadest reasonable interpretation of each of these limitations are construable as abstract ideas being performable in the human mind or on paper (mental processes). A human can certainly (evaluate) translate text within an image in their mind, draw the translation in a layout similar to the image on paper, and choose characters in the translation text to assign a label.
This judicial exception is not integrated into a practical application because the additional limitations of “at least one memory that stores instructions; and at least one processor that executes the instructions” are merely generic computing components with generic functions on which the instructions to implement the abstract idea are applied. The additional limitations or “acquire an input image”, “store the input text to which the label has been assigned, in a storage unit” are insignificant extra-solution activity. Additional limitations directed toward mere instructions to apply the exception to generic computing components and insignificant extra-solution activity, alone or in combination, do not integrate the judicial exception into a practical application (See MPEP§2106.05(f) and §2106.05(g)).
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements identified above, being directed toward mere instructions to apply the exception to generic computing components and insignificant extra-solution activity, alone or in combination, are well-understood routine and conventional, do not provide an inventive concept, and thus, do not amount to significantly more than the judicial exception. Therefore, the independent claims are directed toward ineligible subject matter.
The dependent claims recite additional limitations that are also construed as additional abstract ideas that are performable by a human and on paper, and are therefore, also directed toward ineligible subject matter.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-4 and 7-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Maloney et al. (US 2012/0117467 A1, hereinafter “Maloney”), and further in view of Klima (US 2017/0161246 A1).
Regarding claim 1, Maloney teaches an information processing apparatus comprising: at least one memory that stores instructions; and at least one processor that executes the instructions to:
acquire an input image; acquire a translation text by executing a translation process on an input text included in the input image. More specifically, Maloney discloses a system with a display for showing and interacting with documents (Maloney, abstract). Steps include using a scanner to acquire an image of a document and perform optical character recognition to acquire a text-recognized document (Maloney, [0145], [0277]).
generate a translation image by drawing the translation text to a layout similar to that of the input image. More specifically, Figure 74 depicts a display after a scanned document text-recognized, and then translated into a second language. The display shows two areas; one for an original text-recognized document, and the other for a translated version of the text-recognized document (Maloney, [0305]-0306]). While Figure 74 depicts the text-recognized document in different languages in a flipped orientation for users sitting across from each other, Figure 78 depicts that the different language areas can be oriented side-by-side, and the original and translated language areas are drawn in similar sentence/paragraph layouts (Maloney, [0310]).
select one or more consecutive characters in the translation text as a selection text, based on a user operation on the translation image; More specifically, highlighting text with user input in either language area will automatically highlight the associated text in the opposite language area (Maloney, [0305]-0306]).
Maloney additionally describes that the purpose of highlighting of text is to provide further understanding of conveyed information (Maloney, [0141]).
However, Maloney may not explicitly teach every aspect of
assign a label selected based on a user operation, to a character string in the input text, the character string corresponding to the selection text; and
store the input text to which the label has been assigned, in a storage unit.
Klima discloses a graphical user interface for annotating a textual document in a first language and migrating the annotation to the same document in a second language (Klima, abstract, [0002], [0007], [0022], [0024]). The annotation 312 can be associated with a selected location 307 such as string of words in a document of the first language and is stored in an annotation file (Klima, [0047]-[0048], [0057]). The process of Figure 5 describes the synchronizing of the stored annotation (labels) from the document of a first language to a document of a second language, placing the annotation (labels) at the same location in the document of the second language (Klima, [0062]-[0064]). Textual translation of documents can be performed to identify locations that match between documents of different languages during annotation synchronizing (Klima, [0067], claim 12).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Maloney and Klima that an apparatus for translating documents and selecting text in different languages would include assigning and storing a label to the selected text. With Maloney and Klima disclosing displaying and selecting text in a first language and propagating the selection to a second language, and with Klima additionally disclosing labeling/annotating text in a first language and synchronizing the annotation in the same location of the text in second language, one of ordinary skill in the art of implementing an apparatus for translating documents and selecting text in different languages would include assigning and storing a label to the selected text in order to provide a method of expanding on the understanding of conveyed information. One would therefore be motivated to combine these teachings as in doing so would create this apparatus for translating documents and selecting text in different languages.
Regarding claim 2, Maloney and Klima teach the information processing apparatus according to claim 1, wherein the at least one processor executes the instructions further to: acquire the input text and an input text area representing a coordinate range of the input text, by executing a character recognition process on the input image; and display, based on the translation text and the input text area, the translation image, on a display, wherein the selection text is selected by the user operation on the translation image displayed on the display. More specifically, Maloney discloses a system with a display for showing and interacting with documents (Maloney, abstract). Steps include using a scanner to acquire an image of a document and perform optical character recognition to acquire a text-recognized document (Maloney, [0145], [0277]). The system provides for an input device such as a pen/stylus to be able to select and highlight text in the text-recognized document (Maloney, at least [0267]). Figure 74 depicts a display after a scanned document text-recognized, and then translated into a second language. The display shows two areas; one for an original text-recognized document, and the other for a translated version of the text-recognized document. Highlighting text with user input in either language area will automatically highlight the associated text in the opposite language area (Maloney, [0305]-0306]). While Figure 74 depicts the text-recognized document in different languages in a flipped orientation for users sitting across from each other, Figure 78 depicts that the different language areas can be oriented side-by-side, and the original and translated language areas are drawn in similar sentence/paragraph layouts (Maloney, [0310]).
Regarding claim 3, Maloney and Klima teach the information processing apparatus according to claim 2, wherein the at least one processor executes the instructions further to: acquire, in the translation process, a translation text area by executing translation for each input text area; and assign, in a case where the selection text matches an entire text of the translation text area to which the selection text belongs, the selected label to an entire text of the input text area corresponding to the translation text area selected based on the user operation. More specifically, Klima suggests the annotation from the first language document should end up assigned to the corresponding portion (string of words in the second language) of the second language document. Further, if the text of the second document is not found due to some change in the second document, the smallest granularity for (or closest to) the location is used for the location of the annotation in the second document (Klima, [0048]-[0050], [0062]-[0064], [0067]). Additionally, Maloney suggests that when highlighting text in a first language, the “most appropriate portion” of the translated text is also highlighted (Maloney, [0306]).
Regarding claim 4, Maloney and Klima teach the information processing apparatus according to claim 2, wherein the at least one processor executes the instructions further to: acquire, in the translation process, a translation text area by executing translation for each input text area; search for the character string corresponding to the selection text from a text of the input text area corresponding to the translation text area to which the selection text belongs; and assign, in a case where the character string corresponding to the selection text is found, the selected label to the found character string. More specifically, Klima suggests the annotation from the first language document should end up assigned to the corresponding portion (string of words in the second language) of the second language document. Further, if the text of the second document is not found due to some change in the second document, the smallest granularity for (or closest to) the location is used for the location of the annotation in the second document (Klima, [0048]-[0050], [0062]-[0064], [0067]). Additionally, Maloney suggests that when highlighting text in a first language, the “most appropriate portion” of the translated text is also highlighted (Maloney, [0306]).
Regarding claim 7, Maloney and Klima teach the information processing apparatus according to claim 2, wherein the at least one processor executes the instructions further to: display, on the display, the input image and the translation image side by side in their entirety on a same screen. More specifically, while Figure 74 depicts the text-recognized document in different languages in a flipped orientation for users sitting across from each other, Figures 78 and 79 show that the different language areas can be oriented side by side or on top of one another respectively, in all cases, the original and translated language areas are drawn in similar sentence/paragraph layouts (Maloney, [0310]).
Regarding claim 8, this claim recites the method performed by the information processing apparatus of claim 1, therefore, the same rationale of rejection is applicable.
Regarding claim 9, this claim recites a non-transitory computer-readable storage medium that stores a program of instructions for performing the steps performed by the information processing apparatus of claim 1, therefore, the same rationale of rejection is applicable.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Maloney and Klima, and further in view of Stieglitz et al. (US 2014/0100844 A1, hereinafter “Stieglitz”).
Regarding claim 6, Maloney and Klima teach the information processing apparatus according to claim 2, however, may not explicitly teach every aspect of wherein the at least one processor executes the instructions further to:
display an image acquired by trimming from the input image the input text area corresponding to a translation text area to which the selection text belongs, at a position near the translation text area to which the selection text belongs.
Stieglitz discloses a method and system for displaying translated text for language learning having content in a first language and a translation thereof in a second language. A user can hover over the content with a mouse to display a popup box containing a content component in the second language corresponding to a content component in the first language. The content components are highlighted as the user hovers over them with the mouse to identify corresponding content components between the first language and the second language (Stieglitz, abstract). The first/target language document and the second/source translated phrase are referred to as rendered images (Stieglitz, at least [0012], [0018], claims 5 and 16). Figure 6 depicts the cursor hovering process which causes rendering of an image of a sentence (portion) of the content in a second language (a text area trimmed from the input image) near the content in a first language. Additionally, a more granular selection/highlighting occurs with the cursor hovering over specific words or phrases, referred to as content components, within the content in the first language, and the associated content components in the second language are highlighted (Stieglitz, abstract, [0007], [0057]-[0058], [0066]). The display can operate in two directions, where the first language is the original language and the second language is the translated language and vice versa as depicted in Figure 8 and associated description (Stieglitz, Figure 8, [0083]).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Klima and Maloney with Stieglitz that an apparatus for displaying text in different languages with similar formats would include display an image acquired by trimming from the input image the input text area corresponding to a translation text area to which the selection text belongs, at a position near the translation text area to which the selection text belongs. With Klima, Maloney, and Stieglitz disclosing displaying selecting and text in a first language and propagating the selection to a second language, with Maloney and Stieglitz disclosing being able to display the text in two languages at the same time including the simultaneous highlighting of words or phrases in each language according to selection, and with Stieglitz additionally disclosing rendering an image of just a portion of the second language near the first language responsive to the selection/highlighting, one of ordinary skill in the art of implementing an apparatus for displaying text in different languages with similar formats would include display an image acquired by trimming from the input image the input text area corresponding to a translation text area to which the selection text belongs, at a position near the translation text area to which the selection text belongs in order to allow a user to visualize that the labeling will be propagated properly to the version of a document in a different language when the screen real estate is reduced where there is not enough room to show the full text in both languages. One would therefore be motivated to combine these teachings as in doing so would create this apparatus for assigning labels to text in different languages.
Pertinent Prior Art
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
Al-Onaizan (US 7,895,030 A1) – provides two regions with original text and translated text where highlighting text in one region highlights the corresponding text in the other and additionally allows for manipulating the highlighted text.
Montiel (US 2018/0165987 A1) – provides two regions with original text and translated text where highlighting text in one region highlights the corresponding text in the other.
Kumeran (US 2008/0281578 A1) – provides two regions with original text and translated text where highlighting text in one region highlights the corresponding text in the other.
Pahud (US 2011/0252316 A1) – provides two regions with original text and translated text where highlighting text in one region highlights the corresponding text in the other.
Dozier (US 2021/0097143 A1) – provides two regions with original text and translated text.
Evans (US 2003/0200505 A1) - provides a translation of a document at each line and includes selection resulting in popup with translation options.
Fesbinder (US 2024/0419887 A1) - provides a translation of a document at each line with many styles of presenting it.
Englund (US 8,144,990 A1) – selecting portions of a raw image can provide translation overlays.
Sata (US 2011/0110599 A1) – provides a translation of a document at each line.
Kato (US 2009/0055159 A1) – provides a translation of a document at each line.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK F RIEGLER whose telephone number is (571)270-3625. The examiner can normally be reached M-F 9:30am-6:00pm, ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK F RIEGLER/ Primary Examiner, Art Unit 2171
Prosecution Insights