DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities: Paragraph 00119 Lines 7-8 recites “FIG. 6”, however no Figure 6 is provided in the drawings.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-27 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) elements which under their broadest reasonable interpretation, are directed to mental processes. This judicial exception is not integrated into a practical application as explained below. The claim(s) do not include additional elements that are sufficient to amount to significantly more than the judicial exception as explained below.
Regarding Claim 19, the claim recites A method of operating:
an electronic device that comprises one or more cameras and one or more displays, the method comprising:
obtaining a user input;
(c) determining an intent based on the user input;
(d) and in accordance with a determination that the user intent represents an intent to provide a summary of text in a physical environment:
(e) obtaining a text summary based on information regarding the text, the information regarding the text obtained from one or more images of the physical environment captured using the one or more cameras; and
(f) presenting the text summary on the one more displays.
Claim Interpretation: Under the broadest reasonable interpretation, the terms of the claim are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP 2111.
Claim element a recites that the method is performed by computer components at a high level of generality.
Claim element b as written covers any method for receiving an input from a user. This does not preclude a human from receiving said input.
Claim element c as written covers any method for determining an intent from the input which similarly does not preclude a human from performing this.
Claim element d as written covers making any action based on the intent determined. Nothing precludes a human from performing this
Claim element e as written covers any method for getting a summary based on an image and uses generic computer components.
Claim element f as written likewise utilizes generic computer components to show the summary
Additional elements recited are: electronic device, cameras, displays, images of physical environment captured using the one or more cameras.
Claim 19 is patent ineligible as it is directed to an abstract idea without being significantly more.
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. The claim is directed to a method, which falls within one of the statutory categories of invention. (Step 1: YES).
Step 2A, Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim.
As discussed above, the broadest reasonable interpretation of steps (b)-(e) that those steps fall within the mental process groupings of abstract ideas because they cover concepts performed in the human mind, including observation, evaluation, judgment, and opinion. See MPEP 2106.04(a)(2), subsection III.
Step b is directed to a mental step since a human can receive an input. Step c can be performed by human as a human can provide a response based on request/question. Step c and d can be performed by a human as a human can determine an intent based on an input and act upon that determination. Step e can be performed by human mind as a human can summarize what they see and write it down using pen and paper, additionally the human mind can obtain information regarding text from an image. Hence, these steps can be performed by a human, using “observation, evaluation, judgment, [and] opinion,” because they involve making determinations and identifications, which are mental tasks humans routinely do,' ” and thus can practically be performed in the human mind, In re Killian, 45 F.4th 1373, 1379 (Fed. Cir. 2022). Therefore, these limitations are considered together as an abstract idea for further analysis. (Step 2A, Prong One: YES).
Step 2A, Prong Two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception or whether the claim is “directed to” the judicial exception. This evaluation is performed by (1) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. See MPEP 2106.04(d). Additional elements recited are: electronic device, cameras, displays, one or more images of physical environment captured using the one or more cameras.
The electronic device, cameras and displays disclosed provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. The additional element of one or more images of physical environment captured using the one or more cameras is an insignificant extra-solution activity towards writing a summary. Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application (Step 2A, Prong Two: NO), and the claim is directed to the judicial exception. (Step 2A: YES).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. See MPEP 2106.05.
At Step 2A, the additional elements of electronic device, cameras, displays, images of physical environment captured using the one or more cameras were found to represent no more than mere instructions to apply the judicial exception on a computer using generic computer components. Mere instructions to “apply” the abstract ideas, cannot provide an inventive concept. See MPEP 2106.05(f). The analysis under Step 2A, Prong Two is carried through to Step 2B. Further, one or more images of physical environment captured using the one or more cameras was found to be insignificant extra-solution activity. However, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be re-evaluated in Step 2B. See MPEP 2106.05, subsection I.A. At Step 2B, the re-evaluation of the insignificant extra-solution activity consideration takes into account whether or not the extra-solution activity is well understood, routine, and conventional in the field. See MPEP 2106.05(g). Taking a picture of text prior to obtaining information regarding the text is well understand routine and conventional as disclosed in Ofgang’s publication on “Taking Notes vs. Photographing Slides”. Therefore, this limitation remains insignificant extra solution activity even upon reconsideration and does not amount to significantly more. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept. (Step 2B: NO).
As such Claim 19 is not patent eligible.
Claims 1 and 10 follow analogous analysis aside from the inclusion of computer components recited at a high level of generality to perform the method of Claim 19.
Claims 2, 11, and 20 recite a computer component at a high level of generality to perform the abstract idea.
Claims 3, 12, and 21 comprise a mental process as a human can convert text they see into a format that a machine can understand.
Claims 4, 13, and 22 disclose a computer component at a high level of generality for performing the mental process from Claims 3,12 and 21.
Regarding Claims 5, 14, and 23, analysis analogous to Claim 2 is applicable
Claims 6, 15, and 24 further limits what is provided to the generic computer component. A human can obtain contextual information from an image for the purpose of writing a summary
Claims 7, 16, and 25 recite providing instructions for the length of a summary and adjusting the length of said summary, which covers performance of the limitations in the mind
Regarding Claims 8, 17, and 26, analysis analogous to Claim 2 is applicable as a human can select a subset of text to perform the abstract idea writing a summary.
Claims 9, 18, and 27 recite making a selection on a display to display text at a selected location. A human can make a selection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-7, 10-16, 19-25 are rejected under 35 U.S.C. 103 as being unpatentable over Terman(US PGPub 20130021346), in view of Hwang(US PGPub 20190042551).
Regarding Claim 1, Terman teaches an electronic device comprising:
one or more cameras(front camera, rear camera[Terman Table 1]); one or more displays(Display[Terman Table 1]); one or more processors(Processor[Terman Table 1]); and memory(Memory[Terman Table 1]) storing instructions configured to be executed by the one or more processors, the instructions for:
obtaining a user input(The entire multi-step process is seamlessly accomplished with a single click or tap of the image capturing device[Terman 0028]);
and in accordance with a determination that the user intent represents an intent to provide a summary of text in a physical environment(with a single click of the camera provides a functional summary of the textual images[Terman 0028]);
obtaining a text summary based on information regarding the text, the information regarding the text obtained from one or more images of the physical environment captured using the one or more cameras(Once the image is captured it is then processed seamlessly through OCR and text summarization programs resulting in a user understandable summary of the text in the captured images[Terman 0026]);
and presenting the text summary on the one more displays(the machine summary appears in the monitor screen of the device [Terman 0035]).
Terman does not disclose determining an intent based on the user input.
However, Hwang does teach determining an intent based on the user input (Specifically, if a user input for a document summary is received, the AI agent may operate. The AI agent may acquire the text included in the document based on the user input, and may obtain the summary information through the AI learning model[Hwang 0058]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the determination of intent of Hwang with the device and method of Terman because it would provide the user with various user experiences through a summarization function to summarize a document using a summarization model[Hwang 0009].
Regarding Claim 2, Terman teaches wherein obtaining the text summary based on the information regarding the text comprises providing the information regarding the text to a trained model(The basis for Text Analyst's processing is a neural network technique that analyzes preprocessed text to develop a semantic network. The semantic network provides the core information required for clustering, summarizing, and navigating textual material [0090]).
Regarding Claim 3, Terman teaches wherein providing the information regarding the text to the trained model comprises: performing optical character recognition (OCR) on the text(the software automatically launches an OCR program which converts the image to editable text [0028]), wherein performing optical character recognition on the text comprises converting the text into machine-readable text(the software automatically launches an OCR program which converts the image to editable text [0028], HTML CSV TXT [0046]); and providing the machine-readable text to the trained model(editable text which is then summarized by the machine summarizer software [0028]).
Regarding Claim 4, Terman discloses communication circuitry, wherein providing the machine-readable text to the trained model comprises providing the machine-readable text to the trained model using the communication circuitry(Figure 1. document summarization as provided by SSW 111 may be performed on any electronic documents that may be uploaded to the server from an internet-capable appliance such as from computers 107a, 107b, and 107d [0156], Client station 103 is similar to station 101 in that a computer 107b running a browser instance 112b is provided as well as an OCR scanner 109b[0155]. network backbone 106. Backbone 106 represents all of the lines equipment and access points making up the Internet network as a whole[0153]).
Regarding Claim 5, Terman teaches providing the information regarding the text to the trained model comprises providing the one or more images to the trained model(In some instances, the textual material may be in a format such as WORD or pdf that is transferred directly and converted to a summary by the summarizer program without requiring the intermediary OCR processing [0040]).
Regarding Claim 6, Terman discloses wherein the instructions further comprise instructions for: providing contextual information to the trained model in addition to the information regarding the text, wherein the text summary is based on both the contextual information and the information regarding the text(The user can choose the format of the final summary[0077]) and wherein the contextual information comprises location information, cultural information, reading level information, age information, subject matter knowledge information, education information, historical query information, user preference information(The user therefore first selects the summarizer intervals from a menu comprising: [0075]), temporal information, or calendar information.
Regarding Claim 7, Terman teaches wherein the instructions further comprise instructions for: providing one or more response length parameters(A summary with a compression of 30% would be expected to include more relevant concepts than one with a compression of 10% [0067) to the trained model in addition to the information regarding the text, wherein the text summary is based on both the one or more response length parameters and the information regarding the text and wherein the one or more response length parameters comprises an absolute maximum for a length of the text summary, a relative maximum that is based on a length of the text(The user can compress or condense the summary document to varying degrees determined as a percent of the total word volume of the original material.[0067]), an absolute minimum for a length of the text summary, or a relative minimum that is based on a length of the text.
Regarding Claims 10-16, these claims are directed to non-transitory computer-readable storage medium. Hwang discloses the non-transitory computer-readable storage medium (the software module may be stored in a computer-readable non-transitory computer readable media[0196]). The prior art rejection for Claims 1-7 remains applicable to Claims 10-16.
Regarding Claims 19-25, these claims are directed to the method for performing the same functions as stated in the rejected Claims 1-7. As such the prior art rejection for Claims 1-7 remains applicable to Claims 19-25.
Claim(s) 8,17,26 are rejected under 35 U.S.C. 103 as being unpatentable over Terman(US PGPub 20130021346), and Hwang(US PGPub 20190042551) as applied in prior claims, in view of Du(Us PGPub 20250165728).
Regarding Claim 8, Terman teaches wherein providing information regarding the text to the trained model comprises providing information regarding only a subset of the text to the trained model(The program can automatically summarize individual subsections of the textual material containing relatively homogeneous subject matter [Terman 0034]), wherein the electronic device further comprises one or more input components, and wherein the instructions further comprise instructions for: selecting the subset of the text based on input to the one or more input components, and presenting an indicator on the one more displays that identifies the subset of the text(This is accomplished by using a highlighting the text to be summarized [Terman 0034]).
Terman and Hwang do not teach wherein the one or more input components comprises a gaze detection sensor, a microphone, or a button;
However, Du teaches wherein the electronic device further comprises one or more input components, and wherein the instructions further comprise instructions for: selecting the subset of the text based on input to the one or more input components, wherein the one or more input components comprises a gaze detection sensor, a microphone, or a button(the head mounted wearable device 800 includes a gaze tracking device 815 to detect and track eye gaze direction and movement. Data captured by the gaze tracking device 815 may be processed to detect and track gaze direction and movement as a user input. [Du 00151]);
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Terman and Hwang to further include the use of a gaze sensor as an input for subset selection from Du because the user eye gaze can be tracked not only with respect to objects in the physical environment, but also with respect to the content output for display by the display device [Du 00151].
Regarding Claim 17, this claim is directed to non-transitory computer-readable storage medium. Hwang discloses the non-transitory computer-readable storage medium (the software module may be stored in a computer-readable non-transitory computer readable media[0196]). The prior art rejection for Claim 8 remains applicable to Claim 17.
Regarding Claim 26 , this claim is directed to the method for performing the same functions as stated in the rejected Claim 8. As such the prior art rejection for Claim 8 remains applicable to Claim 26.
Claim(s) 9,18,27 are rejected under 35 U.S.C. 103 as being unpatentable over Terman(US PGPub 20130021346), and Hwang(US PGPub 20190042551) as applied in prior claims, in view of Bala(Us PGPub 20120189190).
Regarding Claim 9, Terman and Hwang do not teach wherein the instructions further comprise instructions for: selecting a position for the text summary on the one more displays based on the one or more images of the physical environment, wherein presenting the text summary on the one more displays comprises presenting the text summary on the one more displays at the selected position.
However, Bala teaches, wherein the instructions further comprise instructions for: selecting a position for the text summary(receiving input regarding a user-selected location in the image for text insertion [0007]) on the one more displays based on the one or more images of the physical environment(receiving an image [0007]), wherein presenting the text summary on the one more displays comprises presenting the text summary on the one more displays at the selected position(In another aspect, a computer implemented method for defining a bounding polygon in an image for text insertion comprises receiving an image for personalization, receiving input regarding a user-selected location in the image for text insertion [0007].
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention having the teachings of Terman and Hwang to further include the placement of text at a selected position from Bala because the user can add value by personalizing documents [Bala 0002]
Regarding Claim 18, this claim is directed to non-transitory computer-readable storage medium. Hwang discloses the non-transitory computer-readable storage medium (the software module may be stored in a computer-readable non-transitory computer readable media[0196]). The prior art rejection for Claim 9 remains applicable to Claim 18.
Regarding Claim 27 , this claim is directed to the method for performing the same functions as stated in the rejected Claim 9. As such the prior art rejection for Claim 9 remains applicable to Claim 27.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARJUN R SWAMY whose telephone number is (571)272-9763. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARJUN SWAMY/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654