DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 10/17/2025 has been entered.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claims are indefinite in that it is unclear how the visual text is displayed in response to the user directing a gaze direction “without input from the user”, in that the user directing a gaze direction to a predetermined location is input from the user.
Claims that are noted above as being rejected but that are not specifically cited below are rejected based on their dependency on rejected independent claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7, 9-15 and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tabata et al., U.S. Patent Publication Number 2024/0119684, in view of Schoen, U.S. Patent Publication number 2023/0026638 A1.
Regarding claim 1, Tabata discloses an eyewear device (FIG. 2) comprising: an image capture system; a display system comprising a display (102, display); a speech recognition system (101, microphone); a processor (12, processor); a memory (11, storage device); and programming in the memory, wherein execution of the programming by the processor configures the processor to perform operations comprising: capture an image of a physical environment using the image capture system (paragraph 0136, captured image captured by the camera); acquire text from a user input, a data input, or the speech recognition system (paragraph 0028, acquire a sound and to display a text image; paragraph 0060-0062, acquires at least one of the following; speech signal collected by microphone); generate, using the display system, visual text graphics from the acquired text (paragraph 0070, generates the text image corresponding the speech sound; paragraph 0136, text image and the symbol image generated based on the speech recognition with the captured image captured by the camera); present, using the display system, the visual text graphics at a predefined location in the display of the display system relative to one or more items in the physical environment (paragraph 0059, presenting an image representing a text corresponding to a speech sound collected by the microphone (hereinafter referred to as a “text image”) at a predetermined position on the display); fixing, using the display system, the visual text graphics at a predefined location in the display (paragraph 0112, area is displayed at a fixed position on the bird’s-eye map, and the symbol associated with the text image is displayed at a position representing the direction of the sound source; paragraph 0114, since the text image corresponding to the sound is collectively displayed in the predetermined text display area regardless of the sound source, the user can easily follow the text image with his/her eyes; further, even when the sound source exists outside the visual filed of the user, the user can recognize the content of the speech uttered from the sound source without facing the direction of the sound source).
However, it is noted that Tabata fixes the text in a predetermined location in the display, but fails to specifically disclose fix, in response to the user directing a gaze direction to a predetermined location that is different from the predefined location in the display without input from the user.
Schoen discloses fix, using the display system, the visual text graphics at the predefined location in the display in response to the user directing a gaze direction to a predetermined location that is different from the predefined location in the display without input from the user (paragraph 0049, can display a head leashed notification at a pre-defined location in an artificial reality device’s field of view; the pre-determined location can, be a place on the edge or in a corner of the field of view; paragraph 0053, determines that the tracked user’s gaze has moved off the notification for a threshold amount of time; when process makes this determination, process can continue; it can return to minimize, head leash, and restore the notification to the pre-defined location).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in the predefined location as disclosed by Tabata for displaying text in a fixed location, fixing the text responsive to a user directing a gaze to a location different from the predefined location without input from the user as disclosed by Schoen, to track a user’s gaze and pre-determined and pre-defined location for moving and restoring notifications or text to a location.
Regarding claim 2, it is noted that Tabata discloses a fixed location, but fails to disclose wherein the predetermined location to which the gaze direction is directed is a floor of a room.
Schoen discloses wherein the predetermined location to which the gaze direction is directed is a floor of a room (paragraph 0049, the pre-determined location can, be a place on the edge or in a corner of the field of view).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further include in the restoring of notification and predetermined location as disclosed by Schoen, the location as an edge or a corner in the field of view, which Examiner interprets as inclusive of a floor, to determine that the tracked user’s gaze has moved and is different from the predefined location for the display of the text.
Regarding claim 3, it is noted that Tabata fails to disclose wherein execution of the programming by the processor further configures the processor to move the visual text graphics from the predefined location relative to the one or more items in the physical environment in response to a second gaze direction or a hand gesture of the user.
Schoen discloses wherein execution of the programming by the processor further configures the processor to move the visual text graphics from the predefined location relative to the one or more items in the physical environment in response to a second gaze direction or a hand gesture of the user (paragraph 0052, the user can move the notification in her field of view (e.g., can move the notification closer to the center of the field of view by direction her head toward the world locked location of the notification)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in the display of visual text as disclosed by Tabata, the moving of the visual text or notification, as disclosed by Schoen, to allow a user to direct the text toward their center field of view.
Regarding claim 4, Tabata discloses wherein execution of the programming by the processor further configures the processor to transmit an image of the display including the visual text graphics fixed at the predefined location in the display relative to the one or more items in the physical environment (figure 10B-10C, disclose text relative to the speaker in the capture image).
Regarding claim 5, Tabata discloses wherein the image capture system captures the image of the physical environment or a sequence of frames of video data and execution of the programming by the processor further configures the processor to fix the visual text graphics at the predefined location in the image of the physical environment or the sequence of frames of video data and to store the image of the physical environment or the sequence of frames of video data with the fixed visual text graphics in the memory (paragraph 0047, display presents (for example, displays) an image under the control of the controller; the image is projected onto the display from a projector; paragraph 0052, the storage device is configured to store programs and data; paragraph 0056-0058, data includes; data obtained by executing information processing, (that is, an execution result of the information processing); paragraph 0059, activates a program store din the storage device to realize a function of presenting an image representation a text corresponding to a speech sound; therefore Examiner interprets the image representation of the text as data obtained by executing the information processing as stored).
Regarding claim 6, Tabata discloses wherein execution of the programming by the processor further configures the processor to alter one or more properties of the visual text graphics in response to text modifying input from the user (paragraph 0101, controller may estimate the emotion of the speaker by speech recognition; and determine the expression or color of the symbol corresponding to the speaker based on the estimated emotion).
Regarding claim 7, Tabata discloses wherein the one or more properties of the visual text graphics include font style or color of the visual text graphics (paragraph 0101, controller may estimate the emotion of the speaker by speech recognition; and determine the expression or color of the symbol corresponding to the speaker based on the estimated emotion; paragraph 0104, texts corresponding to utterances having different sound-arrival direction may be displayed in different colors; paragraph 0062, a user instruction input from an input device connected to the controller).
Regarding claims 9-15, they are rejected based upon similar rational as above claims 1-7. Tabata further discloses an augmented reality method for use with an eyewear device having an image capture system and a display system comprising a display (figure 1; paragraph 0046).
Regarding claims 17-20, they are rejected based upon similar rational as above claims 1-4. Tabata further discloses a non-transitory computer-readable medium storing program code which, when executed, is operative to cause an electronic process to perform the steps (paragraph 0052; 0059).
Claim(s) 8 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tabata in view of Schoen as applied to claims 1 and 9 above, and further in view of Cunningham et al., U.S. Patent 10,031,576 B2, further in view of Sala, U.S. Patent Number 9,965,062 B2.
Regarding claim 8, it is noted that Tabata and Schoen fail to disclose wherein execution of the programming by the processor further configures the processor to arrange attributes of the visual text graphics as a menu of options that are displayed on the display system based on characteristics of the physical environment.
Cunningham discloses col. 3, lines 22-25, producing digitized or synthesized speech; messages may be preconfigured and/or selected and/or composed by a user within a message window displayed on the head mounted display unit. Cunningham further discloses wherein execution of the programming by the processor further configures the processor to arrange attributes of the visual text graphics as a menu of options that are displayed on the display system (co. 3, lines 29-31, head mounted display unit may include a display device for displaying one or more images (e.g., menus, keypads or other graphical user interface); col. 13, lines 57-62, some exemplary graphical user interfaces correspond to conventional “QWERTY” keyboards, numeric keypads, or other customized keypads with alphanumeric identifiers; buttons also may include words, phrases, symbols and other information that can be customized based on user preferences, frequency or other parameters).
It is noted that Tabata, Schoen, and Cunningham fail to disclose attributes of the visual text graphics as a menu of options that are displayed on the display system based on characteristics of the physical environment.
Sala discloses col. 3, lines 3, lines 27-33, user gazes at text item, the text item may be evaluated to determine characteristics such as font, font size, font color, background color, font style, and other such format. Sala discloses options that are displayed on the display system based on characteristics of the physical environment (col. 4, lines 44-51, appearance (e.g., color, brightness, saturation, etc.) of the text and/or the background may be modified to increase a contrast of the text and background, and the visually enhanced text may be displayed as user interface object; col. 4, lines 63-64, display a graphical user interface comprising one or more items).
It would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention to include in the display of text as disclosed by Tabata, and Schoen, the menus for text input as disclosed by Cunningham to allow a text to be customized based on a user’s preferences or frequency or other parameter. It further would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include in the menu for customizing text as disclosed by Cunningham, the characteristics of the environment as disclosed by Sala, to prevent difficulty of a user distinguishing the text from the background in the image.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rimon et al., U.S. Patent Number 9,984,505 B2
Rimon discloses 202A and 202B, image capture devices; col. 8, lines 20-35, some representative examples of sources of text information and/or types of text information include the following: dialogue; col. 8, lines 18-19, rendering of text from any source of text information; col. 8, lines 54-57, text information can be rendered in a virtual environment; text information can be displayed on any type of object or surface that exists within a virtual environment; col. 8, lines 61-65, objects or surfaces that are within a current view of the user may be selected for rendering of text thereon.
Pace et al., U.S. Patent Publication Number 2020/0128231 A1
Pace discloses NED display device 110; 112, processing units; paragraph 0051, one or more computer memory devices; paragraph 0051, one or more computer memory devices storing instructions executable by the logic device(s) to deploy functionalities; paragraph 0045, NED device may anchor the virtual item to the real object; the user is enabled to control virtual item positioning and to anchor virtual item to real objects without needing to user hand gestures or user input device; then, as the user (or other users wearing NED devices for that matter) moves around with respect to the real object, the virtual item remains pinned thereto; paragraph 0098, the user may instruct the virtual item to the specific button by deliberately directing his or her eye gaze direction to the specific button in conjunction with performing a predetermined facial gestures; the user releasing the virtual item in this way is interpreted by the NED device as an instruction to anchor the virtual item to the specific button; the virtual item become anchored with respect tot a real-world coordinate system associated with the real object; the virtual item remains statically anchored (e.g., pinned);
Paragraph 0101; predetermined facial gestures that may serve as indication that the user desires to hold onto a virtual item (e.g., to drag the virtual item through the 3D space of the real-world environment and/or that the user desires to release the virtual item.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Motilewa Good-Johnson whose telephone number is (571)272-7658. The examiner can normally be reached Monday - Friday 6am-2:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 571-272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MOTILEWA . GOOD JOHNSON
Primary Examiner
Art Unit 2616
/MOTILEWA GOOD-JOHNSON/Primary Examiner, Art Unit 2619