DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are currently pending and prosecuted.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after allowance or after an Office action under Ex Parte Quayle, 25 USPQ 74, 453 O.G. 213 (Comm'r Pat. 1935). Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, prosecution in this application has been reopened pursuant to 37 CFR 1.114. Applicant's submission filed on 14 March 2025 has been entered.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 14 March 2025 was considered by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Park et al., US PG-Pub 2019/0339840, hereinafter Park.
Regarding Claim 1, Park teaches a method performed by an apparatus of an extended reality (XR) wearable device (wearable device 101-1) comprising:
capturing, using an image capturing device of the XR wearable device (camera 320), at least one image ([0055], “the camera 320 may obtain the image of an object, which the user watches or which is positioned in a direction close or similar to a direction in which the user's head faces.”);
processing the at least one image to determine context data (Figs. 7A-8, and corresponding descriptions; [0138]-[0139], Operations 815 and 820);
processing audio data ([0135]-[0139], user utterances), from a microphone of the XR wearable device ([0067], “The input device 450 may include, for example, a microphone”), to determine an action from a plurality of actions (Fig. 8, and corresponding descriptions; [0140]-[0146], operations 825, 827 and 830), the audio data comprising utterances of a user of the XR wearable device ([0139], “when the user utterance information included in the first data corresponds to the pre-registered user utterance information (e.g., a wake-up utterance pre-specified by a user or authentication-related information), the server 201 may determine the speech recognition device 102 as a personal device”);
associating one or more applications with the action (Fig. 8, and corresponding descriptions; [0140]-[0146], operations 825, 827 and 830);
presenting indications of the one or more applications (Fig. 8, and corresponding descriptions; [0144]-[0146], “When the user account associated with the speech recognition device 102 is present, the server 201 may generate a list of one or more of applications, skills, or pieces of content, based on the second DB.”);
identifying input of the user as a selection of an application of the one or more applications (Fig. 8, and corresponding descriptions; [0147]-[0151], “the server 201 may determine whether one or more of the reception in operation 810 of the first data, the user input received in operation 830, or a usage history is associated with a specific application or skill”, “When it is determined that a user account is associated with the specific application or skill, in operation 840, the server 201 may select an application from the list of applications associated with the user account.”); and
causing an indication of the action and the context data to be sent to the application (Fig. 8, and corresponding descriptions; [0152]-[0158], “Accordingly, when context such as a voice command, gesture recognition, function change, or the like occurs, the server 201 may change the AR content or may newly generate AR content.”).
Regarding Claim 2, Park teaches the method of claim 1, wherein the processing audio data further comprises:
processing, based on the plurality of actions and context data, the audio data to determine the action ([0155], “the electronic device 101 may receive a user utterance input via the speech recognition device 102 . . . . The electronic device 101 may transmit the received user input to the server 201. The server 201 may perform operation 825 to operation 850 based on the received user input.”).
Regarding Claim 3, Park teaches the method of claim 2 further comprising:
accessing a current time ([0135]-[0137], “ the electronic device 101 may obtain time information using the image and a timer mounted in the electronic device 101”), wherein context data comprises one or more of: the at least one image, a location of the XR wearable device, or the current time ([0135]-[0137], “the electronic device 101 may obtain one or more of the location information of the electronic device 101, the time information, or user utterance information together with the image”).
Regarding Claim 4, Park teaches the method of claim 1, wherein the input of the user comprises a selection of a button of the XR wearable device or a touch of a touchpad of the XR wearable device ([0067], “The input device 450 may include, for example, . . ., a mouse, a keyboard, or a digital pen (e.g., a stylus pen)”, [0155], “the electronic device 101 . . . may receive a user input to select an input device (e.g., the input device 450 of FIG. 4) included in the electronic device 101”).
Regarding Claim 5, Park teaches the method of claim 1, wherein the presenting the indications of the one or more applications comprises:
playing, using a speaker of the XR wearable device (sound output device 455), indications of the one or more applications ([0068], “The sound output device 455 may include, for example, a speaker or a receiver.”), or
presenting, on a display of the XR wearable device (display device 460), the indications of the one or more applications ([0069], “The display device 460 may include, for example, a display, a hologram device, or a projector and control circuitry to control a corresponding one of the display, hologram device, and projector”).
Regarding Claim 6, Park teaches the method of claim 1 further comprising:
presenting a selection shape on a display of the XR wearable device ([0152], “For the purpose of displaying the display candidate content on the display of the electronic device 101, the AR content may be one of an image, video, or a 3D object generated by applying one or more arithmetic operations of object creation, shape, location, size, movement, rotation, color and transparency”); and
identifying the input of the user as the selection of the application based on the indication of the application being within the selection shape ([0152]-[0154], operation 847, 850 and 855), wherein a position of the selection shape remains stable as the XR wearable device moves and positions of the indications of the one or more applications move in accordance with movement of the XR wearable device ([0154], operation 855).
Regarding Claim 7, Park teaches the method of claim 1, wherein the audio data is first audio data, and wherein processing audio data further comprises:
processing, using one or more first processing components, second audio data from the microphone, to determine that the second audio data comprises a keyword ([0135]-[0139], “when the user utterance information included in the first data corresponds to the pre-registered user utterance information (e.g., a wake-up utterance pre-specified by a user or authentication-related information), the server 201 may determine the speech recognition device 102 as a personal device”);
causing, using the one or more first processing components, a signal to be generated to indicate the first audio data comprises the keyword ([0135]-[0139], “when the user utterance information included in the first data corresponds to the pre-registered user utterance information (e.g., a wake-up utterance pre-specified by a user or authentication-related information), the server 201 may determine the speech recognition device 102 as a personal device”); and
responsive to the signal, processing, using one or more second processing components, the second audio data to determine the action ([0135]-[0139], “According to another embodiment, in addition to the location information, the server 201 may determine the type of the speech recognition device 102 based on usage history. According to another embodiment, the server 201 may determine the type of the speech recognition device 102 based on one or more of location information, time, and user utterance information”).
Regarding Claim 8, Park teaches the method of claim 7, wherein the processing, using the one or more second processing components, second audio data further comprises:
sending, across a wireless connection (wireless communication modules 492, 494) to a host system (server 201), the second audio data with an instruction indicating the second audio data is to be processed ([0153]-[0154], “the server 201 may transmit second data including the rendered content to the electronic device 101”);
receiving from the host system a transcription of the second audio data ([0144]-[0146], “the server 201 may generate the content (e.g., an image and/or a text) of the generic device”); and
determining the action from a plurality of actions based on the transcription ([0153]-[0154], “the electronic device 101 may display the content of applications via the display based on the second data in the AR environment. According to an embodiment of the disclosure, the AR content of the second data may be displayed in conjunction with an image of the speech recognition device 102 included in the first data”).
Regarding Claim 9, Park teaches the method of claim 8, wherein the context data is sent with the second audio data ([0189], “For example, the follow-up action may include a follow-up utterance.”, which would be considered second audio data).
Regarding Claim 10, Park teaches the method of claim 7, wherein the one or more first processing components comprises:
a machine learning model to determine the second audio data comprises the keyword, wherein the machine learning model was trained to recognize the keyword ([0180], “ the plan may be generated by an AI system. The AI system may be a rule-based system, or may be a neural network-based system (e.g., a feedforward neural network (FNN) or a recurrent neural network (RNN))”, which include commonly known machine learning models).
Regarding Claim 11, Park teaches the method of claim 10, wherein the machine learning model was trained to recognize the keyword with vocalizations from the user ([0178]-[0179], “the intelligent server 1101 may receive information associated with a user utterance input (or user utterance) from the speech recognition device 102 via a communication network. According to an embodiment, the intelligent server 1101 may change the data associated with the received voice input to text data. According to an embodiment, the intelligent server 1101 may generate a plan for performing a task corresponding to a user utterance input based on the text data”).
Regarding Claim 12, Park teaches the method of claim 7, wherein the first audio data is processed by sending the first audio data to a host device (Figs. 5 and 7A-8, and corresponding descriptions, showing the data is processed by the server 201) and the second audio data is processed by the XR wearable device (Figs. 4 and 7A-8, and corresponding descriptions, showing the data is processed by the server 201 and then processed by the wearable device 101-1 to display content to the user).
Regarding Claim 13, Park teaches the method of claim 1, wherein the action is expressed by an action word or action phrase spoken by the user of the device ([0119], “the electronic device 101 may also either display the content of applications without a user utterance or may display the content of applications in response to a user utterance (e.g., “Hi, Bixby”) for calling the speech recognition device 102.”, noting the use of “Hi, Bixby” is spoken to the device).
Regarding Claim 14, Park teaches the method of claim 1 further comprising:
capturing, from the microphone, third audio data ([0154]-[0158], “the server 201 may specify or select the function in response to a voice signal or a user gesture in the reception in operation 810 of . . . the user input in operation 835. . . . For example, when a user enters a voice command or a user gesture associated with music playback, the server 201 may select the application or skill associated with the music playback from a list and may determine pieces of music content capable of being played from the electronic device 101, the server 201, and the memory of the external electronic device to generate the list.”); and
processing the third audio data to determine the application of a plurality of applications ([0154]-[0158], “the electronic device 101 may receive a user input to select the application, based on the content of the displayed applications.”).
Regarding Claim 15, Park teaches an apparatus of an extended reality (XR) wearable device (wearable device 101-1) comprising:
at least one processor (processor 420); and
at least one memory (internal memory 436 or external memory 438) storing instructions ([0245], “Various embodiments as set forth herein may be implemented as software (e.g., the program 440) including one or more instructions that are stored in a storage medium (e.g., the internal memory 436 or external memory 438) that is readable by a machine”) that, when executed by the at least one processor ([0245]), configure the at least one processor to perform operations comprising:
capturing, using an image capturing device of the XR wearable device (camera 320), at least one image ([0055], “the camera 320 may obtain the image of an object, which the user watches or which is positioned in a direction close or similar to a direction in which the user's head faces.”);
processing the at least one image to determine context data (Figs. 7A-8, and corresponding descriptions; [0138]-[0139], Operations 815 and 820);
processing audio data ([0135]-[0139], user utterances), from a microphone of the XR wearable device ([0067], “The input device 450 may include, for example, a microphone”), to determine an action from a plurality of actions (Fig. 8, and corresponding descriptions; [0140]-[0146], operations 825, 827 and 830), the audio data comprising utterances of a user of the XR wearable device ([0139], “when the user utterance information included in the first data corresponds to the pre-registered user utterance information (e.g., a wake-up utterance pre-specified by a user or authentication-related information), the server 201 may determine the speech recognition device 102 as a personal device”);
associating one or more applications with the action (Fig. 8, and corresponding descriptions; [0140]-[0146], operations 825, 827 and 830);
presenting indications of the one or more applications (Fig. 8, and corresponding descriptions; [0144]-[0146], “When the user account associated with the speech recognition device 102 is present, the server 201 may generate a list of one or more of applications, skills, or pieces of content, based on the second DB.”);
identifying input of the user as a selection of an application of the one or more applications (Fig. 8, and corresponding descriptions; [0147]-[0151], “the server 201 may determine whether one or more of the reception in operation 810 of the first data, the user input received in operation 830, or a usage history is associated with a specific application or skill”, “When it is determined that a user account is associated with the specific application or skill, in operation 840, the server 201 may select an application from the list of applications associated with the user account.”); and
causing an indication of the action and the context data to be sent to the application (Fig. 8, and corresponding descriptions; [0152]-[0158], “Accordingly, when context such as a voice command, gesture recognition, function change, or the like occurs, the server 201 may change the AR content or may newly generate AR content.”).
Regarding Claim 16, Park teaches the apparatus of claim 15, wherein the processing audio data further comprises:
processing, based on the plurality of actions and context data, the audio data to determine the action ([0155], “the electronic device 101 may receive a user utterance input via the speech recognition device 102 . . . . The electronic device 101 may transmit the received user input to the server 201. The server 201 may perform operation 825 to operation 850 based on the received user input.”).
Regarding Claim 17, Park teaches the apparatus of claim 16, wherein the operations further comprise:
accessing a current time ([0135]-[0137], “ the electronic device 101 may obtain time information using the image and a timer mounted in the electronic device 101”), wherein context data comprises one or more of: the at least one image, a location of the XR wearable device, or the current time ([0135]-[0137], “the electronic device 101 may obtain one or more of the location information of the electronic device 101, the time information, or user utterance information together with the image”).
Regarding Claim 18, Park teaches the apparatus of claim 15, wherein the input of the user comprises a selection of a button of the XR wearable device or a touch of a touchpad of the XR wearable device ([0155], “the electronic device 101 . . . may receive a user input to select an input device (e.g., the input device 450 of FIG. 4) included in the electronic device 101”, [0067], “The input device 450 may include, for example, . . ., a mouse, a keyboard, or a digital pen (e.g., a stylus pen)”).
Regarding Claim 19, Park teaches a non-transitory computer-readable storage medium ([0245], “The machine-readable storage medium may be provided in the form of a non-transitory storage medium.”), the computer-readable storage medium including instructions ([0245], “Various embodiments as set forth herein may be implemented as software (e.g., the program 440) including one or more instructions that are stored in a storage medium”) that when executed by at least one processor ([0245], “a processor (e.g., the processor 420) of the machine (e.g., the electronic device 401) may invoke at least one of the one or more instructions stored in the storage medium, and execute it”) of an apparatus of an extended reality (XR) wearable device (wearable device 101-1), cause the apparatus of the XR wearable device to perform operations comprising:
capturing, using an image capturing device of the XR wearable device (camera 320), at least one image ([0055], “the camera 320 may obtain the image of an object, which the user watches or which is positioned in a direction close or similar to a direction in which the user's head faces.”);
processing the at least image to determine context data (Figs. 7A-8, and corresponding descriptions; [0138]-[0139], Operations 815 and 820);
processing audio data ([0135]-[0139], user utterances), from a microphone of the XR wearable device ([0067], “The input device 450 may include, for example, a microphone”), to determine an action from a plurality of actions (Fig. 8, and corresponding descriptions; [0140]-[0146], operations 825, 827 and 830), the audio data comprising utterances of a user of the XR wearable device ([0139], “when the user utterance information included in the first data corresponds to the pre-registered user utterance information (e.g., a wake-up utterance pre-specified by a user or authentication-related information), the server 201 may determine the speech recognition device 102 as a personal device”);
associating one or more applications with the action (Fig. 8, and corresponding descriptions; [0140]-[0146], operations 825, 827 and 830);
presenting indications of the one or more applications (Fig. 8, and corresponding descriptions; [0144]-[0146], “When the user account associated with the speech recognition device 102 is present, the server 201 may generate a list of one or more of applications, skills, or pieces of content, based on the second DB.”);
identifying input of the user as a selection of an application of the one or more applications (Fig. 8, and corresponding descriptions; [0147]-[0151], “the server 201 may determine whether one or more of the reception in operation 810 of the first data, the user input received in operation 830, or a usage history is associated with a specific application or skill”, “When it is determined that a user account is associated with the specific application or skill, in operation 840, the server 201 may select an application from the list of applications associated with the user account.”); and
causing an indication of the action and the context data to be sent to the application (Fig. 8, and corresponding descriptions; [0152]-[0158], “Accordingly, when context such as a voice command, gesture recognition, function change, or the like occurs, the server 201 may change the AR content or may newly generate AR content.”).
Regarding Claim 20, Park teaches the non-transitory computer-readable storage medium of claim 19, wherein the processing audio data further comprises:
processing, based on the plurality of actions and context data, the audio data to determine the action ([0155], “the electronic device 101 may receive a user utterance input via the speech recognition device 102 . . . . The electronic device 101 may transmit the received user input to the server 201. The server 201 may perform operation 825 to operation 850 based on the received user input.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEPHEN T REED whose telephone number is (571)272-7234. The examiner can normally be reached M-F: 0800-1800.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
STEPHEN T. REED
Primary Examiner
Art Unit 2627
/Stephen T. Reed/Primary Examiner, Art Unit 2627