DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114.
Status of Claims
Claims 1, 3-21, 23, 25-28, 37, 43, and 342-344 are pending in this application.
Claims 2, 22, 24, 29-36, 38-42, and 44-341 are canceled.
Response to Arguments
Regarding Rejection under 35 U.S.C. 103
Applicant’s arguments with respect to rejections have been fully considered but they are not persuasive.
Regarding Claim 1, the Applicant argues that the rejection under 35 U.S.C. 103 is improper because 1) Myer silent regarding that, “prior to … reserved processor cores”; and 2) The combination of Myer and Kim does not disclose a processor that binds an IPA to the second stage keyword detector such that the scheduling privileges of the IPA are based on scheduling privilege of the second the keyword detector (On page 10 of 19 1st paragraph - Page, 18 of 19, 3rd paragraph).
However, the Examiner respectfully disagrees because the rejection under 35 U.S.C. 103 is proper. Regarding claim 1, in response to applicant's arguments,
The limitation of “prior to confirmation of the keyword detection by the second-stage keyword detector” does not require that the second stage keyword detector has performed, or even will perform, any confirmation. Thus, this limitation, standing alone, does not establish any particular function or operation performed by the second-stage keyword detector. Moreover, Myer discloses that a lower powered processor may be used to perform the first stage keyword spotting and a second higher powered processor may be used to perform the second stage keyword spotting. When the first stage detects a keyword, the speech feature vectors or the audio corresponding to the keyword may be sent to a second neural network on the device for further processing. A second neural network is used for second stage decoding (Myer, [0074][0129][0130]).
What function is actually performed by the second stage keyword detector? The argument merely asserts that the processor “binds” the voice service and the second stage keyword detector but does not clearly explain what “binds” means in this context, or how the purported binding establishes that the scheduling privileges of the voice service are based on those of the second stage detector.
Myer teaches initiating a transition of a system from a lower powered processor mode to a high powered processor mode. When the system transitions to the other mode, a second stage keyword spotting system operates to perform a voice interface device. Moreover, Kim discloses that a smart device managed a plurality of voice services based on the detected wake word (Kim, Figs. 1 and 3, [0037][0043]). Thus, the rejection is maintained currently. Please, see the rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-10, 12-14, 18-21, 23, 25-28, 37, 43, and 342-344 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Myer et al., (US Pub. 2023/0409102) in view of Kim (US Pub. 2018/0301147).
Regarding claim 1, Myer discloses a device comprising:
a first stage keyword detector ([0130] a lower powered processor may be used to perform the first stage keyword spotting);
a second stage keyword detector ([0129]-[0132] a second higher powered processor may be used to perform the second stage keyword spotting); and
a processor configured to: based on a signal indicating a keyword detection by the first stage keyword detector and prior to confirmation of the keyword detection by the second stage keyword detector, initiate a change of state to enable a voice service that is in a low-power mode to be scheduled for execution on one or more reserved processor cores (Figs. 3 and 6, [0046][0047] “wherein the first core is a low-power processing core and the second core is a high-power processing core, when the first core determines the presence of at least one of a plurality of keywords in the acoustic signal the acoustic signal is provided to the second core for further processing … performing keyword verification”),
wherein, to initiate the change of state, the processor is configured to bind the voice service and the second stage keyword detector ([0132]-[0136][0142] “having additional processing capability to verify the keyword and/or perform additional processing on the acoustic signal to process commands with in the acoustic signal (614). The additional processor can utilize a higher power core or processor to verify the keyword before performing additional processor”; [0074][0129][0130] a lower powered processor may be used to perform the first stage keyword spotting and a second higher powered processor may be used to perform the second stage keyword spotting. When the first stage detects a keyword, the speech feature vectors or the audio corresponding to the keyword may be sent to a second neural network on the device for further processing. A second neural network is used for second stage decoding).
Myer does not explicitly teach however Kim does explicitly teach:
such that scheduling privileges of the voice service are based on scheduling privileges of the second stage keyword detector, and wherein the voice service includes use of a speech recognition model ([0018][0019] each intelligent personal assistant (IPA) services 140,150,160 may be a different commercially available an intelligent personal assistant service, such as Microsoft Cortana™, Apple Siri™, Google Home™, or Amazon Alexa™, among others, such as a web server or another networked computing device; Figs. 1 and 3, [0037][0043] a smart device managed a plurality of voice services based on the detected wake word).
Therefore, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to incorporate the two stages keyword spotting system as taught by Myer with the method of selecting an intelligent personal assistant among a plurality of IPA services based on detecting keywords/wakewords as taught by Kim to provide convenient user experience to interact with multiple human assistants at the same time (Kim, [0004]).
Regarding claim 3, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
wherein to initiate the change of state, the processor is configured to elevate [scheduling privileges] of a second stage keyword detector that is configured to confirm the keyword detection (Figs. 3 and 6, [0132]-[0136][0142] indicating the keyword is detected by using a first and second stage).
Myer does not explicitly teach however Kim does explicitly teach the bracketed limitation:
[scheduling privileges] (Kim, [0045] “order the playback of multiple response audio signals 306 in the order in which the IPA services originating was referenced by activation phrases”).
Regarding claim 4, Myer in view of Kim discloses the device of claim 3, and Myer further discloses:
wherein [to elevate the scheduling privileges] of the second stage keyword detector, the processor is configured to [change a scheduling priority] of the second stage keyword detector (Figs. 3, [0132]-[0136] indicating the keyword is detected by using a second stage).
Myer does not explicitly teach however Kim does explicitly teach including the bracketed limitation:
wherein [to elevate the scheduling privileges] of the second stage keyword detector, the processor is configured to [change a scheduling priority] (Kim, [0045][0063] (Kim, [0045] “order the playback of multiple response audio signals 306 in the order in which the IPA services originating were referenced by activation phrases”).
Regarding claim 5, Myer in view of Kim discloses the device of claim 3, and Myer further discloses:
wherein [to elevate the scheduling privileges] of the second stage keyword detector the processor is configured to [change a designation] of the second stage keyword detector [from a background application to a foreground application] (Figs. 3, [0132]-[0136] indicating the keyword is detected by using a second stage).
Myer does not explicitly teach however Kim does explicitly teach the bracketed limitation:
wherein [to elevate the scheduling privileges] of the second stage keyword detector the processor is configured to [change a designation] of the second stage keyword detector [from a background application to a foreground application] (Kim, [0045][0063] “order the playback of multiple response audio signals 306 in the order in which the IPA services originating were referenced by activation phrases”).
Regarding claim 6, Myer in view of Kim discloses the device of claim 5, and Myer further discloses:
wherein the processor is further configured to execute instructions to delay or inhibit generation of a user notification associated with the change of the designation of the second stage keyword detector from the background application to the foreground application ([0145] “Feedback from the low-power wakeword spotting system may generate output on a display 716, provide audible output 712, or generate instructions to another processor or device… designated for low power processing such as low-power core 707 when the high-power cores are idle 709 or in a power saving state”).
Regarding claim 7, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
wherein to initiate the change of state, the processor is configured to [elevate scheduling privileges] of the voice service (Fig. 6, [0132]-[0136][0142] a method of low-power wakeword spotting).
Myer does not explicitly teach however Kim does explicitly teach the bracketed limitation:
wherein to initiate the change of state the processor is configured to [elevate scheduling privileges] of the voice service (Kim, [0045][0063] “order the playback of multiple response audio signals 306 in the order in which the IPA services originating were referenced by activation phrases”).
Regarding claim 8, Myer in view of Kim discloses the device of claim 7, and Myer further discloses:
wherein [to elevate the scheduling privileges] of the voice service, the processor is configured to change a scheduling priority of the voice service (Figs. 3, [0132]-[0136] indicating the keyword is detected by using a second stage; [0132]-[0136][0142] “having additional processing capability to verify the keyword and/or perform additional processing on the acoustic signal to process commands with in the acoustic signal).
Myer does not explicitly teach however Kim does explicitly teach including the bracketed limitation:
wherein [to elevate the scheduling privileges] of the voice service, the processor is configured to [change a scheduling priority] of the voice service (Kim, [0045][0063] “order the playback of multiple response audio signals 306 in the order in which the IPA services originating were referenced by activation phrases”).
Regarding claim 9, Myer in view of Kim discloses the device of claim 7, and Myer further discloses:
wherein [to elevate the scheduling privileges] of the voice service, the processor is configured to [change a designation] of the voice service [from a background application to a foreground application] (Figs. 3, [0132]-[0136] indicating the keyword is detected by using a second stage; [0132]-[0136][0142] “having additional processing capability to verify the keyword and/or perform additional processing on the acoustic signal to process commands with in the acoustic signal).
Myer does not explicitly teach however Kim does explicitly teach the bracketed limitation:
wherein [to elevate the scheduling privileges] of the voice service, the processor is configured to [change a designation] of the voice service [from a background application to a foreground application] (Kim, [0045][0063] “order the playback of multiple response audio signals 306 in the order in which the IPA services originating were referenced by activation phrases”).
Regarding claim 10, Myer in view of Kim discloses the device of claim 9, and Myer further discloses:
wherein the processor is further configured to execute instructions to delay or inhibit generation of a user notification associated with the change of the designation of the voice service from the background application to the foreground application ([0145] “Feedback from the low-power wakeword spotting system may generate output on a display 716, provide audible output 712, or generate instructions to another processor or device… designated for low power processing such as low-power core 707 when the high-power cores are idle 709 or in a power saving state”).
Regarding claim 12, Myer in view of Kim discloses the device of claim 1, and Kim further discloses:
wherein the processor is further configured to, based on the second stage keyword detector failing to confirm the keyword detection, reverse the change of state (Kim, Fig. 4, steps 403, 409, if any activation phrases are not detected, the method of performing speech recognition is ended).
Regarding claim 13, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
the one or more reserved processor cores and one or more additional processor cores ([0142] “The primary core/processor may be a low power core/processor which the secondary core/processor will have a higher power requirement.”).
Regarding claim 14, Myer in view of Kim discloses the device of claim 13, and Myer further discloses:
wherein enabling the voice service to be scheduled for execution on the one or more reserved processor cores reduces latency of the voice service as compared to restricting execution of the voice service to the one or more additional processor cores ([0103]-[0104] “Lower layers look at smaller time scales and produce higher level features with smaller dimensions to be sent to higher layers. This allows the architecture to look at a large time window, while reducing an amount of computations required”).
Regarding claim 18, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
one or more microphones configured to generate audio data and to provide the audio data to the first stage keyword detector, to the second stage keyword detector, to the voice service, or to any combination thereof ([0142] obtaining an acoustic signal which can be provided by a microphone coupled to the electronic device).
Regarding claim 19, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
one or more sensors configured to generate context data and to provide the context data to the voice service based on the change of state ([0142] obtaining an acoustic signal which can be provided by a microphone coupled to the electronic device).
Regarding claim 20, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
one or more output devices configured to provide a user notification indicating execution of the voice service ([0145] “Feedback from the low-power wakeword spotting system may generate output on a display”).
Regarding claim 21, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
wherein the voice service includes a voice assistant ([0003][0130] keyword spotting system for smart assistants).
Myer does not explicitly teach however Kim does explicitly teach:
wherein the second stage keyword detector is configured to confirm of disconfirm the keyword detection (Kim, Fig. 4, [0054][0063][0066] the method of performing speech recognition is started or ended based on detection of activation phrases).
Regarding claim 23, Myer in view of Kim discloses a method comprising:
generating, by a first stage keyword detector, a signal indicating a keyword detection (Figs. 3 and 6, [0130][0141] obtaining an acoustic signal and performing the first stage keyword spotting); and
based on the signal indicating the keyword detection and prior to confirmation of the keyword detection, initiating a change of state to enable a voice service that is in a low-power mode to be scheduled for execution on one or more reserved processor cores (Figs. 3 and 6, [0046][0047][0130][0141] “wherein the first core is a low-power processing core and the second core is a high-power processing core, when the first core determines the presence of at least one of a plurality of keywords in the acoustic signal the acoustic signal is provided to the second core for further processing … performing keyword verification”).
Regarding claims 25-28, Claims 25-28 are the corresponding medium claims to method claims 3-6. Therefore, claims 25-28 are rejected using the same rationale as applied to claims 3-6 above.
Regarding claim 37, Myer discloses an apparatus comprising:
means for generating a signal indicating a keyword detection (Figs. 3 and 6, [0130][0141] obtaining an acoustic signal and performing the first stage keyword spotting);
means for confirmation of the keyword detection ([0129]-[0132] a second higher powered processor may be used to perform the second stage keyword spotting); and
means for initiating a change of state based on the signal and prior to confirmation of the keyword detection, wherein the change of state enables a voice service that is in a low-power mode to be scheduled for execution on one or more reserved processor cores (Figs. 3 and 6, [0046][0047][0130][0141] “wherein the first core is a low-power processing core and the second core is a high-power processing core, when the first core determines the presence of at least one of a plurality of keywords in the acoustic signal the acoustic signal is provided to the second core for further processing … performing keyword verification”).
wherein, to initiate the change of state includes binding the voice service and the means for confirmation of the keyword detection ([0132]-[0136][0142] “having additional processing capability to verify the keyword and/or perform additional processing on the acoustic signal to process commands with in the acoustic signal (614). The additional processor can utilize a higher power core or processor to verify the keyword before performing additional processor”).
Myer does not explicitly teach however Kim does explicitly teach:
such that scheduling privileges of the voice service are based on scheduling privileges of the means for confirmation of the keyword detection, and wherein the voice service includes use of a speech recognition model ([0018][0019] each intelligent personal assistant (IPA) services 140,150,160 may be a different commercially available an intelligent personal assistant service, such as Microsoft Cortana™, Apple Siri™, Google Home™, or Amazon Alexa™, among others, such as a web server or another networked computing device).
Therefore, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to incorporate the two stages keyword spotting system as taught by Myer with the method of selecting an intelligent personal assistant among a plurality of IPA services based on detecting keywords/wakewords as taught by Kim to provide convenient user experience to interact with multiple human assistants at the same time (Kim, [0004]).
Regarding claim 43, Myer discloses a non-transitory computer-readable medium storing instructions that are executable by one or more processors to cause the one or more processors to:
obtain a signal indicating a keyword detection from a first stage keyword detector (Figs. 3 and 6, [0130][0141] obtaining an acoustic signal and performing the first stage keyword spotting); and
based on the signal indicating the keyword detection and prior to confirmation of the keyword detection by a second stage keyword detector, initiate a change of state to enable a voice service that is in a low-power mode to be scheduled for execution on one or more reserved processor core (Figs. 3 and 6, [0046][0047][0130][0141] “wherein the first core is a low-power processing core and the second core is a high-power processing core, when the first core determines the presence of at least one of a plurality of keywords in the acoustic signal the acoustic signal is provided to the second core for further processing … performing keyword verification”),
wherein initiation of the change of state includes binding the voice service and the second stage keyword detector ([0132]-[0136][0142] “having additional processing capability to verify the keyword and/or perform additional processing on the acoustic signal to process commands with in the acoustic signal (614). The additional processor can utilize a higher power core or processor to verify the keyword before performing additional processor”).
Myer does not explicitly teach however Kim does explicitly teach:
such that scheduling privileges of the voice service are based on scheduling privileges of the second stage keyword detector, and wherein the voice service includes use of a speech recognition model ([0018][0019] each intelligent personal assistant (IPA) services 140,150,160 may be a different commercially available an intelligent personal assistant service, such as Microsoft Cortana™, Apple Siri™, Google Home™, or Amazon Alexa™, among others, such as a web server or another networked computing device).
Therefore, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to incorporate the two stages keyword spotting system as taught by Myer with the method of selecting an intelligent personal assistant among a plurality of IPA services based on detecting keywords/wakewords as taught by Kim to provide convenient user experience to interact with multiple human assistants at the same time (Kim, [0004]).
Regarding claim 342, Myer in view of Kim discloses the device of claim 1, and Kim further discloses:
wherein, after the voice service and the second stage keyword detector are bound, the processor is further configured to:
initiate the second stage keyword detector as a foreground application from being a background application; and in response to confirmation of the keyword detection, initiate the voice service as a foreground application from being a background application (Kim, Fig. 4, step 405, [0054][0063][0066] selecting an IPA service that is referenced by one of the activation phrases detected).
Regarding claim 343, Myer in view of Kim discloses the device of claim 23, and Kim further discloses:
wherein after binding the voice service and the second stage keyword detector, the method further includes:
initiating the second stage keyword detector as a foreground application from being a background application; and in response to confirmation of the keyword detection, initiating the voice service as a foreground application from being a background application (Kim, Fig. 4, step 405, [0054][0063][0066] selecting an IPA service that is referenced by one of the activation phrases detected).
Regarding claim 344, Myer in view of Kim discloses the device of claim 1, and Kim further discloses:
wherein, based on disconfirmation of the keyword detection by the second stage keyword detector and in response to receipt of a reversion signal, the processor is further configured to revert to a state prior to initiation of the change of state (Kim, Fig. 4, steps 403, 409, [0054][0063][0066] if any activation phrases are not detected, the method of performing speech recognition is ended).
Claims 11 and 15-17 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Myer et al., (US Pub. 2023/0409102) in view of Kim (US Pub. 2018/0301147) and further in view of Dong et al., (US Pub. 2010/0088706).
Regarding claim 11, Myer in view of Kim discloses the device of claim 1, and Myer further discloses:
wherein the first stage keyword detector corresponds to includes, is included within, or is executed by a digital signal processor or [a low-power integrated circuit], and wherein the one or more reserved processor cores are cores of one or more application processors (Fig. 7, [0145] Processor(s) 702 includes low-power core 707).
Myer does not explicitly teach however Dong does explicitly teach including the bracketed limitation:
[a low-power integrated circuit], and wherein the one or more reserved processor cores are cores of one or more application processors (Dong, [0037] “The processor 702 may be implemented as one or more CPU chips, or may be part of one or more application specific integrated circuits”).
Therefore, it would have been obvious to one of ordinary skill before the effective filing date of the claimed invention to incorporate the Low-power keyword spotting system as taught by Myer in view of Kim with the method of scheduling the tasks for execution based on the privilege level as taught by Dong to allow a user to specify the thread time tolerance associated with a thread, which allows the scheduler to more intelligently decide when to schedule threads on the processor (Dong, [0018]).
Regarding claim 15, Myer in view of Kim discloses the device of claim 13.
Myer in view of Kim does not explicitly teach however Dong does explicitly teach:
a scheduler configured to assign processes to execute at particular processor cores based on scheduling groups of the processes, wherein the one or more additional processor cores are available for use by each scheduling group and the one or more reserved processor cores are not available for use by one or more of the scheduling groups (Dong, [0021]-[0037] the scheduler may preempt specific tasks to schedule a task).
Regarding claim 16, Myer in view of Kim discloses the device of claim 13.
Myer in view of Kim does not explicitly teach however Dong does explicitly teach:
a scheduler that is configured to allow a first group of processes to execute at the one or more additional processor cores and to allow a second group of processes to execute at the one or more reserved processor cores, and wherein the second group of processes includes or corresponds to a subset of the first group of processes that have a priority level greater than or equal to a threshold priority level (Dong, [0023]-[0025] a scheduler 118 may adjust and readjust queues with migrating tasks to any other cores that have capacity based on exceeding a specified threshold).
Regarding claim 17, Myer in view of Kim discloses the device of claim 13.
Myer in view of Kim does not explicitly teach however Dong does explicitly teach:
a scheduler that is configured to allow foreground and background processes to execute at the one or more additional processor cores and to restrict background processes from executing at the one or more reserved processor cores ([0020]-[0026] the scheduler may determine whether to place each task in RUN queue and may adjust/readjust the queues and migrate tasks to any other cores).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Please see attached form PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEONG-AH A. SHIN whose telephone number is (571)272-5933. The examiner can normally be reached 9 AM-3PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at 571-272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Seong-ah A. Shin
Primary Examiner
Art Unit 2659
/SEONG-AH A SHIN/Primary Examiner, Art Unit 2659
Prosecution Insights