Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to amendment/reconsideration filed 5/21/2026, the amendment/reconsideration has been considered. Claims 1, 3-7 and 9--22 are pending for examination.
Response to Arguments
Applicant's arguments been fully considered but not all arguments are persuasive. The applicant argues the following issues.
(A) Rejection under 35 U.S.C. 112
Issue: The applicant argues that the claimed terms “second timestamp” and “a threshold value” are supported by the originally filed application.
The arguments are considered convincing therefore the 112 rejection have been withdrawn.
(B) Rejection under 35 U.S.C. 103
Issue: The applicant’s arguments are moot in light of the new ground of rejections set forth below.
(C) Double Patenting Rejection
Issue: The applicant argues that the double patenting rejection should be withdrawn.
Examiner respectfully disagrees. See Examiner’s clarification and explanation in light of the newly amended claim scopes, in the Double Patenting rejection section below,.
Double Patenting
3. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
4, Claims 1, 3-7 and 9-10, and 12-14 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-21 of US Patent 10204104 (hereafter “Patent’104”).
As to claim 1, although the conflicting claims are not identical, they are not patentably distinct from each other because all limitations of the independent claim 1 of the instant application are claimed in claim 1 of Patent’104, i.e., claim 1 of Patent’104 is more specific. Thus the invention of claim 1 of Patent’104 is in effect a "species" of the "generic" invention of claim 1 of the instant application. It has been held that the generic invention is “anticipated” by the “species”. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
Specifically, claim 1 of Patent’104 discloses a method, comprising:
receiving, using a mobile device, a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device (see Patent’104, claim 1, ”a media playback device that is presenting media content to a user of the mobile device…. determining whether the text inputs include a trigger term that corresponds to a request to initiate a query relating to the presented media content and the query; in response to determining that the trigger term has been included in the text inputs, determining the media playback information from the media application that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information”);
providing, using the mobile device, a request for media playback information to the media playback device (see Patent’104, claim 1, “determining the media playback information from the media application that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information”);
receiving, at the mobile device from the media playback device, the media playback information including media content identification information related to the media content. a first timestamp determined by the media playback device and corresponding to a first time at which the trigger term was received. and a second timestamp determined by the media playback device and corresponding to a second time after an elapsed time period from when the trigger term was received (see Patent’104, claim 1, “determining the media playback information from the media application that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information, wherein the timing information comprises a first timestamp corresponding to a time at which the trigger term was received and a second timestamp corresponding to a time at which a last input in the one or more text inputs was received”, wherein “a time at which a last input in the one or more text inputs was received” is a second time after an elapsed time period from when the trigger term was received);
in response to receiving the media playback information, the mobile device enabled to cause a search to be performed at a search server that includes the query, the first timestamp, the second timestamp, and the media content identification information (see Patent’104, claim 1, “causing a search to be performed that includes the query, the timing information, and the media content identification information, wherein the query includes at least a portion of the one or more text inputs other than the trigger term”, wherein the timing information comprises the first time stamp and the second timestamp, see rejection to the preceding limitation); and
receiving and presenting, on a display of the mobile device, a portion of a search result responsive to the search (see Patent’104, claim 1, “obtaining a search result that is responsive to the query; and presenting at least a portion of the search result to the query on a mobile display associated with the mobile device”).
As to claim 12, see similar rejection to claim 1.
As to claim 3, Patent’104 discloses the method of claim 1, further comprising: retrieving the media playback information prior to receiving the query; and associating the media playback information with the query in response to receiving the trigger term (see Patent’104, claim 7).
As to claim 13, see similar rejection to claim 3.
As to claim 4, Patent’104 discloses the method of claim 1, further comprising: receiving a request to associate with the media playback device, wherein the media playback device is presenting the media content and executing a media application that exchanges the media playback information with a mobile application executing on the mobile device (Patent’104, claim 1, “receiving, from a mobile device, a request to associate with a media playback device that is presenting media content to a user of the mobile device, wherein a mobile application executing on the mobile device and a media application executing on the media playback device exchange media playback information”);
associating the mobile device with the media playback device (Patent’104, claim 1, “associating the mobile device with the media playback device”); and
activating an audio input device associated with the mobile device to receive a plurality of ambient sounds in response to associating with the media playback device, wherein the plurality of ambient sounds are converted to a plurality of input terms that include the query and the trigger term (Patent’104, claim 1, “activating a microphone associated with the mobile device to receive ambient sounds in response to associating with the media playback device; converting the received ambient sounds to one or more text inputs”).
As to claims 5, Patent’104 discloses the method of claim 4, further comprising: determining whether the audio input device associated with the mobile device is capable of receiving the plurality of ambient sounds; and activating an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the mobile device is not capable of receiving the plurality of ambient sounds (see Patent’104, claim 2).
As to claim 6, Patent’104 discloses the method of claim 4, further comprising extracting the trigger term and the query from the plurality of input terms by processing the plurality of ambient sounds using a speech recognition system (see Patent’104, claim 3).
As to claim 7, Patent’104 discloses the method of claim 4, further comprising:
requesting additional media playback information from the media application executing on the media playback device; and transmitting the query, the media playback information, and the additional media playback information to a search system to receive the search result (see Patent’104, claim 4).
As to claim 9, Patent’104 discloses the method of claim 4, further comprising:
receiving requests from a plurality of mobile devices to associate with the media playback device that is presenting the media content (see Patent’104, claim 8);
activating audio capturing devices associated with the portion of the plurality of mobile devices and the media playback device to receive the plurality of ambient sounds from users proximate to the plurality of mobile devices and the media playback device (see Patent’104, claim 8);
determining that a plurality of queries have been received based on the plurality of ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the media content (see Patent’104, claim 8);
obtaining a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries (see Patent’104, claim 8); and
causing the plurality of search results to be presented (see Patent’104, claim 8).
As to claim 14, see similar rejection to claim 9.
As to claim 10, Patent’104 discloses the method of claim 9, further comprising:
associating at least one search result of the plurality of search results with a respective mobile device from which the plurality of ambient sounds including the query was received (see Patent’104, claim 9); and
causing the portion of the search result to be presented on the respective mobile device (see Patent’104, claim 9).
5, Claims 11, 15-20 and 22 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over Patent’104, as applied to claim 1 above, and further in view of VAN OS et al. (US 20150382047, hereafter VAN).
As to claim 15, Patent’104 discloses the claimed invention substantially as discussed in claim 1, and further discloses that inputs were received continuously (see claim 1, “receive ambient sounds…converting the received ambient sounds to one or more text inputs” indicating that multiple text inputs corresponding to the received ambient sounds were continuously received) but does not expressly disclose that the continuously receiving is according to a threshold value. VAN discloses that continuously receiving is according to a threshold value (see citation in the 103 rejection to claim 15). Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Patent’104 with VAN. The suggestion/motivation of the combination would have been to determine end of a query (VAN, [0094]).
As to claim 11, Patent’104 in view of VAN discloses the method of claim 1, wherein the elapsed time period is set before at the mobile device before receiving the query and the trigger term (VAN, see citation and explanation in the 103 rejection to claim 11).
As to claim 16, Patent’104 in view of VAN discloses the system of claim 15, wherein the threshold value is set to no more than one second between receiving each text input (VAN, see citation and explanation in the 103 rejection to claim 16 below).
As to claim 22, Patent’104 in view of VAN discloses the system of claim 15, wherein the threshold value represents a duration of silence between consecutive inputs (see VAN, as cited in the 103 rejection to claim 22 below).
As to claim 17, see similar rejection to claim 4.
As to claim 18, see similar rejection to claim 5.
As to claim 19, see citation in rejection to claim 6.
As to claim 20, see citation in rejection to claim 7.
6. Claim 21 is rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over Patent’104, as applied to claim 1 above, and further in view of Buerk at al (“AVTransport:1 Service Template Version 1.01 For UPnP™ Version 1.0”, hereafter Buerk).
As to claim 21, Patent’104 discloses the claimed invention substantially as discussed in claim 1, but does not expressly disclose wherein mapping the first time and the second time to a playback timeline of the media content. Buerk discloses a concept for media playback device to map times to a respective playback timeline of a media content (see citation in the 103 rejection to claim 21). Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Patent’104 with Buerk. The suggestion/motivation of the combination would have been to obtain the relative timestamps (Buerk, pages 22-23).
**---------------
7. Claims 1, 3-7 and 9-10, and 12-14 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-21 of US Patent 12216700 (hereafter “Patent’700”).
As to claim 1, although the conflicting claims are not identical, they are not patentably distinct from each other because all limitations of the independent claim 1 of the instant application are claimed in claim 1 of Patent’700, i.e., claim 1 of Patent’700 is more specific. Thus the invention of claim 1 of Patent’700 is in effect a "species" of the "generic" invention of claim 1 of the instant application. It has been held that the generic invention is “anticipated” by the “species”. See In re Goodman, 29 USPQ2d 2010 (Fed. Cir. 1993).
Specifically, claim 1 of Patent’700 discloses a method, comprising:
receiving, using a mobile device, a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device (see Patent’700, claim 1, ” receiving, using a mobile device, a plurality of input terms; determining whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device, the query including at least a portion of the plurality of input terms other than the trigger term; in response to determining that the trigger term has been included in the plurality of input terms”);
providing, using the mobile device, a request for media playback information to the media playback device (see Patent’700, claim 1, “in response to determining that the trigger term has been included in the plurality of input terms, requesting, using the mobile device, media playback information from the media playback device;”);
receiving, at the mobile device from the media playback device, the media playback information including media content identification information related to the media content. a first timestamp determined by the media playback device and corresponding to a first time at which the trigger term was received. and a second timestamp determined by the media playback device and corresponding to a second time after an elapsed time period from when the trigger term was received (see Patent’700, claim 1, “receiving, at the mobile device from the media playback device and in response to the request, the media playback information including media content identification information related to the media content and timing information including a first timestamp corresponding to a first time at which the trigger term was received and a second timestamp corresponding to a second time at which a last term in the query was received”, wherein “a second time at which a last term in the query was received” is a second time after an elapsed time period from when the trigger term was received);
in response to receiving the media playback information, the mobile device enabled to cause a search to be performed at a search server that includes the query, the first timestamp, the second timestamp, and the media content identification information (see Patent’700, claim 1, “in response to receiving the media playback information the mobile device enabled to cause a search to be performed at a search server that includes the query, the timing information, and the media content identification information”, wherein the timing information comprises the first time stamp and the second timestamp, see rejection to the preceding limitation); and
receiving and presenting, on a display of the mobile device, a portion of a search result responsive to the search (see Patent’700, claim 1, “receiving and presenting, on a display of the mobile device, at least a portion of a search result responsive to the search, including the query”).
As to claim 12, see similar rejection to claim 1.
As to claim 3, Patent’700 discloses the method of claim 1, further comprising: retrieving the media playback information prior to receiving the query; and associating the media playback information with the query in response to receiving the trigger term (see Patent’700, claim 3).
As to claim 13, see similar rejection to claim 3.
As to claim 4, Patent’700 discloses the method of claim 1, further comprising: receiving a request to associate with the media playback device, wherein the media playback device is presenting the media content and executing a media application that exchanges the media playback information with a mobile application executing on the mobile device (Patent’700, claim 4);
associating the mobile device with the media playback device (Patent’700, claim 4); and
activating an audio input device associated with the mobile device to receive a plurality of ambient sounds in response to associating with the media playback device, wherein the plurality of ambient sounds are converted to a plurality of input terms that include the query and the trigger term (Patent’700, claim 4).
As to claims 5, Patent’700 discloses the method of claim 4, further comprising: determining whether the audio input device associated with the mobile device is capable of receiving the plurality of ambient sounds; and activating an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the mobile device is not capable of receiving the plurality of ambient sounds (see Patent’700, claim 5).
As to claim 6, Patent’700 discloses the method of claim 4, further comprising extracting the trigger term and the query from the plurality of input terms by processing the plurality of ambient sounds using a speech recognition system (see Patent’700, claim 6).
As to claim 7, Patent’700 discloses the method of claim 4, further comprising:
requesting additional media playback information from the media application executing on the media playback device; and transmitting the query, the media playback information, and the additional media playback information to a search system to receive the search result (see Patent’700, claim 7).
As to claim 9, Patent’700 discloses the method of claim 4, further comprising:
receiving requests from a plurality of mobile devices to associate with the media playback device that is presenting the media content (see Patent’700, claim 9);
activating audio capturing devices associated with the portion of the plurality of mobile devices and the media playback device to receive the plurality of ambient sounds from users proximate to the plurality of mobile devices and the media playback device (see Patent’700, claim 9);
determining that a plurality of queries have been received based on the plurality of ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the media content (see Patent’700, claim 9);
obtaining a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries (see Patent’700, claim 9); and
causing the plurality of search results to be presented (see Patent’700, claim 9).
As to claim 14, see similar rejection to claim 9.
As to claim 10, Patent’700 discloses the method of claim 9, further comprising:
associating at least one search result of the plurality of search results with a respective mobile device from which the plurality of ambient sounds including the query was received (see Patent’700, claim 10); and
causing the portion of the search result to be presented on the respective mobile device (see Patent’700, claim 10).
8. Claims 11, 15-20 and 22 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over Patent’700, as applied to claim 1 above, and further in view of VAN OS et al. (US 20150382047, hereafter VAN).
As to claim 15, Patent’700 discloses the claimed invention substantially as discussed in claim 1, and further discloses that inputs were received continuously (see claim 1, “receiving, using a mobile device, a plurality of input terms” indicating that multiple input terms were continuously received) but does not expressly disclose that the continuously receiving is according to a threshold value. VAN discloses that continuously receiving is according to a threshold value (see citation in the 103 rejection to claim 15). Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Patent’700 with VAN. The suggestion/motivation of the combination would have been to determine end of a query (VAN, [0094]).
As to claim 11, Patent’700 in view of VAN discloses the method of claim 1, wherein the elapsed time period is set before at the mobile device before receiving the query and the trigger term (VAN, see citation and explanation in the 103 rejection to claim 11).
As to claim 16, Patent’700 in view of VAN discloses the system of claim 15, wherein the threshold value is set to no more than one second between receiving each text input (VAN, see citation and explanation in the 103 rejection to claim 16 below).
As to claim 22, Patent’700 in view of VAN discloses the system of claim 15, wherein the threshold value represents a duration of silence between consecutive inputs (see VAN, as cited in the 103 rejection to claim 22 below).
As to claim 17, see similar rejection to claim 4.
As to claim 18, see similar rejection to claim 5.
As to claim 19, see citation in rejection to claim 6.
As to claim 20, see citation in rejection to claim 7.
9. Claim 21 is rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over Patent’700, as applied to claim 1 above, and further in view of Buerk at al (“AVTransport:1 Service Template Version 1.01 For UPnP™ Version 1.0”, hereafter Buerk).
As to claim 21, Patent’700 discloses the claimed invention substantially as discussed in claim 1, but does not expressly disclose wherein mapping the first time and the second time to a playback timeline of the media content. Buerk discloses a concept for media playback device to map times to a respective playback timeline of a media content (see citation in the 103 rejection to claim 21). Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Patent’700 with Buerk. The suggestion/motivation of the combination would have been to obtain the relative timestamps (Buerk, pages 22-23).
***-------------
10. Claims 1, 3-7 and 9-22 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-21 of US Patent 10984038 (hereafter “Patent’038”) in view of VAN OS et al. (US 20150382047, hereafter VAN), and further in view of Buerk at al (“AVTransport:1 Service Template Version 1.01 For UPnP™ Version 1.0”, hereafter Buerk).
As to claim 1, claim 1 of Patent’038 discloses a method, comprising:
receiving, using a mobile device, a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device (see Patent’038, claim 1, ”receiving, using a computing device, a plurality of input terms; determining whether the plurality of input terms include a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented; in response to determining that the trigger term has been included in the plurality of input terms, determining media playback information that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information”);
receiving, at the mobile device, media playback information including media content identification information related to the media content. a first timestamp determined by the media playback device and corresponding to a first time at which the trigger term was received. and a second timestamp determined by the media playback device and corresponding to a second time after an elapsed time period from when the trigger term was received (see Patent’038, claim 1, “in response to determining that the trigger term has been included in the plurality of input terms, determining media playback information that includes timing information corresponding to a time at which the query was received during the presentation of the media content and media content identification information, wherein the timing information comprises a first timestamp corresponding to a time at which the trigger term was received and a second timestamp corresponding to a time at which a last term in the query was received”, wherein “a time at which a last term in the query was received” is a second time after an elapsed time period from when the trigger term was received);
in response to receiving the media playback information, the mobile device enabled to cause a search to be performed at a search server that includes the query, the first timestamp, the second timestamp, and the media content identification information (see Patent’038, claim 1, “causing a search to be performed that includes the query, the timing information, and the media content identification information, wherein the query includes at least a portion of the plurality of input terms other than the trigger term”, wherein the timing information comprises the first time stamp and the second timestamp, see rejection to the preceding limitation); and
receiving and presenting, on a display of the mobile device, a portion of a search result responsive to the search (see Patent’038, claim 1, “obtaining a search result that is responsive to the query; and presenting at least a portion of the search result to the query on a display associated with the computing device”),
but does not expressly disclose providing, using the mobile device, a request for the media playback information to a media playback device or that the receiving of said media playback information is from the media playback device.
VAN discloses providing, using a mobile device, a request for media playback information to the media playback device and receiving media playback information from the media playback device including content identifier information (see citation in the 103 rejection to claim 1). Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Patent’038 with VAN. The suggestion/motivation of the combination would have been to gather context information (VAN, [0072]-[0073]).
Buerk discloses a concept for media playback information received from a media player by another device to comprise a relative first timestamp, and a relative second timestamp, both are determined by the media playback device (see citation in the 103 rejection to claim 1). Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine Patent’038 with Buerk. The result of the modification would have been for the mobile device of Patent’038 to call GetRelTime as disclosed by Buerk to get the relative first timestamp and relative second timestamp respectively. The suggestion/motivation of the combination would have been to convert to relative times (Buerk, pages 22-23).
As to claim 12, see similar rejection to claim 1.
As to claim 3, Patent’038 in view of VAN and Buerk discloses the method of claim 1, further comprising: retrieving the media playback information prior to receiving the query; and associating the media playback information with the query in response to receiving the trigger term (see Patent’038, claim 3).
As to claim 13, see similar rejection to claim 3.
As to claim 4, Patent’038 in view of VAN and Buerk discloses the method of claim 1, further comprising: receiving a request to associate with the media playback device, wherein the media playback device is presenting the media content and executing a media application that exchanges the media playback information with a mobile application executing on the mobile device (Patent’038, claim 4);
associating the mobile device with the media playback device (Patent’038, claim 4); and
activating an audio input device associated with the mobile device to receive a plurality of ambient sounds in response to associating with the media playback device, wherein the plurality of ambient sounds are converted to a plurality of input terms that include the query and the trigger term (Patent’038, claim 4).
As to claims 5, Patent’038 in view of VAN and Buerk discloses the method of claim 4, further comprising: determining whether the audio input device associated with the mobile device is capable of receiving the plurality of ambient sounds; and activating an audio capturing device associated with the media playback device in response to determining that the audio input device associated with the mobile device is not capable of receiving the plurality of ambient sounds (see Patent’038, claim 5).
As to claim 6, Patent’038 in view of VAN and Buerk discloses the method of claim 4, further comprising extracting the trigger term and the query from the plurality of input terms by processing the plurality of ambient sounds using a speech recognition system (see Patent’038, claim 6).
As to claim 7, Patent’038 in view of VAN and Buerk discloses the method of claim 4, further comprising:
requesting additional media playback information from the media application executing on the media playback device; and transmitting the query, the media playback information, and the additional media playback information to a search system to receive the search result (see Patent’038, claim 7).
As to claim 9, Patent’038 in view of VAN and Buerk discloses the method of claim 4, further comprising:
receiving requests from a plurality of mobile devices to associate with the media playback device that is presenting the media content (see Patent’038, claim 9);
activating audio capturing devices associated with the portion of the plurality of mobile devices and the media playback device to receive the plurality of ambient sounds from users proximate to the plurality of mobile devices and the media playback device (see Patent’038, claim 9);
determining that a plurality of queries have been received based on the plurality of ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the media content (see Patent’038, claim 9);
obtaining a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries (see Patent’038, claim 9); and
causing the plurality of search results to be presented (see Patent’038, claim 9).
As to claim 14, see similar rejection to claim 9.
As to claim 10, Patent’038 in view of VAN and Buerk discloses the method of claim 9, further comprising:
associating at least one search result of the plurality of search results with a respective mobile device from which the plurality of ambient sounds including the query was received (see Patent’038, claim 10); and
causing the portion of the search result to be presented on the respective mobile device (see Patent’038, claim 10).
As to claim 15, see similar rejection to claim 1, wherein a hardware processor is implied. In addition, Patent’038 in view of VAN and BUERK further discloses that inputs were received continuously (see Patent’038, claim 1, “receiving, using a computing device, a plurality of input terms” indicating that multiple input terms were continuously received), and that the continuously receiving is according to a threshold value (VAN, see citation in the 103 rejection to claim 15).
As to claim 11, Patent’038 in view of VAN and Buerk discloses the method of claim 1, wherein the elapsed time period is set before at the mobile device before receiving the query and the trigger term (VAN, see citation and explanation in the 103 rejection to claim 11).
As to claim 16, Patent’038 in view of VAN and Buerk discloses the system of claim 15, wherein the threshold value is set to no more than one second between receiving each text input (VAN, see citation and explanation in the 103 rejection to claim 16 below).
As to claim 22, Patent’038 in view of VAN and Buerk discloses the system of claim 15, wherein the threshold value represents a duration of silence between consecutive inputs (see VAN, as cited in the 103 rejection to claim 22 below).
As to claim 17, see similar rejection to claim 4.
As to claim 18, see similar rejection to claim 5.
As to claim 19, see citation in rejection to claim 6.
As to claim 20, see citation in rejection to claim 7.
As to claim 21, Patent’038 in view of VAN and Buerk discloses the method of claim 1, wherein the media playback device determines the first timestamp and the second timestamp by mapping the first time and the second time to a playback timeline of the media content (Buerk, see citation in the 103 rejection to claim 21).
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
14. Claims 1, 3-4, 6-7, 9-17, 19-22 are rejected under 35 U.S.C. 103 as being unpatentable over VAN OS et al. (US 20150382047, hereafter VAN) in view of Buerk at al (“AVTransport:1 Service Template Version 1.01 For UPnP™ Version 1.0”, hereafter Buerk).
As to claim 1, VAN discloses a method, comprising:
receiving, using a mobile device (Fig. 1, User Device 102, see [0057], “User device 102 can include any electronic device, such as a mobile phone (e.g., a smartphone)”),
a query and a trigger term that corresponds to a request to initiate the query relating to media content being presented at a media playback device ([0091], “In one example, a virtual assistant can be triggered to listen for speech input containing a command or query (or to commence recording of speech input for subsequent processing or commence processing in real-time of speech input). Listening can be triggered in a variety of ways, including indications such as a user pressing a physical button on remote control 106, a user pressing a physical button on user device 102, a user pressing a virtual button on user device 102, a user uttering a trigger phrase that is recognizable by an always-listening device (e.g., uttering “Hey Assistant” to commence listening for a command)”; [0160], “user queries can include questions about media content shown on display 112. For example, transcription 1916 includes a query requesting identification of actresses (“Who are those actresses?”)”; [0051], “speech input can be received from a user including a query associated with content show on a television display”; [0057], “Television set-top box 104 can include any media control device, such as cable box, satellite box, video player, video streaming device,… a combination of such devices”);
providing, using the mobile device, a request for media playback information to the media playback device (see Fig.2, wherein the virtual assistant module 264 is on the User Device 102; see [0072], “virtual assistant client module 264 can utilize the various sensors, subsystems, and peripheral devices to gather additional information from the surrounding environment of user device to establish a context associated with a user, the current user interaction and/or the current user input. Such context can include information from other devices, such as from television set-top box 104. In some examples, virtual assistant client module 264 can provide the contextual information or a subset thereof with the user input to the virtual assistant server to hep infer the user’s intent”; [0073], “The contextual information can further include information associated with the state of connected device, or other devices associated with the user (e.g., media content displayed by television set-top box 104)…. Any of these types of contextual information can be provided to virtual assistant server 114”);
receiving, at the mobile device from the media playback device, the media playback information including media content identification information related to the media content ([0072], “virtual assistant client module 264 can utilize the various sensors, subsystems, and peripheral devices to gather additional information from the surrounding environment of user device to establish a context associated with a user, the current user interaction and/or the current user input. Such context can include information from other devices, such as from television set-top box 104. In some examples, virtual assistant client module 264 can provide the contextual information or a subset thereof with the user input to the virtual assistant server to hep infer the user’s intent”; [0073], “The contextual information can further include information associated with the state of connected device, or other devices associated with the user (e.g., media content displayed by television set-top box 104)…. Any of these types of contextual information can be provided to virtual assistant server 114”; [0073], “The contextual information can further include information associated with the state of connected device, or other devices associated with the user (e.g., media content displayed by television set-top box 104”);
in response to receiving the media playback information, the mobile device enabled to cause a search to be performed at a search server that includes the query, a first timestamp corresponding to a first time at which the trigger term was received, and a second timestamp corresponding to a second time after an elapsed time period from when the trigger term was received, and the media content identification information (see [0162], “In one example, a character and/or actor list associated with video 480 can be used to identify all or the most prominent actresses appearing in video 480, which might include actresses 1910, 1912, and 1914. The identified actresses can be returned as a possible result (including fewer or additional actresses if the metadata resolution is coarse). In another example, however, metadata associated with video 480 can include an identification of which actors and actresses appear on screen at a given time, and the actresses appearing at the time of the query can be determined from that metadata (e.g., specifically identifying actresses 1910, 1912, and 1914)”; [0163] In some examples, the content shown on display 112 can change during submission of a query and determination of a response. As such, a viewing history of media content can be used to determine user intent and determine the response to a query. For example, should video 480 move to another view (e.g., with other characters) before a response to the query is generated, the result of the query can be determined based on the user's view at the time the query was spoken (e.g., the characters shown on screen at the time the user initiated the query). In some instances, a user might pause playing media to issue a query, and the content shown when paused can be used with associated metadata to determine user intent and a response to the query”. As disclosed here, the server is provided with the first timestamp corresponding to “the time the user initiated the query” (to handle the situation when “content shown on display 112 can change during submission of a query and determination of a response”), and also the second stamp corresponding to “at the time of the query” which is after an elapsed time period from when the trigger term was received. Here, the time when the trigger term was received can be considered a time that the user initiated the query. The server’s receipt of the content identification information from the client is implied); and
receiving and presenting, on a display of the mobile device, a portion of a search result responsive to the search ([0051], “Television display content can also be used as contextual input for determining user intent from speech input. For example, speech input can be received from a user, including a query associated with content shown on a television display. The user intent of the query can be determined based on the content shown on the television display as well as a viewing history of media content on the television display (e.g., disambiguating the query based on characters in a playing TV show). The results of the query can then be displayed based on the determined user intent”. See also Fig. 14).
However, VAN does not expressly disclose receiving by the mobile device, from the media playback device, the first timestamp and the second timestamp which are determined by the media playback device. Buerk discloses receiving from a media player relative timestamps determined by the media playback in response to one or more requests (see pages 22-23, “2.4.5. GetPositionInfo” with an argument of RelTime as an output, with relatedStateVarable being “RelativeTimePosition”. See page 15, “2.2.22. RelativeTimePosition This state variable contains the current position, in terms of time, from the beginning of the current track”, wherein GetPostionInfo returns the updated RelTime each time it is called therefore multiple RelTime timestamps can be generated by and obtained from the media player. See also page 5, “This service type enables control over the transport of audio and video streams. The service type defines a ‘common’ model for A/V transport control suitable for a generic user interface. It can be used to control a wide variety of disc, tape and solid-state based media devices such as CD players, VCRs and MP3 players”).
Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine VAN with Buerk. The result of the modification would have been for the User System of VAN, at “the time the user initiated the query”, and “at the time of the query”, to call GetRelTime as disclosed by Buerk to obtain the respective relative timestamps. The suggestion/motivation of the combination would have been to obtain the relative timestamps so that the server can directly compare with the times in the metadata of the content to find out user intent (VAN, [0162]-[0163]; Buerk, page 22-23).
As to claim 12, see similar rejection to claim 1.
As to claim 15, see similar rejection to claim 1, and wherein a hardware processor of the mobile device is implied. Regarding the additional limitation of “inputs were received continuously according to a threshold value”, see VAN, paragraph [0094], “Upon detecting that the user has finished speaking (e.g., based on a pause, speech interpretation indicating the end of a query, or any other endpoint detection method, processing confirmation 488…” wherein detecting an end of a query “based on a pause” indicates that inputs were received continuously according to a threshold value, i.e., until a silence duration exceeds the “pause” threshold value.
As to claim 3, VAN in view of Buerk discloses the method of claim 1, further comprising: retrieving the media playback information prior to receiving the query (this limitation conflicts with the parent claim 1, therefore Examiner interprets in light of the parent claim 1 that the retrieving the media playback information is after receiving the query from the user, VAN, see citation in rejection to claim 1); and
associating the media playback information with the query in response to receiving the trigger term (VAN, see citation in rejection to claim 1, wherein both the media playback information and the query are sent to the virtual assistant server so they are associated. It is to be noted that the claim does not require a specific way to associate).
As to claim 13, see similar rejection to claim 3.
As to claim 4, VAN in view of Buerk discloses the method of claim 1, further comprising: receiving a request to associate with the media playback device, wherein the media playback device is presenting the media content and executing a media application that exchanges the media playback information with a mobile application executing on the mobile device (VAN, [0059], “speech input including media-related queries or commands for the virtual assistant can be received at user device 102 and/or remote control 106, and the speech input can be used to cause media-related tasks to be executed on television set-top box 104. Likewise, tactile commands for controlling media on television set-top box 104 can be received at user device 102”; [0091], “a virtual assistant can be triggered to listen for speech input containing a command or query (or to commence recording of speech input for subsequent processing or commence processing in real-time of speech input). Listening can be triggered in a variety of ways, including indications such as a user pressing a physical button on remote control 106, a user pressing a physical button on user device 102, a user pressing a virtual button on user device 102, a user uttering a trigger phrase that is recognizable by an always-listening device (e.g., uttering “Hey Assistant” to commence listening for a command)”, wherein triggering the virtual assistant to listen for speech input containing a command or a query directed towards the set-top box serves as a request to associate with the set-top box which is presenting the media content and executing a media application that exchanges the media playback information with the user device, wherein the association being for the subsequent inputs received by the user device to query or command media-related tasks executed on the television set-top box);
associating the mobile device with the media playback device (see citation above); and
activating an audio input device associated with the mobile device to receive a plurality of ambient sounds in response to associating with the media playback device, wherein the plurality of ambient sounds are converted to a plurality of input terms that include the query and the trigger term (see citation in rejection to limitation 1, wherein triggering the virtual assistant to receive speech inputs indicating activating an audio input device associated with the user device to start to receive a plurality of ambient sounds in response to the triggering of the virtual assistant, wherein the plurality of ambient sounds are converted to a plurality of input terms that include the query and the trigger term. See [0095], “After the captured speech input is interpreted as text (or in response to successfully converting the speech input to text), command receipt confirmation 490 and/or transcription 492 shown in FIG. 4E can be displayed to confirm that the system received and interpreted the speech input”).
As to claim 17, see similar rejection to claim 4.
As to claim 6, VAN in view of Buerk discloses the method of claim 4, further comprising extracting the trigger term and the query from the plurality of input terms by processing the plurality of ambient sounds using a speech recognition system (see citation in rejection to claim 4, e.g., VAN, [0095], “After the captured speech input is interpreted as text (or in response to successfully converting the speech input to text), command receipt confirmation 490 and/or transcription 492 shown in FIG. 4E can be displayed to confirm that the system received and interpreted the speech input”. See also VAN, [0070], “provide the client-side functionalities of the virtual assistant. User data 266 can also be used in performing speech recognition in support of the virtual assistant”).
As to claim 19, see similar rejection to claim 6.
As to claim 7, VAN in view of Buerk discloses the method of claim 4, further comprising:
requesting additional media playback information from the media application executing on the media playback device; and transmitting the query, the media playback information, and the additional media playback information to a search system to receive the search result ([0085], “The contextual information can further include information associated with the physical state of television set-top box 104 (e.g., device location, device temperature, power level, etc.) or the software state of television set-top box 104 (e.g., running processes, installed applications, past and present network activities, background services, error logs, resources usage, etc.). The contextual information can further include information associated with the state of connected devices or other devices associated with the user (e.g., content displayed on user device 102, playable content on user device 102, etc.). Any of these types of contextual information can be provided to virtual assistant server 114 (or used on television set-top box 104 itself) as contextual information associated with a user input”).
As to claim 20, see similar rejection to claim 7.
As to claim 9, VAN in view of Buerk discloses the method of claim 4, further comprising:
receiving requests from a plurality of mobile devices to associate with the media playback device that is presenting the media content (VAN, Fig. 2, a virtual assistant residing in the User Device 102; Fig. 3, a virtual assistant residing in the Set-top box, wherein both are associated with the media player functional entity of the set-top box when the respective virtual assistant is triggered, see citation in rejection to claim 4);
activating audio capturing devices associated with the portion of the plurality of mobile devices and the media playback device to receive the plurality of ambient sounds from users proximate to the plurality of mobile devices and the media playback device (it is unclear what is considered “the portion of the plurality of mobile devices and the media playback device”. For the sake of the examination, Examiner assumes anything related to the plurality of mobile devices and the media playback device. Moreover, since the recited “the plurality of ambient sounds” refers to “a plurality of ambient sounds” as cited in the parent claim 4, see citation therein wherein such a plurality of ambient sounds is received by an audio input device associated with the mobile device and which is converted to “a plurality of input terms that include the query and the trigger term” by the respective virtual assistant, therefore the same functionality apples to the virtual assistant on the user device and on the set-top box. See citation in rejection to claim 4, wherein an audio capturing device associated with each virtual assistant is activated in response to triggering the virtual assistant);
determining that a plurality of queries have been received based on the plurality of ambient sounds, wherein each of the plurality of queries includes the trigger term and the query relating to the media content (see Examiner’s citation in rejection to claim 4 and Examiner’s explanation in the preceding limitations. It is to be noted that the plurality of ambient sounds refers to a plurality of ambient sounds as recited in parent claim 4 that is received by an audio input device associated with the mobile device which is converted to “a plurality of input terms that include the query and the trigger term”, therefore the same functionality apples to the virtual assistant on the user device and on the set-top box);
obtaining a plurality of search results, wherein each of the plurality of search results is responsive to one of the plurality of queries (VAN, see citation in rejection to claim 1; and VAN, Fig 14); and
causing the plurality of search results to be presented (Fig. 14).
As to claim 14, see similar rejection to claim 9.
As to claim 10, VAN in view of Buerk discloses the method of claim 9, further comprising:
associating at least one search result of the plurality of search results with a respective mobile device from which the plurality of ambient sounds including the query was received (VAN, see citation in rejection to claim 4 and claim 9, wherein each search query is associated with a respective virtual assistant therefore respective user device); and
causing the portion of the search result to be presented on the respective mobile device (Fig. 14 showing search results displayed on a user device. See Fig. 8 and Fig. 9 wherein the result is displayed on the display associated with set-top box).
As to claim 11, VAN in view of Buerk discloses the method of claim 1, wherein the elapsed time period is set before at the mobile device before receiving the query and the trigger term (VAN, [0091]-[0095], e.g., “a virtual assistant can be triggered to listen for speech input containing a command or query…Listening can be triggered in a variety of ways, including…a user uttering a trigger phrase that is recognizable by an always-listening device (e.g., uttering “Hey Assistant” to commence listening for a command)…. In response to receiving the indication to listen for speech input, readiness confirmation 476 can be displayed… As the user begins to speak, listening confirmation 487 shown in Fig. 4C can be displayed… readiness confirmation 486 can be displayed for a predetermined amount of time (e.g., 500 milliseconds, 1 second, 3 second, etc.) after which listening confirmation 487 can be displayed… Upon detecting that the user has finished speaking (e.g., based on a pause, speech interpretation indicating the end of a query or any other endpoint detection method) processing confirmation 488 shown in Fig. 4D can be displayed to confirm that the system finishing capturing the speech input is processing the speech input…. After the captured speech input is interpreted as text…. command receipt confirmation 490 and/or transcription 492 shown in Fig. 4E can be displayed to confirm that the system received and interpreted the speech input.”
As disclosed, in response to receiving the indication/trigger phase, readiness confirmation 486 is displayed for a predetermined amount of time, then listening confirmation 487 is displayed, and then upon detecting end of speech inputs, processing confirmation and command receipt confirmation are displayed subsequently, wherein the time of displaying the processing confirmation after detecting end of speech inputs is equivalent to a time of the query (the second time). Because the earlier step of listening confirmation is already after a predetermined amount of time from when the trigger term was received, the time of displaying processing confirmation after detecting end of speck inputs (the second time) is necessarily after the predetermined amount of time from when the trigger term was received, because the query speech input is received after the trigger term was received).
As to claim 16, VAN in view of Buerk discloses the system of claim 15, wherein the threshold value is set to no more than one second between receiving each text input (VAN, paragraph [0094], “Upon detecting that the user has finished speaking (e.g., based on a pause, speech interpretation indicating the end of a query, or any other endpoint detection method, processing confirmation 488…” wherein detecting an end of a query “based on a pause” indicates that inputs were received continuously according to a threshold value, i.e., until a silence duration exceeds the “pause” threshold value, wherein choosing a specific value to serve as a threshold value such as no more than one second is a known design choice, e.g., see [0093], “(e.g., 500 milliseconds…, 1 second, 3 seconds, etc..)”. Before the effective filing date of the invention, it would have been obvious for an ordinary skilled in the art to combine VAN’s teaching of a pause threshold with a known design choice of choosing a threshold value to be less than one second, as supported in VAN, [0093]. The suggestion/motivation would have been to allow various threshold values).
As to claim 21, VAN in view of Buerk discloses the method of claim 1, wherein the media playback device determines the first timestamp and the second timestamp by mapping the first time and the second time to a playback timeline of the media content (VAN in view of Buerk, see rejection to claim 1, including the obviousness analysis contained therein, wherein the respective RelTime is obtained by calling the “GetPositionInfo” to get “RelativeTimePosition” which is a state variable that contains the current position, in terms of time, from the beginning of the current track”, therefore is obtained by mapping the then current time (e.g., the time the user initiated the query”, and “at the time of the query) to a playback timeline of the media content).
As to claim 22, VAN in view of Buerk discloses the system of claim 15, wherein the threshold value represents a duration of silence between consecutive inputs (see VAN, paragraph [0094], “Upon detecting that the user has finished speaking (e.g., based on a pause, speech interpretation indicating the end of a query, or any other endpoint detection method, processing confirmation 488…” wherein detecting an end of a query “based on a pause” indicates that inputs were received continuously according to a threshold value, i.e., until a silence duration meets/exceeds the “pause” threshold value), and
the hardware processor is further configured to identify the second time in response to the duration of silence exceeding the threshold value (see VAN, paragraph [0094], “Upon detecting that the user has finished speaking (e.g., based on a pause, speech interpretation indicating the end of a query, or any other endpoint detection method, processing confirmation 488…” wherein detecting an end of a query “based on a pause” then then processing the query indicates that the second time (the time of the query” is identified in response to a duration of silence exceeding the pause threshold value).
Conclusion
15. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUA FAN whose telephone number is (571)270-5311. The examiner can normally be reached on 9-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nasser Goodarzi, can be reached at (571) 272-4195. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUA FAN/ Primary Examiner, Art Unit 2426
Prosecution Insights