DETAILED ACTION
The amendments filed 7/15/2025 have been entered. Claims 1-36 are pending.
Claim Objections
Claim 13 is objected to because of the following informalities:
Claim 13, last line: “receiving, from the map server, map data.” should not be indented under “transmitting, to a map server:” The indentation of the last line should be decreased.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 9, 11-17, 22, 24-28, 33, 35, and 36 are rejected under 35 U.S.C. 103 as being unpatentable over Gruber et al. (US Publication No. 2012/0245944).
Gruber teaches:
Re claim 1. A computer system (Figure 7; paragraph [0123]: “enable the operation of applications and services via natural language dialog … including … navigation (maps and directions)”; and paragraph [0125]: “the various types of functions … may be implemented at one or more client systems(s), at one or more server systems (s), and/or combinations thereof), comprising:
one or more processors (processor(s) 63, Figure 4; and paragraphs [0075-0078]); and
memory storing one or more programs configured to be executed by the one or more processors (storage device 1208 and memory 1210, Figure 4; and paragraph [0083]), the one or more programs including instructions for:
receiving:
data corresponding to an audio input comprising user speech (paragraph [0123]: “enable the operation of applications and services via natural language dialog” and paragraph [0128]: “speech-to-text and natural language understanding technology”; dialog processing on servers 1340, Figure 7); and
contextual data of an electronic device, wherein the contextual data represents a context of the user speech (paragraphs [0130-0134]: “use information from personal interaction history (e.g., dialog history, previous selections from results, and the like), personal physical context (e.g., user's location and time), and personal information gathered in the context of interaction (e.g., name, email addresses, physical addresses, phone numbers, account numbers, preferences, and the like)”; paragraphs [0142-146]: “Global Positioning System (GPS) and Assisted GPS (A-GPS) on mobile phones. In one embodiment, location information is combined with explicit user input. … Time information from clocks on client devices. … time may be used in the context of user requests, such as for instance, to interpret phrases such as "in an hour" and "tonight".”; and paragraph [0667]: “Language interpreter component 1070 parses the text input and generates a list of possible interpretations of the user's intent 290. In one parse … "around here" is associated with a location parameter describing a distance from a global sensor reading (for example, the user's location as given by GPS on a mobile device).”);
generating a textual representation of the user speech based on the data corresponding to the audio input (paragraph [0128]: “speech-to-text and natural language understanding technology”).
Gruber fails to specifically teach: (re claim 1)
transmitting, to a map server:
data associated with the textual representation of the user speech; and
the contextual data; and
receiving, from the map server, map data.
Gruber teaches, at paragraph [0123], enabling operation of navigation applications via natural language dialog; at paragraph [0125], such functions may be implemented at one or more client systems, at one or more server systems, and/or combinations thereof. Gruber further teaches, at paragraph [0136], interfacing with external services, dynamically determining which services may provide information for a specific user request, mapping parameters of the user request to different service APIs, and calling services. Paragraphs [0130-0134] teach using context information, such as location and time information, to better interpret user language input. Paragraph [0170] teaches the functions may include showing directions on a map. Paragraph [0667] teaches parsing a user input such as “around here” so as to utilize a location given by GPS on a mobile device. The combined teachings of Gruber suggest sending a user request, including context information for the user request (e.g., location information), to an external server executing a navigation application, and receiving the desired navigation results from the external server. This allows for such a system to use natural language to interact with such services, while better interpreting user language input.
In view of Gruber’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the system as taught by Gruber, (re claim 1) transmitting, to a map server: data associated with the textual representation of the user speech; and the contextual data; and receiving, from the map server, map data, with a reasonable expectation of success, since Gruber suggests sending a user request, including context information for the user request (e.g., location information), to an external server executing a navigation application, and receiving the desired navigation results from the external server. This allows for such a system to use natural language to interact with such services, while better interpreting user language input.
Gruber further teaches:
Re claim 2. Wherein the contextual data comprises a location of the electronic device (paragraphs [0130, 0134, 0142, and 0667]).
Re claim 3. Wherein the contextual data comprises a software state of the electronic device (paragraph [0012]).
Re claim 4. Wherein the contextual data comprises a content being displayed by the electronic device (paragraph [0317]: “user selection 136 among the displayed choices can be achieved by any mode of input, including for example any of the modes of multimodal input described in connection with FIG. 16. Such input modes include, without limitation, actively elicited typed input 2610, actively elicited speech input 2620,”; and paragraph [0947]: “context-specific commands, such as "show these on a map"”).
Re claim 9. Wherein the map data comprises:
one or more map tiles;
a set of geographic coordinates;
routing information from a start location to an end location (paragraph [0123]: “navigation (maps and directions)”); or
traffic data (paragraph [0123]: “monitoring traffic”).
Re claim 11. Wherein the electronic device is a mobile phone (abstract).
Re claim 12. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system (Figure 7; paragraph [0123]: “enable the operation of applications and services via natural language dialog … including … navigation (maps and directions)”; and paragraph [0125]: “the various types of functions … may be implemented at one or more client systems(s), at one or more server systems (s), and/or combinations thereof), the one or more programs including instructions for:
receiving:
data corresponding to an audio input comprising user speech (paragraph [0123]: “enable the operation of applications and services via natural language dialog” and paragraph [0128]: “speech-to-text and natural language understanding technology”; dialog processing on servers 1340, Figure 7); and
contextual data of an electronic device, wherein the contextual data represents a context of the user speech (paragraphs [0130-0134]: “use information from personal interaction history (e.g., dialog history, previous selections from results, and the like), personal physical context (e.g., user's location and time), and personal information gathered in the context of interaction (e.g., name, email addresses, physical addresses, phone numbers, account numbers, preferences, and the like)”; paragraphs [0142-146]: “Global Positioning System (GPS) and Assisted GPS (A-GPS) on mobile phones. In one embodiment, location information is combined with explicit user input. … Time information from clocks on client devices. … time may be used in the context of user requests, such as for instance, to interpret phrases such as "in an hour" and "tonight".”; and paragraph [0667]: “Language interpreter component 1070 parses the text input and generates a list of possible interpretations of the user's intent 290. In one parse … "around here" is associated with a location parameter describing a distance from a global sensor reading (for example, the user's location as given by GPS on a mobile device).”);
generating a textual representation of the user speech based on the data corresponding to the audio input (paragraph [0128]: “speech-to-text and natural language understanding technology”).
Gruber fails to specifically teach: (re claim 12)
transmitting, to a map server:
data associated with the textual representation of the user speech; and
the contextual data; and
receiving, from the map server, map data.
Gruber teaches, at paragraph [0123], enabling operation of navigation applications via natural language dialog; at paragraph [0125], such functions may be implemented at one or more client systems, at one or more server systems, and/or combinations thereof. Gruber further teaches, at paragraph [0136], interfacing with external services, dynamically determining which services may provide information for a specific user request, mapping parameters of the user request to different service APIs, and calling services. Paragraphs [0130-0134] teach using context information, such as location and time information, to better interpret user language input. Paragraph [0170] teaches the functions may include showing directions on a map. Paragraph [0667] teaches parsing a user input such as “around here” so as to utilize a location given by GPS on a mobile device. The combined teachings of Gruber suggest sending a user request, including context information for the user request (e.g., location information), to an external server executing a navigation application, and receiving the desired navigation results from the external server. This allows for such a system to use natural language to interact with such services, while better interpreting user language input.
In view of Gruber’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the medium as taught by Gruber, (re claim 12) transmitting, to a map server: data associated with the textual representation of the user speech; and the contextual data; and receiving, from the map server, map data, with a reasonable expectation of success, since Gruber suggests sending a user request, including context information for the user request (e.g., location information), to an external server executing a navigation application, and receiving the desired navigation results from the external server. This allows for such a system to use natural language to interact with such services, while better interpreting user language input.
Gruber further teaches:
Re claim 13. A computer-implemented method comprising:
at a computer system (Figure 7; paragraph [0123]: “enable the operation of applications and services via natural language dialog … including … navigation (maps and directions)”; and paragraph [0125]: “the various types of functions … may be implemented at one or more client systems(s), at one or more server systems (s), and/or combinations thereof):
receiving:
data corresponding to an audio input comprising user speech (paragraph [0123]: “enable the operation of applications and services via natural language dialog” and paragraph [0128]: “speech-to-text and natural language understanding technology”; dialog processing on servers 1340, Figure 7); and
contextual data of an electronic device, wherein the contextual data represents a context of the user speech (paragraphs [0130-0134]: “use information from personal interaction history (e.g., dialog history, previous selections from results, and the like), personal physical context (e.g., user's location and time), and personal information gathered in the context of interaction (e.g., name, email addresses, physical addresses, phone numbers, account numbers, preferences, and the like)”; paragraphs [0142-146]: “Global Positioning System (GPS) and Assisted GPS (A-GPS) on mobile phones. In one embodiment, location information is combined with explicit user input. … Time information from clocks on client devices. … time may be used in the context of user requests, such as for instance, to interpret phrases such as "in an hour" and "tonight".”; and paragraph [0667]: “Language interpreter component 1070 parses the text input and generates a list of possible interpretations of the user's intent 290. In one parse … "around here" is associated with a location parameter describing a distance from a global sensor reading (for example, the user's location as given by GPS on a mobile device).”);
generating a textual representation of the user speech based on the data corresponding to the audio input (paragraph [0128]: “speech-to-text and natural language understanding technology”).
Gruber fails to specifically teach: (re claim 13)
transmitting, to a map server:
data associated with the textual representation of the user speech and the contextual data; and
receiving, from the map server, map data.
Gruber teaches, at paragraph [0123], enabling operation of navigation applications via natural language dialog; at paragraph [0125], such functions may be implemented at one or more client systems, at one or more server systems, and/or combinations thereof. Gruber further teaches, at paragraph [0136], interfacing with external services, dynamically determining which services may provide information for a specific user request, mapping parameters of the user request to different service APIs, and calling services. Paragraphs [0130-0134] teach using context information, such as location and time information, to better interpret user language input. Paragraph [0170] teaches the functions may include showing directions on a map. Paragraph [0667] teaches parsing a user input such as “around here” so as to utilize a location given by GPS on a mobile device. The combined teachings of Gruber suggest sending a user request, including context information for the user request (e.g., location information), to an external server executing a navigation application, and receiving the desired navigation results from the external server. This allows for such a system to use natural language to interact with such services, while better interpreting user language input.
In view of Gruber’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the method as taught by Gruber, (re claim 13) transmitting, to a map server: data associated with the textual representation of the user speech and the contextual data; and receiving, from the map server, map data, with a reasonable expectation of success, since Gruber suggests sending a user request, including context information for the user request (e.g., location information), to an external server executing a navigation application, and receiving the desired navigation results from the external server. This allows for such a system to use natural language to interact with such services, while better interpreting user language input.
Gruber further teaches:
Re claim 14. Wherein the map data that is received from the map server is responsive to an inferred user intent based on the data associated with the textual representation of the user speech and the contextual data (Paragraphs [0130-0134] teach using context information, such as location and time information, to better interpret user language input. Paragraph [0142] teaches location information is combined with explicit user input so that certain inferences may be made about the type of information the user might be interested in. Paragraph [0667]: “Language interpreter component 1070 parses the text input and generates a list of possible interpretations of the user's intent 290. In one parse … "around here" is associated with a location parameter describing a distance from a global sensor reading (for example, the user's location as given by GPS on a mobile device).”).
Re claim 15. Wherein the contextual data comprises a location of the electronic device (paragraphs [0130, 0134, 0142, and 0667]).
Re claim 16. Wherein the contextual data comprises a software state of the electronic device (paragraph [0012]).
Re claim 17. Wherein the contextual data comprises a content being displayed by the electronic device (paragraph [0317]: “user selection 136 among the displayed choices can be achieved by any mode of input, including for example any of the modes of multimodal input described in connection with FIG. 16. Such input modes include, without limitation, actively elicited typed input 2610, actively elicited speech input 2620,”; and paragraph [0947]: “context-specific commands, such as "show these on a map"”).
Re claim 22. Wherein the map data comprises:
one or more map tiles;
a set of geographic coordinates;
routing information from a start location to an end location (paragraph [0123]: “navigation (maps and directions)”); or
traffic data (paragraph [0123]: “monitoring traffic”).
Re claim 24. Wherein the electronic device is a mobile phone (abstract).
Re claim 25. Wherein the map data that is received from the map server is responsive to an inferred user intent based on the data associated with the textual representation of the user speech and the contextual data (Paragraphs [0130-0134] teach using context information, such as location and time information, to better interpret user language input. Paragraph [0142] teaches location information is combined with explicit user input so that certain inferences may be made about the type of information the user might be interested in. Paragraph [0667]: “Language interpreter component 1070 parses the text input and generates a list of possible interpretations of the user's intent 290. In one parse … "around here" is associated with a location parameter describing a distance from a global sensor reading (for example, the user's location as given by GPS on a mobile device).”).
Re claim 26. Wherein the contextual data comprises a location of the electronic device (paragraphs [0130, 0134, 0142, and 0667]).
Re claim 27. Wherein the contextual data comprises a software state of the electronic device (paragraph [0012]).
Re claim 28. Wherein the contextual data comprises a content being displayed by the electronic device (paragraph [0317]: “user selection 136 among the displayed choices can be achieved by any mode of input, including for example any of the modes of multimodal input described in connection with FIG. 16. Such input modes include, without limitation, actively elicited typed input 2610, actively elicited speech input 2620,”; and paragraph [0947]: “context-specific commands, such as "show these on a map"”).
Re claim 33. Wherein the map data comprises:
one or more map tiles;
a set of geographic coordinates;
routing information from a start location to an end location (paragraph [0123]: “navigation (maps and directions)”); or
traffic data (paragraph [0123]: “monitoring traffic”).
Re claim 35. Wherein the electronic device is a mobile phone (abstract).
Re claim 36. Wherein the map data that is received from the map server is responsive to an inferred user intent based on the data associated with the textual representation of the user speech and the contextual data (Paragraphs [0130-0134] teach using context information, such as location and time information, to better interpret user language input. Paragraph [0142] teaches location information is combined with explicit user input so that certain inferences may be made about the type of information the user might be interested in. Paragraph [0667]: “Language interpreter component 1070 parses the text input and generates a list of possible interpretations of the user's intent 290. In one parse … "around here" is associated with a location parameter describing a distance from a global sensor reading (for example, the user's location as given by GPS on a mobile device).”).
Claims 5, 6, 18, 19, 29, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Gruber et al. (US Publication No. 2012/0245944) as applied to claim 1 above, and further in view of Bangalore et al. (US Publication No. 2003/0093419).
The teachings of Gruber have been discussed above. Gruber fails to specifically teach: (re claims 5, 18, and 29) wherein the contextual data comprises a current state of a mapping application running on the electronic device; and (re claims 6, 19, and 30) wherein the current state of the mapping application comprises a current view of the mapping application being displayed on the electronic device.
Bangalore teaches, at 108, Figure 7; and paragraphs [104 and 0108], a user of such a device may combine a spoken request (e.g., “tell me about these places”) with a gesture on a map (e.g., circling two restaurants displayed on a screen) to receive an answer with respect to the indicated displayed areas on the map and the user’s spoken request. This allows such user responsive systems to respond to multimodal inputs of a user, such as spoken language and areas circled on a screen.
In view of Bangalore’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the system as taught by Gruber, (re claims 5, 18, and 29) wherein the contextual data comprises a current state of a mapping application running on the electronic device; and (re claims 6, 19, and 30) wherein the current state of the mapping application comprises a current view of the mapping application being displayed on the electronic device, with a reasonable expectation of success, since Bangalore teaches a user of such a device may combine a spoken request (e.g., “tell me about these places”) with a gesture on a map (e.g., circling two restaurants displayed on a screen) to receive an answer with respect to the indicated displayed areas on the map and the user’s spoken request. This allows such user responsive systems to respond to multimodal inputs of a user, such as spoken language and areas circled on a screen.
Claim 8, 21, and 32 are rejected under 35 U.S.C. 103 as being unpatentable over Gruber et al. (US Publication No. 2012/0245944) as applied to claim 1 above, and further in view of Abramson et al. (US Publication No. 2016/0205238).
The teachings of Gruber have been discussed above. Gruber fails to specifically teach: (re claims 8, 21, and 32) wherein the contextual data comprises length of time since the electronic device received a user input.
Abramson teaches, at paragraph [0143], analyzing the time interval in between the typing of individual letters to determine context regarding the user’s level of distraction.
In view of Abramson’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the system as taught by Gruber, (re claims 8, 21, and 32) wherein the contextual data comprises length of time since the electronic device received a user input, with a reasonable expectation of success, since Abramson teaches analyzing the time interval in between the typing of individual letters to determine context regarding the user’s level of distraction.
Claims 10, 23, and 34 is rejected under 35 U.S.C. 103 as being unpatentable over Gruber et al. (US Publication No. 2012/0245944) as applied to claim 1 above, and further in view of Skillman et al. (US Publication No. 2016/0040996).
The teachings of Gruber have been discussed above. Gruber fails to specifically teach: (re claims 10, 23, and 34) wherein the electronic device is a watch.
Skillman teaches, at paragraphs [0057 and 0061], the end user device in such speech-to text capable mapping systems may be a watch.
In view of Skillman’s teachings, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include, with the system as taught by Gruber, (re claims 10, 23, and 34) wherein the electronic device is a watch, with a reasonable expectation of success, since Skillman teaches the end user device in such speech-to text capable mapping systems may be a watch. Using a watch as the client device of Gruber will yield the predictable result of providing voice-to-text interactions with external services through a watch, which enables users quick and easy access to such services through a device which is easily carried and accessed.
Allowable Subject Matter
Claims 7, 20, and 31 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments, see pages 9-10, filed 7/15/2025, with respect to the rejection(s) of claim(s) 1-4, 9, and 11-13 under 35 USC § 102(a)(1) in view of Gruber (US 2012/0245944) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made under 35 USC § 103 in view of Gruber.
Applicant remarks, on pages 9-10:
Gruber is silent with respect to a map server, much less transmitting data associated with the textual representation or transmitting the contextual data to such as map server, as recited in amended claim 1. At best, at paragraph [0136], Gruber discloses that an assistant can "interface with many external services, dynamically determine which services may provide information for a specific request, map parameters of the user request to different service APIs, call multiple services at once, [and] integrate results from multiple services." Accordingly, Gruber does not disclose "transmitting to a map server: data associated with the textual representation of the user speech; and the contextual data," as recited in amended claim 1.
Gruber does recite enabling operation of navigation applications via natural language dialog at paragraph [0123], and that these functions may be implemented at one or more server systems at paragraph [0125]. This suggests a server enabling navigation, which is equivalent to a map server as claimed. Gruber further teaches, at [0136], mapping parameters of a user request to service APIs; and at paragraphs [0130-0134, 0170, and 0667], such parameters of a user request include context information, such as location and time information, as well as navigation related information required to determine directions to a location. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention for the context information, such as location information, to be sent to a server system implementing a navigation application so as to obtain relevant navigation instructions from the server.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SPENCER D PATTON whose telephone number is (571)270-5771. The examiner can normally be reached Monday to Friday 9:00-5:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached at (571)272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SPENCER D PATTON/Primary Examiner, Art Unit 3656