DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 4/21/26 have been fully considered but they are not persuasive.
Regarding the 35 U.S.C. 101 rejection of the claims, Applicant argues that the amendments to the claims ensures the claims are directed to statutory subject matter (Arguments, pg. 10-11). Examiner respectfully disagrees as the claims still recite data gathering and data analysis steps without significantly more as provided by the rejection below.
Regarding the 35 U.S.C. 103 rejection of the claim with references Park and Paek, Applicant argues that para. [0322] of Paek discloses applying a weighting to factors including the user’s gaze, distance, dwell time and utterance and that the target word “I” in Paek’s figure 9W and para. [0322] does not exist in the utterance, and as such argues that the Paek does not disclose adding an attention bias weigh to words in the user prompt, and as such, argues that Paek fails to disclose limitation “add an attention bias weight to words in the user prompt based on the determined attention” (Arguments, pg. 11, third para. – pg. 15, first para.).
Examiner respectfully disagrees as the figure of Paek that Applicant provides (i.e., fig. 9W) includes utterance 916 “Are not I” (i.e., user prompt/utterance including “I”) where the utterance is an attempt to correct the user gaze area that includes words “dogs I definitely” (i.e., the displayed word “I” is included in user’s utterance “Are not I”). Although Paek’s para. [0322] discloses assigning higher weights to words included in a display/user’s gaze that the user wishes to edit 902 even in instances where the received utterance does not include a gazed at word (see fig. 9AC and para. [0302]), Paek discloses multiple instances in which the provided utterance includes the words present in the display/user’s gaze (see fig. 9W; fig. 9AK; fig. 9AL; para. [0322]; para. [0369]-[0370]) i.e., the displayed/user gaze words that the user wishes to edit are sometimes included in the user utterance/prompt in addition to the displayed words that the user wishes to edit not being included in the user utterance/prompt (see fig. 9AC and para. [0302]). Therefore, because Paek discloses assigning higher weights to words included in a display/user gaze area that the user wishes to edit based on the user’s gaze 902, and the words in the user gaze/display in some instances correspond to words present in the user’s utterance, Paek discloses adding weights to words that are present in the user prompt (words in the user prompt) based on the determined user gaze, and as such, limitation “add an attention bias weight to words in the user prompt based on the determined attention”.
Regarding the dependent claims, Applicant argues that the claims are allowable as a result of their dependency from argued claims 1, 16, 30 above (Arguments, pg. 15). Examiner respectfully disagrees that claims 1, 16, 30 are allowable as presented above, and absent any argument as to why the cited portions of the references fail to disclose limitations recited in the dependent claims, Examiner maintains that the rejections of the dependent claims are appropriate.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-16 and 18-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to the abstract idea of prompt analysis without significantly more. The claims 1, 16 and 30 recite steps of: receive, from a user, a user prompt for a generative artificial intelligence model (LXM) (i.e., a data gathering step), determine an attention of the user to subject matter when or prior to receiving the user prompt (i.e., a data analysis step observing a user), add an attention bias weight to words in the user prompt based on the determined attention (i.e., a data analysis step of assigning values to words), generate an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight (i.e., a data analysis step of assigning values to text/input), and submitting the enhanced prompt to the LXM (i.e., a data transmission/post solutional step), corresponding to steps achievable by a human in analyzing gathered data and context information to select a profile, and providing a combined output for submission, and as such, the mental processes category of abstract ideas. This judicial exception is not integrated into a practical application because the claims are directed to an abstract idea with additional generic computer elements (computing device, LXM, memory, processor, processor readable-medium), where the generically recited computer elements do not add a meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea on a computer. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because steps to: “generate an enhanced prompt based on the user prompt by applying an adaptive weighting to portions of the user prompt based on the attention bias weight” and “submit the enhanced prompt to the LXM” correspond to the well-understood, routine, conventional computer functions of “collecting information, analyzing it, and displaying certain results of the collection and analysis“ and “receiving or transmitting data over a network” as recognized by the court decisions listed in MPEP § 2106.05, and as presented by cited references Park, Paek, Prasad and Yang (See PTO 892 form).
The dependent claims 3-15 and 18-29 also recite mental processes and do not add significantly more than the abstract idea and are as such similarly rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1, 5, 6, 9-14, 16-20, 23-28 and 30 rejected under 35 U.S.C. 103 as being unpatentable over Park et al US 2024/0420491 A1 (“Park”) in view of Paek et al US 2024/0185856 A1 (“Paek”)
Per claim 1, Park discloses a computing device, comprising:
a memory (para. [0185]); and
at least one processor coupled to the memory and configured (para. [0185]; para. [0222]-[0223]) to:
receive, from a user, a user prompt for a generative artificial intelligence model (LXM) (fig. 1; FIG. 1 is a graphical representation of conventional large language model (LLM) operation. As shown, a user 102 provides an input prompt to a client device 104 …, para. [0038]; para. [0218]);
determine an attention of the user to subject matter when or prior to receiving the user prompt (para. [0093]-[0094]; Looking at the menu, the user may follow up with a question to their virtual assistant about the number of calories of a food item (“how many calories is a taco?”)…. The reply may include a contextually relevant response, based on the location information of the user, with the calories of the food item on the restaurant's menu …, para. [0138]; para. [0207]; Consider, for example, a user that asked “What can I cook with this ingredient?” at the grocery store. They bought the ingredient and returned home. In the intervening time, their previous LLM session may have timed out. Here, the session management logic may reconstruct the previous conversation, so that when the user asks, “can I add this spice to the recipe?” the question is answered in the context of the same recipe that they were shown at the grocery store, para. [0292]; the session management logic may pre-emptively trigger an image capture of the user's gaze point and send LLM queries to e.g., prime the conversation state with information about the user's environment. These initial LLM queries may be performed before the user has said anything …, para. [0294], user looking/gazing at menu item prior to asking question and user paying attention to “this spice”/ingredient determined in grocery store prior to user question “can I add this spice to the recipe? at home as implying limitation, user’s gaze as attention);
generate an enhanced prompt based on the user prompt (para. [0047]; para. [0060]; the smart glasses 402 may also gather contextual information about the user, their environment, and/or objects of interest, that may be useful to augment the user prompt. As but one such example, smart glasses 402 may use eye-tracking cameras and/or forward-facing cameras to obtain gaze information …, para. [0069]; para. [0137]; para. [0207]; para. [0219]; para. [0239]; para. [0292]-[0294]); and
submit the enhanced prompt to the LXM (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0069]; para. [0075]-[0076]; For example, an LLM input specializer may be used to augment user context with additional input for an LLM. Functionally, an LLM input specializer augments the user's prompt in view of captured data ..., para. [0206]-[0207]; para. [0292]-[0294]).
Park does not explicitly disclose to add an attention bias weight to words in the user prompt based on the determined attention or generate an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight
However, these features are taught by Paek:
add an attention bias weight to words in the user prompt based on the determined attention (fig. 8; fig. 9O; fig. 9AK; fig. 10; para. [0008]; For example, as shown in FIG. 9O, user gaze 902 is directed at words displayed on screen 901 …, para. [0269]; para. [0302]; para. [0317]; target selector 830 determines the word displayed on the screen of the electronic device to edit based on utterance 801. For example, as shown in FIG. 9W, when utterance 916 of “are not I” is received, target selector 830 determines that the user intends to edit the word “I” based on the use of “I” in utterance 916. In some examples, multiple words to edit are determined based on utterance 801…, para. [0318]; target selector 830 assigns a weight to each of the factors discussed above (e.g., user gaze 802, the distance between the location of user gaze 802 and the word, the dwell time, and utterance 801 … For example, user gaze 902 may be weighted more heavily because user gaze 902 heavily indicates what word the user wishes to edit. Accordingly, target selector 830 may assign a high weight to the word “I” based on user gaze 902 …, para. [0322] …, para. [0322]; For example, as shown in FIG. 9K, when utterance 929 of “change that to Tuesday evening” is received while user gaze 902 is directed at the word “Monday” displayed on display 901 of electronic device 900, system 800 determines that the user is intending to edit “Monday” and thus “that” refers to “Monday.”, para. [0369]-[0370]; para. [0379]; para. [0383], multiple words to edit as determined from user utterance 801, user’s gaze 902 directed at words displayed on screen 901 as determined attention, weights assigned to utterance that includes words “and” and “I” (in utterance “are not I”) and “that” and “Tuesday” (in utterance “change that to Tuesday evening”), higher/heavy weight assigned to words present in user’s input/prompt based on user gaze 902/attention)
generate an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight (fig. 9W; para. [0299]; para. [0318]; para. [0321]; target selector 830 assigns a weight to each of the factors discussed above (e.g., user gaze 802, the distance between the location of user gaze 802 and the word, the dwell time, and utterance 801) and determines based on the assigned weights the word to edit. For example, user gaze 902 may be weighted more heavily because user gaze 902 heavily indicates what word the user wishes to edit. Accordingly, target selector 830 may assign a high weight to the word “I” based on user gaze 902 …, para. [0322]; para. [0355]; para. [0383]; para. [0396], heavily weighting displayed words corresponding to words of user prompt based on areas of a user gaze as implying adaptive important weighting)
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Paek with the device of Park in arriving at the missing features of Park, because such combination would have resulted in improving dictation services (Paek, para. [0002]; para. [0317]-[0322]).
Per claim 5, Park in view of Paek discloses the computing device of claim 1,
Park discloses wherein the at least one processor is further configured to generate the enhanced prompt by generating a summary prompt that includes words assigned greater weight based on the attention of the user to the subject matter when or prior to receiving the user prompt (para. [0047]; para. [0060]; para. [0219]; para. [0239]; para. [0250]; para. [0291).
Per claim 6, Park in view of Paek d discloses the computing device of claim 1,
Park discloses wherein the determined attention comprises the attention of the user paid to words or phrases in the subject matter prior to entry of the user prompt (para. [0138]; para. [0292])
Paek discloses wherein the determined attention comprises the attention of the user paid to words or phrases in the subject matter prior to entry of the user prompt (fig. 9P; para. [0092]; para. [0229]; para. [0256]; para, [0263]).
Per claim 9, Park in view of Paek discloses the computing device of claim 1,
Park discloses wherein the at least one processor is further configured to generate the enhanced prompt based on the user prompt and the subject matter to which the user is paying attention by including in the enhanced prompt information regarding the subject matter to which the user paid attention when or prior to receiving the user prompt (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0068]-[0069]; para. [0138]; para. [0207]).
Per claim 10, Park in view of Paek discloses the computing device of claim 9,
Park discloses wherein the at least one processor is further configured to include in the enhanced prompt information regarding the subject matter to which the user paid attention when or prior to receiving the user prompt by: generating text describing a portion of the subject matter to which the user paid attention when or prior to receiving the user prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0138]; para. [0207]); and
including at least a portion of the generated text in the enhanced prompt (para. [0065]; para. [0068]-[0069]).
Per claim 11, Park in view of Paek discloses the computing device of claim 10,
Park discloses wherein the at least one processor is further configured to include in the enhanced prompt information regarding the subject matter to which the user paid attention when or prior to receiving the user prompt by: generating text summarizing the subject matter to which the user paid attention when or prior to receiving the user prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0138]; para. [0207]); and
including at least a portion of the generated text in the enhanced prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0206]).
Per claim 12, Park in view of Paek discloses the computing device of claim 1,
Park discloses wherein the at least one processor is further configured to determine the user attention to subject matter when or prior to receiving the user prompt by determining the user’s attention to subject matter associated with the computing device when or prior to receiving the user’s prompt (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0068]-[0069]; para. [0138]; para. [0207]).
Per claim 13, Park in view of Paek discloses the computing device of claim 1,
Park discloses wherein the at least one processor is further configured to determine the attention of the user to subject matter when or prior to receiving the user prompt by determining the user’s attention to subject matter associated with another nearby device when or prior to receiving the user prompt (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0068]-[0069]; para. [0138]; para. [0207]).
Per claim 14, Park in view of Paek discloses the computing device of claim 1,
Park discloses wherein the LXM is a large language model (LLM) (para. [0053]).
Per claim 16, Park discloses a method performed by a computing device for generating a prompt for a generative artificial intelligence model (LXM), comprising:
receiving a user prompt for the LXM (fig. 1; FIG. 1 is a graphical representation of conventional large language model (LLM) operation. As shown, a user 102 provides an input prompt to a client device 104 …, para. [0038]; para. [0218]);
determine an attention of the user to subject matter when or prior to receiving the user prompt (para. [0093]-[0094]; Looking at the menu, the user may follow up with a question to their virtual assistant about the number of calories of a food item (“how many calories is a taco?”)…. The reply may include a contextually relevant response, based on the location information of the user, with the calories of the food item on the restaurant's menu …, para. [0138]; para. [0207]; Consider, for example, a user that asked “What can I cook with this ingredient?” at the grocery store. They bought the ingredient and returned home. In the intervening time, their previous LLM session may have timed out. Here, the session management logic may reconstruct the previous conversation, so that when the user asks, “can I add this spice to the recipe?” the question is answered in the context of the same recipe that they were shown at the grocery store, para. [0292]; the session management logic may pre-emptively trigger an image capture of the user's gaze point and send LLM queries to e.g., prime the conversation state with information about the user's environment. These initial LLM queries may be performed before the user has said anything …, para. [0294], user looking/gazing at menu item prior to asking question and user paying attention to “this spice”/ingredient determined in grocery store prior to user question “can I add this spice to the recipe? at home as implying limitation);
generating an enhanced prompt based on the user’s prompt and the subject matter to which the user is paying attention when or prior to receiving the user prompt (para. [0060]; the smart glasses 402 may also gather contextual information about the user, their environment, and/or objects of interest, that may be useful to augment the user prompt. As but one such example, smart glasses 402 may use eye-tracking cameras and/or forward-facing cameras to obtain gaze information …, para. [0069]; para. [0137]; para. [0207]; para. [0292]-[0294]); and
submitting the enhanced prompt to the LXM (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0069]; para. [0075]-[0076]; For example, an LLM input specializer may be used to augment user context with additional input for an LLM. Functionally, an LLM input specializer augments the user's prompt in view of captured data ..., para. [0206]-[0207]; para. [0292]-[0294])
Park does not explicitly disclose to adding an attention bias weight to words in the user prompt based on the determined attention or generating an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight
However, these features are taught by Paek:
adding an attention bias weight to words in the user prompt based on the determined attention (fig. 8; fig. 9O; fig. 9AK; fig. 10; para. [0008]; For example, as shown in FIG. 9O, user gaze 902 is directed at words displayed on screen 901 …, para. [0269]; para. [0302]; para. [0317]; target selector 830 determines the word displayed on the screen of the electronic device to edit based on utterance 801. For example, as shown in FIG. 9W, when utterance 916 of “are not I” is received, target selector 830 determines that the user intends to edit the word “I” based on the use of “I” in utterance 916. In some examples, multiple words to edit are determined based on utterance 801…, para. [0318]; target selector 830 assigns a weight to each of the factors discussed above (e.g., user gaze 802, the distance between the location of user gaze 802 and the word, the dwell time, and utterance 801 … For example, user gaze 902 may be weighted more heavily because user gaze 902 heavily indicates what word the user wishes to edit. Accordingly, target selector 830 may assign a high weight to the word “I” based on user gaze 902 …, para. [0322] …, para. [0322]; For example, as shown in FIG. 9K, when utterance 929 of “change that to Tuesday evening” is received while user gaze 902 is directed at the word “Monday” displayed on display 901 of electronic device 900, system 800 determines that the user is intending to edit “Monday” and thus “that” refers to “Monday.”, para. [0369]-[0370]; para. [0379]; para. [0383], multiple words to edit as determined from user utterance 801, user’s gaze 902 directed at words displayed on screen 901 as determined attention, weight assigned to utterance that includes words “and” and “I” (in utterance “are not I”) and “that” and “Tuesday” (in utterance “change that to Tuesday evening”), higher/heavy weight assigned to words present in user’s input/prompt based on user gaze 902/attention)
generating an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight (fig. 9W; para. [0299]; para. [0318]; para. [0321]; target selector 830 assigns a weight to each of the factors discussed above (e.g., user gaze 802, the distance between the location of user gaze 802 and the word, the dwell time, and utterance 801) and determines based on the assigned weights the word to edit. For example, user gaze 902 may be weighted more heavily because user gaze 902 heavily indicates what word the user wishes to edit. Accordingly, target selector 830 may assign a high weight to the word “I” based on user gaze 902 …, para. [0322]; para. [0355]; para. [0383]; para. [0396], heavily weighting displayed words corresponding to words of user prompt based on areas of a user gaze as implying adaptive important weighting)
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Paek with the method of Park in arriving at the missing features of Park, because such combination would have resulted in improving dictation services (Paek, para. [0002]; para. [0317]-[0322]).
Per claim 18, Park in view of Paek discloses the method of claim 16,
Park discloses wherein determining the attention of the user to the subject matter comprises one or more of tracking an eye gaze of the user on the subject matter, tracking a mouse cursor location on the subject matter, or tracking locations of touch input from the user on a touch sensitive display (para. [0223]; para. [0250]).
Per claim 19, Park in view of Paek discloses the method of claim 16,
Park discloses wherein generating the enhanced prompt comprises generating a summary prompt that includes words assigned greater weight based on the attention of the user to the subject matter a when or prior to receiving the user prompt (para. [0047]; para. [0060]; para. [0219]; para. [0239]; para. [0250]; para. [0291).
Per claim 20, Park in view of Paek discloses the method of claim 16,
Park discloses wherein the determined attention comprises the attention of the user paid to words or phrases in the subject matter prior to entry of the user prompt (para. [0138]; para. [0292])
Paek discloses wherein the determined attention comprises the attention of the user paid to words or phrases in the subject matter prior to entry of the user prompt (fig. 9P; para. [0092]; para. [0229]; para. [0256]; para, [0263]).
Per claim 23, Park in view of Paek discloses the method of claim 16,
Park discloses wherein generating the enhanced prompt based on the user prompt and the subject matter to which the user is paying attention comprises including in the enhanced prompt information regarding the subject matter to which the user is paying attention when or prior to receiving the user prompt (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0068]-[0069]; para. [0292]-[0294]).
Per claim 24, Park in view of Paek discloses the method of claim 23,
Park discloses wherein including in the enhanced prompt information, information regarding the subject matter to which the user is paying attention when or prior to receiving the user prompt comprises: generating text describing a portion of the subject matter to which the user is paying attention when or prior to receiving the user prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0292]-[0294]); and
including at least a portion of the generated text in the enhanced prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0206]; para. [0292]-[0294]);
Per claim 25, Park in view of Paek discloses the method of claim 23,
Park discloses wherein including in the enhanced prompt information regarding the subject matter to which the user is paying attention when or prior to receiving the user prompt comprises: generating text summarizing the subject matter to which the user is paying attention when or prior to receiving the user prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0138]; para. [0207]; para. [0292]-[0294]); and
including at least a portion of the generated text in the enhanced prompt (para. [0060]; in the context of a large language model (LLM) the different modalities of information may first be converted to a common comparison domain (text) …, para. [0065]; para. [0068]-[0069]; para. [0206]; para. [0292]-[0294]).
Per claim 26, Park in view of Paek discloses the method of claim 16,
Park discloses wherein determining the attention of the user to the subject matter when or prior to receiving the user prompt comprises determining the user’s attention to subject matter associated with the computing device when or prior to receiving the user prompt (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0068]-[0069]; para. [0138]; para. [0207]; para. [0292]; para. [0294]).
Per claim 27, Park in view of Paek discloses the method of claim 16,
Park discloses wherein determining the user attention to the subject matter when or prior to receiving the user prompt comprises determining the user’s attention to subject matter associated with another nearby device when or prior to receiving the user prompt (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0068]-[0069]; para. [0138]; para. [0207]; para. [0292]; para. [0294]).
Per claim 28, Park in view of Paek discloses the method of claim 16,
Park discloses wherein: receiving the user prompt for the LXM comprises receiving the user prompt for a large language model (LLM) (fig. 1; FIG. 1 is a graphical representation of conventional large language model (LLM) operation. As shown, a user 102 provides an input prompt to a client device 104 …, para. [0038]; para. [0218]); and
submitting the enhanced prompt to the LXM comprises submitting the enhanced prompt to the LLM (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0069]; para. [0075]-[0076]; For example, an LLM input specializer may be used to augment user context with additional input for an LLM. Functionally, an LLM input specializer augments the user's prompt in view of captured data ..., para. [0206]; para. [0292]; para. [0294]).
Per claim 30, Park discloses a computing device, comprising:
means for receiving a user’s prompt for a generative artificial intelligence model (LXM) (fig. 1; FIG. 1 is a graphical representation of conventional large language model (LLM) operation. As shown, a user 102 provides an input prompt to a client device 104 …, para. [0038]; para. [0185]; para. [0218]; para. [0222]-[0223]);
means for determining an attention of the user to subject matter when or prior to receiving the user prompt (para. [0093]-[0094]; Looking at the menu, the user may follow up with a question to their virtual assistant about the number of calories of a food item (“how many calories is a taco?”)…. The reply may include a contextually relevant response, based on the location information of the user, with the calories of the food item on the restaurant's menu …, para. [0138]; para. [0207]; Consider, for example, a user that asked “What can I cook with this ingredient?” at the grocery store. They bought the ingredient and returned home. In the intervening time, their previous LLM session may have timed out. Here, the session management logic may reconstruct the previous conversation, so that when the user asks, “can I add this spice to the recipe?” the question is answered in the context of the same recipe that they were shown at the grocery store, para. [0292]; the session management logic may pre-emptively trigger an image capture of the user's gaze point and send LLM queries to e.g., prime the conversation state with information about the user's environment. These initial LLM queries may be performed before the user has said anything …, para. [0294], user looking/gazing at menu item prior to asking question and user paying attention to “this spice”/ingredient determined in grocery store prior to user question “can I add this spice to the recipe? at home as implying limitation);
means for generating an enhanced prompt based on the user’s prompt and the subject matter to which the user is paying attention when or prior to receiving the user prompt (para. [0060]; the smart glasses 402 may also gather contextual information about the user, their environment, and/or objects of interest, that may be useful to augment the user prompt. As but one such example, smart glasses 402 may use eye-tracking cameras and/or forward-facing cameras to obtain gaze information …, para. [0069]; para. [0137]; para. [0207]; para. [0292]-[0294]); and
means for submitting the enhanced prompt to the LXM (During online operation, the smart glasses 402 and smart phone 404 capture instantaneous user context …, para. [0060]; para. [0069]; para. [0075]-[0076]; For example, an LLM input specializer may be used to augment user context with additional input for an LLM. Functionally, an LLM input specializer augments the user's prompt in view of captured data ..., para. [0206]-[0207]; para. [0218]; para. [0222]-[0223]; para. [0292]-[0294])
Park does not explicitly disclose means for adding an attention bias weight to words in the user prompt based on the determined attention or generating an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight
However, these features are taught by Paek:
means for adding an attention bias weight to words in the user prompt based on the determined attention (fig. 8; fig. 9O; fig. 9AK; fig. 10; para. [0008]; For example, as shown in FIG. 9O, user gaze 902 is directed at words displayed on screen 901 …, para. [0269]; para. [0302]; para. [0317]; target selector 830 determines the word displayed on the screen of the electronic device to edit based on utterance 801. For example, as shown in FIG. 9W, when utterance 916 of “are not I” is received, target selector 830 determines that the user intends to edit the word “I” based on the use of “I” in utterance 916. In some examples, multiple words to edit are determined based on utterance 801…, para. [0318]; target selector 830 assigns a weight to each of the factors discussed above (e.g., user gaze 802, the distance between the location of user gaze 802 and the word, the dwell time, and utterance 801 … For example, user gaze 902 may be weighted more heavily because user gaze 902 heavily indicates what word the user wishes to edit. Accordingly, target selector 830 may assign a high weight to the word “I” based on user gaze 902 …, para. [0322] …, para. [0322]; For example, as shown in FIG. 9K, when utterance 929 of “change that to Tuesday evening” is received while user gaze 902 is directed at the word “Monday” displayed on display 901 of electronic device 900, system 800 determines that the user is intending to edit “Monday” and thus “that” refers to “Monday.”, para. [0369]-[0370]; para. [0379]; para. [0383], multiple words to edit as determined from user utterance 801, user’s gaze 902 directed at words displayed on screen 901 as determined attention, weight assigned to utterance that includes words “and” and “I” (in utterance “are not I”) and “that” and “Tuesday” (in utterance “change that to Tuesday evening”), higher/heavy weight assigned to words present in user’s input/prompt based on user gaze 902/attention)
means for generating an enhanced prompt based on the user prompt by applying an adaptive important weighting to portions of the user prompt based on the attention bias weight (fig. 9W; para. [0299]; para. [0318]; para. [0321]; target selector 830 assigns a weight to each of the factors discussed above (e.g., user gaze 802, the distance between the location of user gaze 802 and the word, the dwell time, and utterance 801) and determines based on the assigned weights the word to edit. For example, user gaze 902 may be weighted more heavily because user gaze 902 heavily indicates what word the user wishes to edit. Accordingly, target selector 830 may assign a high weight to the word “I” based on user gaze 902 …, para. [0322]; para. [0355]; para. [0383]; para. [0396], heavily weighting displayed words corresponding to words of user prompt based on areas of a user gaze as implying adaptive important weighting)
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Paek with the device of Park in arriving at the missing features of Park, because such combination would have resulted in improving dictation services (Paek, para. [0002]; para. [0317]-[0322]).
2. Claims 3, 4, 7, 15, 21 and 29 are rejected under 35 U.S.C. 103 as being unpatentable over Park in view of Paek as applied to claims 1 and 16 above, and further in view of Prasad et al US 2025/0004544 A1 (“Prasad”)
Per claim 3, Park in view of Paek discloses the computing device of claim 1,
Park discloses a user camera, wherein the at least one processor is further configured to determine the attention of the user to the subject matter by tracking an eye gaze of the user on the subject matter (para. [0223]; para. [0250])
Park does not explicitly disclose the use of a user facing camera
However, this feature is taught by Prasad (fig. 1B, element 118)
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Prasad with the device of Park in arriving at the missing features of Park, because such combination would have resulted in determining whether a front facing user is facing an audio capture device for the purposes of determining whether user speech/input is system-directed (Prasad, para. [0127])
Per claim 4, Park in view of Paek discloses the computing device of claim 1,
Park discloses further comprising a touch sensitive display (para. [0223]),
Park does not explicitly disclose wherein the at least one processor is further configured to determine the attention of the user to the subject matter by tracking locations of touch input from the user on the touch sensitive display
However, this feature is taught by Prasad (para. [0054])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Prasad with the device of Park in arriving at the missing features of Park, because such combination would have resulted in determining what content is more of interest to a user (Prasad, para. [0054]; para. [0064])
Per claim 7, Park in view of Paek and Prasad discloses the computing device of claim 6,
Park does not explicitly disclose wherein the at least one processor is further configured to increase the attention bias weight responsive to a duration the user focused on particular words and decreasing the attention bias weight with time after the user focus shifts away from the particular words
However, this feature is suggested by Prasad disclosing changing the attention coloring of a gazed area of text/words from green to yellow due to the time between gaze instances, to respectively indicate a detected maintained gaze or a gaze shift away from the text/words (A gaze event is one where the system 100 has determined that the user has actually looked at a particular location/region of the display 102 for a sufficient period of time to consider it an actual gaze …, para. [0034]; In response to the gaze event meeting the initial threshold the gaze manager 150/device manager 160 may control the display 102 to present a first visual indicator 322. For example, the first visual indicator 322 may correspond to a colored border surrounding the first GUI element 104. The border may change color or otherwise animate (e.g., through flashing, pulsing, or other animation) to indicate an active gaze. Such color of the border may also change as different gaze thresholds are met (for example starting at yellow and progressing to green or the like)…. The first visual indicator 322 and second visual indicator 324 may also animate or change to indicate a gaze away from the display 102 (such as a color change from green to yellow, para. [0051])
It would have been obvious to one of ordinary skill in the art to try to implement wherein the at least one processor is further configured to increase the attention bias weight responsive to a duration the user focused on particular words and decreasing the attention bias weight with time after the user focus shifts away from the particular words, because such implementation would have resulted in indicating an active gaze.
Per claim 15, Park in view of Paek discloses the computing device of claim 1,
Park discloses a display coupled to the at least one processor (para. [0222]-[0223]),
Paek discloses wherein the at least one processor is configured to: determine attention of the user to the subject matter when or prior to receiving the user prompt by determining the user’s attention to subject matter presented on the display when or prior to receiving the user’s prompt (para. [0317]-[0322])
Park does not explicitly disclose generate the enhanced prompt based on the user prompt and the subject matter to which the user is paying attention when or prior to receiving the user prompt by generating the enhanced prompt based on the user prompt and subject matter presented on the display to which the user is paying attention when or prior to receiving the user prompt
However, this feature is taught by Prasad:
generate the enhanced prompt based on the user prompt and the subject matter to which the user is paying attention when or prior to receiving the user prompt by generating the enhanced prompt based on the user prompt and subject matter presented on the display to which the user is paying attention when or prior to receiving the user prompt (fig. 1B; para. [0056]; para. [0074])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Prasad with the device of Park in arriving at the missing features of Park, because such combination would have resulted in determining what content is more of interest to a user (Prasad, para. [0054]; para. [0064])
Per claim 21, Park in view of Paek discloses the method of claim 20,
Park does not explicitly disclose increasing the attention bias weight responsive to a duration the user focused on particular words and decreasing the attention bias weight with time after the user focus shifts away from the particular words
However, this feature is suggested by Prasad disclosing changing the attention coloring of a gazed area of text/words from green to yellow due to the time between gaze instances, to respectively indicate a detected maintained gaze or a gaze shift away from the text/words (A gaze event is one where the system 100 has determined that the user has actually looked at a particular location/region of the display 102 for a sufficient period of time to consider it an actual gaze …, para. [0034]; In response to the gaze event meeting the initial threshold the gaze manager 150/device manager 160 may control the display 102 to present a first visual indicator 322. For example, the first visual indicator 322 may correspond to a colored border surrounding the first GUI element 104. The border may change color or otherwise animate (e.g., through flashing, pulsing, or other animation) to indicate an active gaze. Such color of the border may also change as different gaze thresholds are met (for example starting at yellow and progressing to green or the like)…. The first visual indicator 322 and second visual indicator 324 may also animate or change to indicate a gaze away from the display 102 (such as a color change from green to yellow, para. [0051])
It would have been obvious to one of ordinary skill in the art to try to implement wherein the at least one processor is further configured to increase the attention bias weight responsive to a duration the user focused on particular words and decreasing the attention bias weight with time after the user focus shifts away from the particular words, because such implementation would have resulted in indicating an active gaze.
Per claim 29, Park in view of Paek discloses the method of claim 16,
Paek discloses wherein: determining the user’s attention to the subject matter when or prior to receiving the user prompt comprises determining the attention of the user to subject matter presented on a display of the computing device when or prior to receiving the user prompt (para. [0317]-[0322])
Park does not explicitly disclose generating the enhanced prompt based on the user prompt and the subject matter to which the user is paying attention a when or prior to receiving the user prompt comprises generating the enhanced prompt based on the user prompt and subject matter presented on the display to which the user is paying attention when or prior to receiving the user’s prompt
However, this feature is taught by Prasad:
generating the enhanced prompt based on the user prompt and the subject matter to which the user is paying attention at the time or prior to receipt of the user prompt comprises generating the enhanced prompt based on the user prompt and subject matter presented on the display to which the user is paying attention at the time or prior to receipt of the user prompt (fig. 1B; para. [0056]; para. [0074])
It would have been obvious to one of ordinary skill in the art before the effective filing of the invention to combine the teachings of Prasad with the method of Park in arriving at the missing features of Park, because such combination would have resulted in determining what content is more of interest to a user (Prasad, para. [0054]; para. [0064]).
Allowable Subject Matter
Claims 8 and 22 are objected to as being dependent upon a rejected base claim, but would be allowable (pending Applicant addressing the 35 U.S.C. 101 rejection) if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO 892 form.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLUJIMI A ADESANYA whose telephone number is (571)270-3307. The examiner can normally be reached Monday-Friday 8:30-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Richemond Dorvil can be reached at 571-272-7602. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OLUJIMI A ADESANYA/Primary Examiner, Art Unit 2658
Prosecution Insights