DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is in response to the amendment filed on 03/11/2026. Claims 1-20 remain pending in the application. Claims independent.
Drawings
Applicant's amendment to specification corrects previous objections; therefore, the previous objections are withdrawn.
Specification
Applicant's amendment to specification corrects some of previous objections; therefore, some of the previous objections are withdrawn. The remaining objections are shown below.
The use of the term "Wi-Fi" in ¶ [0068], which is a trade name or a mark used in commerce, has been noted in this application. The term should be accompanied by the generic terminology; furthermore the term should be capitalized wherever it appears or, where appropriate, include a proper symbol indicating use in commerce such as ™, SM , or ® following the term.
Although the use of trade names and marks used in commerce (i.e., trademarks, service marks, certification marks, and collective marks) are permissible in patent applications, the proprietary nature of the marks should be respected and every effort made to prevent their use in any manner which might adversely affect their validity as commercial marks.
Claim Objections
Applicant's amendment to claims corrects previous objections; therefore, the previous objections are withdrawn. Applicant's amendment to claims also raises the following new objections.
Claims 1, 8-9, 12, 15, 17, and 19-20 are objected to because of the following informalities:
in Claim 1, lines 5-11, "… identifying … a set of keywords representative of customer characteristics from the personal data … generating … the personalized virtual content from the set of keywords representative of the customer characteristics combined with …" appears to be "… identifying … a set of keywords representing customer characteristics from the personal data … generating … the personalized virtual content from the set of keywords representing the customer characteristics combined with …" because "the set of keywords" is also recited in the claim;
in Claim 8, line 3, "… generate user-specific content based upon the set of keywords representative of the customer characteristics" appears to be "… generate user-specific content based upon the set of keywords representing the customer characteristics";
in Claim 9, lines 2-3, "… generate the personalized virtual content based upon an additional set of additional keywords directed to content messaging" appears to be "… generate the personalized virtual content based upon an additional set of keywords directed to content messaging";
in Claim 12, lines 7-13, "… identify a set of keywords representative of customer characteristics from the personal data … generate … the personalized virtual content from the set of keywords representative of the customer characteristics combined with …" appears to be "… identify a set of keywords representing customer characteristics from the personal data … generate … the personalized virtual content from the set of keywords representing the customer characteristics combined with …" because "the set of keywords" is also recited in the claim;
in Claim 15, lines 3-4, "… generate user-specific content based upon the set of keywords representative of the customer characteristics" appears to be "… generate user-specific content based upon the set of keywords representing the customer characteristics";
in Claim 17, lines 5-11, "… identify a set of keywords representative of customer characteristics from the personal data … generate … the personalized virtual content from the set of keywords representative of the customer characteristics combined with …" appears to be "… identify a set of keywords representing customer characteristics from the personal data … generate … the personalized virtual content from the set of keywords representing the customer characteristics combined with …" because "the set of keywords" is also recited in the claim;
in Claim 19, lines 3-5, "… generate user-specific content based upon the set of keywords representative of the customer characteristics" appears to be "… generate user-specific content based upon the set of keywords representing the customer characteristics";
in Claim 20, lines 9-10, "… training content to a user via and additional virtual headset" appears to be "… training content to a user via an additional virtual headset".
Appropriate correction is required.
Claim Rejections - 35 USC § 112
Applicant's amendment to claims corrects previous rejections; therefore, the previous rejections are withdrawn. Applicant's amendment to claims also raises the following new rejections.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "… combining … the set of keywords with a predetermined set of subject-matter keywords … generating … using a machine learning model trained to generate virtual content based upon keywords, the personalized virtual content from the set of keywords representative of the customer characteristics combined with the predetermined set of subject-matter keywords …" in lines 7-12, which rendering the claim indefinite because it is unclear whether "the set of keywords" (used to combine with "a predetermined set of subject-matter keywords") is the same as or different to "keywords" (used to generate virtual content using a trained machine learning model). For examination purpose, "… combining … the set of keywords with a predetermined set of subject-matter keywords … generating … using a machine learning model trained to generate virtual content based upon the set of keywords, the personalized virtual content from the set of keywords representing the customer characteristics combined with the predetermined set of subject-matter keywords …" is considered (see also Claim Objections to Claim 1).
Claims 2-11 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 12 recites the limitation "… combine the set of keywords with a predetermined set of subject-matter keywords … generate, using a machine learning model trained to generate virtual content based upon keywords, the personalized virtual content from the set of keywords representative of the customer characteristics combined with the predetermined set of subject-matter keywords …" in lines , which rendering the claim indefinite because it.
Claims 13-16 are rejected for fully incorporating the deficiency of their respective base claims.
Claim 17 recites the limitation "… combine the set of keywords with a predetermined set of subject-matter keywords … generate, using a machine learning model trained to generate virtual content based upon keywords, the personalized virtual content from the set of keywords representative of the customer characteristics combined with the predetermined set of subject-matter keywords …" in lines , which rendering the claim indefinite because it.
Claims 18-20 are rejected for fully incorporating the deficiency of their respective base claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Independent Claims 1, 12, and 17
Step 1: Claim 1 is a process claim, Claim 12 is a system claim, and Claim 17 is a non-transitory computer-readable medium claim. These claims are fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) recite(s) "identifying/identify a set of keywords representative of customer characteristics from the personal data" (i.e., a person can mentally identify keywords from data), "combining/combine the set of keywords with a predetermined set of subject-matter keywords associated with an area of interest of the customer" (i.e., a person can mentally combine keywords from data with predetermined subject keywords of interest), and "generating/generate personalized virtual content from the set of keywords representative of the customer characteristics combined with the predetermined set of subject-matter keywords" (i.e., a person can mentally generate personalized content based on combined keywords) which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper").
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) recite(s) additional elements/limitations of "a processor" (Claim 1), "a virtual headset", "computing system" (Claim 12), "one or more processors" (Claims 12 and 17), "one or more non-transitory memories" (Claim 12), "one or more non-transitory computer-readable media" (Claim 17), "one or more computing systems" (Claim 17) (i.e., generic computer components), "collecting/collect personal data of a customer" (i.e., insignificant extra solution activity of data gathering), "using a machine learning model trained to generate virtual content based upon keywords" (i.e., apply it), and "providing/provide the personalized virtual content to the customer in a virtual environment " (i.e., insignificant extra solution activity of transmitting data/presenting results) which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) the additional limitation/element "collecting/collect personal data of a customer" is well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"); (b) the additional limitation/element "using a machine learning model trained to generate virtual content based upon keywords" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"); and (c) the additional limitation/element "providing/provide the personalized virtual content to the customer in a virtual environment" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 2 and 13
Step 1: Claim 2 is a process claim and Claim 13 is a system claim. These claims are fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) further recite(s) "the personalized virtual content is further generated based upon the received inquiry" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper").
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "receiving an inquiry from the customer" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "receiving an inquiry from the customer" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 3
Step 1: Claim 3 is a process claim. The claim is fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "collecting/collect personal data of a customer, wherein the personal data includes at least one of health data, biometric data, ethnic data, race data, age, sex, gender, income bracket, credit score, personal training history, data associated with indications of knowledge of the customer, personalized multimodal learning data, or user specific required training" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "collecting/collect personal data of a customer, wherein the personal data includes at least one of health data, biometric data, ethnic data, race data, age, sex, gender, income bracket, credit score, personal training history, data associated with indications of knowledge of the customer, personalized multimodal learning data, or user specific required training" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 4
Step 1: Claim 4 is a process claim. The claim is fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "collecting/collect personal data of a customer, wherein the personal data includes at least one of social media posts, voice recordings, photographs, images, or videos" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "collecting/collect personal data of a customer, wherein the personal data includes at least one of social media posts, voice recordings, photographs, images, or videos" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 5, 14, and 18
Step 1: Claim 5 is a process claim, Claim 14 is a system claim, and Claim 18 is a non-transitory computer-readable medium claim. These claims are fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "collecting/collect personal data of a customer, wherein the personal data comprises data regarding past communications of the customer" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "collecting/collect personal data of a customer, wherein the personal data comprises data regarding past communications of the customer" is also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 6
Step 1: Claim 6 is a process claim. The claim is fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "providing/provide the personalized virtual content to the customer in a virtual environment, wherein the personalized virtual content comprises a virtual object in the virtual environment, a video, an image, synthetically produced audio, or a voice recording" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "providing/provide the personalized virtual content to the customer in a virtual environment, wherein the personalized virtual content comprises a virtual object in the virtual environment, a video, an image, synthetically produced audio, or a voice recording" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 7
Step 1: Claim 7 is a process claim. The claim is fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) further recite(s) "identifying one or more characteristics of a synthetic agent" and "the personalized virtual content is further generated based upon at least one of the one or more characteristics of the synthetic agent" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper").
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional elements/limitations of "the personalized virtual content is provided to the customer by the synthetic agent" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "the personalized virtual content is provided to the customer by the synthetic agent" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 8, 15, and 19
Step 1: Claim 8 is a process claim, Claim 15 is a system claim, and Claim 19 is a non-transitory computer-readable medium claim. These claims are fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "training/train the machine learning model to generate user-specific content based upon the set of keywords representative of the customer characteristics" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "training/train the machine learning model to generate user-specific content based upon the set of keywords representative of the customer characteristics" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 9
Step 1: Claim 9 is a process claim. The claim is fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "the machine learning model is further trained to generate the personalized virtual content based upon an additional set of additional keywords directed to content messaging" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "the machine learning model is further trained to generate the personalized virtual content based upon an additional set of additional keywords directed to content messaging" is also well-understood, routine and conventional (WURC) activity similar to "performing repetitive calculation" (see MPEP 2106.05(d), "Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199 (recomputing or readjusting alarm limit values)"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claims 10, 16, and 20
Step 1: Claim 10 is a process claim, Claim 16 is a system claim, and Claim 20 is a non-transitory computer-readable medium claim. These claims are fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) further recite(s) "generating/generate the virtual environment based upon the indicated/a desired training module", "determining/determine training content from the desired training module and the personal data", and "generating/generate one or more virtual objects associated with the determined training content" which can be reasonably considered as mental processes (i.e., which "can be performed in the human mind, or by a human using a pen and paper").
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "an additional virtual headset", "receiving, by a user interface, an indication of a desired training module" (Claim 10), "a desired training module indicated via a user interface" (Claims 16 and 20), and "providing/provide the (i) virtual environment, (ii) one or more virtual objects, and (iii) training content to a user of the additional virtual headset " which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because (a) the additional limitations/elements "receiving, by a user interface, an indication of a desired training module" (Claim 10) and "a desired training module indicated via a user interface" (Claims 16 and 20) are also well-understood, routine and conventional (WURC) activity similar to "receiving or transmitting data over a network" (see MPEP 2106.05(d), "Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)"); and (b) the additional limitation/element "providing/provide the (i) virtual environment, (ii) one or more virtual objects, and (iii) training content to a user of the additional virtual headset " is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim 11
Step 1: Claim 11 is a process claim. The claim is fall within at least one of the four categories of patent eligible subject matter
Step 2A Prong 1: the claim(s) does not further recite(s) any mentor processes.
Step 2A Prong 2: this judicial exception is not integrated into a practical application because the claim(s) further recite(s) additional element/limitation of "the user interface is a virtual user interface provided to the user by the additional virtual headset" which only amount to "apply it" with the use of generic computer components or insignificant extra solution activity. None of the additional elements/limitations, taken alone or in combination, integrate the abstract idea into a practical application.
Step 2B: the claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional limitation/element "the user interface is a virtual user interface provided to the user by the additional virtual headset" is also well-understood, routine and conventional (WURC) activity similar to "presenting offers and gathering statistics" (see MPEP 2106.05(d), "Presenting offers and gathering statistics, OIP Techs., 788 F.3d at 1362-63, 115 USPQ2d at 1092-93"). Thus, none of the additional limitations, taken either alone or combined, amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9, 12-15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Abdel-Wahab et al. (US 2022/0335476 A1, pub. date: 10/20/2022, filed on 04/20/2021), hereinafter Abdel-Wahab in view of Zheng et al. (US 2015/0112918 A1, pub. date: 04/23/2015), hereinafter Zheng.
Independent Claims 1, 12, and 17
Abdel-Wahab discloses a computer-implemented method of generating personalized virtual content for a customer in a virtual environment (Abdel-Wahab, ¶¶ [0001] and [0013]-[0014]: providing interactive personalized immersive content; provide an immersive content platform that is capable of enhancing immersive experiences (e.g., via extended reality (XR), such as augmented reality (AR), virtual reality (VR), or mixed reality (MR))) with location-based and/or personalized content (or asset) renderings), the computer-implemented method comprising:
collecting, by a processor (Abdel-Wahab, ¶ [0124] with 404 in FIG. 4: the processing unit 404 can be any of various commercially available processors; dual microprocessors and other multiprocessor architectures can also be employed as the processing unit 404), personal data of the customer (Abdel-Wahab, ¶¶ [0014] and [0033]-[0045] with 210 and 215 in FIG. 2A: obtain contextual (or situational) information relating to a user; the contextual information may be related to people, objects, and/or events occurring in proximity to, or associated with, the user; the contextual information may include data regarding a location of the user, data regarding a media content item (e.g., video, audio, etc.) that the user has requested or is presently/currently consuming, calendar/travel-related data associated with the user, data regarding a voice-based input provided by the user, data regarding a gesture-based input provided by the user, data regarding a present time of day, weather data, data regarding structures (e.g., buildings or other objects) at or near the location, and/or the like; determine a likely mood of the user based on a present time of day, weather information, genre of music that the user is currently consuming, the user's voice-based inputs, the user's gesture-based inputs, and/or the like; identify (and/or retrieve) media content that relates to the contextual information and to profile data associated with the user; the profile data can include data relating to user preferences (e.g., historical explicit preferences, including advertisement placement policy restrictions, opt-in or opt-out preferences, or the like), data relating to user behaviors and/or interests (e.g., historical behaviors, such as Internet browsing activities, content consumption (e.g., videos, games, etc.), purchase histories, and/or the like), demographic data associated with the user (e.g., age of the user, gender of the user, etc.), and/or the like; the profile data can additionally, or alternatively, include data relating to prior locations of the user ( e.g., places that the user has visited, performances/shows/conferences that the user has attended, etc.), which may, e.g., be determined based on historical location (e.g., GPS) data, based on Exif (Exchangeable image file) data from photos previously captured by a camera of the user's smartphone, based on historical calendar data, etc.; the profile data can additionally, or alternatively, include data relating to prior conversations, discussions, and/or engagements of the user, data relating to advertisement responses of the user (e.g., advertisement exposures, click-through actions, affinities between users and advertisements and/or advertisement types) and/or other data representative or indicative of user activities, preferences, and/or behaviors (e.g., Interactive Advertising Bureau (IAB)-related data, tag data, genre data, embedding data, and/or the like); the profile data can include XR domain data, such as data relating to user behavior in immersion environments (e.g., user activities or interactions associated with objects in immersion environments, including objects that are native to the immersion environment and/or advertising objects included, or embedded, in the immersion environment); the profile data may include social profile information associated with the user (e.g., the user's social media/networking profile); the social profile information may include data regarding actions, preferences, activities, and/or the like relating to the user's friends, family, or other connections, such as other users that the user may be following, etc.; some data/information described as being contextual information may instead be characterized as profile data, and vice versa; the media content may include audio content (e.g., stereo audio, surround sound, binaural audio, 3D audio, etc.), such as a song, a tune, a speech, a soundscape, and/or the like; the media content may include image/video/object content (e.g., video clips, graphics, computer-generated objects, etc., which may, e.g., include 2D/3D VR, AR, MR objects or the like); the media content may include, or relate to, a scene from a film or television show, a recording of an event (e.g., a concert, a sports competition, etc.), people/characters (e.g., represented by virtual objects, such as avatars), a product to be marketed or advertised, etc.; ¶¶ [0077]-[0079] and [0081]-[0082] with 230 in FIG. 2A: obtain user interaction data from one or more of the target device(s) 204 based upon user interactions with personalized media content rendered by the one or more target device(s) 204; user interactions can include gestures (e.g., hand- or finger-based selections or movements), utterances or other voice-based commands, etc. relative to the rendered content; obtain other data relating to user movements (e.g., position data, such as gyroscope data, provided by a user's smartphone, smartwatch, or the like; image/video data associated with the user captured and provided by nearby cameras (e.g., IoT cameras); etc.), such as dance-related movements (e.g., swaying, rocking, or the like of the user's body, such as the arms, shoulders, hips, legs, etc.) or the like; obtain user interaction data associated with, e.g., the user touching the particular branded shirt worn by the 3D avatar of the person/character in the parade, the user uttering that the user likes the song being played back during the parade, and/or the like; enable a user to bookmark, add to a playlist, or otherwise record, an immersion and/or user interaction data therein for future playback; e.g., in a case where the user utters that the user likes a song that is being played back during an immersion or where the user manually selects a rendered object corresponding to the song (such as a 3D music symbol icon/object presented in the immersion and corresponding to the song), the immersive content platform 202 may cause the song to be added to the user's personal playlist; ¶ [0094] with 270a and 270 b in FIG. 2E: obtain contextual information (270b) relating to a user (e.g., location data, data regarding a media content item that the user has requested or is currently consuming, calendar data, travel-related data, voice-based input data associated with the user, gesture-based input data associated with the user, time of day information, weather information, etc.), and identify media content (e.g., audio content, image content, video content, XR objects, etc.) (270d) that relates to the contextual information and to profile data (270a) associated with the user (e.g., information regarding preferences of the user, interests of the user, a browsing history of the user, a media consumption history of the user, a purchase history of the user, an advertising response history of the user, historical immersion-related behavior of the user, etc.); ¶¶ [0097]-[0098] with 270c and 207h in FIG. 2E: detect user interaction data (e.g., information regarding movements of the user, gesture-based inputs of the user, voice-based inputs of the user, etc.) (270c) relating to the immersion environment, and perform action(s) relating to the personalized media content based on the user interaction data; orchestrate (e.g., via a follow-up orchestrator 270h) one or more follow-up actions/activities, such as recording the personalized media content and/or the user interaction data (e.g., in a media content library associated with the user, in a general archive, and/or the like) for future playback, sharing the personalized media content and/or the user interaction data with one or more other users associated with the user, associating the personalized media content and/or the user interaction data with the location of the user for future use (e.g., in an immersion associated with a different user, etc.), facilitating a reservation for a service for the user, facilitating a purchase of a product for the user, scheduling a service appointment for the user, facilitating additional media location guidance ( e.g., presenting data or content associated with additional follow-up activities at a subsequent location), etc.; ¶¶ [0102]-[0103] with 290a and 290b in FIG. 2F: at 290a, obtaining contextual information relating to a user, where the contextual information comprises location data that identifies a location of the user; at 290b, identifying media content that relates to the contextual information and to profile data associated with the user; ¶ [0106] with 290e in FIG. 2F: at 290e, detecting user interaction data relating to the immersion environment);
identifying, by the processor, a set of keywords representative of customer characteristics from the personal data (Abdel-Wahab, ¶¶ [0048]-[0054] with FIGS. 2B-2D: in a case where the contextual information (e.g., location data) indicates that the user is located at, or is approaching or located proximate to (e.g., within a threshold distance from), a particular landmark (e.g., the Empire State Building), and where the profile data indicates that the user has an affinity for a particular character ( e.g., a fictional character, such as Superman), identify and/or retrieve media content that involves the particular landmark and the particular character ( e.g., a reenactment of a scene from a film in which Superman flew by the Empire State Building); in a case where the contextual information (e.g., location data, data regarding a present time of day, etc.) indicates that it is a suitable time for the user to consume a meal (e.g., noontime, etc.) and that the user is lingering or located at, or is approaching or located proximate to (e.g., within a threshold distance from), a particular restaurant (e.g., in Little Italy, New York City), and where the profile data indicates that the user has an affinity for a particular person (e.g., a celebrity, such as Frank Sinatra) associated with the particular restaurant, identify and/or retrieve media content that involves the particular restaurant and the particular person (e.g., a 3D representation of Frank Sinatra at or near the restaurant); in a case where the contextual information includes a voice-based input from the user that identifies a person, a place, a thing, an event, etc., such as an utterance of a name of a location or establishment (e.g., "Rudy's," "Max's Kansas City," "Copacabana," etc.), a name or title of an historical event ( e.g., "Woodstock," etc.), or the like; ¶ [0082]: based upon detecting user interaction in the immersion, such as an utterance of "I want some coffee now" or the like) cause one or more offers for products of that particular brand to be provided to the user; cause the one or more offers to be provided to the user via a text message, via e-mail, by adding a product of that particular brand to the user's virtual shopping cart, by placing an order for a product of that particular brand at a nearby store, and/or the like; ¶ [0092]: the user uttering positive phrases (e.g., "wow," "I like that," etc.) in relation to some media content in an immersion, but uttering negative phrases (e.g., "I don't like that," etc.) in relation to other media content; etc.));
combining, by the processor, the set of keywords with a preference associated with an area of interest of the customer; generating, by the processor, using a machine learning model trained to generate virtual content based upon keywords, the personalized virtual content from the set of keywords representative of the customer characteristics combined with the mail, by adding a product of that particular brand to the user's virtual shopping cart, by placing an order for a product of that particular brand at a nearby store, and/or the like; ¶¶ [0092]-[0093] with FIG. 2A: the immersive content platform 202 can employ machine learning algorithm(s) that are configured to learn a user's behavior or preferences relating to interactive, personalized media content experiences (including, e.g., immersive experiences); this can include, e.g., the user's reactions (e.g., selections, movements, utterances, etc.) relating to presented media content, the user's preferences for objects (e.g., AR objects, etc.), storylines, etc. in media content, and/or the like; adjust, based on the learned information, future actions performed by, or outputs provided by, the immersive content platform 202 to enhance the user's immersive experiences; provide information regarding a user's preferences or behavior as input to one or more machine learning algorithms, which may perform machine learning to automate future determinations or predictions of user preferences or behavior; e.g., train a machine learning algorithm based on known inputs (e.g., identified, generated, and/or provided media content) and known outputs (e.g., the user engaging, such as moving toward, reaching out for, touching, etc., an AR object in an immersion, but not engaging other media content in the immersion; the user uttering positive phrases (e.g., "wow," "I like that," etc.) in relation to some media content in an immersion, but uttering negative phrases (e.g., "I don't like that," etc.) in relation to other media content; etc.); refine a machine learning algorithm based on feedback received from a user of the immersive content platform 202 and/or from one or more other devices (e.g., management device(s)); e.g., provide feedback indicating whether predictions of user preferences or behavior, made by the machine learning algorithm based on new inputs, are accurate and/or helpful; predict user preferences or behavior based on one or more machine learning algorithms, which improves the accuracy of the predictions, and conserves processor and/or storage; resources that may otherwise be used to generate and store rules for predicting user preferences or behavior; ¶¶ [0014], [0041]-[0043], and [0055]-[0073] with 220 in FIG. 2A: derive, from the media content, personalized media content based on the profile data, wherein the profile data may be used as personalization cues or triggers, and may include information regarding preferences of the user, interests of the user, a browsing history of the user, a media consumption history of the user, a purchase history of the user, an advertising response history of the user, historical immersion-related behavior of the user, etc.; generate personalized media content by defining, adapting, or adjusting one or more aspects or parameters of identified media content or objects within the identified media content; define or adjust an appearance (e.g., tone, nature, etc.) of, or one or more visual characteristics of, an immersion environment or objects (e.g., AR objects or the like) to be included in the immersion environment; modify an appearance of a character in a film or television show scene, such as by substituting a shirt worn by the character for a shirt of a brand that the user recently searched for on the Internet and/or clicked through in an advertisement, substituting a shoe worn by the character with a shoe that the user previously observed at a store and uttered "this shoe looks nice," and/or the like; define or adjust content based on demographic data associated with the user; customize an immersion environment with background audio to provide a more entertaining immersive experience for the user; adjust a quality of the content, or otherwise adapt the content to a less computationally-intensive version of the content, to suit the network conditions and/or the capabilities of the available target device(s) 204; identify different media content and/or generate different personalized media content for different available target device(s) 204 to render or present, based on the capabilities of the different target device(s) 204, orientations of the different target device(s) 204, etc. so as to provide a more realistic and enjoyable immersive experience for the user; define or adjust how media content (or an overall immersion) is to be rendered based on environmental factors or obstacles (e.g., local weather, local time of day, lighting, presence of potential obstacles proximate to the user for occlusion purposes, etc., as may be identified in available data, such as from local information sources, image/video data captured and provided by nearby cameras (e.g., IoT cameras, etc.), and/or the like) so as to provide a more realistic and enjoyable immersive experience for the user; rank or score available media content, and select media content to present to the user based on the rankings or scores, which can aid the immersive content platform 202 in determining which media content to present to the user, particularly in a case where the immersive content platform 202 identifies numerous relevant media content that matches the contextual information and/or the user's profile data; rank media content based on a determined level of user affinity relating to a given media content; rank media content based on recency of user affinity relating to a given media content; rank media content based on advertising or marketing opportunities; ¶ [0094] with 270c in FIG. 2E: derive, from the media content, personalized media content based on the profile data associated with the user (e.g., information regarding preferences of the user, interests of the user, a browsing history of the user, a media consumption history of the user, a purchase history of the user, an advertising response history of the user, historical immersion-related behavior of the user, etc.), and cause one or more target device(s) (e.g., a smartphone, a smartwatch, an XR-based user equipment, and/or one or more proximal devices, such as proximal displays, proximal speakers, etc.) (270c) to provide an immersion environment that includes the personalized media content; ¶ [0104] with 290c in FIG. 2F: at 290c, deriving, from the media content, personalized media content based on the profile data associated with the user); and
providing, via a virtual headset (Abdel-Wahab, ¶ [0030] with 204 in FIG. 2A: the target device(s) 204 may include one or more devices capable of receiving, generating, storing, processing, and/or providing data (e.g., audio data, video data, XR data, text data, control data, etc.) relating to the immersive content platform 202; a target device 204 can include a wearable communication device (e.g., a smart wristwatch, a pair of smart eyeglasses, XR gear (e.g., a pair of AR, VR, MR glasses, a headset, headphones, and/or the like)), the personalized virtual content to the customer in the virtual environment (Abdel-Wahab, ¶¶ [0014] and [0075]-[0076] and [0079]-[0091] with 225 in FIG. 2A: provide different portions of the immersion data to different ones of the target device(s) 204 to render so as to provide a (e.g., fully) immersive experience for the user; personalize the media content to include a particular song by the music artist during rendering of the parade and/or to substitute a shirt worn by a character/person in the parade with a shirt of the user's preferred brand; in a case where the user utters that the user likes a song that is being played back during an immersion or where the user manually selects a rendered object corresponding to the song (such as a 3D music symbol icon/object presented in the immersion and corresponding to the song), the immersive content platform 202 may cause the song to be added to the user's personal playlist; initiate a ( e.g., real world) transaction, such as a purchase request or the like, based on user interaction data, which can enhance the user's overall immersive experience and/or broaden marketing channels; in a case where the user touches a particular item (e.g., a branded shirt, a meal, etc.) in an immersion, the immersive content platform 202 may cause the item (e.g., shirt) to be added to the user's virtual shopping cart and/or facilitate a purchase or order of the item (e.g., shirt, meal, etc.) with a local merchant nearby and/or over the Internet; based upon detecting user interaction in the immersion, such as an utterance of "I want some coffee now" or the like) cause one or more offers for products of that particular brand to be provided to the user; cause the one or more offers to be provided to the user via a text message, via e-mail, by adding a product of that particular brand to the user's virtual shopping cart, by placing an order for a product of that particular brand at a nearby store, and/or the like; generate, and cause the target device(s) 204 to present, data or content associated with additional follow-up activities relating to a particular location; automatically schedule a particular appointment for the user (e.g., based on an immersion including media content relating to dentistry and based on the user's calendar data indicating that the user is past due for dental cleaning, etc.), set a reminder for the user to complete a task, prepare and/or submit a social media post to "check-in," and/or the like; modify an immersion (e.g., in real-time or near real-time) based upon detecting user movement or reactions (e.g., gazing, walking, running, slowing down, stopping movement, etc.) during the immersion, user interaction with media content in the immersion (e.g., selections of objects, utterances, etc.), user engagement with one or more other users also experiencing the immersion, etc.; e.g., modify an immersion by adjusting a scene (e.g., graphics) of the immersion as the user moves about, turns around, or the like; by switching playback of certain portions of media content between different target device(s) as the user moves about, turns around, or the like; and/or by removing or adding additional media content to the immersion, such as based upon detecting user movements, gestures, commands, utterances, etc. (which enables in-painting of the immersion, such as by adding a virtual basketball hoop object in response to the user performing movements indicative of a basketball shooting motion and/or the like); modify an immersion ( e.g., in real-time or near real-time) according to a (e.g., predicted) mood or sentiment of the user, which may be determined based upon the user's voice-based inputs, based upon the user's gesture-based inputs, and/or based upon content that the user typically consumes during certain activities and/or during certain times of day, such as during workouts in the morning, during the evening after dinner, and/or the like; alter media content in an immersion to improve the user's mood; generate personalized media content to include trigger(s) that coax, or otherwise entice, a user to perform an action; modify an immersion by controlling a visual appearance and/or movements of one or more characters in an immersion based on the present user's movements (e.g., dance moves), speech, manner of speech, etc.; XR content creators or authors may enable pre-splitting of dialog points for user interaction, e.g., to allow a user to pick up acting within a scene, such as speaking a line intended to be spoken by an original character, but where the user's speaking of the line triggers the scene to begin/continue; ¶¶ [0095]-[0097] in 270e in FIG. 2E: a rendering/re-creation engine 270e configured to generate immersion data, that includes the personalized media content, for the target device(s) to render; access to and/or identify content creator-provided triggers (270/ and 270g), and combine, integrate, or otherwise include, such triggers with corresponding media content in an immersion to provide opportunities for new content generation and enhanced immersive experiences for users; interleave (e.g., using the rendering/re-creation engine 270e) real-time media content or assets (e.g., adapted based on user movements, adapted based on predicted user activities, and/or the like) with original or previously-rendered versions of the media content (including, e.g., by linking to such prior content or assets as needed) to provide a smooth or seamless immersive experience for the user, while, for example, maintaining a prior narrative in the media content; as an example, for interleaved rendering, a previously-created version of an immersion for a basketball team may have been created (e.g., a historical rendering) and may be served to the user, but the immersion may be rendered by the rendering/re-creation engine 270e with real-time, or near real-time, updates (e.g., featuring highlights or occasional updates from a game that is currently occurring) and/or timely, user-specific content (e.g., a visual of the user in a case where the user was previously present at a stadium and posed for a "fan-cam" or the like); ¶ [0105} with 290d in FIG. 2F: At 290d, causing a target device to provide an immersion environment that includes the personalized media content).
Abdel-Wahab further discloses a computing system configured to generate personalized virtual content (Abdel-Wahab, ¶¶ [0020], [0116], and [0143] with 100 in FIG. 1, 400 in FIG. 4, and 60 in FIG. 6: a system 100/a computing environment 400/a communication device 600 can facilitate, in whole or in part, identifying, generating, and/or providing of interactive, personalized media content based on contextual information and user profile data; ¶¶ [0029] and [0094] with 200 in FIG.2A and 270 in FIG. 2E: the system 200/270 can include an immersive content platform 202 which can include one or more devices (e.g., server device(s) or the like) configured to provide one or more functions or capabilities, such as identifying, generating, and/or providing interactive, personalized media content based on contextual information and user profile data), the computing system comprising: one or more processors (Abdel-Wahab, ¶ [0124] with 404 in FIG. 4: the processing unit 404 can be any of various commercially available processors; dual microprocessors and other multiprocessor architectures can also be employed as the processing unit 404; ¶¶ [0144] and [0150] with 606 in FIG. 6: the controller 606 can utilize computing technologies such as a microprocessor, a digital signal processor (DSP), programmable gate arrays, application specific integrated circuits, and/or a video processor); and one or more non-transitory memories (Abdel-Wahab, ¶¶ [0124]-[0126] with 406/414/416/420 in FIG. 4: the system memory 406 comprises ROM 410 and RAM 412; the computer 402 further comprises an internal hard disk drive (HDD) 414 (e.g., EIDE, SATA), which internal HDD 414 can also be configured for external use in a suitable chassis (not shown), a magnetic floppy disk drive (FDD) 416, (e.g., to read from or write to a removable diskette 418) and an optical disk drive 420, (e.g., reading a CD-ROM disk 422 or, to read from or write to other high capacity optical media such as the DVD)) storing computer-executable instructions (Abdel-Wahab, ¶¶ [0127]-[0128]: the drives and their associated computer-readable storage media provide nonvolatile storage of data, data structures, computer-executable instructions, and so forth) that, when executed via the one or more processors, cause the computing system to perform the method described above (Abdel-Wahab, ¶ [0150] with FIG. 6: the controller 606 can utilize computing technologies such as a microprocessor, a digital signal processor (DSP), programmable gate arrays, application specific integrated circuits, and/or a video processor with associated storage memory such as Flash, ROM, RAM, SRAM, DRAM or other storage technologies for executing computer instructions, controlling, and processing data supplied by the aforementioned components of the communication device 600).
Abdel-Wahab also discloses one or more non-transitory computer-readable media storing computer-executable instructions (Abdel-Wahab, ¶¶ [0124]-[0128] with 406/414/416/420 in FIG. 4: the system memory 406 comprises ROM 410 and RAM 412; the computer 402 further comprises an internal hard disk drive (HDD) 414 (e.g., EIDE, SATA), which internal HDD 414 can also be configured for external use in a suitable chassis (not shown), a magnetic floppy disk drive (FDD) 416, (e.g., to read from or write to a removable diskette 418) and an optical disk drive 420, (e.g., reading a CD-ROM disk 422 or, to read from or write to other high capacity optical media such as the DVD); the drives and their associated computer-readable storage media provide nonvolatile storage of data, data structures, computer-executable instructions, and so forth) that, when executed via one or more processors, cause one or more computing systems to perform the method described above (Abdel-Wahab, ¶ [0150] with FIG. 6: the controller 606 can utilize computing technologies such as a microprocessor, a digital signal processor (DSP), programmable gate arrays, application specific integrated circuits, and/or a video processor with associated storage memory such as Flash, ROM, RAM, SRAM, DRAM or other storage technologies for executing computer instructions, controlling, and processing data supplied by the aforementioned components of the communication device 600).
Abdel-Wahab fails to explicitly disclose combining the set of keywords with a predetermined set of subject-matter keywords associated with an area of interest of the customer; generating the personalized virtual content from the set of keywords representative of the customer characteristics combined with the predetermined set of subject-matter keywords.
Zheng teaches a system and a method relating to providing personalized content (Zheng, ¶¶ [0005]-[0006] and [0043]), wherein combining the set of keywords with a predetermined set of subject-matter keywords associated with an area of interest of the customer; generating the personalized virtual content from the set of keywords representative of the customer characteristics combined with the predetermined set of subject-matter keywords (Zheng, ¶ [0045]-[0047] with FIGS.1(a)-1(b): a user 106 may send a request and provide basic user information to the content portal 104 (e.g., a search engine, a social network site, etc.) and receive recommended content from the content portal 104; the content recommendation engine 102 may work as backend support (e.g., FIG. 1(a)) to provide estimated topics of interest for the user 106 to the content portal 104 based on basic information of the user 106; more than one topics of interest may be ranked and provided to the content portal such that the content portal 104 may retrieve content for each topic and present content to the user 106 based on the ranking of the multiple estimated topics of interest; both the content recommendation engine 102 and content portal 104 may access information from any of the content sources 110-a, 110-b, ... , 110-c to obtain dynamic information related to the users 106 or to identify and retrieve content based on estimated interests of the user; the user content activity monitor 112 act as a service provider, independent of the content recommendation engine 102, that monitors and gathers dynamic user-related content and activities and provides such collected information to the content recommendation engine 102 as the basis for continuously updating the recommendation model for recommending up-to-date personalized content to user 106; the content recommendation engine 102 is configured as an independent service provider (e.g., FIG. 1(b)) that interacts with the users 106 directly to provide personalized content recommendation service; i.e., the content recommendation engine 102 may receive a request with some basic information from a user 106 and/or dynamic content associated with users and provide recommended content to the user directly without going through a third-party content portal 104; ¶ [0048] with FIG. 2: for previous (existing) users of the content recommendation engine 102, three types of user information may be obtained by the content recommendation engine 102: (1) monitored user-related content, such as content consumed by the user, content contributed by the user such as the user's own blog or microblog entry, user's comments on others' biogs or micro-blog entries, etc.; (2) user profiles, i.e., user's basic attributed stored in the user information archive of the content recommendation engine 102 or collected from other content sources 110; and (3) monitored user online activities, such as clicks, lack of clicks, online purchases made, or online gaming activities, etc.; based on characterization of information associated with existing users, the content recommendation engine 102 may build and refine a model for estimating topics of interest for such users (or for new users); when a new user 204 signs up with the content recommendation engine 102 with some basic user information such as attributes, the content recommendation engine 102 may estimate the new user's topics of interest based on the new user information received, and present the recommended content on such estimated interests to the new user 204; the received new user's information as well as the initial recommended content on estimated topics of interest may then be user to expand the recommendation model so that the recommendation model now incorporates the new user's information to form an aggregated or integrated model (integrated in the sense that it is appended to the existing model for all other existing users); any further online activities (including content consumed or contributed) of the new user are monitored and used to further refine the recommendation model so that content recommendations can be made adaptive; ¶¶ [0049]-[0052] with FIGS. 3(a)-3(b): any user information may be dynamically monitored and gathered by the user content/activity monitor 112; user's online activities are thus informative as to the user's interests, particularly dynamic interests; different types of user information may also be monitored and gathered through different mechanisms employed by the user content/activity monitor 112; e.g., the user-related content and user profile may be collected by a web crawler 302, and the user activities may be monitored by an activity monitor 304; a topic monitor/propagator 306 may be responsible for inferring the user's interests through, e.g., the user's activities and social graph in a social network setting; e.g., topic monitor/propagator 306 may be configured to apply link propagation methods to propagate labels (e.g., categories) of celebrities to every user in the social network based on the social graph and user activities (e.g., following, friends); in general, the importance of a user in a social network can be inferred and the personal interests of an importance user who have significant number of followers may be used to infer the interests of the followers; the identified users and corresponding gathered content and activities may be saved in a user-related information archive 312 and a content/activity information archive 314 for future use; the user content/activity monitor 112 may be continuously collecting dynamic user-related content and dynamic user activities to enable the continuous update of the recommendation model; ¶¶ [0053]-[0060] with FIGS. 4(a)-(b): in response to receiving a request to recommend content to a new user or an existing user, the content recommendation engine 102 is configured to obtain information associated with user, whether such information is static, dynamic, explicit or implicit, and identify one or more topics of interest for the user based on a model that maps from users to topics of interest; the model is established based on information related to the existing users of the content recommendation engine 102; for a new user who just signed up to the system, basic attributes of the new user, such as age, gender, profession, residency, etc., is sufficient for the content recommendation engine 102 to make recommendation based on the recommendation model; for an existing user, every time when the existing user signs in the system, the content recommendation engine 102 is able to provide an up-to-date recommendation based on the continuously refined recommendation model and/ or the dynamically refreshed user information and online behaviors; the content recommendation engine 102 includes a user characterization module 402, a modeling module 404, a feature database (or user feature database) 406, a topic database (or content feature database) 408, a user request processing unit 410, a topic estimation module 412, a content recommendation module 414, and a user profile archive 416; the user characterization module 402 includes three units, each of which is responsible for processing one type of input dynamic user information; the dynamic user-related content and user activities are characterized by the user-related content characterization unit 418 and the user activity characterization unit 420, respectively, and are converted to content feature information, including topics/categories and keywords (e.g., represented by a content feature matrix B); the user profile (attributes) is characterized and converted to user feature information (e.g., represented by a user feature matrix A) by the user information characterization unit 422; both the user and content feature information are fed into the modeling module 404 to generate a recommendation model for the topic estimation module 412; the modeling module 404 is configured to establish a model that maps from users to topics of interest based on the user and content feature information fed from the user characterization module 402; the model may be established based on a user feature matrix A representing user features with respect to the existing users and a content feature matrix B representing content features with respect to the existing users; the user feature and content feature database 406, 408 may store a larger amount of information related to user attributes, topics, and keywords, as compared with the user and content feature information used by the modeling module 404 to establishing or refining the model; over time, a user's interest may change; this may be observed when recommended content has not been selected by the user; in this case, topics or keywords associated with such unselected content may be removed from content feature matrix B and new interest may be retrieve from the archive to replace the staled interests; user to user matching based on interested topics or keywords may be achieved by identifying rows in content feature matrix B that are similar; i.e., users corresponding to such rows have shared interests; on the other hand, if one is query about other users who have similar attributes, user to user matching may be performed by analyzing user feature matrix A on the similarity between the row representing the querying user and rows representing other users; this is also a mechanism that enables recommendations of content of interests for a new user; whenever a new user signs up, look for one or more "similar" users through the model and consider recommending content consumed or contributed by the "similar" users to the new user, as they likely share the same implicit interest on such content; inferring and/or deriving a user's implicit interests and preferences via, e.g., derivation by the nine-way query-result matching or propagation by social graph, to obtain user-related information based on, e.g., information related to other existing users; the user request processing unit 410 may perform attribute preprocessing and normalization operations to generate a user feature vector for each user when they first time sign up or every time when they update their attributes; the topic estimation module 412 is configured to provide estimated topics of interest for new or existing users based on the recommendation model from the modeling module 404 and the user profile (e.g., feature vector) obtained from the user request processing unit 410; the estimated topics may be provided to the content recommendation module 414 to determine content that the user is most likely interested in; the estimated topics may be continuously fed back to the modeling module 404 for model refinement; user information, such as user profile, user-related content, and activities, is received by the content recommendation engine 102; the received user information is characterized by the user characterization module 402, and the characterized user information (e.g., represented as the content feature matrix B and user feature matrix A) is used to establish an initial recommendation model by the modeling unit 404; the model may be used by the topic estimation module 412 in conjunction with the content recommendation module 414 to recommend content for any online existing user (currently signing-in the content recommendation engine 102); e content recommendation engine 102 may continuously receive dynamically collected information of the existing users; the content recommendation engine 102 may continuously receive dynamically collected information of the existing users; refine the initial recommendation model based on the dynamically updated information of the existing user; once a new user signs up to the content recommendation engine 102, the information of the new user is collected and characterized in order to recommend content of interest to the new user; the information of the new user may be also integrated into the recommendation model by appending the new user's features and topics of interest to the existing user feature and content feature matrices in order to continuously enhance the recommendation model; ¶¶ [0061]-[0068] with FIGS. 5(a)-(d): the user information characterization unit 422 may include a user feature analysis unit 502, a user feature categorization unit 504, and a user feature quantification unit 506; the user feature analysis unit 502 is responsible for extracting basic user features from received user profiles without any dimension reduction process; the features extracted by the user feature analysis unit 502 may be represented as a user feature vector for each user[ the user information characterization unit 422 generates user feature information, which may be represented as an m×n matrix A; the user information characterization unit 422 generates user feature information, which may be represented as an mxn matrix A, with rows corresponding to users and columns corresponding to user features in a reduced dimension; the dimension of user feature vectors in the matrix A may be reduced by the user feature categorization unit 504 and user feature quantification unit 506 compared with the original dimension of user feature vectors; the user feature categorization unit 504 is configured to derive categorical features for each user base on correlation of its values with content interests, i.e., predefined feature categorization configuration 508; the user feature quantification unit 506 is configured to quantify each feature into value ranges according to data analysis, i.e., predefined feature quantification configuration 510; based on astrology and numerology, features such as a user's personality may be interred based on his/her constellation and Chinese Zodiac, which may be further combined with other features to infer the user's possible social roles and topics of interest; user information, such as user profiles, is received by the user information characterization unit 422; the user information is then analyzed by the user feature analysis unit 502 to extract original user features; the original user features may be archived in the feature database 406 for future use, such as model refinement; user features may be categorized to derive features based on predefined feature categorization configuration; categorical user features may be further quantified with respect to predefined feature quantification configuration to reduce the dimension of the feature vectors; eventually, user feature information with a reduced dimension is generated by the user information characterization unit 422; the user-related content characterization unit 418 and user activity characterization unit 420 are responsible for generating content feature information that indicates each user's interest profile; the content feature information may be represented as an m×n content feature matrix B, with rows corresponding to users and columns corresponding to topics/categories and keywords in a reduced dimension; the user-related content characterization unit 418 may include a user content analyzer 532 responsible for performing keywords selection from the user-related dynamic content based on language models 534 and vocabulary 536 and storing the extracted keywords in a user keywords storage 538; the user-related content may be any content consumed or contributed by the user; the user content analyzer 532 may apply any known language models to extract keywords and/or identify topics of interest from the content, e.g., by feature selection methods in text classification, such as document frequency (how many documents in the corpus a word occurs in), mutual information, information gain, chi-square, etc.; all those feature selection methods may help selecting of the most indicative keywords or key phrases from various candidate keywords with respect to any predefined category (topic of interest) from the user-related dynamic content; the user activity characterization unit 420 may include a user activity analyzer 540 responsible for analyzing the user's dynamic activities based on activity context information and topic hierarchy 542; the activity context information indicates the context of each user activity, such as the time when the activity occurs, the site where the activity happens, etc., which may have different weights when different user activities are aggregated; the user activity characterization unit 420 may also include a topic/keyword determiner 544 configured to determine content features, such as topics of interest or keywords, based on the user activities, the extracted keywords from the user keywords storage 538, and the topic hierarchy 542; the activities and keywords may be classified under predefined topics in terms of the same taxonomy in the topic hierarchy 542 by any known classifier; activities and keywords related to the same user may be aggregated through weighted linear combination into a single topic vector; in addition to explicit interests, topics of interest for each user may be also inferred as implicit interests by topic propagation methods in the social network setting; the determined topics of interest may be represented as a real-value vector (i.e., a vector of weights with respect to keywords and topics) for each user and stored in the content feature database 408 in their original dimensions; in order to reduce the dimension of topics in the content feature matrix B, a topic/keyword dimension reduction unit 546 may be applied in conjunction with predefined reduction aggressiveness configuration 548; known feature selection methods in text classification may be applied to calculate scores for each <topic, keyword> pair; the scores are then used to rank all the keywords for each topic; by setting a threshold on the scores or the number of keywords selected for each topic, the dimension of the topic vectors may be reduced; e.g., keywords such as "football," "basketball," "Michael Jordan," "NBA" may be considered as the most indicative keywords for "sports" topic and thus, are included in the content feature matrix B); at block 552, dynamic user-related content is received by the user-related content characterization unit 418; at block 554, the received content is analyzed to extract keywords based on language models and vocabulary; topics of interest related to the received dynamic content are then estimated at block 556 by, e.g., statistical classifiers; a block 558, dynamic user activities are also received by the user activity characterization unit 420; the received activities are analyzed at block 560, and their natures, such as whether an activity supports or negates a topic, are determined at block 562; implicit topics of interest (e.g., supporting or negating a topic) may be identified at block 564 based on the estimated topics and determined natures of activities; proceeding to block 566, all the identified topics associated with each user, whether explicit or implicit, and the keywords may be archived in the content feature database 408 as content feature for each user; at block 568, dimensionality reduction may be performed to reduce the dimension of content features in the content feature matrix B; eventually, at block 570, content feature information with a reduced dimension is generated by the user-related content characterization unit 418 and user activity characterization unit 420; ¶¶ [0069]-[0074] with FIGS. 6(a)-6(e): the modeling module 404 includes an initial modeling unit 602, a model integration unit 604, and a model refinement unit 606; the initial modeling unit 602 is configured to provide an initial model to the topic estimation module 412 based the user feature matrix A and content feature matrix B of the existing users; the modeling module 404 further includes a user feature matrix generator 608 and a content feature matrix generator 610; the user feature matrix generator 608 and content feature matrix generator 610 then combine user feature vectors and content feature vectors for all existing users to generate the user feature matrix A and content feature matrix B, respectively; the model integration unit 604 is configured to generate an integrated model by continuously appending the information of each new user (e.g., new user attributes, estimated topics of interest) to the user and content feature matrices of the existing model; given that online activities continuous occur and change, the model refinement unit 606 is responsible for dynamically refining the content recommendation model based on dynamic user-related content and activities and characterized user and content features; in addition, the discrepancy between the estimated topics and the actual user selected content may be used by the model refinement unit 606 for adjusting the current recommendation model to improve prediction accuracy; the up-to-date model may be always provided to the topic estimation module 412 for topics stimulation; the model refinement unit 606 includes a discrepancy detector 620, a refinement mode determiner 622, a user subgroup filter 624, a time-based interest filter 626, a content feature matrix remapping unit 628, a matrix update unit 630, and a model refiner 632; a user on week days and weekend/evening times may have different interested topics when online; sub-models for each user may be divided into such time frames and used accordingly depending on the time at which a recommendation needs to be made; more similar users may be grouped together to more precisely model the interests of this subgroup; the hierarchical models applied by the user subgroup filter 624 or the time-based interest filter 626 may be fed into the matrix update unit 630 to cause the model refiner 632 to adjust the content recommendation models; starting from block 640, user feature information and content feature information are received by the modeling module 404; at block 642, the user feature matrix A and content feature matrix B are generated based on the received user feature information and content feature information (e.g., topics, keywords), respectively; a recommendation model is then established at block 644 using the matrices A and B; proceeding to block 646, dynamic user online behavior information such as dynamic user-related content and user activities are continuously monitored and received by the model refinement unit 606; at block 648, the mode for refining the recommendation model is determined; if a gradual refinement mode is chosen, at block 650, matrices A and B are updated by the model refinement unit 606 using the dynamically updated user information; if estimated topics of interest provided by the current recommendation model are deemed to be undesired at block 652, the content feature matrix B may be updated using the next-best topics at block 654; otherwise, a hierarchical model may be applied to adjust the current recommendation model. At block 656, the current recommendation model may be divided into sub-models with respect to time or user; new sets of user feature matrices A's and content feature matrices B's may be generated at block 658 for the divided sub-models; eventually, at block 660, a refined recommendation model may be established using the new sets of user feature matrices A's and content feature matrices B's).
Abdel-Wahab and Zheng are analogous art because they are from the same field of endeavor, a system and a method relating to providing personalized content. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Zheng to Abdel-Wahab. Motivation for doing so would provide an improved solution for personalized content recommendation based on information associated with users, whether such information is static, dynamic, explicit or implicit, all in a systematic and effective manner .
Claims 2 and 13
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claims 1 and 12 and further discloses receiving, via the virtual headset, an inquiry from the customer, and wherein the personalized virtual content is further generated based upon the received inquiry (Abdel-Wahab, ¶¶ [0061]-[0063] with FIG. 2A: implement or enable user interactivity in an immersion by generating, or otherwise including, triggers that, upon being activated or engaged by a user, may cause the immersive content platform 202 to perform adjustments to the media content and/or to obtain and provide other (e.g., related) media content in an immersion; e.g., generate a 3D representation of a character for guiding a user to a particular location; enable triggering of, or user interaction with, the 3D representation---e.g., speaking in a direction toward the 3D representation may cause the 3D representation to respond (e.g., speak back, turn its head, etc.); touching of the 3D representation may cause the 3D representation to traverse several steps towards the particular location; and so on; provide an interactive, automated response mechanism (e.g., a chat-bot or the like) configured to respond to detected user inputs ( e.g., speech, movements, etc.) and/or guide a user through one or more triggers that are available in an immersion; e.g., generates a 3D representation of a character for guiding a user to a particular location, the automated response mechanism may, based upon detecting user movement toward the 3D representation, entice the user (e.g., visually, audibly, etc.) to move closer to the 3D representation to cause the 3D representation to perform an action ( e.g., to jump, to dance, to proceed several steps in a direction towards the particular location, etc.); ¶¶ [0077]-[0091] with 230 and 235 in FIG. 2A: detect user interaction data; user interactions can include gestures (e.g., hand- or finger-based selections or movements), utterances or other voice-based commands, etc. relative to the rendered content; obtain other data relating to user movements (e.g., position data, such as gyroscope data, provided by a user's smartphone, smartwatch, or the like; image/video data associated with the user captured and provided by nearby cameras (e.g., IoT cameras); etc.), such as dance-related movements (e.g., swaying, rocking, or the like of the user's body, such as the arms, shoulders, hips, legs, etc.) or the like; obtain user interaction data associated with, e.g., the user touching the particular branded shirt worn by the 3D avatar of the person/character in the parade, the user uttering that the user likes the song being played back during the parade, and/or the like; perform an action relating to the personalized media content based on the user interaction data; in a case where the user utters that the user likes a song that is being played back during an immersion or where the user manually selects a rendered object corresponding to the song (such as a 3D music symbol icon/object presented in the immersion and corresponding to the song), the immersive content platform 202 may cause the song to be added to the user's personal playlist; initiate a ( e.g., real world) transaction, such as a purchase request or the like, based on user interaction data, which can enhance the user's overall immersive experience and/or broaden marketing channels; in a case where the user touches a particular item (e.g., a branded shirt, a meal, etc.) in an immersion, the immersive content platform 202 may cause the item (e.g., shirt) to be added to the user's virtual shopping cart and/or facilitate a purchase or order of the item (e.g., shirt, meal, etc.) with a local merchant nearby and/or over the Internet; based upon detecting user interaction in the immersion, such as an utterance of "I want some coffee now" or the like) cause one or more offers for products of that particular brand to be provided to the user; cause the one or more offers to be provided to the user via a text message, via e-mail, by adding a product of that particular brand to the user's virtual shopping cart, by placing an order for a product of that particular brand at a nearby store, and/or the like; generate, and cause the target device(s) 204 to present, data or content associated with additional follow-up activities relating to a particular location; automatically schedule a particular appointment for the user (e.g., based on an immersion including media content relating to dentistry and based on the user's calendar data indicating that the user is past due for dental cleaning, etc.), set a reminder for the user to complete a task, prepare and/or submit a social media post to "check-in," and/or the like; modify an immersion (e.g., in real-time or near real-time) based upon detecting user movement or reactions (e.g., gazing, walking, running, slowing down, stopping movement, etc.) during the immersion, user interaction with media content in the immersion (e.g., selections of objects, utterances, etc.), user engagement with one or more other users also experiencing the immersion, etc.; e.g., modify an immersion by adjusting a scene (e.g., graphics) of the immersion as the user moves about, turns around, or the like; by switching playback of certain portions of media content between different target device(s) as the user moves about, turns around, or the like; and/or by removing or adding additional media content to the immersion, such as based upon detecting user movements, gestures, commands, utterances, etc. (which enables in-painting of the immersion, such as by adding a virtual basketball hoop object in response to the user performing movements indicative of a basketball shooting motion and/or the like); modify an immersion ( e.g., in real-time or near real-time) according to a (e.g., predicted) mood or sentiment of the user, which may be determined based upon the user's voice-based inputs, based upon the user's gesture-based inputs, and/or based upon content that the user typically consumes during certain activities and/or during certain times of day, such as during workouts in the morning, during the evening after dinner, and/or the like; alter media content in an immersion to improve the user's mood; generate personalized media content to include trigger(s) that coax, or otherwise entice, a user to perform an action; modify an immersion by controlling a visual appearance and/or movements of one or more characters in an immersion based on the present user's movements (e.g., dance moves), speech, manner of speech, etc.; XR content creators or authors may enable pre-splitting of dialog points for user interaction, e.g., to allow a user to pick up acting within a scene, such as speaking a line intended to be spoken by an original character, but where the user's speaking of the line triggers the scene to begin/continue; ¶¶ [0097]-[0098] with 270c, 270e, and 270h in FIG. 2E: detect user interaction data (e.g., information regarding movements of the user, gesture-based inputs of the user, voice-based inputs of the user, etc.) (270c) relating to the immersion environment, and perform action(s) relating to the personalized media content based on the user interaction data; ¶¶ [0106]-[0107] with 290e and 290f in FIG. 2F: at 290e, detecting user interaction data relating to the immersion environment; at 290f, performing an action relating to the personalized media content based on the detecting the user interaction data).
Claim 3
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claim 1 and further discloses wherein the personal data includes at least one of health data, biometric data, ethnic data, race data, age, sex, gender, income bracket, credit score, personal training history, data associated with indications of knowledge of the customer, personalized multimodal learning data, or user specific required training (Abdel-Wahab, ¶¶ [0014] and [0033]-[0045] with 210 and 215 in FIG. 2A: obtain contextual (or situational) information relating to a user; the contextual information may be related to people, objects, and/or events occurring in proximity to, or associated with, the user; the contextual information may include data regarding a location of the user, data regarding a media content item (e.g., video, audio, etc.) that the user has requested or is presently/currently consuming, calendar/travel-related data associated with the user, data regarding a voice-based input provided by the user, data regarding a gesture-based input provided by the user, data regarding a present time of day, weather data, data regarding structures (e.g., buildings or other objects) at or near the location, and/or the like; determine a likely mood of the user based on a present time of day, weather information, genre of music that the user is currently consuming, the user's voice-based inputs, the user's gesture-based inputs, and/or the like; identify (and/or retrieve) media content that relates to the contextual information and to profile data associated with the user; the profile data can include data relating to user preferences (e.g., historical explicit preferences, including advertisement placement policy restrictions, opt-in or opt-out preferences, or the like), data relating to user behaviors and/or interests (e.g., historical behaviors, such as Internet browsing activities, content consumption (e.g., videos, games, etc.), purchase histories, and/or the like), demographic data associated with the user (e.g., age of the user, gender of the user, etc.), and/or the like; the profile data can additionally, or alternatively, include data relating to prior locations of the user ( e.g., places that the user has visited, performances/shows/conferences that the user has attended, etc.), which may, e.g., be determined based on historical location (e.g., GPS) data, based on Exif (Exchangeable image file) data from photos previously captured by a camera of the user's smartphone, based on historical calendar data, etc.; the profile data can additionally, or alternatively, include data relating to prior conversations, discussions, and/or engagements of the user, data relating to advertisement responses of the user (e.g., advertisement exposures, click-through actions, affinities between users and advertisements and/or advertisement types) and/or other data representative or indicative of user activities, preferences, and/or behaviors (e.g., Interactive Advertising Bureau (IAB)-related data, tag data, genre data, embedding data, and/or the like); the profile data can include XR domain data, such as data relating to user behavior in immersion environments (e.g., user activities or interactions associated with objects in immersion environments, including objects that are native to the immersion environment and/or advertising objects included, or embedded, in the immersion environment); the profile data may include social profile information associated with the user (e.g., the user's social media/networking profile); the social profile information may include data regarding actions, preferences, activities, and/or the like relating to the user's friends, family, or other connections, such as other users that the user may be following, etc.; some data/information described as being contextual information may instead be characterized as profile data, and vice versa; the media content may include audio content (e.g., stereo audio, surround sound, binaural audio, 3D audio, etc.), such as a song, a tune, a speech, a soundscape, and/or the like; the media content may include image/video/object content (e.g., video clips, graphics, computer-generated objects, etc., which may, e.g., include 2D/3D VR, AR, MR objects or the like); the media content may include, or relate to, a scene from a film or television show, a recording of an event (e.g., a concert, a sports competition, etc.), people/characters (e.g., represented by virtual objects, such as avatars), a product to be marketed or advertised, etc.; ¶ [0094] with 270a and 270 b in FIG. 2E: obtain contextual information (270b) relating to a user (e.g., location data, data regarding a media content item that the user has requested or is currently consuming, calendar data, travel-related data, voice-based input data associated with the user, gesture-based input data associated with the user, time of day information, weather information, etc.), and identify media content (e.g., audio content, image content, video content, XR objects, etc.) (270d) that relates to the contextual information and to profile data (270a) associated with the user (e.g., information regarding preferences of the user, interests of the user, a browsing history of the user, a media consumption history of the user, a purchase history of the user, an advertising response history of the user, historical immersion-related behavior of the user, etc.)).
Claim 4
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claim 1 and further discloses wherein the personal data includes at least one of social media posts, voice recordings, photographs, images, or videos (Abdel-Wahab, ¶¶ [0014] and [0033]-[0045] with 210 and 215 in FIG. 2A: recording the personalized media content and/or the user interaction data (e.g., in a media content library associated with the user, in a general archive, and/or the like) for future; obtain contextual (or situational) information relating to a user; the contextual information may be related to people, objects, and/or events occurring in proximity to, or associated with, the user; the contextual information may include data regarding a location of the user, data regarding a media content item (e.g., video, audio, etc.) that the user has requested or is presently/currently consuming, calendar/travel-related data associated with the user, data regarding a voice-based input provided by the user, data regarding a gesture-based input provided by the user, data regarding a present time of day, weather data, data regarding structures (e.g., buildings or other objects) at or near the location, and/or the like; determine a likely mood of the user based on a present time of day, weather information, genre of music that the user is currently consuming, the user's voice-based inputs, the user's gesture-based inputs, and/or the like; identify (and/or retrieve) media content that relates to the contextual information and to profile data associated with the user; the profile data can include data relating to user preferences (e.g., historical explicit preferences, including advertisement placement policy restrictions, opt-in or opt-out preferences, or the like), data relating to user behaviors and/or interests (e.g., historical behaviors, such as Internet browsing activities, content consumption (e.g., videos, games, etc.), purchase histories, and/or the like), demographic data associated with the user (e.g., age of the user, gender of the user, etc.), and/or the like; the profile data can additionally, or alternatively, include data relating to prior locations of the user ( e.g., places that the user has visited, performances/shows/conferences that the user has attended, etc.), which may, e.g., be determined based on historical location (e.g., GPS) data, based on Exif (Exchangeable image file) data from photos previously captured by a camera of the user's smartphone, based on historical calendar data, etc.; the profile data can additionally, or alternatively, include data relating to prior conversations, discussions, and/or engagements of the user, data relating to advertisement responses of the user (e.g., advertisement exposures, click-through actions, affinities between users and advertisements and/or advertisement types) and/or other data representative or indicative of user activities, preferences, and/or behaviors (e.g., Interactive Advertising Bureau (IAB)-related data, tag data, genre data, embedding data, and/or the like); the profile data can include XR domain data, such as data relating to user behavior in immersion environments (e.g., user activities or interactions associated with objects in immersion environments, including objects that are native to the immersion environment and/or advertising objects included, or embedded, in the immersion environment); the profile data may include social profile information associated with the user (e.g., the user's social media/networking profile); the social profile information may include data regarding actions, preferences, activities, and/or the like relating to the user's friends, family, or other connections, such as other users that the user may be following, etc.; some data/information described as being contextual information may instead be characterized as profile data, and vice versa; the media content may include audio content (e.g., stereo audio, surround sound, binaural audio, 3D audio, etc.), such as a song, a tune, a speech, a soundscape, and/or the like; the media content may include image/video/object content (e.g., video clips, graphics, computer-generated objects, etc., which may, e.g., include 2D/3D VR, AR, MR objects or the like); the media content may include, or relate to, a scene from a film or television show, a recording of an event (e.g., a concert, a sports competition, etc.), people/characters (e.g., represented by virtual objects, such as avatars), a product to be marketed or advertised, etc.; ¶ [0094] with 270a and 270 b in FIG. 2E: obtain contextual information (270b) relating to a user (e.g., location data, data regarding a media content item that the user has requested or is currently consuming, calendar data, travel-related data, voice-based input data associated with the user, gesture-based input data associated with the user, time of day information, weather information, etc.), and identify media content (e.g., audio content, image content, video content, XR objects, etc.) (270d) that relates to the contextual information and to profile data (270a) associated with the user (e.g., information regarding preferences of the user, interests of the user, a browsing history of the user, a media consumption history of the user, a purchase history of the user, an advertising response history of the user, historical immersion-related behavior of the user, etc.)).
Claims 5, 14, and 18
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claims 1, 12, and 17 respectively and further discloses wherein the personal data comprises data regarding past communications of the customer (Abdel-Wahab, ¶¶ [0014] and [0033]-[0045] with 210 and 215 in FIG. 2A: obtain contextual (or situational) information relating to a user; the contextual information may be related to people, objects, and/or events occurring in proximity to, or associated with, the user; the contextual information may include data regarding a location of the user, data regarding a media content item (e.g., video, audio, etc.) that the user has requested or is presently/currently consuming, calendar/travel-related data associated with the user, data regarding a voice-based input provided by the user, data regarding a gesture-based input provided by the user, data regarding a present time of day, weather data, data regarding structures (e.g., buildings or other objects) at or near the location, and/or the like; determine a likely mood of the user based on a present time of day, weather information, genre of music that the user is currently consuming, the user's voice-based inputs, the user's gesture-based inputs, and/or the like; identify (and/or retrieve) media content that relates to the contextual information and to profile data associated with the user; the profile data can include data relating to user preferences (e.g., historical explicit preferences, including advertisement placement policy restrictions, opt-in or opt-out preferences, or the like), data relating to user behaviors and/or interests (e.g., historical behaviors, such as Internet browsing activities, content consumption (e.g., videos, games, etc.), purchase histories, and/or the like), demographic data associated with the user (e.g., age of the user, gender of the user, etc.), and/or the like; the profile data can additionally, or alternatively, include data relating to prior locations of the user ( e.g., places that the user has visited, performances/shows/conferences that the user has attended, etc.), which may, e.g., be determined based on historical location (e.g., GPS) data, based on Exif (Exchangeable image file) data from photos previously captured by a camera of the user's smartphone, based on historical calendar data, etc.; the profile data can additionally, or alternatively, include data relating to prior conversations, discussions, and/or engagements of the user, data relating to advertisement responses of the user (e.g., advertisement exposures, click-through actions, affinities between users and advertisements and/or advertisement types) and/or other data representative or indicative of user activities, preferences, and/or behaviors (e.g., Interactive Advertising Bureau (IAB)-related data, tag data, genre data, embedding data, and/or the like); the profile data can include XR domain data, such as data relating to user behavior in immersion environments (e.g., user activities or interactions associated with objects in immersion environments, including objects that are native to the immersion environment and/or advertising objects included, or embedded, in the immersion environment); the profile data may include social profile information associated with the user (e.g., the user's social media/networking profile); the social profile information may include data regarding actions, preferences, activities, and/or the like relating to the user's friends, family, or other connections, such as other users that the user may be following, etc.; some data/information described as being contextual information may instead be characterized as profile data, and vice versa; the media content may include audio content (e.g., stereo audio, surround sound, binaural audio, 3D audio, etc.), such as a song, a tune, a speech, a soundscape, and/or the like; the media content may include image/video/object content (e.g., video clips, graphics, computer-generated objects, etc., which may, e.g., include 2D/3D VR, AR, MR objects or the like); the media content may include, or relate to, a scene from a film or television show, a recording of an event (e.g., a concert, a sports competition, etc.), people/characters (e.g., represented by virtual objects, such as avatars), a product to be marketed or advertised, etc.; ¶ [0094] with 270a and 270 b in FIG. 2E: obtain contextual information (270b) relating to a user (e.g., location data, data regarding a media content item that the user has requested or is currently consuming, calendar data, travel-related data, voice-based input data associated with the user, gesture-based input data associated with the user, time of day information, weather information, etc.), and identify media content (e.g., audio content, image content, video content, XR objects, etc.) (270d) that relates to the contextual information and to profile data (270a) associated with the user (e.g., information regarding preferences of the user, interests of the user, a browsing history of the user, a media consumption history of the user, a purchase history of the user, an advertising response history of the user, historical immersion-related behavior of the user, etc.); ¶ [0155]: information regarding use of services can be generated including services being accessed, media consumption history, user preferences, and so forth; this information can be obtained by various methods including user input, detecting types of communications (e.g., video content vs. audio content), analysis of content streams, sampling, and so forth; the generating, obtaining and/or monitoring of this information can be responsive to an authorization provided by the user; an analysis of data can be subject to authorization from user(s) associated with the data, such as an opt-in, an opt-out, acknowledgement requirements, notifications, selective authorization based on types of data, and so forth).
Claim 6
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claim 1 and further discloses wherein the personalized virtual content comprises a virtual object in the virtual environment, a video, an image, synthetically produced audio, or a voice recording (Abdel-Wahab, ¶ [0014]: the personalized media content may include audio content, image content, video content, XR object(s), etc.; e.g., the personalized media content may include, or relate to, a scene from a film or television show (e.g., a reenactment thereof), a song, a video clip, a person or character (e.g., represented by a three-dimensional (3D) representation, such as an avatar), a brand, a branded product, and/or the like; ¶ [0045]: the media content may include audio content (e.g., stereo audio, surround sound, binaural audio, 3D audio, etc.), such as a song, a tune, a speech, a soundscape, and/or the like; the media content may include image/video/object content ( e.g., video clips, graphics, computer-generated objects, etc., which may, for example, include 2D/3D VR, AR, MR objects or the like); the media content may include, or relate to, a scene from a film or television show, a recording of an event (e.g., a concert, a sports competition, etc.), people/characters (e.g., represented by virtual objects, such as avatars), a product to be marketed or advertised, etc.; ¶ [0055]-[0072] with 220 in FIG. 2A: generate personalized media content by defining, adapting, or adjusting one or more aspects or parameters of identified media content or objects within the identified media content; define or adjust one or more audio characteristics of one or more portions of a video or audio clip (e.g., amplitude, frequency, etc.) and/or replace audio (e.g., by substituting a particular song originally presented during a Thanksgiving parade with a different song that the user's profile data indicates that the user has an affinity for); define or adjust one or more visual characteristics of one or more frames of a video clip ( e.g., brightness, contrast, color, etc.); define or adjust an appearance (e.g., tone, nature, etc.) of, or one or more visual characteristics of, an immersion environment or objects (e.g., AR objects or the like) to be included in the immersion environment; modify an appearance of a character in a film or television show scene, such as by substituting a shirt worn by the character for a shirt of a brand that the user recently searched for on the Internet and/or clicked through in an advertisement, substituting a shoe worn by the character with a shoe that the user previously observed at a store and uttered "this shoe looks nice," and/or the like; define or adjust content based on demographic data associated with the user; identify and/or obtain media content associated with activities of other user(s) who have previously engaged in an immersion at the particular location ( e.g., music, a soundscape, a video, or the like that other user(s) have experienced at the particular location (and, for example, were determined to have reacted positively to); dances that other user(s) have performed while experiencing an immersion at the particular location; etc.), and may include such media content in the immersion for the user to consume/experience; customize an immersion environment with background audio to provide a more entertaining immersive experience for the user; include, in an immersion, a particular song that the user tends to repeat on the user's playlist, a piece of music that aligns with a determined present mood of the user, or the like; generate, provide, or otherwise facilitate, gameplay relating to media content presented in an immersion environment, such as a trivia game relating to a song or a video being played back, etc., which can further enrich a user's immersive experience; define or adjust how media content (or an overall immersion) is to be rendered based on environmental factors or obstacles (e.g., local weather, local time of day, lighting, presence of potential obstacles proximate to the user for occlusion purposes, etc., as may be identified in available data, such as from local information sources, image/video data captured and provided by nearby cameras (e.g., IoT cameras, etc.), and/or the like) so as to provide a more realistic and enjoyable immersive experience for the user).
Claim 7
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claim 1 and further discloses identifying, by the processor, one or more characteristics of a synthetic agent, and wherein the personalized virtual content is further generated based upon at least one of the one or more characteristics of the synthetic agent, and wherein the personalized virtual content is provided to the customer by the synthetic agent (Abdel-Wahab, ¶¶ [0051]-[0052], [0058], [0061]-[0062], [0081], [0089]-[0090], and [0097] with FIG. 2A and 2D: in a case where the contextual information (e.g., location data) indicates that the user is located at, or is approaching or located proximate to (e.g., within a threshold distance from), a shopping mall, and where the profile data (e.g., social profile information) indicates that the user prefers a particular brand of products (e.g., has "liked" the particular brand) or that a particular person that the user is following on social media prefers (or has "liked") the particular brand, identify and/or generate navigation information (e.g., in the form of directional arrows and/or guided paths) to present to the user to guide the user to a store in the shopping mall that sells the particular brand of products; generate the navigation information to include a 3D rendering of a character (e.g., for which the user may have a determined affinity, such as Mickey Mouse, Spiderman, etc.) for guiding the user to the store; generates a 3D representation of a character for guiding a user to a particular location, enable triggering of, or user interaction with, the 3D representation, e.g., speaking in a direction toward the 3D representation may cause the 3D representation to respond (e.g., speak back, turn its head, etc.); touching of the 3D representation may cause the 3D representation to traverse several steps towards the particular location; and so on; generates a 3D representation of a character for guiding a user to a particular location, the automated response mechanism may, based upon detecting user movement toward the 3D representation, entice the user (e.g., visually, audibly, etc.) to move closer to the 3D representation to cause the 3D representation to perform an action (e.g., to jump, to dance, to proceed several steps in a direction towards the particular location, etc.); provide a particular song to be played back in an immersion, which may include an icon/object corresponding to the particular song and/or an instruction to a user to tap the object/icon for details regarding the particular song; in a case where the user performed a dance during an immersion, the immersive content platform 202 may store data relating to the user's dance moves in the user's library (or archive) of personal content or in a library of content associated with the present location of the user; this enables replaying of the user interaction/activity at a later time and/or sharing of the immersion and/or user interaction data with other users, and may allow other user to view/experience the user's immersion interactions at the location; in a case where a first user's immersion-related dancing is recorded and later retrieved and replayed in a different immersion for a second user, the immersive content platform 202 may generate an avatar that has a visual appearance (likeness) of the first user and dance moves corresponding to the recorded dancing; determine that it is a suitable time for the user to consume a meal (e.g., noontime or the like), identify a marketing opportunity involving a restaurant within walking distance from the user's current location, and identify that the user has an affinity for a particular person (e.g., a celebrity, such as Frank Sinatra), the immersive content platform 202 may identify and/or define media content that includes an interactive 3D avatar or representation of the celebrity configured to guide the user to the restaurant; XR content creators or authors may enable puppeteering of character(s) in media content, e.g., by enabling a character to be replaced (or substituted) with a 3D avatar having a present user's likeness and/or by enabling a character to mimic or otherwise exhibit actions of the present user; modify an immersion by controlling a visual appearance and/or movements of one or more characters in an immersion based on the present user's movements (e.g., dance moves), speech, manner of speech, etc.; present to the user, based on determining, from contextual information (e.g., an itinerary or the like relating to a current context and/or a future context), that the user is at the airport and enroute to a vacation destination (e.g., FIG. 2D) and based on determining, from the user's profile data, the user's interests, advertising preferences, etc.; in a case where the user approaches a travel display at an airport (e.g., FIG. 2D), the rendering/re-creation engine 270e may provide, or otherwise (e.g., fully) encompass, the user with vacation-related content, such as beach visuals, sounds, a wayward sand crab near the user's feet, etc.; e.g., Virtual objects (for example, AR objects. holograms, etc.) representing a guide, local setting relating to vacation destination. etc.)).
Claims 8, 15, and 19
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claims 1, 12, and 17 respectively and further discloses training, by the processor, the machine learning model to generate user-specific content based upon the set of keywords representative of the customer characteristics (Abdel-Wahab, ¶¶ [0092]-[0093] with FIG. 2A: the immersive content platform 202 can employ machine learning algorithm(s) that are configured to learn a user's behavior or preferences relating to interactive, personalized media content experiences (including, e.g., immersive experiences); this can include, e.g., the user's reactions (e.g., selections, movements, utterances, etc.) relating to presented media content, the user's preferences for objects (e.g., AR objects, etc.), storylines, etc. in media content, and/or the like; adjust, based on the learned information, future actions performed by, or outputs provided by, the immersive content platform 202 to enhance the user's immersive experiences; provide information regarding a user's preferences or behavior as input to one or more machine learning algorithms, which may perform machine learning to automate future determinations or predictions of user preferences or behavior; e.g., train a machine learning algorithm based on known inputs (e.g., identified, generated, and/or provided media content) and known outputs (e.g., the user engaging, such as moving toward, reaching out for, touching, etc., an AR object in an immersion, but not engaging other media content in the immersion; the user uttering positive phrases (e.g., "wow," "I like that," etc.) in relation to some media content in an immersion, but uttering negative phrases (e.g., "I don't like that," etc.) in relation to other media content; etc.); refine a machine learning algorithm based on feedback received from a user of the immersive content platform 202 and/or from one or more other devices (e.g., management device(s)); e.g., provide feedback indicating whether predictions of user preferences or behavior, made by the machine learning algorithm based on new inputs, are accurate and/or helpful; predict user preferences or behavior based on one or more machine learning algorithms, which improves the accuracy of the predictions, and conserves processor and/or storage; resources that may otherwise be used to generate and store rules for predicting user preferences or behavior; ¶¶ [0014] and [0055]-[0073] with 220 in FIG. 2A: derive, from the media content, personalized media content based on the profile data; generate personalized media content by defining, adapting, or adjusting one or more aspects or parameters of identified media content or objects within the identified media content; define or adjust an appearance (e.g., tone, nature, etc.) of, or one or more visual characteristics of, an immersion environment or objects (e.g., AR objects or the like) to be included in the immersion environment; modify an appearance of a character in a film or television show scene, such as by substituting a shirt worn by the character for a shirt of a brand that the user recently searched for on the Internet and/or clicked through in an advertisement, substituting a shoe worn by the character with a shoe that the user previously observed at a store and uttered "this shoe looks nice," and/or the like; define or adjust content based on demographic data associated with the user; customize an immersion environment with background audio to provide a more entertaining immersive experience for the user; adjust a quality of the content, or otherwise adapt the content to a less computationally-intensive version of the content, to suit the network conditions and/or the capabilities of the available target device(s) 204; identify different media content and/or generate different personalized media content for different available target device(s) 204 to render or present, based on the capabilities of the different target device(s) 204, orientations of the different target device(s) 204, etc. so as to provide a more realistic and enjoyable immersive experience for the user; define or adjust how media content (or an overall immersion) is to be rendered based on environmental factors or obstacles (e.g., local weather, local time of day, lighting, presence of potential obstacles proximate to the user for occlusion purposes, etc., as may be identified in available data, such as from local information sources, image/video data captured and provided by nearby cameras (e.g., IoT cameras, etc.), and/or the like) so as to provide a more realistic and enjoyable immersive experience for the user; rank or score available media content, and select media content to present to the user based on the rankings or scores, which can aid the immersive content platform 202 in determining which media content to present to the user, particularly in a case where the immersive content platform 202 identifies numerous relevant media content that matches the contextual information and/or the user's profile data; rank media content based on a determined level of user affinity relating to a given media content; rank media content based on recency of user affinity relating to a given media content; rank media content based on advertising or marketing opportunities; ¶ [0094] with 270c in FIG. 2E: derive, from the media content, personalized media content based on the profile data, and cause one or more target device(s) (e.g., a smartphone, a smartwatch, an XR-based user equipment, and/or one or more proximal devices, such as proximal displays, proximal speakers, etc.) (270c) to provide an immersion environment that includes the personalized media content; ¶ [0097] with 270e in FIG. 2F: interleave (e.g., using the rendering/re-creation engine 270e) real-time media content or assets (e.g., adapted based on user movements, adapted based on predicted user activities, and/or the like) with original or previously-rendered versions of the media content (including, e.g., by linking to such prior content or assets as needed) to provide a smooth or seamless immersive experience for the user, while, for example, maintaining a prior narrative in the media content; e.g., for interleaved rendering, a previously-created version of an immersion for a basketball team may have been created (e.g., a historical rendering) and may be served to the user, but the immersion may be rendered by the rendering/re-creation engine 270e with real-time, or near real-time, updates (e.g., featuring highlights or occasional updates from a game that is currently occurring) and/or timely, user-specific content (e.g., a visual of the user in a case where the user was previously present at a stadium and posed for a "fan-cam" or the like); ¶ [0104] with 290c in FIG. 2F: at 290c, deriving, from the media content, personalized media content based on the profile data associated with the user; ¶¶ [00156]-[0157]: employ artificial intelligence (AI) to facilitate automating one or more features; classification can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to determine or infer an action that a user desires to be automatically performed; employ classifiers that are explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via observing UE behavior, operator preferences, historical information, receiving extrinsic information)).
Claim 9
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claim 1 and further discloses wherein the machine learning model is further trained to generate the personalized virtual content based upon an additional set of additional keywords directed to content messaging (Abdel-Wahab, ¶¶ [0048]-[0054] with FIGS. 2B-2D: in a case where the contextual information (e.g., location data) indicates that the user is located at, or is approaching or located proximate to (e.g., within a threshold distance from), a particular landmark (e.g., the Empire State Building), and where the profile data indicates that the user has an affinity for a particular character ( e.g., a fictional character, such as Superman), identify and/or retrieve media content that involves the particular landmark and the particular character ( e.g., a reenactment of a scene from a film in which Superman flew by the Empire State Building); in a case where the contextual information (e.g., location data, data regarding a present time of day, etc.) indicates that it is a suitable time for the user to consume a meal (e.g., noontime, etc.) and that the user is lingering or located at, or is approaching or located proximate to (e.g., within a threshold distance from), a particular restaurant (e.g., in Little Italy, New York City), and where the profile data indicates that the user has an affinity for a particular person (e.g., a celebrity, such as Frank Sinatra) associated with the particular restaurant, identify and/or retrieve media content that involves the particular restaurant and the particular person (e.g., a 3D representation of Frank Sinatra at or near the restaurant); in a case where the contextual information includes a voice-based input from the user that identifies a person, a place, a thing, an event, etc., such as an utterance of a name of a location or establishment (e.g., "Rudy's," "Max's Kansas City," "Copacabana," etc.), a name or title of an historical event ( e.g., "Woodstock," etc.), or the like; ¶¶ [0061]-[0062] with FIG. 2A: implement or enable user interactivity in an immersion by generating, or otherwise including, triggers that, upon being activated or engaged by a user, may cause the immersive content platform 202 to perform adjustments to the media content and/or to obtain and provide other (e.g., related) media content in an immersion; generate a 3D representation of a character for guiding a user to a particular location, the immersive content platform 202 may enable triggering of, or user interaction with, the 3D representation, e.g., speaking in a direction toward the 3D representation may cause the 3D representation to respond (e.g., speak back, turn its head, etc.); touching of the 3D representation may cause the 3D representation to traverse several steps towards the particular location; and so on; provide an interactive, automated response mechanism (e.g., a chat-bot or the like) configured to respond to detected user inputs (e.g., speech, movements, etc.) and/or guide a user through one or more triggers that are available in an immersion; ¶ [0082]: based upon detecting user interaction in the immersion, such as an utterance of "I want some coffee now" or the like) cause one or more offers for products of that particular brand to be provided to the user; cause the one or more offers to be provided to the user via a text message, via e-mail, by adding a product of that particular brand to the user's virtual shopping cart, by placing an order for a product of that particular brand at a nearby store, and/or the like; ¶¶ [0092]-[0093] with FIG. 2A: the immersive content platform 202 can employ machine learning algorithm(s) that are configured to learn a user's behavior or preferences relating to interactive, personalized media content experiences (including, e.g., immersive experiences); this can include, e.g., the user's reactions (e.g., selections, movements, utterances, etc.) relating to presented media content, the user's preferences for objects (e.g., AR objects, etc.), storylines, etc. in media content, and/or the like; adjust, based on the learned information, future actions performed by, or outputs provided by, the immersive content platform 202 to enhance the user's immersive experiences; provide information regarding a user's preferences or behavior as input to one or more machine learning algorithms, which may perform machine learning to automate future determinations or predictions of user preferences or behavior; e.g., train a machine learning algorithm based on known inputs (e.g., identified, generated, and/or provided media content) and known outputs (e.g., the user engaging, such as moving toward, reaching out for, touching, etc., an AR object in an immersion, but not engaging other media content in the immersion; the user uttering positive phrases (e.g., "wow," "I like that," etc.) in relation to some media content in an immersion, but uttering negative phrases (e.g., "I don't like that," etc.) in relation to other media content; etc.); refine a machine learning algorithm based on feedback received from a user of the immersive content platform 202 and/or from one or more other devices (e.g., management device(s)); e.g., provide feedback indicating whether predictions of user preferences or behavior, made by the machine learning algorithm based on new inputs, are accurate and/or helpful; predict user preferences or behavior based on one or more machine learning algorithms, which improves the accuracy of the predictions, and conserves processor and/or storage; resources that may otherwise be used to generate and store rules for predicting user preferences or behavior).
Claims 10-11, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Abdel-Wahab in view of Zheng as applied to Claims 1, 12, and 17 respectively above, and further in view of ALLEN et al. (US 2022/0207830 A1, pub. date: 06/30/2022; filed on 12/31/2020), hereinafter ALLEN.
Claims 10, 16, and 20
Abdel-Wahab in view of Zheng discloses all the elements as stated in Claims 1, 12, and 17 respectively and further discloses receiving, by a user interface, an indication of a desired service module; generating, by the processor, the virtual environment based upon the indicated desired service module; determining, by the processor, service content from the desired training module and the personal data; generating, by the processor, one or more virtual objects associated with the determined service content; and providing, via an additional virtual headset, the (i) virtual environment, (ii) one or more virtual objects, and (iii) service content to a user of the additional virtual headset (Abdel-Wahab, ¶ [0015]: media content or assets (in an immersion or otherwise) based on user profile data or the like, broadens marketing channels, avoids a need to explicitly query users for their content preferences, and personalizes immersive experiences---e.g., enables user discovery of location context and consumption of media content (e.g., for entertainment purposes, for educational (i.e., training/learning) purposes, for socializing purposes, etc.; ¶ [0026]: the content sources 175 include broadcast television and radio sources, video on demand platforms and streaming video and audio services platforms, one or more content data networks, data servers, web servers and other content servers, and/or other sources of media; ¶ [0030] with FIG. 2A: a target device 204 can include a wearable communication device ( e.g., a smart wristwatch, a pair of smart eyeglasses, XR gear ( e.g., a pair of AR, VR, MR glasses, a headset, headphones, and/or the like), etc.); ¶¶ [0033]-[0054] with FIG. 2A: obtain contextual (or situational) information relating to a user; data regarding a media content item that the user has requested or is presently consuming may include a title of the media content item, a description of the media content item, information regarding a source of the media content item, and/or information regarding a determined fingerprint of audio/video in the media content item; identify (and/or retrieve) media content that relates to the contextual information and to profile data associated with the user; identify and/or obtain media content from any suitable source, such as publicly available database(s), private database(s), the user's personal library or collection, and/or the like; the media content may include content that the user may or may not be previously aware of; identify and/or retrieve media content that relates to the contextual information and that is consistent with the profile data (e.g., based on a determined linkage between media content and indication(s) in the contextual information and indication(s) in the profile data); analyze the contextual information and the profile data, and perform a comparison/match thereof with external data relating to media content (e.g., metadata, notes, archives, or other information) to identify and/or retrieve relevant media content; identifying and/or retrieving entertainment, informational, educational, demonstrational (e.g., demonstrate/teach a user for how to use a product and its functionality), and/or promotional content, such as cultural music, product brochures, ad placements, etc. (e.g., FIG. 2D) to present to the user, based on determining, from contextual information (e.g., an itinerary or the like relating to a current context and/or a future context), that the user is at the airport and enroute to a vacation destination and based on determining, from the user's profile data, the user's interests, advertising preferences, etc.; ¶¶ [0055]-[0060] and [0064]-[0073] with FIG. 2A: define or adjust one or more audio characteristics of one or more portions of a video or audio clip (e.g., amplitude, frequency, etc.) and/or replace audio ( e.g., by substituting a particular song originally presented during a Thanksgiving parade with a different song that the user's profile data indicates that the user has an affinity for); define or adjust one or more visual characteristics of one or more frames of a video clip (e.g., brightness, contrast, color, etc.); define or adjust an appearance ( e.g., tone, nature, etc.) of, or one or more visual characteristics of, an immersion environment or objects (e.g., AR objects or the like) to be included in the immersion environment; identify and/or generates navigation information, e.g., to present to the user to guide the user to a store in the shopping mall that sells the user's preferred brand of products, generate the navigation information to include a 3D rendering of a character (e.g., for which the user may have a determined affinity, such as Mickey Mouse, Spiderman, etc.) for guiding the user to the store; define or adjust content based on demographic data associated with the user; identify and/or obtain media content associated with activities of other user(s) who have previously engaged in an immersion at the particular location (e.g., music, a soundscape, a video, or the like that other user(s) have experienced at the particular location (and, for example, were determined to have reacted positively to); dances that other user(s) have performed while experiencing an immersion at the particular location; etc.), and may include such media content in the immersion for the user to consume/experience; customize an immersion environment with background audio to provide a more entertaining immersive experience for the user; generate, provide, or otherwise facilitate, gameplay relating to media content presented in an immersion environment, such as a trivia game relating to a song or a video being played back, etc., which can further enrich a user's immersive experience; define or adjust how media content (or an overall immersion) is to be rendered based on environmental factors or obstacles (e.g., local weather, local time of day, lighting, presence of potential obstacles proximate to the user for occlusion purposes, etc., as may be identified in available data, such as from local information sources, image/video data captured and provided by nearby cameras (e.g., IoT cameras, etc.), and/or the like) so as to provide a more realistic and enjoyable immersive experience for the user; ¶¶ [0061]-[0063] with FIG. 2A: implement or enable user interactivity in an immersion by generating, or otherwise including, triggers that, upon being activated or engaged by a user, may cause the immersive content platform 202 to perform adjustments to the media content and/or to obtain and provide other (e.g., related) media content in an immersion; e.g., generate a 3D representation of a character for guiding a user to a particular location; enable triggering of, or user interaction with, the 3D representation---e.g., speaking in a direction toward the 3D representation may cause the 3D representation to respond (e.g., speak back, turn its head, etc.); touching of the 3D representation may cause the 3D representation to traverse several steps towards the particular location; and so on; provide an interactive, automated response mechanism (e.g., a chat-bot or the like) configured to respond to detected user inputs ( e.g., speech, movements, etc.) and/or guide a user through one or more triggers that are available in an immersion; e.g., generates a 3D representation of a character for guiding a user to a particular location, the automated response mechanism may, based upon detecting user movement toward the 3D representation, entice the user (e.g., visually, audibly, etc.) to move closer to the 3D representation to cause the 3D representation to perform an action ( e.g., to jump, to dance, to proceed several steps in a direction towards the particular location, etc.); a content creator may define triggers in media content that, upon selection or activation in an immersion, links to new media content and/or immersive experiences (e.g., similar to hidden or cut scenes of a film); ¶ [0077]-[0091]: user interactions can include gestures (e.g., hand- or finger-based selections or movements), utterances or other voice-based commands, etc. relative to the rendered content; obtain user interaction data associated with, e.g., the user touching the particular branded shirt worn by the 3D avatar of the person/character in the parade, the user uttering that the user likes the song being played back during the parade, and/or the like; initiate a (e.g., real world) transaction, such as a purchase request or the like; enable a user to bookmark, add to a playlist, or otherwise record, an immersion and/or user interaction data therein for future playback; in a case where the user utters that the user likes a song that is being played back during an immersion or where the user manually selects a rendered object corresponding to the song (such as a 3D music symbol icon/object presented in the immersion and corresponding to the song), the immersive content platform 202 may cause the song to be added to the user's personal playlist; initiate a ( e.g., real world) transaction, such as a purchase request or the like, based on user interaction data, which can enhance the user's overall immersive experience and/or broaden marketing channels; in a case where the user touches a particular item (e.g., a branded shirt, a meal, etc.) in an immersion, cause the item (e.g., shirt) to be added to the user's virtual shopping cart and/or facilitate a purchase or order of the item (e.g., shirt, meal, etc.) with a local merchant nearby and/or over the Internet; in a case of determining that the user has a preference for a particular brand (e.g., a particular coffee brand), identify, generate, and/or provide media content involving characters consuming a product of that particular brand, and may (e.g., either without detecting any user interaction data in the immersion or, alternatively, based upon detecting user interaction in the immersion, such as an utterance of "I want some coffee now" or the like) cause one or more offers for products of that particular brand to be provided to the user; cause the one or more offers to be provided to the user via a text message, via e-mail, by adding a product of that particular brand to the user's virtual shopping cart, by placing an order for a product of that particular brand at a nearby store, and/or the like; in a case where a user has experienced an immersion at a particular location, and historical data of other users who have visited the particular location (e.g., archives of other users' immersion-related interaction data associated with the particular location, other users' social media posts, and/or the like) indicates that the other users have typically subsequently visited a proximal location (e.g., an ice cream shop, a restaurant, etc.) after experiencing an immersion at the particular location, the immersive content platform 202 may generate, and cause one or more of the target device(s) 204 to present, a recommendation or navigation information to guide the user to such a "next" proximal location; facilitate follow-up activities and/or interactions in the physical world, such as via IoT systems, point-of-sale systems, or the like; in a case where the above-described "next" proximal location is a service-based establishment, such as an ice cream shop, a restaurant, etc., the immersive content platform 202 may, based upon user confirmation, time of data, calendar data of the user, user preferences in the user's profile data, etc., place a reservation for the user at the "next" proximal location; automatically schedule a particular appointment for the user (e.g., based on an immersion including media content relating to dentistry and based on the user's calendar data indicating that the user is past due for dental cleaning, etc.), set a reminder for the user to complete a task, prepare and/or submit a social media post to "check-in," and/or the like; modify an immersion (e.g., in real-time or near real-time) based upon detecting user movement or reactions (e.g., gazing, walking, running, slowing down, stopping movement, etc.) during the immersion, user interaction with media content in the immersion (e.g., selections of objects, utterances, etc.), user engagement with one or more other users also experiencing the immersion, etc.; modify an immersion by adjusting a scene (e.g., graphics) of the immersion as the user moves about, turns around, or the like; by switching playback of certain portions of media content between different target device(s) as the user moves about, turns around, or the like; and/or by removing or adding additional media content to the immersion, such as based upon detecting user movements, gestures, commands, utterances, etc. (which enables in-painting of the immersion, such as by adding a virtual basketball hoop object in response to the user performing movements indicative of a basketball shooting motion and/or the like); upon detecting a low level of user engagement (e.g., inattentiveness and/or the like) in an immersion, determine that lower quality media content (e.g., audio-only content or low resolution video rather than 4K video) can be presented to the user, and may adjust presentation of the immersion accordingly; modify an immersion ( e.g., in real-time or near real-time) according to a (e.g., predicted) mood or sentiment of the user, which may be determined based upon the user's voice-based inputs, based upon the user's gesture-based inputs, and/or based upon content that the user typically consumes during certain activities and/or during certain times of day, such as during workouts in the morning, during the evening after dinner, and/or the like; alter media content in an immersion to improve the user's mood; generate personalized media content to include trigger(s) that coax, or otherwise entice, a user to perform an action; determine that it is a suitable time for the user to consume a meal (e.g., noontime or the like), identify a marketing opportunity involving a restaurant within walking distance from the user's current location, and identify that the user has an affinity for a particular person (e.g., a celebrity, such as Frank Sinatra), the immersive content platform 202 may identify and/or define media content that includes an interactive 3D avatar or representation of the celebrity configured to guide the user to the restaurant; modify an immersion by controlling a visual appearance and/or movements of one or more characters in an immersion based on the present user's movements (e.g., dance moves), speech, manner of speech, etc.; XR content creators or authors may enable pre-splitting of dialog points for user interaction, e.g., to allow a user to pick up acting within a scene, such as speaking a line intended to be spoken by an original character, but where the user's speaking of the line triggers the scene to begin/continue; ¶¶ [0097]-[0098] with FIG. 2E: interleave (e.g., using the rendering/re-creation engine 270e) real-time media content or assets (e.g., adapted based on user movements, adapted based on predicted user activities, and/or the like) with original or previously-rendered versions of the media content (including, e.g., by linking to such prior content or assets as needed) to provide a smooth or seamless immersive experience for the user, while, for example, maintaining a prior narrative in the media content; for interleaved rendering, a previously-created version of an immersion for a basketball team may have been created (e.g., a historical rendering) and may be served to the user, but the immersion may be rendered by the rendering/re-creation engine 270e with real-time, or near real-time, updates (e.g., featuring highlights or occasional updates from a game that is currently occurring) and/or timely, user-specific content (e.g., a visual of the user in a case where the user was previously present at a stadium and posed for a "fan-cam" or the like); orchestrate (e.g., via a follow-up orchestrator 270h) one or more follow-up actions/activities, such as recording the personalized media content and/or the user interaction data (e.g., in a media content library associated with the user, in a general archive, and/or the like) for future playback, sharing the personalized media content and/or the user interaction data with one or more other users associated with the user, associating the personalized media content and/or the user interaction data with the location of the user for future use (e.g., in an immersion associated with a different user, etc.), facilitating a reservation for a service for the user, facilitating a purchase of a product for the user, scheduling a service appointment for the user, facilitating additional media location guidance (e.g., presenting data or content associated with additional follow-up activities at a subsequent location), etc.); Claim 17: the personalized media content comprises an advertising object corresponding to a product to be marketed, wherein the user interaction data is associated with a user selection of the advertising object, and wherein the performing the action comprises facilitating a purchase of the product for the user).
Abdel-Wahab in view of Zheng fails to explicitly disclose wherein receiving an indication of a desired training module; generating a virtual environment based upon the indicated desired training module; determining training content from the desired training module; generating one or more virtual objects associated with the determined training content; and providing the training content to a user.
ALLEN teaches a system and a method relating to virtual reality environments (ALLEN ¶ [0001]), wherein receiving an indication of a desired training module (ALLEN, ¶ [0076]: a user may search for and retrieve a virtual training scenario that includes, within metadata of an object, a specified model number or type of equipment (e.g. personal protective equipment or breathing apparatus); ¶¶ [0071]-[0072] with FIG. 4A: browser application 414 may be used to log in to a remote server and download a virtual reality training scenario for local storage and execution; VR application 416 may comprise a plug-in executed by a browser application 414); generating a virtual environment based upon the indicated desired training module; determining training content from the desired training module; generating one or more virtual objects associated with the determined training content; and providing the training content to a user (ALLEN, ¶¶ [0023]-[0032] with FIGS. 1A-B: virtual reality environments allow for training and certification of users and operators in environments that would be hazardous in reality, such as nuclear power or chemical processing plants, simulated emergencies such as fires or gas leaks, or other such environments; the virtual reality environment 10 may comprise a three-dimensional environment and may be viewed from the perspective of a virtual camera, which may correspond to a viewpoint of a user or operator; the virtual camera may be controlled via tracking of a head-mounted display (e.g. virtual reality goggles or headset) or similar head tracking such that the user's view within the virtual environment corresponds to their physical movements and orientation; the virtual reality environment may comprise one or more objects 20, which may include buttons, levers, wheels, panels, screens, gauges, pipes, ladders, signage, or any other type and form of object; objects 20 may have three-dimensional boundaries in many implementations, and may include textures, shading, or coloring for realism, including photo-realistic textures or images; objects 20 may be interactive or allow a user to interact with the objects to control various aspects of a simulation; a user may select an object (e.g. physically, in implementations where a user's movements are tracked, by reaching for the object; with a user interface device such as a joystick, mouse, tablet, pointer, or other device; verbally, according to a speech-to-command interface; visually, by directing the virtual camera towards the object and pressing a selection button or waiting a predetermined period; or any other such method), and various functions may be executed; provide for a dynamic, reconfigurable virtual reality environment with in-environment access to external data and resources; one of the most important aspects of training, the supplemental materials available to students, will be configurable by the end customer without the need for additional vendor engagement; implementations of these systems also provide an external mechanism for modifying other aspects of the virtual reality experience with no need to recode or compile the experience; this can alter the primary flow of the experience, change its behavior based on the specific user accessing it and add branded or customer-specific aspects to the application; the same level or environment can provide drastically different experiences for various users from beginners through experts, even allowing the option of random or ordered events, controllable by an instructor or administrator, through simple configuration; . objects having associated metadata may be annotated with icons 30 as shown in the example of FIG. 1A, or may be otherwise identified for interaction; changing values in the configuration may be reflected in the experience the next time it is launched and the construct (and associated key/value pairs) are read and interpreted; for each object that is encoded with these parameters, if a value exists ( or a value of a certain type, indicating a help file or other information), an icon 30 may be displayed for the information; this may be limited to specific run modes, such as a training mode or guest mode; responsive to the user selecting the icon, instantiate an in-environment web browser or other interface 40, which may be rendered within the virtual environment to display a view of the corresponding content or resource, in a configurable presentation style, with relevant controls; delivery of dynamic content in virtual reality, with no need to recreate existing content, while providing real time updates of information and access to legacy data, such as documents, audio, video and other file types, which can still be utilized, as-is; allow for updating of URI addresses or endpoint resources through reconfiguration of the external configuration construct, without requiring programming knowledge or the need to recode or recompile an executable application; as a student 160 or other user (or a computing device operated by or on behalf of a student or other user) executes the compiled virtual reality experience, their computing device may identify metadata comprising resource identifiers or GUIDs of objects within the virtual environment; read the linked URI addresses from the configuration construct 158; and retrieve the associated resource for display within an embedded browser or renderer 154 in the virtual environment; to dynamically change the scenario or environment, the configuration construct 158 may be edited without changing the compiled virtual reality runtime package 156, allowing for selection and embedding of different resources, triggering of additional functions, etc.; the external linked resource may be changed or replaced without changing the configuration construct, similarly resulting in embedding of different resources; every object within an environment or scenario may have a unique resource identifier or GUID, but may not necessarily have a linked resource URI in the configuration construct 158; such linked resources may be added after compilation, adding additional functionality or data to the virtual environment without requiring modification of the runtime package or code; ¶¶ [0039]-[0051] with FIG. 2B: a set of templates are created for objects represented in the virtual environment which include a GUID or other identifier, a name for the object, and other metadata that is common to the specific object type; objects may be associated with leaf nodes of a hierarchical tree, with higher layers or levels including a group/user layer; project layer; portal layer; organization layer; and site layer; User/Group: at this level, a configuration construct can be added to provide distinctive modifications specifically for users or groups of users, based on login environment, providing URLs and metadata that are tailored for the individual experience; this metadata guides the behavior of the experience by managing launch parameters based on the loaded data, to direct which links are presented, the training mode that the experience launches in, and other relevant actions; if metadata is not set for a required property, a menu may be presented to allow the user to choose the metadata options for that instance of the application; other metadata can be manipulated at various levels in the path to delivery to alter the virtual experience, enabling different users to execute the same virtual reality application, but interact with a vastly different training event; ¶¶[0054]-[0067] with FIGS. 3A-B: executing the runtime application to allow students to run a virtual training session; upon execution of the virtual reality experience, the configuration construct is read to seed the data for all of the metadata values, allowing the correct endpoints to be identified and displayed during runtime; all information, supplementary and help icons are displayed on objects that have corresponding metadata for those keys; determine whether the object has a locked property; if so, the object may be unavailable for interaction; this may be due to the mode of operation, for example (e.g. in test modes, additional help resources may be unavailable that would be available in study or guided modes), or may be due to a scenario (e.g. where the user is being trained in performing a sequence of operations, some objects or elements may initially be locked and unlocked later in the scenario); the modes (e.g. Study, Guided, and Test, for example) may be specified within the metadata and may have different values for different modes, (e.g. different visibility during different modes), such that values can be shown or ignored based on the modes supported for that metadata property; in a testing mode, help icons may be disabled; however, information icons may still be enabled or visible to provide answers to questions or actions for which incorrect responses have been given; icon visibility within the experience may be guided by a number of environmental and metadata factors, to aid in maintaining a realistic setting; voice commands, controller actions and key combinations are three examples, and/or metadata may specify how information about that object is displayed, such as Always On, Always Off, On Failure, Distance, etc.; if the object is not locked, then the object metadata may be read from the configuration construct and applied to the object; detect an interaction of the user with an object which may comprise pressing a button, pulling lever, rotating a knob or dial, etc., and may be performed in any suitable manner (e.g. by tracking a hand position of the user and determining an intersection between a corresponding hand position of a virtual avatar of the user and the object; by tracking a position of a virtual "laser pointer" or other device; by selection via a mouse, keyboard, joystick, or other interface element; via a verbal command received a speech-to-text or speech-to-command engine (e.g. "press blue button" or "tum dial to 20"); by selection via head tracking (e.g. looking at a particular button and holding the user's head position for several seconds); or any other such method or combination of methods); upon detecting an interaction with an object, a local agent or handler may identify in the metadata for the object a resource path or address and identifier for a resource to display; the handler or local agent may instantiate a browser within the virtual environment).
Abdel-Wahab in view of Zheng, and ALLEN are analogous art because they are from the same field of endeavor, a system and a method relating to virtual reality environments. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of ALLEN to Abdel-Wahab in view of Zheng. Motivation for doing so would provide .
Claim 11
Abdel-Wahab in view of Zheng and ALLEN discloses all the elements as stated in Claim 10 and further discloses wherein the user interface is a virtual user interface provided to the user by the additional virtual headset (Abdel-Wahab, ¶ [0045]: the media content may include, or relate to, a scene from a film or television show, a recording of an event (e.g., a concert, a sports competition, etc.), people/characters (e.g., represented by virtual objects, such as avatars), a product to be marketed or advertised, etc.; ¶ [0051] with FIGS. 2B-2D: identifying and/or retrieving media content relating to a famous dance performance (e.g., FIG. 2B) based on determining, from contextual information, that the user is located at or near (e.g., within a threshold distance from) Central Park, New York City and determining, from the user's profile data, that the user has a preference for romantic comedies and/or a dancing program (e.g., Dancing with the Stars or the like); the immersive content platform 202 identifying and/or retrieving media content relating to a (e.g., historical) Thanksgiving parade (e.g., FIG. 2C) based on determining, from contextual information, that the user is located at or near (e.g., within a threshold distance from) a Macy's store and determining, from the user's profile data, that the user has a preference for a certain musical artist, a certain music genre, etc.; and the immersive content platform 202 identifying and/or retrieving entertainment, informational, educational, demonstrational, and/or promotional content, such as cultural music, product brochures, ad placements, etc. (e.g., FIG. 2D) to present to the user, based on determining, from contextual information (e.g., an itinerary or the like relating to a current context and/or a future context), that the user is at the airport and enroute to a vacation destination and based on determining, from the user's profile data, the user's interests, advertising preferences, etc.; ¶ [0087]: based upon detecting user movements, gestures, commands, utterances, etc. (which enables in-painting of the immersion, such as by adding a virtual basketball hoop object in response to the user performing movements indicative of a basketball shooting motion and/or the like)).
Response to Arguments
Applicant’s arguments filed on 0 regarding 102 & 103 rejections with respect to Claims 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant's arguments filed on 03/11/2026 regarding 101 rejections have been fully considered but they are not persuasive.
Applicant argues on Pages 11-13 of the Remarks that the amended claims further integrate any such abstract idea into a practical application under Prong Two by improving the technical field of generating and presenting personalized virtual content (e.g., ¶¶ [0026] and [00119] of the specification) by combining a first set of keywords derived from the personal data of a user with a second set of predetermined subject-matter keywords for an area of interest, then generating the personalized virtual content from the combined keywords.
In response, examiner respectfully disagrees. In order for a claim reciting a judicial exception and not directed to the judicial exception, "additional elements" (i.e., non-abstract idea elements) must be integrated with other "judicial exception" elements (i.e., abstract idea elements) in a meaningful way (i.e., not just "apply it") so that an improvement of a practical technology described in the specification is reflected in the claim as a whole. Since "combining a first set of keywords derived from the personal data of a user with a second set of predetermined subject-matter keywords for an area of interest" to "generate personalized content" can be performed in the human mind, or by a human using a pen and paper, they are not "additional elements" (i.e., they are "judicial exception" elements) and cannot be used to integrate with other "judicial exception" elements into a practical application. The three "additional elements" in the claim are (1) collecting personal data (i.e., insignificant extra solution activity of data gathering); (2) using (i.e., applying) "a trained machine learning model" to "generate personalized content" in replace of mental process; and (3) providing the generated content to the customer (i.e., insignificant extra solution activity of presenting results). Therefore, considering the claim as a whole, the improvement indicated in ¶ [0026] of the specification (e.g., "the personalized content may be generated automatically to remove the requirement for a user to update or reprogram a training module according to a specific user") is not reflected in the claim.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
LuVogt et al. (US 2013/0290110 A1, pub. date: 10/31/2013) discloses in ABSTRACT that (1) users receive content recommendations from a personalized, generalized recommendation service that aggregates and selects content of high personal relevance to each individual user from a large pool of both personal and public content; (2) the received content is filtered and the content determined to be relevant is cached; (3) when a user request for content is received, the cached content is rescored and the content determined to be most relevant based on satisfaction of a relevance threshold is selected and forwarded to the user; (4) feedback methodologies are also implemented so that a user's actions are taken into consideration in real time and can affect subsequent recommendations to the user. • LuVogt further discloses in ¶¶ [0003]-[0019] that (1) provide for personalized, generalized recommendation systems and methodologies that facilitate determining relevance and recommending relevant content selected from different public and private data; (2) receiving a plurality of content items obtained from content streams of different content types to be forwarded to at least one user; (3) the content streams of different content types are received from multiple content sources; (4) attributes of the received content items are obtained and content items to be forwarded to the user are obtained based on the attributes; (5) the content items determined to be relevant based on the attributes are forwarded to a user model associated with the user; (6) the content items forwarded to the user model are scored and content items determined to be relevant based on the score are added to the lists of unseen content items; (7) at least a first subset of the scored content items are added to respective lists of unseen content items for the user based at least on the type of content in each content item of the subset; (8) when a request is received from the user for relevant unseen content items, the first subset of content items are rescored based on a current context of the user; (9) top scoring content items are selected from the first subset of content items and forwarded to the user; (9) a user selection of a content item from the selected top scoring content items is received that recalculates the relevance of the user selected content item to user models of other users on receiving the user selection; (10) generating a user model in terms of categories of interest to the user and providing recommendations based on such a user model; (11) receiving, a user request for content recommendations along with information regarding the user which can comprise at least a subset of content sources from which the user desires content recommendations; (12) the information regarding the user can also comprise information associated with prior user activity, such as, previous user searches, selections of content or prior user feedback; (13) based on such user information, a plurality of categories are provided for selection to the user; (13) the user selections of categories are received and the respective category vectors of the user selected categories are aggregated; (14) a user model representing the user's interests which is employed in making relevant content item recommendations is generated from the aggregated category vectors in combination with vector representations of the user information; (15) keywords, entities, content item features and other user information can be represented as vectors which are combined with the aggregated category vectors to generate the user model; (16) based on a determination of relevance by the user model, content recommendations are forwarded to the user; (17) the recommended content items are selected from the subset of content sources wherein each content source provides a respective content type different from other sources; (18) the user model is updated based on a user selection of a recommended content item; (19) this can trigger discovery of new content and consequently new content recommendations; (20) such updates to user model in response to user selections and recommendations of new content can occur in real time; (21) hence, in response to a user selection of a particular content item, updated recommendations of content items can be transmitted to the user; (22) such updated recommendations of content items can comprise recommendations for new content items identified as being relevant to the user in accordance with the updated user model; (23) updated recommendations of content items can be based on updates to other user models that are similar to the generated user model; (24) updated recommendations of content items can be based on updates to the category vectors included in the user model; (25) the user model can be updated on a periodic basis on a daily basis; (26) user vectors associated with the user model over the preceding 'N' days are obtained, N being a natural number, e.g., thirty; (27) the user vectors of the preceding thirty days are obtained and are weighed by a function of the number of days back and aggregated to generate an aggregated user vector; (28) current category vectors of the respective categories are combined with the aggregated user vector to generate an updated user vector for the day; (29) the user model parameters such as relevance threshold are adjusted based on user behavior and/or system response; (30) the number of times that a user requests new content is recorded; (40) if the user requests new content frequently, a relevance threshold associated with determination of relevance for providing the recommendations is lowered, such that more content items can be recommended; (41) conversely, if the user requests new content less frequently, the relevance threshold is increased such that fewer relevant content items can be recommended; (42) types of content requested by the user can be stored and the user model can be updated such that a greater number of category vectors are aggregated within the user model if the user is requesting greater variety of content or content of different content types; (43) receiving terms from the user for explicitly adding to or deleting from the user model; (44) respectively weighing favorably and unfavorably content items comprising the received terms when providing the recommendations; (45) suggesting terms from the content items to the user for adding to the user model and/or for content searches that may be issued by the user; (46) comparing the user model to a disparate user model of a second user; and (47) suggest the second user as a contact to the user based on a similarity of the user model to the disparate user model of the second user based on the similarity between the two user models crossing a predetermined threshold value. LuVogt also discloses in ¶¶ [0060]-[0072] that (1) a query-less search procedure can be implemented based on an understanding of a user's needs in real-time, adapting to user's interests changes, location changes and changes in the time of the day by employing a filtering mechanism that is adaptive and personalized on a per-user basis; (2) a search based on a user query for a particular type of content from a particular content stream or for content items having user-specified attributes from various content streams can be aggregated for presentation to the user; (3) in order to build the personalized content recommendation system 100 wherein the content is constantly changing, hyper-personally relevant, the following are examples of preferred considerations to be addressed: A. Cold-start: ensuring that the recommended items are relevant the first time a user begins to use the recommendation system 100 in order to mitigate the users from abandoning usage of the recommendation system 100; B. Implicit model: automatically building a robust representation of the users' interests with as little explicit customization as possible since users typically will only invest a small amount of cognitive energy on customizing the recommendation system 100; C. Learning & forgetting: adjusting the user model based on user behavior (or absence of behavior); D. Discovery: ensuring that the user model does not get "trapped" in a local optimum, and allows the user to discover content on topics they may not have previously seen, including trending topics; E. Transparency: representing the user model in such a way as to aid interpretation, as well as supporting a user-friendly view (for example, to let the user know what the model thinks are relevant topics for him/her); and F. De-duping: detecting duplicate or near-duplicate content, and being smart about when to recommend it and when to not recommend it; (4) a recommendation system or methodology may address at least partially the aforementioned challenges by combining the following features: 1. an Information Retrieval vector-space model to represent users, articles, and categories, which allows for a simple similarity criterion used for both recommendation and duplicate detection; 2. Leveraging lightweight explicit customization to prime the cold-start model, while also providing a mechanism to support discovery of new topics as well as trending topics; 3. exploiting simple "Rocchio" style feedback in real-time to update the model based on user behavior; and 4. adapting parameters of the model in real-time to target desired user and system behavior, for example, based on click-through rate of the user or recommendation rate of the recommendation system. LuVogt further teaches in ¶¶ [0080]-[0112] with FIGS. 3-4 that (1) the recommendations module 210 comprises three modules, the item processing module 310, the feature processing module 320 and the user processing module 330; (2) each time a new piece or item of content is received by the recommendation module 210, a representation of the item or an item processing element 312 of the content item is generated by the item processing module 310; (3) the data 3120 included in the item processing element 312 can comprise the keywords or features associated with the content item represented by the item processing element 312; (4) in addition, the data 3120 can comprise attributes of the content item which can include, for example, the title, abstract, source, author, or location associated with or referred to in the content item, in addition to other characterizing and/or statistical data; (5) when a new user registers with the recommendation module 210, information related to the user is received and processed by the user processing module 330 which generates a representation of the user as the user processing element 332; (6) the data 3320 can include an initial user model 3324 that is generated from the information provided explicitly by the user, e.g., while signing up to use the recommendation module 210 and implicit information gathered by observing the user's interaction as will be detailed further infra; (7) the user model 3324 thus generated is further constantly updated based on user interaction with the recommendations module 210; (8) a feature processing module 320 that maintains a correspondence between users 254 and the received items of content 252; (9) when an item of content is received, features such as keywords and/or metadata as described herein regarding the item are extracted, e.g., via natural language processing or other techniques; (10) a feature can generally include any piece of data derived from an item or a user and used as input to the filtering decision; (11) features can include explicit data of content items such as words, or entities; (12) entities can comprise information associated with the content item such as a source of the content item, author of the content item, a location, a category or a sentiment associated with the content item; (13) for each of the recognized features, the feature processing module 320 maintains, e.g., in the data 3220 of the corresponding feature processing element 322, a list of content items 3222 associated with a given feature e.g., a word or phrase and a list of users 3224 who express interest in the given feature; e.g., for a feature, such as a word "micro-hybrids", the feature processing module 320 maintains a list of a subset of the received content items 252 which include or are otherwise associated with the word "micro-hybrids" and a list of active users of the recommendation module 210 who have either implicitly or explicitly expressed interest in the content items that include the term "micro-hybrids"; (14) the different processing elements are generated by modeling the users, categories/features and items of content within the recommendations module 210 as vectors; (15) the feature space for user, item, and category vectors consists of terms (single words or bi-grams) that have been stripped of some special and non-ASCII characters, and stopped (common, non-meaningful words are ignored, and bigrams with one or more stop-words are also ignored); (16) every item or article is preferably represented by a vector Vg,c,d, where the item has a guid (global unique identifier) g, is possibly labeled with a category c, and was published/received on day d; (17) a modified version of tf-idf (term frequency-inverse document frequency) weighting is used to populate the item vector; (18) all new users are asked to select from a list of preferred categories, e.g., from a list of eight possible categories; (19) the initial user model for user x who has selected a list of categories C, is simply a term vector that is taken from a set of cold-start vectors: Vc; (20) these cold-start vectors are computed on an ongoing basis; (21) category vectors Vc are therefore, used as both the initial user vector, as well as a component of the evolving user vector. This provides a direct way to give the users reasonable content recommendations for the outset thereby addressing the cold start challenge, with an implicit model that requires little input from the user; (22) it also directly helps set user expectations by making explicit to them what types of content they can expect to receive as recommendations; (23) as the category vectors Vc are computed continuously, they always represent the recent content in any category most strongly, and thus, as part of the user vector, they help make sure users are able to discover new, trending content thereby ensuring that a user model does not get "trapped" in a local optimum; (24) furthermore, since different users may use the recommendations module 210 in different ways (e.g., some users may want hyper-relevance while others may expect more discovery), an explicit parameter αx as detailed further infra to trade-off how much of the category vectors are used in the user vector and thereby will allow the recommendations module 210 (or the user) to adjust this property; (25) the user processing element 322 includes data 3320 and code 3322 associated with a particular representation of the user, which aid in determination of accurate recommendations for the user represented by the user PE 322; (26) the data 3320 can comprise user attributes such as age, location, demographic information of the user in addition to different item collections 410 that are unique to the user who is represented by the user PE 332, which can include without limitation, collections of seen items, unseen items, saved items, deleted items, items recommended to other users, liked or disliked items; (27) the data 3320 also includes user preferences 420 that are collected from the user, for example, via a settings screen, which can include, for example, the privacy setting of the user, the themes to be used for a user interface, the information to be displayed for various screens of the user interface and other user preferences; (28) in addition to simply providing the stream of recommended content, the recommendations module 210 also provides for "lenses" which are filters based on different content attributes; (29) the user can therefore, "drill down" on specific attributes of recommended content, so they see only trending content, content from a particular publisher or author, content from a specified time period in the past or future, content associated with a given location, content containing specific keywords, content based on sentiment (positive, negative, happy, sad, shocking, etc.) and content based on any other meta-data; and (30) when the user is issuing a keyword search (either within or outside of the recommendations module 210), terms from the respective user model can be used as search suggestions or as input to the ranking functions so that content items including terms from the user model 3324 are ranked higher in the result set. LuVogt also teaches in ¶¶ [0113]-[0114] with FIG. 5 that (1) a global user processing element (user PE) 500 is a hypothetical entity that is designed to address the cold start challenge associated with new users of the recommendation module 210 by aiding in the exploration aspects; (2) in particular, the global user PE comprises models of various categories, e.g., news categories that can include business 510, entertainment 520, health 530, or other categories like world 580; (3) in general, the global user PE 500 maintains a model each for a finite number of categories associated with various content items that can be received by the recommendations module 210; (4) in case a new document or content item D is received by the recommendations module 210, a vector representation 540 of the content item D is generated and a category associated with the content item D is retrieved; e.g., if the content item D is associated with the health category 530, the vector representation 502 of the content item Dis added to the health category vector and the new health category vector thus generated is decayed as detailed supra; (5) thus, at any given time, the global user PE 500 conveys a substantially accurate representation of all the words and phrases in the latest news articles of a particular category; (6) when a new user registers with the recommendations module 210, the new user selects specific categories for receiving content items and those specific user selected categories are included in the initial user model thereby addressing the cold start challenge; (7) additionally, since the user is also represented by a vector, typical clustering techniques can be used to create groups of related users and aggregate their vectors to create the vector for a new user, again by looking at other attributes they share in common like age, sex, location, News category selections, Avatar selection, or other selections the user makes when configuring the recommendations module 210 for personal use; (8) moreover, the updated category vector, e.g., the updated health category vector described above is also folded into the user models of the users who indicated their interest in the health category; and (9) thus, the user models of all the users are automatically updated with the new features from the health category 530 in real-time and/or on a periodic basis. LuVogt further discloses in ¶¶ [0123]-[0131] with FIGS. 9-10 that (1) the user PE 332 upon receiving a vector representation of a content item D, scores the content item D for its relevance to the user in real time as shown at 952; (2) in particular, a modified dot product of a vector representation 540 of the content item D 502 is obtained in combination with the user vector 944 as shown at 952; (3) the resultant of the operation at 952 includes the user vector 944 which represents the user in combination with user preferences including but not limited to keywords, categories and sources; (4) if the received content item D 502 is determined to be relevant, it is added to one of the unseen item lists 954; (5) each of the item types has a respective unseen items queue in addition to being associated with a particular item mixture weight; (6) thus, the news item type has a respective item mixture weight associated therewith; (7) when a request for content, e.g., in the form of a 'gimme' gesture is received from the user as shown at 956, the various items from the different item queues 954 are rescored in accordance with respective weights and combined as shown at 958; (8) in this instance depicted in FIG. 9, content item D 502 is found to be relevant upon rescoring at 958 and accordingly, it is forwarded for display to the user in the seen items queue 960; (9) if the user, upon receiving the content item D 502 in the seen queue 960, clicks on it, the click event 962 is received by the user PE 332 and used as a feedback to update item mixture weights in real time as shown at 964; (10) each time a user selects an item of content, the event generated by such selection affects the user model 3324 in real time; (11) the selection information of the user can also affect other user models which are similar to the user model 3324 so that the content items forwarded to the other users are also affected in real-time; (12) all category-level vectors for content categories that a user has selected are aggregated and this aggregate is assigned as the initial user model; (13) the user PE 332 includes content categories 1002 that are explicitly declared by the user for example, as shown in the user preferences 420, or are implicitly derived as being of interest to the user via various user actions and/or input from other related users or users with similar interests; (14) on a periodic basis, e.g., on a nightly basis, the features such as word/terms which are included in the categories 1002 in addition the content items from the sources specified in the user preferences 420 are folded or added to the user vector 944 as shown at 1004; (15) by folding in the cold-start model or the latest version of the categories included in the user PE 332 on an continuing basis, the recommendations module 210 ensures that content on new topics is recommended to the user associated with the user PE 332, especially as new topics are always being created; and (16) in addition to being automatically updated in real time and on periodic basis, the recommendations module 210 allows a user to manipulate the user vector 944 to include keywords directly as shown at 1010 so that the user vector 944 can be configured to better represent the user.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HWEI-MIN LU whose telephone number is (313)446-4913. The examiner can normally be reached Mon - Fri: 9:00 AM - 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D. Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HWEI-MIN LU/Primary Examiner, Art Unit 2142
Prosecution Insights