DETAILED ACTION
Claims 1-20 are pending and have been examined.
--
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 09/10/2024, 10/02/2024, 12/16/2024 and 12/19/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
-
Claims 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more
Step 1: Claims 1-9 recite a system. Claims 10-17 recite a method. Claims 18-20 recite a non-transitory machine-readable storage medium. Therefore, claims 1-9 are directed to a machine, claims 10-17 are directed to a process, and claims 18-20 are directed to a manufacture.
With respect to claims 1, 10 and 18:
2A Prong 1: The claim recites a judicial exception.
based on the interaction data and profile data associated with the user, predicting a journey phase of a user journey for the user, the journey phase indicative of user intent to perform a first type of interaction with a first online platform (mental process – evaluation or judgement,--- a human can manually predict a journey phase indicative user intent based on interaction and profile data)
determining whether to cause the first piece of content to be presented to the user via the first online platform based on the score (mental process – evaluation or judgement,--- a human can manually determine if the content to be presented to the user)
2A Prong 2: The judicial exception is not integrated into a practical application.
(claim 1) at least one processor; and at least one non-transitory computer-readable medium having instructions stored thereon, which, when executed by the at least one processor, cause the system to perform operations comprising (claim 18) A non-transitory machine-readable storage medium comprising instructions which, when implemented by one or more machines, cause the one or more machines to perform operations comprising (mere instructions to apply an exception, (2) Whether the claim invokes computers - MPEP 2106.05(f); generic computer components)
accessing interaction data regarding one or more interactions between a user and digital content presented on one or more online platforms (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting)
accessing a first piece of content presented on the first online platform (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting)
feeding the first piece of content into a generative artificial intelligence (GAI) model, the GAI outputting an embedding corresponding to the first piece of content, the embedding being a representation of a meaning of the content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a GAI model that takes content as input and generates an embedding as output)
feeding the journey phase and the embedding into a machine learning model trained separately from the GAI model, the machine learning model outputting a score for the first piece of content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a machine learning model that takes the journey phase and the embedding as input and generates a score as an output)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
(claim 1) at least one processor; and at least one non-transitory computer-readable medium having instructions stored thereon, which, when executed by the at least one processor, cause the system to perform operations comprising (claim 18) A non-transitory machine-readable storage medium comprising instructions which, when implemented by one or more machines, cause the one or more machines to perform operations comprising (mere instructions to apply an exception, (2) Whether the claim invokes computers - MPEP 2106.05(f); generic computer components)
accessing interaction data regarding one or more interactions between a user and digital content presented on one or more online platforms (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
accessing a first piece of content presented on the first online platform (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i))
feeding the first piece of content into a generative artificial intelligence (GAI) model, the GAI outputting an embedding corresponding to the first piece of content, the embedding being a representation of a meaning of the content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a GAI model that takes content as input and generates an embedding as output)
feeding the journey phase and the embedding into a machine learning model trained separately from the GAI model, the machine learning model outputting a score for the first piece of content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using a machine learning model that takes the journey phase and the embedding as input and generates a score as an output)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 2, 11 and 19:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the machine learning model takes as input one or more features corresponding to the user in addition to the embedding (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 1 recites “feeding the journey phase and the embedding into a machine learning model,” which is mere instructions to apply an exception. Specifying more details about the input does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the machine learning model takes as input one or more features corresponding to the user in addition to the embedding (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 1 recites “feeding the journey phase and the embedding into a machine learning model,” which is mere instructions to apply an exception. Specifying more details about the input does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 3, 12 and 20:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the one or more features corresponding to the user are extracted from a user profile (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 2 recites “the machine learning model takes as input one or more features” which is mere instructions to apply an exception. Specifying more details about the features does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the one or more features corresponding to the user are extracted from a user profile (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 2 recites “the machine learning model takes as input one or more features” which is mere instructions to apply an exception. Specifying more details about the features does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 4 and 13:
2A Prong 1: The claim recites a judicial exception.
wherein the operations further comprise recommending the first piece of content be displayed to the user based on a prediction of a likelihood that the user will interact with the first piece of content (mental process – evaluation or judgement,--- a human can manually recommend the content based on a prediction of a likelihood)
With respect to claims 5 and 14:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the GAI model is further utilized to generate a textual description of why the first piece of content was recommended to the user (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using the GAI model to generate a description)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the GAI model is further utilized to generate a textual description of why the first piece of content was recommended to the user (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of using the GAI model to generate a description)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 6 and 15:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the feeding the first piece of content includes feeding the first piece of content and a list of categories into the GAI model, and the embedding represents a selection of a category from the list of categories, the category determined by the GAI model to be a closest match for the meaning of the content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 1 recites “feeding the first piece of content into a generative artificial intelligence (GAI) model… ,” which is mere instructions to apply an exception. Specifying more details about the input and output of the GAI model does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the feeding the first piece of content includes feeding the first piece of content and a list of categories into the GAI model, and the embedding represents a selection of a category from the list of categories, the category determined by the GAI model to be a closest match for the meaning of the content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 1 recites “feeding the first piece of content into a generative artificial intelligence (GAI) model… ,” which is mere instructions to apply an exception. Specifying more details about the input and output of the GAI model does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claims 7 and 16:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the feeding the first piece of content includes additionally providing the GAI model with a text question about the first piece of content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 1 recites “feeding the first piece of content into a generative artificial intelligence (GAI) model… ,” which is mere instructions to apply an exception. Specifying more details about the input to the GAI model does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the feeding the first piece of content includes additionally providing the GAI model with a text question about the first piece of content (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; claim 1 recites “feeding the first piece of content into a generative artificial intelligence (GAI) model… ,” which is mere instructions to apply an exception. Specifying more details about the input to the GAI model does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 8:
2A Prong 1: The claim recites a judicial exception.
wherein the journey phase is predicted by mapping behavioral signals into a time dimension and a signal strength dimension (mental process – evaluation or judgement,--- a human can manually predict the journey phase by mapping behavioral signals into a time dimension and a signal strength dimension)
With respect to claim 9:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the interaction data includes data regarding interactions with a plurality of different content types (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting; claim 1 recites “accessing interaction data,” which is insignificant extra-solution activity. Specifying more details of the interaction data does not cause the limitation to integrate the exception into a practical application.)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the interaction data includes data regarding interactions with a plurality of different content types (insignificant extra-solution activity – MPEP 2106.05(g), (3) data gathering and outputting, and WURC: receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 - MPEP 2106.05(d)(II)(i); claim 1 recites “accessing interaction data,” which is insignificant extra-solution activity. Specifying more details of the interaction data does not cause the limitation to be significantly more than the judicial exception.)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
With respect to claim 17:
2A Prong 2: The judicial exception is not integrated into a practical application.
wherein the GAI model is trained to understand content from different domains (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of training the GAI model)
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is directed to an abstract idea.
2B: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
wherein the GAI model is trained to understand content from different domains (mere instructions to apply an exception – MPEP 2106.05(f), (3) The particularity or generality of the application of the judicial exception; high level recitation of training the GAI model)
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-7 and 9-20 rejected under 35 U.S.C. 103 as being unpatentable over Cui ("M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems" 20220519) in view of Xie ("Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning" 20221019)
In regard to claims 1, 10 and 18, Cui teaches: A system comprising: at least one processor; and at least one non-transitory computer-readable medium having instructions stored thereon, which, when executed by the at least one processor, cause the system to perform operations comprising: (Cui, p. 11, A APPENDIX "We train M6-Rec in a distributed manner on four machines, where each machine is equipped with eight Nvidia Tesla V100 GPUs.")
accessing interaction data regarding one or more interactions between a user and digital content presented on one or more online platforms; (Cui, p. 4, Scoring tasks" The most common tasks in a recommender system [online platforms] are about scoring the plausibility of an event, for example, clickthrough rate (CTR) prediction or conversion rate (CVR) prediction where the goal is to estimate the probability of a user clicking or purchasing an item. M6-Rec expresses an example sample for CTR prediction as follows and sends it into M6’s Transformer: [BOS′] December. Beijing, China. Cold weather. A male user in early twenties, searched 'winter stuff' 23 minutes ago, clicked a product of category 'jacket' named 'men’s lightweight warm winter hooded jacket' 19 minutes ago, clicked a product of category 'sweatshirt' named 'men’s plus size sweatshirt stretchy pullover hoodies' 13 minutes ago, clicked... [interaction data regarding one or more interactions between a user and digital content] [EOS′]... The text between [BOS'] and [EOS'] describes the user-side features, which corresponds to the first region of M6’s input, i.e., the bidirectional region in Figure 1.")
based on the interaction data and profile data associated with the user, (Cui, p. 4, 3.2 Behavior Modeling as Language Modeling "We may put information about the user [profile data associated with the user] along with some basic facts between [BOS'] and [EOS'] if necessary."; see the above limitation for [the interaction data] between [BOS'] and [EOS']) predicting a journey phase of a user journey for the user, the journey phase indicative of user intent to perform a first type of interaction with a first online platform; (Cui, p. 3, Figure 3 "The figure showcases our implementation of late interaction for click-through rate (CTR) prediction. To support tasks that emphasize low-latency real-time inference, M6-Rec pre-computes and caches the first L' layers’ results, while computing the last L-L' layers for performing late interaction when the request arrives... we can dynamically incorporate the user’s latest activities. [real-time inference, latest activities, a journey phase of a user journey]"; p. 4 Zero-shot scoring tasks)
accessing a first piece of content presented on the first online platform; (Cui, p. 3, Figure 3 "The special token [BOS′] before the user feature text and the two special tokens [EOS′] [BOS] before the candidate feature text are omitted for clarity."; see Fig. 3, User Features. Candidate features. User-candidate cross features. [EOS] [a first piece of content], which includes products presented on the platform)
feeding the first piece of content into a generative artificial intelligence (GAI) model, the GAI outputting an embedding corresponding to the first piece of content, the embedding being a representation of a meaning of the content; (Cui, p. 1, 1 Introduction "In this paper, we extend our existing generative pretrained language model M6 [28–30] and present M6-Rec, a foundation model for recommender systems."; p. 5, 3.3 Late Interaction for Low-Latency Inference "We illustrate the design of multi-segment late interaction in Figure 3 using CTR prediction as an example. The core idea is to precompute and cache the computation results of the Transformer’s first L' layers [the first L' layers of the Transformer, a generative artificial intelligence (GAI) model]... The first L' layers do not model the interaction among the segments of a request. We hence concatenate these segments’ results precomputed by the first L' layers to form a single sequence, [the GAI outputting an embedding]
PNG
media_image1.png
426
1522
media_image1.png
Greyscale
and...")
feeding the journey phase and the embedding into a machine learning model..., (Cui, p. 5, 3.3 Late Interaction for Low-Latency Inference "... run the Transformer’s last L-L' layers [a machine learning model] in real time when a user request arrives. [the journey phase: real- time inference] We use L-L' ≤ 3 for low latency... It processes the segments individually, and caches the segments’ results for future reuse... We hence concatenate these segments’ results precomputed by the first L' layers to form a single sequence, [the embedding] and run the last L-L' layers of M6’s Transformer on the said sequence to take the interaction into account [the embedding (the sequence, precomputed) + the journey phase (real-time inference)].") the machine learning model outputting a score for the first piece of content; and (Cui, p. 4, Zero-shot scoring tasks "language models can estimate the likelihood of an event [the score] in a zero-shot manner, as long as the event is described in natural language."; zero-shot (real-time inference))
determining whether to cause the first piece of content to be presented to the user via the first online platform based on the score. (Cui, p. 4, Zero-shot scoring tasks "language models can estimate the likelihood of an event [the score] in a zero-shot manner, as long as the event is described in natural language. For example, to estimate whether a user who clicks hiking shoes prefers trekking poles or yoga knee pads, [determining a recommendation decision based on the score] we can construct the following two sentences: [BOS′] A user clicks hiking shoes [EOS′] [BOS] also... M6-Rec sends each sentence into M6’s Transformer and obtains the probability of each token. Let... be the output probabilities [the score] corresponding to token 'trekking', 'poles', 'yoga'... respectively.")
PNG
media_image2.png
236
298
media_image2.png
Greyscale
Cui does not teach, but Xie teaches: … a machine learning model trained separately from the GAI model (Xie, p. 4, Figure 2 "We present our strategies with a toy model of L = 5 layers and l∗ = 3. The green layers will be tuned (e.g. fine-tuned or adapter-tuned) during the task-specific training [a machine learning model trained separately] while the grey layers are not."; tuning only the last few layers)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Cui to incorporate the teachings of Xie by including layer-selecting strategies. Doing so would result in significant reduction of computation cost in fine-tuning. (Xie, p. 3, 3.2 Layer-Selecting Strategies "Only using the selected layers will result in significant reduction of computation cost: In fine-tuning, only the parameters of the selected layers will be updated.")
Claims 10 and 18 recite substantially the same limitation as claim 1, therefore the rejection applied to claim 1 also apply to claims 10 and 18. In addition, Cui teaches: A non-transitory machine-readable storage medium comprising instructions which, when implemented by one or more machines, cause the one or more machines to perform operations comprising: (Cui, p. 11, A APPENDIX "We train M6-Rec in a distributed manner on four machines, where each machine is equipped with eight Nvidia Tesla V100 GPUs.")
In regard to claims 2, 11 and 19, Cui teaches: wherein the machine learning model takes as input one or more features corresponding to the user in addition to the embedding. (Cui, p. 5, Zero-shot scoring tasks "M6-Rec supports other zero-shot tasks, such as building tag-based user profiles [features corresponding to the user] in a zero-shot manner, in a similar vein."; structured prompt can include features from user profile by embedding structured metadata (e.g., using XML tags like <user_type> or <interests>); zero-shot (real-time inference))
In regard to claims 3, 12 and 20, Cui teaches: wherein the one or more features corresponding to the user are extracted from a user profile. (Cui, p. 5, Zero-shot scoring tasks "M6-Rec supports other zero-shot tasks, such as building tag-based user profiles [a user profile] in a zero-shot manner, in a similar vein."; structured prompt can include features from user profile by embedding structured metadata (e.g., using XML tags like <user_type> or <interests>); zero-shot (real-time inference))
In regard to claims 4 and 13, Cui teaches: wherein the operations further comprise recommending the first piece of content be displayed to the user based on a prediction of a likelihood that the user will interact with the first piece of content. (Cui, p. 4, Generation task "We can further send the title into a text-to-image [using image: displayed to the user] synthesis pipeline such as M6-UFC [63]."; p. 4, Zero-shot scoring tasks "language models can estimate the likelihood of an event [a prediction of a likelihood that the user will interact with the first piece of content] in a zero-shot manner, as long as the event is described in natural language. For example, to estimate whether a user who clicks hiking shoes prefers trekking poles or yoga knee pads, we can construct the following two sentences: [BOS′] A user clicks hiking shoes [EOS′] [BOS] also... M6-Rec sends each sentence into M6’s Transformer and obtains the probability of each token. Let... be the output probabilities corresponding to token 'trekking', 'poles', 'yoga'... respectively.")
In regard to claims 5 and 14, Cui teaches: wherein the GAI model is further utilized to generate a textual description of why the first piece of content was recommended to the user. (Cui, p. 4, Generation tasks "M6-Rec uses the following plain text format to support both personalized product design [18] and explainable recommendation [why the first piece of content was recommended to the user.] [61]: [BOS′]... [EOS′] [BOS] The user now purchases a product of category '...' named '...'. Product details: ... The user likes it because ...[EOS] ... The product description is included here to provide basic facts for drafting an explanation text when making a recommendation decision")
In regard to claims 6 and 15, Cui teaches: wherein the feeding the first piece of content includes feeding the first piece of content and a list of categories into the GAI model, and the embedding represents a selection of a category from the list of categories, the category determined by the GAI model to be a closest match for the meaning of the content. (Cui, p. 5, Retrieval tasks "M6-Rec feeds the information of a user into M6’s Transformer using the following format: [BOS′] December. Beijing, China. Cold weather. A male user in early twenties, searched 'winter stuff' 23 minutes ago, clicked a product of category 'jacket' named 'men’s lightweight warm winter hooded jacket' 19 minutes ago,... [EOS′] [BOS] [EOS] The output corresponding to [EOS′] is linearly projected into a 128-dimensional vector and l2-normalized to form the user’s vector representation x. [a selection of a category from the list of categories, closest match for the meaning of the content] An item’s text is fed into the model as follows: [BOS′] [EOS′] [BOS] A product of category 'boots' named 'waterproof hiking shoes mens outdoor'. High CTR among the top 5%. Product details:...[EOS] Again, the output at [EOS] is projected into a 128-dimensional vector and l2-normalized to serve as the item’s vector representation y. We use two different learnable matrices for linearly projecting user vectors and item vectors... "; embedding selection or token-level selection, fixed-length vector that summarizes the entire input)
In regard to claims 7 and 16, Cui teaches: wherein the feeding the first piece of content includes additionally providing the GAI model with a text question about the first piece of content. (Cui, p. 4 "Generation task: conversational recommendation. M6-Rec supports this task by marking the speaker of each sentence:[BOS']... [EOS'] [BOS] USER: Hi! SYSTEM: What kind of movie do you like? USER: I like horror movies. SYSTEM: How about The Shining (1980)?... [a text question][EOS]")
In regard to claim 9, Cui teaches: wherein the interaction data includes data regarding interactions with a plurality of different content types. (Cui, p. 4, Scoring tasks"M6-Rec expresses an example sample for CTR prediction as follows and sends it into M6’s Transformer: [BOS′] December. Beijing, China. Cold weather. A male user in early twenties, searched 'winter stuff' 23 minutes ago, clicked a product of category 'jacket' named 'men’s lightweight warm winter hooded jacket' 19 minutes ago, clicked a product of category 'sweatshirt' named 'men’s plus size sweatshirt stretchy pullover hoodies' 13 minutes ago, clicked... [interactions with a plurality of different content types]")
In regard to claim 17, Cui teaches: wherein the GAI model is trained to understand content from different domains. (Cui, p. 1, Abstract "In this paper, we explore the possibility of developing a unified foundation model to support open-ended domains [understand content from different domains] and tasks in an industrial recommender system… we propose an improved version of prompt tuning [the GAI model is trained] that outperforms fine-tuning with negligible 1% task-specific parameters"; p. 1, 1 Introduction "we propose option tuning, [the GAI model is trained] an improved variant of prompt tuning [22], in place of fine-tuning. It adds negligible task-specific parameters and makes no modification to the pre-trained model’s parameters")
Claim 8 rejected under 35 U.S.C. 103 as being unpatentable over Cui and Xie, as applied to claim 1, and in further view of Cao ("Time-aware Prompting for Text Generation" 20221103)
In regard to claim 8, Cui and Xie do not teach, but Cao teaches: wherein the journey phase is predicted by mapping behavioral signals into a time dimension and a signal strength dimension. (Cao, p. 1, 1 Introduction "this work aims to study the effects of presenting temporal information to generation models. Concretely, to include timestamps in model inputs, we consider prepending two types of time aware prompts to the encoder or the decoder."; p. 3, Figure 2 "A linear prompt treats year/month/day as separate scalars [a time dimension] and projects them into continuous prompt vectors [a signal strength dimension] to be used on encoder or decoder. The vector’s scale reflects their temporal orderings.")
PNG
media_image3.png
358
572
media_image3.png
Greyscale
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Cui and Xie to incorporate the teachings of Cao by including temporal information with separate scalars in model inputs. Doing so would improve the generation quality. (Cao, p. 1, Abstract "we show that linear prompts on encoder and textual prompts improve the generation quality on all datasets. Despite having less performance drop… are less sensitive to the given timestamps…")
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SU-TING CHUANG whose telephone number is (408)918-7519. The examiner can normally be reached Monday - Thursday 8-5 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SU-TING CHUANG/Examiner, Art Unit 2146
Prosecution Insights