DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The following is a Final Office Action in response to communications received on 1/27/2026. Claims 1-20 are currently pending and have been examined. Claims 1-11 and 13-20 have been amended.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Step 1: The claims 1-12 are a system, claims 13-19 are a method, and claims 20 is a computer readable medium. Thus, each independent claim, on its face, is directed to one of the statutory categories of 35 U.S.C. §101. However, the claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 2A Prong 1: The independent claims (1, 13 and 20, taking claim 1 as a representative claim) recite:
An electronic device, comprising: circuitry configured to:
receive first history information associated with a set of users for a first item of a set of items;
determine first similarity information associated with each user of the set of users with respect to remaining users of the set of users;
receive social network information associated with each user of the set of users with respect to remaining users of the set of users;
determine a first embedding associated with each user of the set of users for the first item, based on the received first history information, the determined first similarity information, and the received social network information;
apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model;
determine a masked user from the set of users based on the application of the shared BERT model, wherein the masked user is associated with the first item;
train the shared BERT model based on the determined masked user corresponding to the first item;
and render, based on the trained shared BERT model, first recommendation information including the determined masked user.
These limitations, except for the italicized portions, under their broadest reasonable interpretations, recite certain methods of organizing human activity for managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) as well as commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations). The claimed invention recites steps for receiving historical information about a user and user item interactions, determining an embedding for the user item interactions, applying a transformer model to determine and user-item pair. The steps under its broadest reasonable interpretation specifically fall under sales activities. The Examiner notes that although the claim limitations are summarized, the analysis regarding subject matter eligibility considers the entirety of the claim and all of the claim elements individually, as a whole, and in ordered combination.
Prong 2: This judicial exception is not integrated into a practical application. In particular, the claims recite the additional elements of
An electronic device, comprising: circuitry configured to: (claim 1)
A method, comprising: in an electronic device: (claim 13)
A non-transitory computer-readable medium having stored thereon, computer-executable instructions that when executed by an electronic device, causes the electronic device to execute operations, the operations comprising: (claim 20)
apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model;
determine a masked user from the set of users based on the application of the shared BERT model, wherein the masked user is associated with the first item;
train the shared BERT model based on the determined masked user corresponding to the first item;
and render, based on the trained shared BERT model, first recommendation information including the determined masked user.
The additional elements emphasized above are recited at a high-level of generality (i.e., as a generic processor performing a generic computer function of processing data) such that it amounts no more than mere instructions to apply the exception using a generic computer component. The limitations not impose any meaningful limits on practicing the abstract idea, and therefore do not integrate the abstract idea into a practical application – MPEP 2106.05(f).
Accordingly, these additional elements when considered individually or as a whole do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The independent claims are directed to an abstract idea.
Step 2B: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed with respect to Step 2A Prong two, the additional elements in the claims amount to no more than mere instructions to apply the judicial exception using a generic computer component.
Even when considered as an ordered combination, the additional elements of claim 1, 13, and 20 do not add anything that is not already present when they are considered individually. Therefore, under Step 2B, there are no meaningful limitations in claims 1, 13, and 20 that transform the judicial exception into a patent eligible application such that the claims amount to significantly more than the judicial exception itself (see MPEP 2106.05).
As such, independent claims 1, 13, and 20 are ineligible.
Dependent claims 2-12 and 14-19 when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. §101 because the additional recited limitations fail to establish that the claims are not directed to the same abstract idea of Independent Claims 1, 13 and 19 without significantly more.
Claim 2 recites wherein the circuitry is further configured to: receive second history information associated with the set of items for a specific user of the set of users;
determine second similarity information associated with each item of the set of items with respect to remaining items of the set of items; determine a second embedding associated with each item of the set of items for the specific user, based on the received second history information and the determined second similarity information; apply a second transformer model on the determined second embedding; determine at least one second item from the set of items based on the application of the second transformer model;
and render second recommendation information including the determined at least one second item for the user. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 3 recites wherein the second transformer model corresponds to the shared BERT model. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 4 recites wherein the circuitry is further configured to: receive first correlation information associated with the set of items for the specific user of the set of users, and determine the second similarity information based on the received first correlation information. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 5 recites wherein the circuitry is further configured to: apply a user sequence header on the received first correlation information associated with the specific set of items for the user of the set of users, wherein the user sequence header corresponds to the determined at least one second item. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 6 recites wherein the first correlation information corresponds to a masked item from the set of items, for the specific user, and the circuitry is further configured to train the second transformer model based on the masked item corresponding to the specific user. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 7 recites wherein the circuitry is further configured to determine first neighborhood information associated with the set of items for the specific user of the set of users, based on the determined first correlation information, the determination of the at least one second item from the set of items is further based on the determined first neighborhood information, and the first neighborhood information is indicative of each item of the set of items correlated with the specific user. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 8 recites wherein the circuitry is further configured to: receive second correlation information associated with the set of users for the first item of the set of items; and determine the first similarity information
Claim 9 recites wherein the circuitry is further configured to apply an item sequence header on the received second correlation information associated with the set of users for the first item of the set of items, and the item sequence header corresponds to the determined masked user. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 10 recites wherein the second correlation information
Claim 11 recites wherein the circuitry is further configured to determine second neighborhood information associated with the set of users for the first item of the set of items, based on the second correlation information, the determination of the masked user from the set of users is further based on the determined second neighborhood information, and the second neighborhood information is indicative of each user of the set of users correlated with the first item. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claim 12 recites wherein the social network information includes at least one of: a set of relationships between the set of users on a set of social network platforms, or a set of preferences corresponding to the set of items for each user of the set of users. The limitation merely further limits that abstract idea and does not integrate the judicial exception into a practical application.
Claims 14-19 recite parallel claim language and therefore are also rejected for the reasons set forth above.
For these reasons claims 1-20 are rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 6-10, 12-15, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ahmed (US 20220245161) in view of Rossi (US 20200342006) in view of GUIM - General User and Item Embedding with Mixture of Representation in E-commerce in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation.
Regarding claims 1, 13, and 20, Ahmed discloses:
An electronic device, comprising: circuitry configured to: (claim 1) (shown in Figure 1)
A method, comprising: (claim 13) (shown in Figure 1)
A non-transitory computer-readable medium having stored thereon, computer-executable instructions that when executed by an electronic device, causes the electronic device to execute operations, the operations comprising: (claim 20) (shown in Figure 1)
receive first history information associated with a set of users for a first item of a set of items; [0028] As previously mentioned, user information 138 may be collected and/or gathered to model user interest to identify content that is most likely to be of interest to the user. User information 138 may include, but is not limited to, a user id 138A, explicit user interests 138B, browser search history 138C, search engine clicks 138D, search engine queries 138E, other content 138F consumed by an application utilized by the user, and/or other user metric information 138G (e.g., dwell time, telemetry data, etc.) that may be used to model user behaviors and user interests. [0035] The user information 222 may include user content information 224 including, but not limited to, user age information, user gender information, user language information, user ethnicity information, user education information, user job information, user income information, user location information, and/or additional information related to a user. The user interest information 226 may include user interest content information 228 including, but not limited, to topics that the user likes and topics that the user does not like. In some examples, the user interest content information 228 includes topics that the user likes. In examples, the user interest information 226 may be derived from one or more models configured to identity topics of interest and topics of disinterest based on user interaction information with content items. For example, based on metric information previously discussed, a user dwell time associated with particular content that is high indicates that a topic associated with the content may be of interest to the user. As another example, a user may spend little time viewing content that is of no interest to them. The user history information 230 may include history content information 232 corresponding to information related to documents viewed by the user together with a timestamp. The user history information 230 may be obtained from browser logs, search history, clicks, etc.
determine first similarity information associated with each user of the set of users with respect to remaining users of the set of users; [0039] FIG. 4 depicts an example of a collaborative document filtering module 404 in accordance with examples of the present disclosure. The collaborative document filtering module 404 may be the same as or similar to the collaborative document filtering module 148 previously described. Collaborative filtering techniques use a database of preferences for items by users to predict additional topics or products a new user might like. A fundamental assumption of content filtering is that if users X and Y rate n items similarly, or have similar behaviors and/or interests (e.g., viewing the same or similar content, buying the same or similar items, watching the same or similar videos, and/or listening to the same or similar audio content), a new user having the same and/or similar behaviors and/or interests as users X and Y may act in a similar manner.
determine a first embedding associated with each user of the set of users for the first item, based on the received first history information, the determined first similarity information, […] [0046] The user embedding, which may be the same as or similar to the user embedding 236 (FIG. 2) may be a relatively low-dimensional space representation of a translated higher-dimensional vector derived from information associated with a user. For example, the user embedding may be based on user information, such as user information 222 (FIG. 2), user interest information, such as user interest information 226 (FIG. 2), and user history information, such as user history information 230 (FIG. 2). The user information, user interest information, and history information may be provided to a machine learning model, such as the transformer 234 (FIG. 2). And see [0045] and see Figure 2 236
Ahmed discloses the collection of user interactions information with websites, such as indicating likes and clicks, but does not expressly disclose:
receive social network information associated with each user of the set of users with respect to remaining users of the set of users;
apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model;
determine a masked user from the set of users based on the application of the shared BERT model, wherein the masked user is associated with the first item;
train the shared BERT model based on the determined masked user corresponding to the first item;
and render, based on the trained shared BERT model, first recommendation information including the determined masked user.
However Rossi discloses:
receive social network information associated with each user of the set of users with respect to remaining users of the set of users; and the received social network information; [0036] For example, in a scenario where the heterogeneous graph 116 represents a social network where different members of the social network interact with one another, the node associations 124 may represent various interactions between two of the members of the social network, such as a message exchange between two members of the social network, establishing a friendship between two members of the social network, and so forth. In another example, the heterogeneous graph 116 may represent any type of network that evolves with the addition, deletion, and updates of various network entities and associations. Thus, by considering temporal values associated with node associations, the techniques described herein can account for changes to a network over time and generate clusterings and embeddings for use by a recommendation system to generate real-time recommendations that account for a current state of the network.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information in Ahmed to include receive social network information associated each user of the set of users with respect to remaining users of the set of users; and the received social network information, as taught in Rossi, in order to account for changes to a network over time and generate clusterings and embeddings for use by a recommendation system to generate real-time recommendations that account for a current state of the network (paragraph 0036).
Ahmed discloses determining similarities based on user embeddings and Rossi teaches the collection of social networking information, the combination does not expressly disclose:
apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model;
determine a masked user from the set of users based on the application of the shared BERT model, wherein the masked user is associated with the first item;
train the shared BERT model based on the determined masked user corresponding to the first item;
and render, based on the trained shared BERT model, first recommendation information including the determined masked user.
However GUIM teaches:
apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model; [page 3] and an embedding of each item. We represent the target embedding for item 𝑗 as vector 𝒗𝒋 ∈ R𝑑, a vector of length 𝑑. We propose to represent the target embedding for user u𝑖 and The second layer of our model is "Masking & Time Fusion Layer". The logic of this layer is similar to that in the original BERT. In the first step, with some specified probability it randomly chooses some items from 𝑗1to 𝑗𝑁 in the sequence for the masking purpose, that is, replaces the embedding of each chosen item with the embedding of a special token "MASK". In our model, we use a default setting of 15% masking probability, just as the original BERT. In the second step, for each embedding of𝐶𝐿𝑆𝑐 or an item before the cutoff time 𝑇, it adds with the corresponding time embedding to generate the final embedding which will be fed to the consequent Transformer layers [page 5]
determine a masked user from the set of users based on the application of the shared BERT model, wherein the masked user is associated with the first item; We include two pre-training tasks in out model: one is the MLM task, the other is the matching task. Since the MLM task is similar to that in the original BERT (except that the InfoNCE is used to avoid the computation-intractable full softmax operation), we focus on describing our proposed matching task which is based on user 𝑖 and his/her interacted items in the time period [𝑇,𝑇 + 𝐷2). [page 5]
train the shared BERT model based on the determined masked user corresponding to the first item; [page 7] For each CPP task, we partition the labelled dataset into training set and test set. There are roughly 10M records in the training set for each task, and the sizes of three test sets are 100k, 200k, 100k respectively. For each task, we obtain the general user embeddings from our pre-trained GUIM model, and then feed each user embedding as the only features of the user into a three-layer fully connected feed forward network (FFN) for the training. Thus, for our GUIM model, each user has totally 𝐶 ·𝑑 input features to train the FFN. After the training, we get the prediction accuracy of the trained FFN on each test dataset.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information for determining similarities of users through embeddings in Ahmed in view of Rossi to include apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model; determine a masked user from the set of users based on the application of the shared BERT model, wherein the masked user is associated with the first item; train the shared BERT model based on the determined masked user corresponding to the first item, as taught in GUIM, in order to reduce computational costs while maintaining good performance learning (page 2).
Ahmed discloses determining similarities based on user embeddings and Rossi teaches the collection of social networking information, the combination does not expressly disclose:
and render, based on the trained shared BERT model, first recommendation information including the determined masked user.
However “Pre-training of Context-aware Item Representation for Next Basket Recommendation” teaches:
and render, based on the trained shared BERT model, first recommendation information including the determined masked user. [page 2]The goal of the next basket recommendation is to predict the items that the user I would probably purchase in his next visit, given his historical records… BERT has shown its effectiveness in a variety of NLP tasks including general language understanding, question answering, named entity recognition and grounded common sense inference .In this paper, we propose to adapt the BERT model for the task of next basket recommendation.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information for determining similarities of users through embeddings in Ahmed in view of Rossi in further view of GUIM to include and render, based on the trained shared BERT model, first recommendation information including the determined masked user, as taught in Pre-training of Context-aware Item Representation for Next Basket Recommendation, in order to increase the effectiveness of the recommendation of next basket representation. (abstract).
Regarding claims 2 and 14, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above. Ahmed further discloses:
wherein the circuitry is further configured to: receive second history information associated with the set of items for a specific user of the set of users; [0024] Various metrics are collected based on user interaction, and non-interaction, with content items provided to and/or displayed to the user. The collected metrics are used to further tune, or otherwise change, the content items that are presented to the user by the targeted search and display system. For example, metrics indicating that that the content is not relevant to the user may be used to change one or more parameters associated with the generation of the document embeddings or the user embedding.
determine second similarity information associated with each item of the set of items with respect to remaining items of the set of items; [0049] the machine learning model generates a pool of ranked documents, where each document in the pool of ranked documents includes a ranking based on the document embedding and the user embedding. In some examples, the content identification and ranking module identify and select documents from the pool of ranked documents that are most likely to be relevant to the user. A detrimental point process can be applied to ensure the ranked results are diverse and do not refer to the same content or similar content. Accordingly, a user-content quality scoring and a content-content similarity score may be used to select a subset of the pool of ranked documents.
determine a second embedding associated with each item of the set of items for the specific user, based on the received second history information and the determined second similarity information; (see 218 document embedding Figure 2)
apply a second transformer model on the determined second embedding; determine at least one second item from the set of items based on the application of the second transformer model; and render second recommendation information including the determined at least one second item for the specific user. [0053] At 810, the user embedding together with the document embeddings for the set of selected documents may be provided to a machine learning model, such as a transformer. The machine learning model generates content rankings for the content in the set of contents. For example, each document in the set of documents includes a ranking based on the document embedding and the user embedding. In some examples, the documents classified as most likely to receive a user interaction include a higher ranking than other documents. In addition, the documents may be ranked in accordance with relevance, novelty, serendipity, diversity, and explainability with respect to the user. In some examples, a detrimental point process may be applied to ensure the ranked results are diverse and do not refer to the same content or similar content. The method 800 proceeds to 814, where recommended content is selected.
Regarding claim 3, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above.
Ahmed discloses generating user embeddings, document embeddings and applying a transformer model and Rossi teaches the collection of additional user related information to include social networking information, but the combination does not expressly teach:
wherein the second transformer model corresponds to the shared BERT model.
However GUIM teaches:
wherein the second transformer model corresponds to the shared BERT model. [page 3] and an embedding of each item. We represent the target embedding for item 𝑗 as vector 𝒗𝒋 ∈ R𝑑, a vector of length 𝑑. We propose to represent the target embedding for user u𝑖 and The second layer of our model is "Masking & Time Fusion Layer". The logic of this layer is similar to that in the original BERT. In the first step, with some specified probability it randomly chooses some items from 𝑗1to 𝑗𝑁 in the sequence for the masking purpose, that is, replaces the embedding of each chosen item with the embedding of a special token "MASK". In our model, we use a default setting of 15% masking probability, just as the original BERT. In the second step, for each embedding of𝐶𝐿𝑆𝑐 or an item before the cutoff time 𝑇, it adds with the corresponding time embedding to generate the final embedding which will be fed to the consequent Transformer layers [page 5]
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information for determining similarities of users through embeddings in Ahmed in view of Rossi to include wherein the second transformer model corresponds to the shared BERT model., as taught in GUIM, in order to reduce computational costs while maintaining good performance learning (page 2).
Regarding claims 4 and 15, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above. Ahmed further discloses:
wherein the circuitry is further configured to: receive first correlation information associated with the set of items for the specific user of the set of users, and determine the second similarity information based on the received first correlation information. [0024] Various metrics are collected based on user interaction, and non-interaction, with content items provided to and/or displayed to the user. The collected metrics are used to further tune, or otherwise change, the content items that are presented to the user by the targeted search and display system. For example, metrics indicating that that the content is not relevant to the user may be used to change one or more parameters associated with the generation of the document embeddings or the user embedding.
Regarding claim 6, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above.
Ahmed discloses generating user embeddings, document embeddings and applying a transformer model using correlating information regarding items and users related to the items and Rossi discloses the collection of additional user related information to include social networking information, but the combination does not expressly disclose:
wherein the first correlation information corresponds to a masked item from the set of items, for the specific user, and the circuitry is further configured to train the second transformer model
However GUIM teaches:
wherein the first correlation information corresponds to a masked item from the set of items, for the specific user, and the circuitry is further configured to train the second transformer model [page 3] and an embedding of each item. We represent the target embedding for item 𝑗 as vector 𝒗𝒋 ∈ R𝑑, a vector of length 𝑑. We propose to represent the target embedding for user u𝑖 and The second layer of our model is "Masking & Time Fusion Layer". The logic of this layer is similar to that in the original BERT. In the first step, with some specified probability it randomly chooses some items from 𝑗1to 𝑗𝑁 in the sequence for the masking purpose, that is, replaces the embedding of each chosen item with the embedding of a special token "MASK". In our model, we use a default setting of 15% masking probability, just as the original BERT. In the second step, for each embedding of𝐶𝐿𝑆𝑐 or an item before the cutoff time 𝑇, it adds with the corresponding time embedding to generate the final embedding which will be fed to the consequent Transformer layers [page 5] We include two pre-training tasks in out model: one is the MLM task, the other is the matching task. Since the MLM task is similar to that in the original BERT (except that the InfoNCE is used to avoid the computation-intractable full softmax operation), we focus on describing our proposed matching task which is based on user 𝑖 and his/her interacted items in the time period [𝑇,𝑇 + 𝐷2). [page 5]
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information for determining similarities of users through embeddings in Ahmed in view of Rossi to include wherein the first correlation information corresponds to a masked item from the set of items, for the specific user, and the circuitry is further configured to train the second transformer modeluser, as taught in GUIM, in order to reduce computational costs while maintaining good performance learning (page 2).
Regarding claims 7 and 17, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above. Ahmed further discloses:
wherein the circuitry is further configured to determine first neighborhood information associated with the set of items for the specific user of the set of users, based on the determined first correlation information, the determination of the at least one second item from the set of items is further based on the determined first neighborhood information, and the first neighborhood information is indicative of each item of the set of items correlated with the specific user. [0031, 0036, 0037] The similarity filter 312 utilizes a similarity metric to efficiently search for the documents that may be similar to the user. In examples, a nearest neighbor search 314 is performed on the similar documents. The nearest neighbor search identifies the similar documents 315 of the document embeddings 308 that are most similar to the user embedding 310.
Regarding claims 8 and 18, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above. Ahmed further discloses:
wherein the circuitry is further configured to: receive second correlation information associated with the set of users for the first item of the set of items; and determine the first similarity information based on the received second correlation information. [0028] As previously mentioned, user information 138 may be collected and/or gathered to model user interest to identify content that is most likely to be of interest to the user. User information 138 may include, but is not limited to, a user id 138A, explicit user interests 138B, browser search history 138C, search engine clicks 138D, search engine queries 138E, other content 138F consumed by an application utilized by the user, and/or other user metric information 138G (e.g., dwell time, telemetry data, etc.) that may be used to model user behaviors and user interests. [0035] The user information 222 may include user content information 224 including, but not limited to, user age information, user gender information, user language information, user ethnicity information, user education information, user job information, user income information, user location information, and/or additional information related to a user. The user interest information 226 may include user interest content information 228 including, but not limited, to topics that the user likes and topics that the user does not like. In some examples, the user interest content information 228 includes topics that the user likes. In examples, the user interest information 226 may be derived from one or more models configured to identity topics of interest and topics of disinterest based on user interaction information with content items. For example, based on metric information previously discussed, a user dwell time associated with particular content that is high indicates that a topic associated with the content may be of interest to the user. As another example, a user may spend little time viewing content that is of no interest to them. The user history information 230 may include history content information 232 corresponding to information related to documents viewed by the user together with a timestamp. The user history information 230 may be obtained from browser logs, search history, clicks, etc.
Regarding claim 10, Ahmed in view of Rossi teaches the limitations set forth above.
Ahmed discloses generating user embeddings, document embeddings and applying a transformer model using correlating information regarding items and users related to the items and Rossi discloses the collection of additional user related information to include social networking information, but does not expressly disclose:
wherein the second correlation information is associated with the masked user correspondinq to the first item
However GUIM teaches:
wherein the second correlation information is associated with the masked user correspondinq to the first item [page 3] and an embedding of each item. We represent the target embedding for item 𝑗 as vector 𝒗𝒋 ∈ R𝑑, a vector of length 𝑑. We propose to represent the target embedding for user u𝑖 and The second layer of our model is "Masking & Time Fusion Layer". The logic of this layer is similar to that in the original BERT. In the first step, with some specified probability it randomly chooses some items from 𝑗1to 𝑗𝑁 in the sequence for the masking purpose, that is, replaces the embedding of each chosen item with the embedding of a special token "MASK". In our model, we use a default setting of 15% masking probability, just as the original BERT. In the second step, for each embedding of𝐶𝐿𝑆𝑐 or an item before the cutoff time 𝑇, it adds with the corresponding time embedding to generate the final embedding which will be fed to the consequent Transformer layers [page 5] We include two pre-training tasks in out model: one is the MLM task, the other is the matching task. Since the MLM task is similar to that in the original BERT (except that the InfoNCE is used to avoid the computation-intractable full softmax operation), we focus on describing our proposed matching task which is based on user 𝑖 and his/her interacted items in the time period [𝑇,𝑇 + 𝐷2). [page 5]
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information for determining similarities of users through embeddings in Ahmed in view of Rossi to include wherein the second correlation information is associated with the masked user correspondinq to the first item, as taught in GUIM, in order to reduce computational costs while maintaining good performance learning (page 2).
Regarding claim 12, Ahmed in view of Rossi in view of GUIM in further view of Pre-training of Context-aware Item Representation for Next Basket Recommendation teaches the limitations set forth above.
Ahmed discloses the collection of user interactions information with websites, such as indicating likes and clicks, but does not expressly disclose:
wherein the social network information includes at least one of: a set of relationships between the set of users on a set of social network platforms, or a set of preferences corresponding to the set of items for each user of the set of users.
However Rossi discloses:
wherein the social network information includes at least one of: a set of relationships between the set of users on a set of social network platforms, or a set of preferences corresponding to the set of items for each user of the set of users. [0036] For example, in a scenario where the heterogeneous graph 116 represents a social network where different members of the social network interact with one another, the node associations 124 may represent various interactions between two of the members of the social network, such as a message exchange between two members of the social network, establishing a friendship between two members of the social network, and so forth. In another example, the heterogeneous graph 116 may represent any type of network that evolves with the addition, deletion, and updates of various network entities and associations. Thus, by considering temporal values associated with node associations, the techniques described herein can account for changes to a network over time and generate clusterings and embeddings for use by a recommendation system to generate real-time recommendations that account for a current state of the network.
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the user collected information in Ahmed to include wherein the social network information includes at least one of: a set of relationships between the set of users on a set of social network platforms, or a set of preferences corresponding to the set of items for each user of the set of users, as taught in Rossi, in order to account for changes to a network over time and generate clusterings and embeddings for use by a recommendation system to generate real-time recommendations that account for a current state of the network (paragraph 0036).
Subject Matter Free of Prior Art
Claims 5, 9, 11, 16, 19 are determined to have overcome the prior art of rejection and are free of prior art, however the claims remain rejected under 35 USC 101, as set forth above.
The closest art of record was found to be as follows:
For claims 5, 9 16 and 19, Narahara (US 20050144499) shows the labeling of the matrix in Figure 4 and Figure 5 “recommended user” “similar user” vs. Programs labeled by category as headers that correspond to the groups of users.
For claim 11, HybridBERT4Rec discloses ITEM/USER MASKING In CF-HybridBERT4rec, we aim to extract the target item representation which contains the similarity level between all neighbors and the target user. For each target item, there is a user sequence (see Fig. 6) that includes all users who have rated a target item, including the target user. Thus, we assume that other users besides target user are neighbors. In training, we utilize a random user masking to user sequence, aiming to allow the model to reconstruct the masked user as close to its original embedding as possible. After training is complete, we receive the network which is able to construct the next user representation based on the characteristics of users who interact with the target item. Therefore, in testing, we construct the target item representation by masking the target user and adding it to the end of the user sequence. The target item representation contains a comparison value between each neighboring user in the sequence and the target user. In other words, the target item representation represents the similarity level between all neighbors and the target user.
It was found that no references alone or in combination, neither anticipates, reasonable teaches, nor renders obvious the features of Applicant’s invention of the above claims. Therefore, none of the cited references disclose or render obvious each and every feature of the claimed invention and the claimed invention is determined to be free of the prior art. Although individually the claimed features could be taught, any combination of references would teach the claimed limitations using a piecemeal analysis, since references would only be combined and deemed obvious based on knowledge gleaned from the applicant's disclosure. Such a reconstruction is improper (i.e., hindsight reasoning). See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). The examiner emphasizes that it is the interrelationship of the limitations that renders these claims free of the prior art/additional art.
Therefore, it is hereby asserted by the Examiner that, in light of the above, that the claims are free of prior art as the references do not anticipate the claims and do not render obvious any further modification of the references to a person of ordinary skill in art.
Relevant Art Not Cited
Hsieh (US 20210174164) discloses processing embeddings through a matchmaking neural network training to map user embeddings to relevant content
Kopru (US20230101174) discusses determining similarities in combinations of user-item, item-item, query- item
Xu US20230206076 discloses the user to item mapping in Figure 1
CN110162706 discloses the construction of a user history behavior matrix showing user-item interaction for determining recommendations (see particularly claim 1)
Response to Arguments
Applicant's arguments filed 1/27/2026 have been fully considered but they are not persuasive.
With respect to the remarks directed to prong 1, the language presented in the arguments regarding amended language “[a]n electronic device, comprising: circuitry configured to ... apply a shared Bidirectional Encoder Representations from Transformers (BERT) model on the determined first embedding, wherein the shared BERT model is a neural network model; determine a masked user from the set of users based on the application of the shared BERT model ... train the shared BERT model based on the determined masked user corresponding to the item ... render, based on the trained shared BERT model, first recommendation information including the determined masked user.", the examiner has updated the rejection above to address this language and has not considered the language to be part of the abstract idea. However the language has been considered as part of the additional elements, addressed further below.
With respect to the remarks directed to prong 2, while the amended language recites additional elements, these elements are merely recited at the apply it level and does not integrate the judicial exception into a practical application. At the level of detail recited in the claims, the BERT model is merely applied to the abstract idea to output more accurate recommendations. Herein, the improvement lies in the abstract idea, better or more accurate recommendations and not the improvement of the BERT or neural network technology itself. That is, the data sets themselves are improved and thereby produce an improved result by applying the BERT model. That is not an integration into a practical application.
With respect to Step 2B, first, the examiner asserts that no limitation has been considered extra solution activity and therefore the Berkheimer considerations of whether limitations are well-understood, routine, and conventional is not required at this step. Furthermore, as discussed with prong 2, at most, the data set itself is being improved. If the data set (part of the abstract idea) is improved and then a machine learning model is applied and a better result is output (i.e. better recommendations), the abstract idea itself is improved, but not the additional element (i.e. the BERT model). As stated in the prong 2 remarks, the BERT model is merely applied to the abstract idea and not integrated into a practical application when the claim is taken as a whole.
The dependent claims remain rejected for the same reasons above.
With respect to remarks directed to 35 USC 103, the rejection has been updated in view of the claim amendments and no longer relies on the teachings of Lamba. Therefore the remarks directed to Lamba are considered moot. The claims remain rejected in view of the newly applied prior art combination under 35 USC 103.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTORIA E. FRUNZI whose telephone number is (571)270-1031. The examiner can normally be reached Monday- Friday 7-4 (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Marissa Thein can be reached at (571) 272-6764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
VICTORIA E. FRUNZI
Primary Examiner
Art Unit TC 3689
/VICTORIA E. FRUNZI/Primary Examiner, Art Unit 3689 4/7/2026
Prosecution Insights