Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant's claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 7 is rejected under 35 U.S.C. 112(d) as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Regarding dependent Claim 7, all of the claim elements are already included in Independent Claim 1 and thus fail to further limit Claim 1. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over Claims 1-20 of U.S. Patent No. 12,493,653 (‘653 Patent). Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the instant application are broader than the ‘653 Patent and are thus anticipated by the ‘653 Patent.
Regarding Claim 1, ‘653 Patent discloses a computer-implemented method for calculating a relevancy score for a candidate item of a plurality of candidate items based on a query, comprising:
providing, by at least one computer processor, input to a context tower of a two-tower deep machine learning (ML) model that comprises the context tower and an item tower, the input including (see ‘653 Patent, Claim 1, providing, by at least one computer processor, input to a context tower of a two-tower deep machine learning (ML) model that comprises the context tower and an item tower, the input including);
the query, and (see ‘653 Patent, Claim 1, the query, and)
a graph user embedding corresponding to a user associated with the query (see ‘653 Patent, Claim 1, a graph user embedding corresponding to a user associated with the query);
generating, by the context tower and based at least on the input, a context embedding in a vector space, wherein the generating the context embedding comprises (see ‘653 Patent, Claim 1, generating, by the context tower and based at least on the input, a context embedding in a vector space, wherein the generating the context embedding comprises):
generating a first output vector by providing demographics information associated with the user to a fully connected layer of the context tower (see ‘653 Patent, Claim 1, generating a first output vector by providing demographics information associated with the user to a fully connected layer of the context tower);
generating a second output vector by providing, to a factorization-machine based neural network of the context tower, the first output vector and a plurality of query projection embeddings (see ‘653 Patent, Claim 1, generating a second output vector by providing, to a factorization-machine based neural network of the context tower, the first output vector and a plurality of query projection embeddings); and
generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower (see ‘653 Patent, Claim 1, generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower);
determining, by the two-tower deep ML model, a measure of similarity between the context embedding and an item embedding associated with a corresponding item in the plurality of candidate items (see ‘653 Patent, Claim 1, determining, by the two-tower deep ML model, a measure of similarity between the context embedding and each of a plurality of item embeddings in the vector space, wherein each item embedding of the plurality of item embeddings is generated by the item tower and represents a corresponding item in the plurality of candidate items); and
calculating the relevancy score for the candidate item of the plurality of candidate items based on the determined measure of similarity between the context embedding and the item embedding corresponding to the respective candidate item (see ‘653 Patent, Claim 1).
Regarding Claim 2, ‘653 Patent discloses the computer-implemented method of Claim 1, further comprising selecting a subset of the candidate items for retrieval based on the relevancy scores associated therewith (see ‘653 Patent, Claim 2, selecting a subset of the candidate items for retrieval based on the relevancy scores associated therewith).
Regarding Claim 3, ‘653 Patent discloses the computer-implemented method of Claim 1, further comprising ranking some or all of the candidate items based on the relevancy scores associated therewith (see ‘653 Patent, Claim 2, ranking some or all of the candidate items based on the relevancy scores associated therewith).
Regarding Claim 4, ‘653 Patent discloses the computer-implemented method of Claim 1, wherein the input further comprises a query embedding corresponding to the query and generated by a neural-network-based language model trained on a text corpus and then tuned based on training data generated from logs of previously-submitted queries and associated user-item interactions (see ‘653 Patent, Claim 4, a query embedding corresponding to the query and generated by a neural-network-based language model trained on a text corpus and then tuned based on training data generated from logs of previously-submitted queries and associated user-item interactions).
Regarding Claim 5, ‘653 Patent discloses the computer-implemented method of Claim 1, further comprising demographics information associated with the user (see ‘653 Patent, Claim 4, wherein the input further comprises the demographics information associated with the user).
Regarding Claim 6, ‘653 Patent discloses the computer-implemented method of Claim 5, wherein the generating the context embedding comprises:
projecting the query into a plurality of query embedding spaces to generate a corresponding plurality of query projection embeddings (see ‘653 Patent, Claim 5, generating the corresponding plurality of query projection embedding by projecting the query into a plurality of query embedding spaces); and
generating the context embedding based at least on the graph user embedding, the query embedding generated by the neural-network-based language model, the demographics information associated with the user, and the plurality of query projection embeddings (see ‘653 Patent, Claim 5, generating the context embedding based at least on the graph user embedding, the query embedding generated by the neural-network-based language model, the demographics information associated with the user, and the plurality of query projection embeddings).
Regarding Claim 7, ‘653 Patent discloses the computer-implemented method of Claim 6, wherein the generating the context embedding comprises:
providing the demographics information associated with the user to a fully connected layer that generates a first output vector based thereon (see ‘653 Patent, Claim 1, generating a first output vector by providing demographics information associated with the user to a fully connected layer of the context tower);
providing the first output vector and the plurality of query projection embeddings to a factorization-machine based neural network that generates a second output vector based thereon (see ‘653 Patent, Claim 1, generating a second output vector by providing, to a factorization-machine based neural network of the context tower, the first output vector and a plurality of query projection embeddings); and
providing the graph user embedding, the second output vector, and the query embedding generated by the neural-network-based language model to a series of fully connected layers that generate the context embedding based thereon (see ‘653 Patent, Claim 1, generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower).
Regarding Claim 8, ‘653 Patent discloses the computer-implemented method of Claim 1, further comprising:
generating, by the item tower, the item embedding based at least on (see ‘653 Patent, Claim 7, generating, by the item tower, each item embedding of the plurality of item embeddings based at least on):
information indicative of a popularity level of the corresponding candidate item (see ‘653 Patent, Claim 7, information indicative of a popularity level of the corresponding candidate item),
an item identifier (ID) associated with the corresponding candidate item (see ‘653 Patent, Claim 7, an item identifier (ID) associated with the corresponding candidate item),
metadata associated with the corresponding candidate item (see ‘653 Patent, Claim 7, metadata associated with the corresponding candidate item), and
a graph item embedding generated by the GNN and associated with the corresponding candidate item (see ‘653 Patent, Claim 7, a graph item embedding generated by the GNN and associated with the corresponding candiate item).
Regarding Claim 9, the ‘653 Patent discloses the computer-implemented method of Claim 8, wherein the generating of the item embedding comprises:
providing the information indicative of the popularity level of the corresponding candidate item to a fully connected layer that generates an output vector based thereon (see ‘653 Patent, Claim 8, providing the information indicative of the popularity level of the corresponding candidate item to a fully connected layer that generates an output vector based thereon);
generating an embedding representation of the item ID associated with the corresponding candidate item (see ‘653 Patent, Claim 8, generating an embedding representation of the item ID associated with the corresponding candidate item);
generating an embedding representation of the metadata associated with the corresponding candidate item (see ‘653 Patent, Claim 8, generating an embedding representation of the metadata associated with the corresponding candidate item); and
providing the graph item embedding associated with the corresponding candidate item, the output vector, the embedding representation of the item ID associated with the corresponding candidate item, and the embedding representation of the metadata associated with the corresponding candidate item to a series of fully connected layers that generate the item embedding for the corresponding candidate item (see ‘653 Patent, Claim 8, providing the graph item embedding associated with the corresponding candidate item, the output vector, the embedding representation of the item ID associated with the corresponding candidate item, and the embedding representation of the metadata associated with the corresponding candidate item to another series of fully connected layers that generate the item embedding for the corresponding candidate item).
Regarding Claim 10, ‘653 Patent discloses a system for calculating a relevancy score for a candidate item of a plurality of candidate items based on a query, comprising:
one or more memories (see ‘653 Patent, Claim 11, one or more memories);
at least one processor each coupled to at least one of the memories and configured to perform operations comprising (see ‘653 Patent, Claim 11, at least one processor each coupled to the at least one memories and configured to perform operations comprising):
providing input to a context tower of a two-tower deep machine learning (ML) model that comprises the context tower and an item tower, the input including (see ‘653 Patent, Claim 11, providing input to a context tower of a two-tower deep machine learning (ML) model that comprises the context tower and an item tower, the input including):
the query (see ‘653 Patent, Claim 11, the query); and
a query embedding corresponding to the query (see ‘653 Patent, Claim 11, a query embedding corresponding to the query);
generating, by the context tower and based at least on the input, a context embedding in a vector space, wherein the generating the context embedding comprises (see ‘653 Patent, Claim 11, generating, by the context tower and based at least on the input, a context embedding in a vector space, wherein the generating the context embedding comprises):
generating a first output vector by providing demographics information associated with the user to a fully connected layer of the context tower (see ‘653 Patent, Claim 11, generating a first output vector by providing demographics information associated with a user associated with the query to a fully connected layer of the context tower);
generating a second output vector by providing, to a factorization-machine based neural network of the context tower, the first output vector and a plurality of query projection embeddings (see ‘653 Patent, Claim 11, generating a second output vector by providing the first output vector and a plurality of query projection embeddings to a factorization-machine based neural network of the context tower); and
generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower (see ‘653 Patent, Claim 11, generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower);
determining, by the two-tower deep ML model, a measure of similarity between the context embedding and an item embedding associated with a corresponding item in the plurality of candidate items (see ‘653 Patent, Claim 11, determining, by the two-tower deep ML model, a measure of similarity between the context embedding and each of a plurality of item embeddings in the vector space, wherein each item embedding of the plurality of item embeddings is generated by the item tower and represents a corresponding item in the plurality of candidate items); and
calculating the relevancy score for the candidate item of the plurality of candidate items based on the determined measure of similarity between the context embedding and the item embedding corresponding to the respective candidate item (see ‘653 Patent, Claim 11, calculating the relevancy score for each candidate item in the plurality of candidate items based on the determined measure of similarity between the context embedding and the item embedding corresponding to the respective candidate item).
Regarding Claim 11, ‘653 Patent discloses the system of Claim 10, wherein the operations further comprise selecting a subset of the candidate items for retrieval based on the relevancy scores associated therewith (see ‘653 Patent, Claim 12, selecting a subset of the candidate items for retrieval based on the relevancy scores associated therewith).
Regarding Claim 12, ‘653 Patent discloses the system of Claim 10, wherein the operations further comprise ranking some or all of the candidate items based on the relevancy scores associated therewith (see ‘653 Patent, Claim 12, ranking some or all of the candidate items based on the relevancy scores associated therewith).
Regarding Claim 13, ‘653 Patent discloses the system of Claim 10, wherein the input futher comprises a graph user embedding corresponding to a user associated with the query and generated by a graph neural network (GNN) based at least on past interactions between the user and one or more candidate items in the plurality of candidate items (see ’653 Patent, Claim 13, a graph user embedding corresponding to the user and generated by a graph neural network (GNN) based at least on past interactions between the user and one or more candidate items in the plurality of candidate items).
Regarding Claim 14, ‘653 Patent discloses the system of Claim 10, wherein the input further comprises demographics information associated with the user (see ‘653 Patent, Claim 13, wherein the input further comprises the demographics information associated with the user).
Regarding Claim 15, ‘653 Patent discloses the system of Claim 14, wherein the generating the context embedding comprises:
projecting the query into a plurality of query embedding spaces to generate a corresponding plurality of query projection embeddings (see ‘653 Patent, Claim 15, generating the corresponding plurality of query projection embedding by projecting the query into a plurality of query embedding spaces); and
generating the context embedding based at least on the graph user embedding, the query embedding generated by the neural-network-based language model, the demographics information associated with the user, and the plurality of query projection embeddings (see ‘653 Patent, Claim 15, generating the context embedding based at least on the graph user embedding, the query embedding generated by the neural-network-based language model, the demographics information associated with the user, and the plurality of query projection embeddings).
Regarding Claim 16, ‘653 Patent discloses the system of Claim 10, wherein the operations further comprise:
generating, by the item tower, the item embedding based at least on (see ‘653 Patent, Claim 17, generating, by the item tower, each item embedding of the plurality of item embeddings based at least on):
information indicative of a popularity level of the corresponding candidate item (see ‘653 Patent, Claim 17, information indicative of a popularity level of the corresponding candidate item),
an item identifier (ID) associated with the corresponding candidate item (see ‘653 Patent, Claim 17, an item identifier (ID) associated with the corresponding candidate item),
metadata associated with the corresponding candidate item (see ‘653 Patent, Claim 7, metadata associated with the corresponding candidate item), and
a graph item embedding generated by the GNN and associated with the corresponding candidate item (see ‘653 Patent, Claim 17, a graph item embedding generated by the GNN and associated with the corresponding candiate item).
Regarding Claim 17, ‘653 Patent discloses a non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations for calculating a relevancy score for a candidate item of a plurality of candidate items based on a query, the operations comprising:
providing input to a context tower of a two-tower deep machine learning (ML) model that comprises the context tower and an item tower, the input including (see ‘653 Patent, Claim 21, providing input to a context tower of a two-tower deep machine learning (ML) model that comprises the context tower and an item tower, the input including):
the query (see ‘653 Patent, Claim 21, the query); and
a query embedding corresponding to the query (see ‘653 Patent, Claim 21, a query embedding corresponding to the query);
generating, by the context tower and based at least on the input, a context embedding in a vector space, wherein the generating the context embedding comprises (see ‘653 Patent, Claim 21, generating, by the context tower and based at least on the input, a context embedding in a vector space, wherein the generating the context embedding comprises):
generating a first output vector by providing demographics information associated with the user to a fully connected layer of the context tower (see ‘653 Patent, Claim 21, generating a first output vector by providing demographics information associated with a user associated with the query to a fully connected layer of the context tower);
generating a second output vector by providing, to a factorization-machine based neural network of the context tower, the first output vector and a plurality of query projection embeddings (see ‘653 Patent, Claim 21, generating a second output vector by providing the first output vector and a plurality of query projection embeddings to a factorization-machine based neural network of the context tower); and
generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower (see ‘653 Patent, Claim 21, generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower);
determining, by the two-tower deep ML model, a measure of similarity between the context embedding and an item embedding associated with a corresponding item in the plurality of candidate items (see ‘653 Patent, Claim 21, determining, by the two-tower deep ML model, a measure of similarity between the context embedding and each of a plurality of item embeddings in the vector space, wherein each item embedding of the plurality of item embeddings is generated by the item tower and represents a corresponding item in the plurality of candidate items); and
calculating the relevancy score for the candidate item of the plurality of candidate items based on the determined measure of similarity between the context embedding and the item embedding corresponding to the respective candidate item (see ‘653 Patent, Claim 21, calculating the relevancy score for each candidate item in the plurality of candidate items based on the determined measure of similarity between the context embedding and the item embedding corresponding to the respective candidate item).
Regarding Claim 18, ‘653 Patent discloses the non-transitory computer-readable medium of Claim 17, wherein the operations further comprise selecting a subset of the candidate items for retrieval based on the relevancy scores associated therewith (see ‘653 Patent, Claim 22, selecting a subset of the candidate items for retrieval based on the relevancy scores associated therewith).
Regarding Claim 19, ‘653 Patent discloses the non-transitory computer-readable medium of Claim 17, wherein the operations further comprise ranking some or all of the candidate items based on the relevancy scores associated therewith (see ‘653 Patent, Claim 22, ranking some or all of the candidate items based on the relevancy scores associated therewith).
Regarding Claim 20, ‘653 Patent discloses the non-transitory computer-readable medium of Claim 17, wherein the input further comprises demographics information associated with the user (see ‘653 Patent, Claim 23, wherein the input further comprises the demographics information associated with the user).
Allowable Subject Matter
Claims 1-20 would be allowable if rewritten to overcome the rejection for non-statutory double patenting. In addition, Claim 7 would be allowable if rewritten to overcome the rejection under 35 U.S.C. 112(d).
The following is an Examiner’s Statement of Reasons for Indicating Allowable Subject Matter:
The prior art of record does not teach, disclose, or fairly suggest, all of the elements of Independent Claims 1, 11, and 21. Specifically, the prior art does not disclose:
generating a first output vector by providing demographics information associated with the user to a fully connected layer of the context tower;
generating a second output vector by providing, to a factorization-machine based neural network of the context tower, the first output vector and a plurality of query projection embeddings; and
generating the context embeddings by providing at least the second output vector to a series of fully connected layers of the context tower.
Guda (PG Pub. No. 2022/0129621 A1) discloses providing demographic information to the input layer of a neural network (see Guda, paragraph [0027], where example of a classification head is a set of one or more dense layers for receiving an encoded input (e.g., a concatenated version of the input text vector and the input demographics vector)). It is the position of the Examiner that inputting to an input layer is not patentably distinguishable from providing to a fully connected layer, as fully connected layers are, by definition, connected to the input layer of a neural network, which is, by definition, configured to provide input to the other layers of a neural network. However, Guda is silent as to providing the first output vector and the plurality of query projection embeddings to a factorization-machine based neural network that generates a second output vector based thereon and providing the second output vector and the query embedding generated by the neural-network-based language model to a series of fully connected layers that generate the context embedding based thereon.
Yu (PG Pub. No. 2020/0272913 A1) is directed toward recommending a target object to the user using factorization machine (FM) models, field-aware factorization machine (FFM) models, and deep learning FM (DeepFM) models, and deep learning FFM (DeepFFM) models. However, Yu is silent as to providing the first output vector and the plurality of query projection embeddings to a factorization-machine based neural network that generates a second output vector based thereon and providing the second output vector and the query embedding generated by the neural-network-based language model to a series of fully connected layers that generate the context embedding based thereon.
In addition, it is the position of the Examiner that in constructing and training a neural network that Independent Claims 1, 10, and 17 are not directed to an abstract idea without significantly more at least in view of MPEP 2106.04(a)(1)(vii).
Dependent Claims 2-9, 11-16, and 18-20, being definite, further limiting, and fully enabled by the Specification, would also be allowable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to the Applicant’s disclosure:
Gesmundo (PG Pub. No. 2020/0104687 A1), which concerns a hybrid neural architecture search.
Ding (PG Pub. No. 2022/0100786 A1), which concerns training a retrieval model.
Ojo (PG Pub. No. 2023/0297625 A1), which concerns utilizing a graph neural network to generate visualization and attribute recommendations.
Hsieh (PG Pub. No. 2021/0174164 A1), which concerns a personalized search and discovery engine.
Stabler (PG Pub. No. 2021/0224486 A1), which concerns structured adversarial training for natural language machine learning tasks.
Li (PG Pub. No. 2022/0215032 A1), which concerns an AI-based recommendation method and apparatus.
Guda (PG Pub. No. 2022/0129621 A1), which concerns a BERT-based machine-learning tool for predicting emotional response to text.
Yu (PG Pub. No. 2020/0272913 A1), which concerns a recommendation method and apparatus.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FARHAD AGHARAHIMI whose telephone number is (571)272-9864. The examiner can normally be reached M-F 9am - 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Apu Mofiz can be reached at 571-272-4080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FARHAD AGHARAHIMI/Examiner, Art Unit 2161
/APU M MOFIZ/Supervisory Patent Examiner, Art Unit 2161
