Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claims 1, 2, 4-20 have been examined.
Response to Arguments
Applicant's arguments with respect to the claims have been considered but are moot in view of the new ground(s) of rejection.
On page 22, 23, of the 9/8/25 Remarks, Applicant argues the cross layer and attention data and score features. See the added citations and explanation for this features in the rejection below.
Also, on page 24, Applicant states that Garimella does not disclose the low rank approximations and a cross-later of a deep and cross network. However, it is the combination of prior art that renders the features obvious in a 103 rejection. And, Wang already discloses the DCN and cross layer features. Garimella further shows how low rank matrices can be used to reduce data set size and computations needed.
Also, Claims are given their broadest reasonable interpretation in light of Applicant’s Spec. And, the actually stated claim language is what is interpreted.
Also, the 101 is still found to apply. The generic inputs/features into the DCN or the generic output of general recommendations are not considered specific enough of a technical solution for a technical problem to pass 101.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Independent Claims 1, 11, 16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims are in a statutory category of invention. However, the claims recite generating an input feature vector for a set of features by using an embedding layer of a residual deep and cross network (DCN); generating a first output feature vector representing explicit feature crosses of the input feature vector using a set of cross layers of a cross network of the residual DCN, with at least one cross layer comprising a set of attention data structures to generate attention scores for feature crosses of the set of features from the input feature vector, wherein the at least one cross layer comprises a set of low-rank matrices representing low-rank approximations of a full-rank weight matrix, the set of low-rank matrices comprising a first low-rank matrix representing a first subspace of the full-rank weight matrix and a second low-rank matrix representing a second subspace of the full-rank weight matrix; generating a prediction vector for the defined prediction task based, at least in part, on the first output feature vector; and providing a recommendation for a connections networking system based on the prediction vector.
This is considered in the Abstract Idea grouping of certain methods of organizing human activity - advertising, marketing or sales activities or behaviors. This judicial exception is not integrated into a practical application because the claim is directed to an abstract idea with additional generic computer elements. The claim uses AI but it is considered generic use of AI/machine learning. The generic inputs/features into the DCN or the generic output of general recommendations are not considered specific enough of a technical solution for a technical problem to pass 101. These are considered generic. The generically recited computer elements do not add a practical application or meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea on a computer.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional limitations only perform well-understood, routine, conventional computer functions as recognized by the court decisions listed in MPEP § 2106.05(d). Also, the additional hardware elements are: (i) mere instructions to implement the idea on a computer, and/or (ii) recitation of generic computer structure that serves to perform generic computer functions. Viewed separately or as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amounts to significantly more than the abstract idea itself. The claim does not provide significantly more than the identified abstract idea, in that there is no improvement to another technology or technical field, no improvement to the functioning of a computer, no application with, or by use of a particular machine, no transformation or reduction of a particular article to a different state or thing, no specific limitation other than what is well-understood, routing and conventional in the field, no unconventional step that confines the claim to a particular useful application, or meaningful limitations that amount to more than generally linking the use of the abstract idea to a particular technological environment. Therefore, the claims are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Dependent claims 2, 4, 7, 8, 14, 19 are not considered directed to any additional non-abstract claim elements. Rather, these claims offer further descriptive limitations of elements found in the independent claims and addressed above. While these descriptive elements may provide further helpful description for the claimed invention, these elements do not confer subject matter eligibility to the invention since their individual and combined significance is still not more than the abstract concepts identified in the claimed invention. Hence, these dependent claims are also rejected under 101.
The Not Listed dependent claims preceding are considered to pass 101 because of their more detailed use of Machine Learning in combination with their parent claims.
Please see the 35 USC 101 section at the Examination Guidance and Training Materials page on the USPTO website.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1, 2, 4-8, 10-14, 16-19 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wang (20240242127) in view of Garimella (20150170020).
Claims 1, 11, 16. Wang discloses a method, comprising:
generating an input feature vector for a set of features by using an embedding layer of a residual deep and cross network (DCN) ([6], also see cross and deep network at Figs. 6a, 6b);
generating a first output feature vector representing explicit feature crosses of the input feature vector using a set of cross layers of a cross network of the residual DCN, with at least one cross layer comprising a set of attention data structures to generate attention scores for feature crosses of the set of features from the input feature vector (see vector at [6, 12, 21]; see score and article/comment/rating at [11], see score at [147]; see cross and deep layers at Figs. 6a, 6b). And, Wang further discloses that the attention/recommendation and score at [11] uses a recommendation model that uses cross layers [12, 19, 41, 42] and also vectors and cross layers with the recommendation model [67]. Hence, Examiner interprets that the cross layer has data that is used for the attention/recommendation score.
Wang 127 does not explicitly disclose wherein the at least one cross layer comprises a set of low-rank matrices representing low-rank approximations of a full-rank weight matrix, the set of low-rank matrices comprising a first low-rank matrix representing a first subspace of the full-rank weight matrix and a second low-rank matrix representing a second subspace of the full-rank weight matrix. However, Examiner notes in Applicant Spec at [39] that a full rank matrix is a weight matrix. And, Wang 127 discloses ranked higher and lower [174] and parameter matrix [152] and a weight matrix [178, 180, 323, 326] and using matrices for recommending [152, 154, 178, 323] and vector multiplication [328] and cross and deep layers at Figs. 6a, 6b. And, Garimella discloses replacing a matrix with a lowrank approximation [13] and a weight matrix approximated as two low-rank matrices (Abstract, [12, 14, 24], Fig. 3). These citations from Garimella read on Applicant Spec at [41] which is cited as relevant by Applicant in the 9/4/25 interview. Therefore, it would have been obvious to one having ordinary skill in the art at the time the invention was made to add Garimella AI and low rank matrices approximating a full rank/weight matrix to Wang 127 AI and weight matrix and recommending and ranking. One would have been motivated to do this in order to more efficiently train because of large amounts of data and computation (as Garimella says at [12]).
Wang further discloses generating a prediction vector for the defined prediction task based, at least in part, on the first output feature vector (see prediction and vector at [118]); and
providing a recommendation for a connections networking system based on the prediction vector (see recommendation at [118]; comment or like at [11, 192] show the connections networking system).
Claim 2. Wang further discloses the method of claim 1, wherein the set of features comprises one or more numerical features, categorical features, categorical feature embeddings from a lookup table, dense embeddings, sparse identifier embeddings, or member history features defined for the connections networking system (see history at [4], see category and type at [11, 87], see plurality embedding vectors [216]).
Claim 4. Wang does not explicitly disclose the method of claim 1, wherein the set of attention data structures comprise an attention score matrix and a value matrix, the attention score matrix comprising a combination of a query matrix and a key matrix. However, Wang discloses using score for recommending [11, 147, 152] and using matrices for recommending [152, 154, 178, 323]. Therefore, it would have been obvious to one having ordinary skill in the art at the time the invention was made to add Wang’s use of matrices for recommending to Wang’s scores and recommending. One would have been motivated to do this in order to better use common AI techniques for recommending.
Claim 5, 12, 17. Wang 127 does not explicitly disclose the method of claim 1, comprising: generating a cross layer input feature vector based on the input feature vector; multiplying the cross layer input feature vector with the first low-rank matrix of the set of low-rank matrices to form a query matrix; multiplying the cross layer input feature vector with the first low-rank matrix of the set of low-rank matrices to form a key matrix; and multiplying the query matrix and the key matrix to form an attention score matrix for the at least one cross layer. However, Wang 127 discloses ranked higher and lower [174] and parameter matrix [152] and using matrices for recommending [152, 154, 178, 323] and vector multiplication [328]. And, Garimella discloses AI and low-rank matrices [12] and multiplying low rank matrices [12, 29, 36]. Therefore, it would have been obvious to one having ordinary skill in the art at the time the invention was made to add Garimella AI and low rank matrices features and AI to Wang 127 AI and recommending and ranking. One would have been motivated to do this in order to more efficiently train (as Garimella says at [12]).
Claim 6, 13, 18. Wang 127 does not explicitly disclose the method of claim 1, comprising generating a cross layer output feature vector by the at least one cross layer using a set of operations comprising: multiplying a first cross layer input feature vector with the first low-rank matrix of the set of low-rank matrices and an attention score matrix to form a first intermediate result; multiplying the first intermediate result with a second low-rank matrix of the set of low-rank matrices to form a second intermediate result; adding a bias vector to the second intermediate result to form a third intermediate result; multiplying the third intermediate result with the input feature vector to form a fourth intermediate result; and adding the first cross layer input feature vector to the fourth intermediate result via a residual connection to form the cross layer output feature vector, the residual connection comprising a skip connection. However, Wang 127 discloses ranked higher and lower [174] and parameter matrix [152] and using matrices for recommending [152, 154, 178, 323] and vector multiplication [328]. And, Garimella discloses AI and low-rank matrices [12] and multiplying low rank matrices [12, 29, 36] and a bias vector [16]. Therefore, it would have been obvious to one having ordinary skill in the art at the time the invention was made to add Garimella AI and low rank matrices and recommending to Wang 127 AI and recommending and ranking. One would have been motivated to do this in order to more efficiently train (as Garimella says at [12]).
Claim 7. Wang further discloses the method of claim 1, comprising: generating a first cross layer output feature vector by a first cross layer based on the input feature vector and a first cross layer input feature vector; generating a second cross layer output feature vector by a second cross layer based on the input feature vector and the first cross layer output feature vector; and providing the second cross layer output feature vector to an output layer of the cross network (see vector and network at [12, 31, 34, 36]).
Claim 8, 14, 19. Wang further discloses the method of claim 1, comprising: generating a second output feature vector representing implicit feature crosses of the input feature vector using a deep neural network (DNN) of the residual DCN; combining the first output feature vector and the second output feature vector into a final output feature vector by a final layer of the residual DCN; and generating the prediction vector based on the final output feature vector (see vector and network at [12, 31, 34, 36]).
Claim 10. Wang further discloses the method of claim 1, comprising: accessing a training dataset comprising a set of datapoints to train the set of cross layers of the cross network for the residual DCN, the set of datapoints comprising input feature vectors and output feature vectors, the input feature vectors representing a set of features for a connections networking system and the output feature vectors representing a set of feature crosses for the set of features; generating a candidate output feature vector for an input feature vector of a datapoint by the set of cross layers of the cross network (see train at [31, 40]; see vector and network at [12, 31, 34, 36]); determining a difference value between the candidate output feature vector and an output feature vector associated with the input feature vector of the datapoint; and updating the attention parameters for the attention data structures of the at least one cross layer based on the difference value and a loss function (see loss function and difference at “(3)Loss Function” and [179]; see loss and selection and update and recommendation at [43, 60, 61, 109]).
Claims 9, 15, 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wang (20240242127) in view of Garimella (20150170020) in view of Liu (20170178181).
Claim 9, 15, 20. Wang does not explicitly disclose the method of claim 8, comprising calibrating a set of predicted values from the prediction vector using a calibration model co-trained with the residual DCN using operations comprising: mapping the set of predicted values to a corresponding set of intervals associated with a set of calibrated scores using an isotonic calibration layer of a calibration model, the isotonic calibration layer using an isotonic regression function that is monotonically increasing or decreasing to preserve an order of the set of prediction values; and replacing the set of prediction values with the set of calibrated scores based on the mapping. However, Wang discloses AI and calibration and layers (see calibrate at [36, 200, 221]; see layers citations above). And Liu discloses AI and click thru prediction and (Title) and calibration mapping and isotonic regression. Therefore, it would have been obvious to one having ordinary skill in the art at the time the invention was made to add Liu’s common Ai techniques and predicting and calibrating to Wang’s Ai techniques and predicting and calibrating. One would have been motivated to do this in order to better calibrate.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
a) Wang 394 discloses low rank [75];
b) Garimella and Sainath discloses low rank matrices and multiplication.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARTHUR DURAN whose telephone number is (571)272-6718. The examiner can normally be reached Mon-Thurs, 7-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ilana Spar can be reached on (571) 270-7537. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARTHUR DURAN/Primary Examiner, Art Unit 3621 9/23/25