Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/11/2026 has been entered.
Response to Amendments
Claims 1-20 remain pending in the application.
Claims 1-3, 5, 8-10, 12, and 15-17 have been amended.
Argument 1, regarding the prior art rejections, applicant argues that the combination of Ying, Perry, and Agravante does not teach “obtaining at least one target resource from the candidate resource set based on the target recommendation model and the preference feature, wherein each target resource of the at least one target resource is obtained using simultaneously both the first target recommendation model corresponding to the channel preference feature and the second target recommendation model corresponding to the content preference feature” because Agravante recites using hierarchical machine learning to perform a sequence of tasks. Examiner respectfully disagrees because Agravante recites multiple blocks within the hierarchical reinforcement learning structure may be executed concurrently (see Agravante P0089). Thus, the combination of the recommendation models taught by Ying and Perry with the hierarchical reinforcement learning structure of Agravante may result in the recommendation models operating concurrently and not sequentially.
Applicant also argues that none of the cited art teaches “wherein: the first target recommendation model is trained based on a sample channel feature and a first enhancement value set of first enhancement values, wherein each of the first enhancement values is obtained based on click information of a sample pushed-resource, the second target recommendation model is trained based on a sample content feature and a second enhancement value set of second enhancement values, wherein each of the second enhancement values is obtained based on (1) click information of the sample pushed-resource and (2) at least one of reading duration information, diversity information, or novelty information of the sample pushed-resource”. Examiner respectfully disagrees because Perry teaches wherein: the first target recommendation model is trained based on a sample channel feature and a first enhancement value set of first enhancement values, wherein each of the first enhancement values is obtained based on click information of a sample pushed-resource, (parameters of a model may be adjusted based on a marketing channel, such as based on product-based marketing, audience-based marketing, and the like. Parameters may be adjusted based on values associated with the marketing channel including clicks per sales for a search engine or tracking visits per new audience member for a social network. P0062). Ying teaches the second target recommendation model is trained based on a sample content feature and a second enhancement value set of second enhancement values, wherein each of the second enhancement values is obtained based on (1) click information of the sample pushed-resource and (2) at least one of reading duration information, diversity information, or novelty information of the sample pushed-resource (resource pushing model may be adjusted based on user click history associated with resources, click action data of users, resource diversity, and reading behavior of a user, page 17 paragraphs 8-13, page 18 paragraphs 1-2, page 10 paragraphs 7-8, page 11 paragraphs 1-2, page 16 paragraph 1, page 2 paragraphs 1-2 of background).
The full prior art rejections are outlined below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6-9, 13-16 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ying et al (Pub. No.: CN 112182460 B), hereafter Ying, in view of Perry et al (Pub. No.: US 20200294108 A1), hereafter Perry and Agravante et al (Pub. No.: US 20200034704 A1), hereafter Agravante.
Regarding claims 1, 8, and 15, Ying teaches obtaining a target recommendation model and a preference feature and a candidate resource set corresponding to a target object, the preference feature comprising at least… a content preference feature (User preferences may be analyzed such that to be pushed resources meet the preference of the user, page 15, full paragraph 2), the target recommendation model comprising … a second target recommendation model (Machine learning model in the form of a target vector model may be used to recommend to a user resources to be pushed, page 2, full paragraph 2), and the candidate resource set comprising at least one candidate resource (“wherein each target resource in the target resource set belongs to a plurality of training samples, target vector model output, the resource vector to be pushed corresponding to the to-be-pushed resource vector and historical resource vector similarity is higher than the second threshold value”, page 3, paragraph 2), … the second target recommendation model is trained based on a sample content feature and a second enhancement value set of second enhancement values, wherein each of the second enhancement values is obtained based on (1) click information of the sample pushed-resource and (2) at least one of reading duration information, diversity information, or novelty information of the sample pushed-resource (resource pushing model may be adjusted based on user click history associated with resources, click action data of users, resource diversity, and reading behavior of a user, page 17 paragraphs 8-13, page 18 paragraphs 1-2, page 10 paragraphs 7-8, page 11 paragraphs 1-2, page 16 paragraph 1, page 2 paragraphs 1-2 of background); obtaining at least one target resource from the candidate resource set based on the target recommendation model and the preference feature, wherein each target resource of the at least one target resource is obtained using simultaneously both the first target recommendation model corresponding to the channel preference feature and the second target recommendation model corresponding to the content preference feature (“obtaining the resource to be pushed from the target resource set, wherein each target resource in the target resource set belongs to a plurality of training samples; the target vector model output, the resource vector to be pushed corresponding to the to-be-pushed resource vector and historical resource vector similarity is higher than the second threshold value; pushing the resource to be pushed to the target object; each training sample used by the training initial vector model comprises a plurality of sample resources of different types of the training object browsed in the second time period; the similarity between the plurality of sample resource vector corresponding to a plurality of sample resources in the same training sample is higher than the first threshold value, so that the obtained to-be-pushed resource not only can meet the user preferences”, page 4, paragraph 1. The resources to be pushed are obtained using both the initial vector model as well as the target vector model, meaning more than one model may be used in conjunction to obtain the resources); and pushing the at least one target resource to the target object (“a pushing module, used for pushing the resource to be pushed to the target object”, page 3, paragraph 2).
Ying does not appear to explicitly teach “a channel preference feature and…
a first target recommendation model…wherein: the first target recommendation model is trained based on a sample channel feature and a first enhancement value set of first enhancement values, wherein each of the first enhancement values is obtained based on click information of a sample pushed-resource”.
Perry teaches a channel preference feature and (Machine learning model may be used to recommend what channel may be used to deliver content based on user preferences, P0053)… a first target recommendation model (Machine learning model may be used to recommend a channel to the user, with different channels having corresponding content such as emails or web content, the recommendation based on user preferences, P0053)… wherein: the first target recommendation model is trained based on a sample channel feature and a first enhancement value set of first enhancement values, wherein each of the first enhancement values is obtained based on click information of a sample pushed-resource (parameters of a model may be adjusted based on a marketing channel, such as based on product-based marketing, audience-based marketing, and the like. Parameters may be adjusted based on values associated with the marketing channel including clicks per sales for a search engine or tracking visits per new audience member for a social network. P0062).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Ying and Perry before them, to include Perry’s specific teaching of a machine learning model being used to recommend channels in Ying’s method of resource pushing. One would have been motivated to make such a combination of a machine learning model being used to recommend a channel to a user (see Perry P0053), and using a machine learning model to determine type of pushing to be performed on the to be pushed resource in order to meet the user’s preferences (see Ying page 15, full paragraph 2).
Ying in view of Perry does not appear to explicitly teach “…that are trained as two levels of a hierarchical reinforcement learning process”.
Agravante teaches …that are trained as two levels of a hierarchical reinforcement learning process (two distinct neural networks are trained using hierarchical reinforcement learning, P0037, claims 1 and 5).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Ying, Perry, and Agravante before them, to include Agravante’s specific teaching of two distinct neural networks being trained using hierarchical reinforcement learning in Ying’s method of resource pushing. One would have been motivated to make such a combination of using constraints to reduce training time of reinforcement learning (see Agravante P0043), and using a machine learning model to determine type of pushing to be performed on the to be pushed resource in order to meet the user’s preferences (see Ying page 15, full paragraph 2).
Regarding claims 2, 9, and 16, Ying in view of Perry and Agravante teaches the limitations of claims 1, 8, and 15 as outlined above. Ying further teaches obtaining a training sample set, the training sample set comprising at least one training sample, the training sample comprising …, a sample content feature, and feedback information corresponding to at least one sample pushed-resource; and training an initial recommendation model based on …, the sample content feature, and the feedback information in the training sample, to obtain the target recommendation model, the initial recommendation model comprising a first initial recommendation model and a second initial recommendation model (vector model may be trained with a set of training samples comprising a plurality of sample resources of different types of training objects, obtaining a to be pushed resource for the user, ultimately to improve user satisfaction of the pushed resource, page 15, full paragraph 2).
Perry further teaches the training sample comprising the sample channel feature… training an initial recommendation model based on the sample channel feature (training data used to train the model may include channel data, P0069).
Regarding claims 6, 13, and 19, Ying in view of Perry and Agravante teaches the limitations of claims 1, 8, and 15 as outlined above. Perry further teaches obtaining at least one target channel from a candidate channel set based on the first target recommendation model and the channel preference feature, one candidate resource corresponding to one candidate channel, and the candidate channel set comprising candidate channels corresponding to candidate resources in the candidate resource set (Machine learning model may be used to recommend a channel to the user, with different channels having corresponding content such as emails or web content, the recommendation based on user preferences, P0053); and … the content preference feature and the at least one target channel (Machine learning model may be used to recommend what channel may be used to deliver content based on user preferences, P0053).
Ying further teaches obtaining the at least one target resource from the candidate resource set based on the second target recommendation model (vector model compares each sample resources in the set of sample resources to a threshold value such that the to be pushed resource meets the preferences of the user, page 15, full paragraph 2).
Regarding claims 7, 14, and 20, Ying in view of Perry and Agravante teaches the limitations of claims 1, 8, and 15 as outlined above. Ying further teaches obtaining at least one target content from a candidate content set based on the second target recommendation model and the content preference feature, one candidate resource corresponding to one candidate content, and the candidate content set comprising candidate contents corresponding to candidate resources in the candidate resource set (“each training sample used by the training initial vector model comprises a plurality of sample resources of different types of training object browsed in the second time period, the similarity between a plurality of sample resource vectors corresponding to a plurality of sample resources in the same training sample is higher than the first threshold value; so that the obtained to-be-pushed resource not only meets the preference of the user, and can have a plurality of types, solves the problem that the resource push method in the related technology to the user push resource is too single, enriches the type of pushing the resource to the user, improves the user satisfaction of the push resource”, page 15, full paragraph 2); and obtaining the at least one target resource from the candidate resource set based on the first target recommendation model (vector model recommends a resource by comparing the resource to a threshold value dependent upon user preferences, page 15, full paragraph 2).
Perry further teaches the channel preference feature, and the at least one target content (Machine learning model may be used to recommend what channel may be used to deliver content based on user preferences, P0053).
Claims 3-5, 10-12, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Ying in view of Perry and Agravante and further in view of Wang et al (Pub. No.: US 20180374147 A1), hereafter Wang.
Regarding claims 3, 10, and 17, Ying in view of Perry and Agravante teaches the limitations of claims 1, 8, and 15 as outlined above. Ying further teaches the second initial recommendation model comprises a second initial recommendation sub- model and a second initial evaluation sub-model (Machine learning model in the form of a target vector model may be used to recommend to a user resources to be pushed, page 2, full paragraph 2);… obtaining at least one initial content recommendation result based on the sample content feature in the training sample and the second initial recommendation sub-model (vector model may be trained with a set of training samples comprising a plurality of sample resources of different types of training objects, obtaining a to be pushed resource for the user, ultimately to improve user satisfaction of the pushed resource, page 15, full paragraph 2); obtaining a second evaluation value set for the at least one initial content recommendation result based on the second initial evaluation sub-model (vector similarity between different resources may be obtained, page 11, paragraph 3); … updating a parameter of the second initial recommendation sub-model based on the second evaluation value set (vector similarity may be used to update parameters of the trained vector model, page 11, paragraph 3);
Perry further teaches wherein the first initial recommendation model comprises a first initial recommendation sub-model and first initial evaluation sub-model (Machine learning model may be used to recommend what channel may be used to deliver content based on user preferences, P0053),… and the training an initial recommendation model based on the sample channel feature, the sample content feature, and the feedback information in the training sample comprises: obtaining the first enhancement value set and the second enhancement value set based on the feedback information in the training sample (a marketing effectiveness normalization factor, such as a multiplier, may be used to re-engage an audience to meet product goals. An example is tracking and normalizing clicks per sales for a search engine such as Google or tracking visits per new audience member for a social network such as Facebook, P0062); obtaining at least one initial channel recommendation result based on the sample channel feature in the training sample and the first initial recommendation sub-model (Machine learning model may be used to recommend a channel to the user, with different channels having corresponding content such as emails or web content, the recommendation based on user preferences, P0053); obtaining a first evaluation value set for the at least one initial channel recommendation result based on the first initial evaluation sub-model (Merchant constraints may be used to evaluate content recommendation, P0068);…updating a parameter of the first initial recommendation sub-model based on the first evaluation value set (Model inputs may include merchant parameters that may be updated based on merchant constraints, P0068).
Ying in view of Perry does not appear to explicitly teach obtaining a channel loss function based on the first enhancement value set and the first evaluation value set; obtaining a content loss function based on the second enhancement value set and the second evaluation value set; obtaining a target loss function based on the channel loss function and the content loss function; and updating a parameter of the first initial evaluation sub-model and a parameter of the second initial evaluation sub-model based on the target loss function.
Wang teaches obtaining a channel loss function based on the first enhancement value set and the first evaluation value set (Click-through rate loss function is calculated. CTR loss function is expanded upon and includes content loss, the content being a user interaction with an advertisement and the price of the advertisement. P0009, P0013, P0067-P0068); obtaining a content loss function based on the second enhancement value set and the second evaluation value set (Click-through rate loss function is calculated. CTR loss function is expanded upon and includes loss related to the advertisement used to deliver website content to the user, with different advertisement slots being interpreted as channels, P0009, P0013, P0067-P0068); obtaining a target loss function based on the channel loss function and the content loss function (CTR loss function is expanded upon and includes content loss, the content being a user interaction with an advertisement and the price of the advertisement. The advertisement is used to deliver website content to the user, with different advertisement slots being interpreted as channels. P0009, P0013, P0067, P0068); and updating a parameter of the first initial evaluation sub-model and a parameter of the second initial evaluation sub-model based on the target loss function (Model parameter may be adjusted based upon the loss function which takes into account content loss dependent upon different advertisement slots, or channels, P0065-P0071).
Accordingly, it would have been obvious to a person having ordinary skill in the
art before the effective filing date of the claimed invention, having the teachings of
Ying, Perry, Agravante, and Wang before them, to include Wang’s specific teachings of channel, content, and target loss functions in Ying’s system of Object Pushing. One would have been motivated to make such a combination of calculating loss for content recommendations (see Wang P0009, P0013, P0067, P0068), and adjusting parameters of a trained vector model based on vector similarity in order to improve satisfaction of the resource of the user to push (see Ying page 16, paragraph 5).
Regarding claims 4, 11, and 18, Ying in view of Perry, Agravante, and Wang teaches the limitations of claims 3, 10, and 17 as outlined above. Ying further teaches wherein the training sample further comprises the at least one sample pushed-resource (“obtaining the resource to be pushed from the target resource set, wherein each target resource in the target resource set belongs to a plurality of training samples”, page 4, paragraph 1).
Wang further teaches and the obtaining a target loss function based on the channel loss function and the content loss function comprises: obtaining at least one of a click-through rate (CTR) loss function or a similarity loss function based on the at least one initial content recommendation result and the at least one sample pushed-resource in the training sample (Click-through rate loss function is calculated, P0013, P0067-P0068); and obtaining the target loss function based on the at least one of the CTR loss function or the similarity loss function, as well as the channel loss function and the content loss function (CTR loss function is expanded upon and includes content loss, the content being a user interaction with an advertisement and the price of the advertisement. The advertisement is used to deliver website content to the user, with different advertisement slots being interpreted as channels. P0009, P0013, P0067, P0068).
Regarding claims 5 and 12, Ying in view of Perry, Agravante, and Wang teaches the limitations of claims 3 and 10 as outlined above. Perry further teaches obtaining at least one of reading duration information, diversity information, or novelty information of the sample pushed-resource and click/tap information of the sample pushed-resource based on the feedback information in the training sample (tracking and normalizing clicks per sales for a search engine such as Google, P0062); obtaining a first enhancement value corresponding to the sample pushed-resource based on the click/tap information of the sample pushed-resource (a marketing effectiveness normalization factor, such as a multiplier, may be used to re-engage an audience to meet product goals, P0062); obtaining a second enhancement value corresponding to the sample pushed-resource based on the at least one of the reading duration information, the diversity information, or the novelty information of the sample pushed-resource and the click/tap information of the sample pushed-resource (a marketing effectiveness normalization factor, such as a multiplier, may be used to re-engage an audience to meet product goals. An example is tracking visits per new audience member for a social network such as Facebook, P0062); using a set of first enhancement values respectively corresponding to sample pushed- resources as the first enhancement value set (tracking and normalizing clicks per sales pitch for a search engine such as Google may be used to optimize a product-based marketing channel to gain an initial product goal, P0062); and using a set of second enhancement value respectively corresponding to the sample pushed-resources as the second enhancement value set (tracking visits per new audience member for a social network such as Facebook may be used to optimize a product-based marketing channel to gain an initial product goal, P0062).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHAN MOUNDI whose telephone number is (703)756-1547. The examiner can normally be reached 8:30 A.M. - 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights