Detailed Action
This action is in response to the amendment filed on 03/23/2026 for application 17/688,409, in which:
Claims 1, 13, and 20 are independent claims.
Claims 1, 11, 13, and 20 are currently amended.
Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 03/23/2026 have been fully considered but they are not persuasive.
Regarding the 35 U.S.C. § 103 Rejections:
Applicant's arguments regarding the 35 U.S.C. § 103 rejections of the previous office action have been fully considered, but are unpersuasive.
Applicant traverses (Pages 8-10) the 103 rejections and requests reconsideration and withdrawal of the rejections. Amended independent claim recites "predict a value of the informational category for an ingested job listing using the neural network" and "receive implicit prediction feedback regarding the predicted value of the neural network" which are not taught by the prior art references. Paragraph 102 of Hanna discusses secondary training datasets. However, a training dataset is used to train a neural network. It is not actual data that is ingested by the trained neural network to make a prediction during inferencing operations. By way of contrast, claim 1 recites "predict a value of the informational category for an ingested job listing using the neural network" (where ingesting a job listing is taught within [0004] in the specification). The other cited references fail to remedy the deficiency of Hanna. Consequently, the cited references fail to provide an identical disclosure of at least this language of the claimed subject matter. Absence from the cited references of the above-mentioned claim elements negates the rejections. Accordingly, Applicant respectfully requests removal of the rejections with respect to the independent claims and dependent claims (due to dependency).
Examiner respectfully disagrees. The limitation of predict a value of the informational category for an ingested job listing using the neural network is taught by Wang as the Deep Internest Network teaches predicting values for the informational categories (ex. items within mother and infant; items within sports goods; items within books …). Thus, using the neural network (utilizing the Deep Internest Network) to make predictions based on user interest within an e-commerce dataset for a value of the informational categories. For the newly added limitation receive implicit prediction feedback regarding the predicted value of the neural network is now taught by a newly added reference (Liu) and the rejection has been updated. Applicant’s arguments regarding the other independent and dependent claims rely upon the same assertions as with respect to Claim 1, and are thus likewise unpersuasive. Therefore, for the reasons given above and in the rejections below, the rejection to all Claims (including Claim 1, similar independent claims, and all dependent Claims) are maintained. More specific details are discussed below within the 35 USC § 103 Rejections.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5-6, 9, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., “Research of Recommendation System Based on Deep Interest Network”, in view of Tran et al., “Predicting Job Titles from Job Descriptions with Multi-label Text Classification”, in view of Liu et al., “Neural Matrix Factorization Recommendation for User Preference Prediction Based on Explicit and Implicit Feedback”.
Regarding Claim 1:
Wang teaches:
A system comprising:
(see Wang, Page 5, [5. Summary], Paragraph 1, “In this paper, a recommendation system on deep interest network is proposed …”).
a non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the system to perform operations comprising:
(see Wang, Page 2, [2.2 Recommendation Method], Paragraph 1, “Wide-deep networks are trained models that have the ability to acquire memory and generalization ...”; Page 4, [4. Experiment], Paragraph 1, “TensorFlow, an open source code, is used for model training.”. The Recommendation System comprises trained models that acquire data from memory utilizing TensorFlow (machine learning software library used for training neural networks on multiple operating systems within a computer); thus, a non-transitory computer-readable medium which performs the Recommendation Method (interpreted as instructions) to perform operations).
obtain a training set of one or more … having a value of an informational category,
(see Wang, Page 4, [4. Experiment], “We use e-commerce data as training-testing set for one month, totaling 7 million, With time dimension as the criterion for training, the28 days of user data as the training set …”; Page 2, Table 1-4; [3.1 Feature Representation], “The features in UGC data of e-commerce platform are user features, user behavior, purchase items and context”. Training set includes e-commerce UGC (user-generated content) data with one or more e-commerce products within the dataset. Table 1 shows the types of data for the input characteristics from the e-commerce dataset to determine the recommendation based on informational category (types of products to recommend) shown in Table 2-4).
train a neural network using a machine-learning algorithm that performs operations, for each of the one or more … in the training set: obtain a plurality of sparse features from … ;
(see Wang, Page 1, [Abstract], “First, the user's purchased goods and the user's search are embedded coded, and the sparse features are transformed into low-dimensional dense features”; Page 2, Table 1; Page 3, Figure 1. Figure 1 shows the structure of the neural network (MLP) model which performs the operations of obtaining features to process and predict an output. Table 1 shows the sparse features within the categories: User Features, Purchase Items, and Context. They are interpreted as sparse features as they are one-hot/multi-hot types denoted in the table and not the embedded dense features such as User Behavior which is an embedded feature (dense feature) containing Purchase Items (sparse features)).
pass a first portion of the plurality of sparse features into an encoding layer of the neural network to convert the first portion of the plurality of sparse features into a first vector;
(see Wang, Page 2, [3.1 Feature Representation], Paragraph 2, “We transform these features into high-dimensional sparse features by encoding”; Page 3, [3.2 Baseline Model], Paragraph 2, “According to the different input vectors, the following operations are performed: 1) If ti is the j-th element ti[j] = 1 of one-hot vector, then the embedded vector of ti is represented by a single vector …”; Table 1; Figure 1 & 2. User Profile Features and Context Features are interpreted as a first portion of the plurality of sparse features (one-hot type sparse features not including the multi-hot type sparse features) are encoded into single vectors which is shown in Figures 1 & 2 and considered a first vector).
pass a second portion of the plurality of sparse features into an embedding layer of the neural network to embed the second portion of the plurality of sparse features into an embedding vector, the embedding layer comprising a …;
(see Wang, Page 3, [3.2 Baseline Model], Paragraph 2, “According to the different input vectors, the following operations are performed: … 2) If ti is multi-hot vector and ti[j] = 1 … then the embedded vector of ti is represented by a list of vectors …”; Table 1; Figure 2. Purchase Items are interpreted as a second portion of the plurality of sparse features (a multi-hot type sparse feature not including the one-hot type sparse features) into an embedding layer (as shown within Figure 2: Embedding Layer which outputs embedding vectors) as they are embedded within the User Behavior embedding vectors).
concatenate the first vector and the embedding vector into a concatenated vector, in a concatenated dense layer of the neural network;
(see Wang, Page 3, [3.2 Baseline Model], “Finally, all dense vectors are connected together and fed into MLP”. Figure 1 & 2. The concatenated vector is connected together in Figure 1 & 2 within the Concat & Flatten layer which connects all dense vectors; thus, a concatenated dense layer of the neural network).
pass the concatenated vector serially through a plurality of rectifier linear units (ReLUs),
(see Wang, Page 3, [3.2 Baseline Model], “Finally, all dense vectors are connected together and fed into MLP”. Figure 1 & 2. Once the dense vectors are combined, they are then fed into the MLP within a plurality of PReLUs (interpreted as a ReLUs as they are type of ReLUs named Parametric-ReLUs) which is shown in Figure 1 & 2).
each ReLU using a set of parameters to apply a function to an input of the ReLU; and
(see Wang, Figure 1 & 2. The PReLUs are Parametric-ReLUs that utilize learnable parameters to enhance model accuracy and convergence where the activation function is adaptable using the set of parameters).
use a softmax layer to predict a predicted value for the informational category for the … , based on output of the plurality of ReLUs;
(see Wang, Figure 1 & 2; Tables 2-4; Page 3, [3.2 Baseline Model], Paragraph 4, “… softmax function is used to output the predicted items (we only select five types of goods for prediction).”. Figure 1 & 2 shows the Softmax (5) layer within the model to predict a value for the categorical recommendation based on the previous ReLU layers. Tables 2-4 shows the 5 categories (which the examiner interprets as the informational categories as they provide insight/information on the categories that should be recommended) that are predicted based on the softmax layer).
predict a value of the informational category for an … using the neural network;
(see Wang, Figure 1 & 2; Tables 2-4. The Deep Internest Network (shown in Figure 2) is utilized for the recommendation system of Wang. As the predictions are for items within the informational categories (ex. items within mother and infant; items within sports goods; items within books …). Thus, using the neural network (utilizing the Deep Internest Network) to make predictions based on user interest within an e-commerce dataset for a value of the informational categories).
While Wang teaches the noted limitations of Claim 1 above within an e-commerce dataset. Wang fails to teach:
… ingested job listing …
training data set of one or more multilingual job listings … a job listing of the one or more multilingual job listings … and a multilingual pre-trained text encoder within the embedding layer … ; and
receive implicit prediction feedback regarding the predicted value of the neural network; and
retrain the neural network using the machine-learning algorithm to change a value for a parameter of the neural network based on the feedback
However, Tran teaches:
… ingested job listing …
(see Tran, Page 3, Fig. 3, Column 1, Paragraph 1, “… our task is defined as below, … Input: Job descriptions as texts … ”; Page 1, Column 1, Paragraph 2, “This task takes the input as a post of job descriptions, then predicts the suitable job titles for job description post as the output. … output for a job description post are not only one job title but also several of related job titles corresponding to the job description content …”. Fig. 3 indicates the architecture of the model where the Input layer is where the job descriptions are ingested as text values for predicting job titles based off job descriptions and several other titles associated with each job listing description; thus, interpreted by the examiner as ingested job listing).
… multilingual job listings …
(see Tran, Page 3, Table 1; Page 2, Column 2, Paragraph 4, “… in the dataset, the job descriptions are written in both Vietnamese and English”; Page 1, Column 1, Paragraph 1, “… recruiters find suitable candidates by posting job description and requirements for each job title”. Table 1 shows an example of job descriptions and associated job titles from the training dataset which exemplifies the multilingual (Vietnamese and English ) job listings (which is interpreted by the examiner as a request for an open position within an entity to find potential candidates; thus, the job postings within Tran) used to train the model for multi-label classification).
… a job listing of the one or more multilingual job listings …
(see Tran, Page 2, Column 2, Paragraph 4, “… in the dataset, the job descriptions are written in both Vietnamese and English”; Page 3, Fig. 3. Fig. 3 teaches taking input job descriptions for job postings (listings) in Vietnamese and English (multilingual); thus, a job listing (posting) of the one or more multilingual job listings is used within the input layer to obtain features and output a list of job titles suitable for the input job descriptions (Fig. 3)).
… multilingual pre-trained text encoder …
(see Tran, Page 3, Fig. 3; Page 4, Column 1, Paragraph 5, “For the transformers pre-trained models, we use bertmultilingual- cased for m-BERT …”. m-BERT is a pre-trained model for multiple languages (multilingual), including Vietnamese and English, which is implemented within embedding layer to encode the input to multiple languages via the transformer model).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the job listings, multilingual job listings, and multi-lingual text encoder disclosed and taught by Tran, in the system taught by Wang to increase performance by training with a dataset with records/listings/postings including job descriptions represented in multiple languages (see Tran, Page 4, Column 2, Paragraph 1, “… it can be seen that Transformer models obtained higher performance than pre-trained embeddings. … two multilingual models includes BERT and XLM-R showed the highest results … This is because our dataset contains both job description written in Vietnamese and English …”).
Wang/Trang does not explicitly disclose:
receive implicit prediction feedback regarding the predicted value of the neural network; and
retrain the neural network using the machine-learning algorithm to change a value for a parameter of the neural network based on the feedback
However, Liu teaches:
receive implicit prediction feedback regarding the predicted value of the neural network; and
(see Liu, Page 5, Column 1, Paragraph 1, “… to predict the degree of user preferences for non-interacted items and utilizes a new loss function based on explicit and implicit feedback proposed in this paper and the forward and backward propagation of the neural network model to update the relevant parameters of the EINMF model”; Page 5, Column 2, Paragraph 2, “ … the preference prediction value of the interaction between user u and item i based on explicit and implicit feedback …”. Liu teaches receiving both explicit AND implicit feedback regarding the neural network for the predicted values; thus, the examiner interprets the structure of the EINMF model to receive implicit prediction feedback regarding the predicted value of the neural network).
retrain the neural network using the machine-learning algorithm to change a value for a parameter of the neural network based on the feedback.
(see Liu, Page 5, Column 1, Paragraph 1, “… to predict the degree of user preferences for non-interacted items and utilizes a new loss function based on explicit and implicit feedback proposed in this paper and the forward and backward propagation of the neural network model to update the relevant parameters of the EINMF model”; Page 6, Column 2, Paragraph 2, “… we use both explicit feedback and implicit feedback information together in the objective optimization function to optimize and update the recommendation model parameters”. Based on the implicit feedback, the EINMF model taught by Liu updates the recommendation model’s parameters (change a value for a parameter of the neural network); thus, interpreted by the examiner as retraining the neural network as the model parameters are now updated based on the implicit feedback).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the system taught by Wang/Tran with Liu’s architecture which incorporates receiving feedback and incorporates retraining based on the feedback received to improve the model for predictions, improve accuracy, integration of different kinds of feedback, and preference prediction (see Liu, Page 10, Column 2, Paragraphs 3 “… To improve the accuracy of recommendation systems and enhance user satisfaction, this paper proposes a user preference prediction neural matrix factorization algorithm integrating explicit feedback and implicit feedback … and solve the defects of the current deep learning algorithm in training the model using only one feedback data … which improves the accuracy of the recommendation system for user preference prediction … implicit feedback data fusion method to further alleviate the sparse data …”).
Regarding Claim 3:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the embedding vector is a coordinate in an n-dimensional space.
(see Wang, Page 3, [3.2 Baseline Model], Paragraph 2, “Embedded layer are represented high-dimensional vectors. the embedded vector of ti is represented by a list of vectors …”; Table 1; Figure 1 & 2. Purchase Items (multi-hot vectors) are embedded within the User Behavior embedding vectors which is shown in Figures 1 & 2 and considered an embedding vector in an n-dimensional space (high-dimensional space as the set of vectors can combined/multiplied for concatenation)).
Regarding Claim 5:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the second portion of the plurality of sparse features includes …
Wang teaches the sparse features from the e-commerce dataset but does not explicitly disclose job description.
However, Tran explicitly discloses:
job description
(see Tran, Page 3, Table 1: Column 2 ‘Job description’.).
The motivation of Claim 1’s combination of Wang/Tran/Liu is still maintained.
Regarding Claim 6:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the operations further comprise: obtain one or more dense features from the job listing;
(see Wang, Page 1, [Abstract], “First, the user's purchased goods and the user's search are embedded coded …”; Page 2, Table 1. Table 1 shows the dense feature within the category User Behavior which is embedded with the sparse features of purchased goods (multi-hot type). User Behavior is interpreted as a dense feature as it is embedded with a list of vectors (purchase items/goods and search)).
concatenate the one or more dense features, the first vector, and the embedding vector into the concatenated vector.
(see Wang, Page 3, [3.2 Baseline Model], “Finally, all dense vectors are connected together and fed into MLP”. Figure 1 & 2. The concatenated vector is connected together in Figure 1 & 2 within the Concat & Flatten layer which connects all dense vectors; thus, a concatenated dense layer of the neural network).
Regarding Claim 9:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the encoding layer is a one-hot encoding layer.
(see Wang, Page 2, [3.1 Feature Representation], Paragraph 2, “We transform these features into high-dimensional sparse features by encoding”; Page 3, [3.2 Baseline Model], Paragraph 2, “According to the different input vectors, the following operations are performed: 1) If ti is the j-th element ti[j] = 1 of one-hot vector, then the embedded vector of ti is represented by a single vector …”; Table 1; Figure 1 & 2. User Profile Features and Context Features are one-hot type sparse features which are encoded into single vectors which is shown in Figures 1 & 2 and done within the embedding layer (specific type of encoding layer)).
Regarding Claim 13:
Claim 13 incorporate substantively all the limitations of Claim 1 in a method (see Wang, Page 2, [2.2 Recommendation Method]. The Recommendation System comprises trained models that acquire data from memory utilizing TensorFlow (machine learning software library used for training neural networks on multiple operating systems within a computer); thus, the Recommendation System of Wang utilizes a method for recommendation); and further recites no new limitations; thus, Claim 13 is rejected for reasons set forth in the rejections of Claim 1.
Claims 2, 4, 7-8, 10-12, and 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., “Research of Recommendation System Based on Deep Interest Network”, in view of Tran et al., “Predicting Job Titles from Job Descriptions with Multi-label Text Classification”, in view of Liu et al., “Neural Matrix Factorization Recommendation for User Preference Prediction Based on Explicit and Implicit Feedback”, in view of Hanna et al., US PG PUB 2021/0383308 A1.
Regarding Claim 2:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the operations further comprise:
… obtaining a plurality of sparse features, passing the first portion, passing the second portion, concatenating the first vector and the embedding vector to the concatenated vector, passing the concatenated vector, and using the softmax layer.
Wang and Tran teach the limitations of Claim 2. However, Wang and Tran do not explicitly disclose the determination of minimizing the loss function and repeating the process.
However, Hanna teaches:
determine if a loss function applied to the predicted value and to the value for the informational category has been minimized; and
(see Hanna, Fig. 3: 312 -> 215; Page 13, [0103], “At step 312, the process 300 includes analyzing output … comparing the output to an expected output and … computing one or more accuracy or error metrics (also referred to as loss functions)”; Page 13, [0105], “… in response to determining that the predetermined threshold is not met, the process 300 proceeds to step 306 … to reduce model error or otherwise improve the model performance …”. Fig 3. shows step 312 which does the comparison and computation of the loss functions for the predicted outputs (values) to the informational category. The computed loss functions are used to determine if a threshold has been met (reduce model error is interpreted as minimizing model error for the predicted output for the informational category)).
in response to a determination that the loss function has not been minimized, change values for a set of parameters and repeating the …
(see Hanna, Fig. 3: 315 -> 306; Page 12, [0095], “At step 306, the process 300 includes … adjusting one or more parameters to reduce an error metric … to reduce the error metrics, the model service 109 may perform actions … identifying … excluding one or more identified most-erroneous parameters … increasing and/or decreasing various parameter weights such that … error metrics may be reduce, and executing one or more loss function optimization algorithms”. If the threshold is not met within Fig 3: 315 then the method goes back to step 306 to configure/change/update parameters and repeats the process).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the determination of minimizing loss disclosed and taught by Hanna and repeating the process, in the System taught by Wang and Tran to reduce error (see Hanna, Page 13, [0105], “… in response to determining that the predetermined threshold is not met, the process 300 proceeds to step 306 … to reduce model error or otherwise improve the model performance …”).
Regarding Claim 4:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the first portion of the plurality of sparse features comprises one or more of …
Wang teaches the sparse features but does not disclose job company, job title, and job location.
However, the dataset of Hanna comprises:
job company, job title, and job location.
(see Hanna, Fig. 1; Page 6, [0051], “The model service 109 or data service 107 … generate input data for machine learning … the model service 109 performs entity resolution on location data for a plurality of locations to standardize terms such as position titles, company names …”; The Model/Data Service obtain and manipulate data within the Data Store (113) within Fig. 1. The dataset of Hanna includes company names (the examiner interprets as job company features) and position titles (the examiner interprets as job title features) by performing entity resolution on location data (117 which is interpreted as job location features)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the explicit disclosure of job company/title/location/description taught by Hanna, in the System taught by Wang and Tran for specific features to determine specific feature associations for prediction (see Hanna, Page 5, [0047], “… in response to a request for information on a particular job position, the data service 107 analyzes historical location data 117 and position data 119 and determines position titles, types, and tasks that were associated with the particular job position …”).
Regarding Claim 7:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the informational category is a …
Wang teaches the informational category from the e-commerce dataset (for the 5 categories) but does not disclose workplace type within a dataset about job listings.
However, the dataset of Hanna comprises:
workplace type
(Hanna, Fig. 1; Page 12, Column 2, [0093], “… training dataset for predicting a measurable metric for whether a job position is …”. The prediction that is generated is for the type of job position (which is interpreted by the examiner as workplace type as it includes remote, on-site, and hybrid)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the explicit disclosure of job company/title/location/description, and workplace type including on-site/remote/hybrid types taught by Hanna, in the System taught by Wang and Tran for specific features to determine specific feature associations for prediction (see Hanna, Page 5, [0047], “… in response to a request for information on a particular job position, the data service 107 analyzes historical location data 117 and position data 119 and determines position titles, types, and tasks that were associated with the particular job position …”).
Regarding Claim 8:
Wang/Tran/Liu/Hanna teach the system of Claim 7. Hanna further teaches:
wherein possible values for the workplace type include on-site, remote, and hybrid.
(see Hanna, Page 12, Column 2, [0093], “… training dataset for predicting a measurable metric for whether a job position is remote, on-site, or any degree between the two aforementioned states”. Job position includes on-site, remote, and hybrid (hybrid is interpreted by the examiner as any degree between the two states (remote/on-site))).
The motivation of Claim 7’s combination of Wang/Tran/Liu/Hanna is still maintained.
Regarding Claim 10:
Wang/Tran/Liu teach the system of Claim 1. Wang further teaches:
wherein the operations further comprise: train the embedding layer … into the machine-learning algorithm.
(see Wang, Page 4, [4 Experiment], “We use e-commerce data as training-testing set for one month, totaling 7 million, With time dimension as the criterion for training, the28 days of user data as the training set, the next 2 days of data as the test set.”; Figure 1 & 2. The concatenated vector is connected together which is utilized in the training of the neural networks (including the embedding layer) shown in Figures 1 & 2).
Wang teaches the training of the embedding layer but does not disclose pairs of job listings in a second training set.
However, Hanna teaches:
… by passing pairs of job listings in a second training set of job listings, each pair including a label indicative of similarity of the second portion of the plurality of sparse features between each job listing in the pair …
(see Hanna, Page 12, [0094], “… the model service 109 may generate a second training set including candidate and/or position data describing both known on-site job positions and known remote job positions … second training set may be absent information that identifies job positions therein as on-site job positions or remote job positions”; Page 5, [0049], “… the model service 109 generates and trains machine learning models for recommending if a particular task or job position can be performed remotely … the machine learning models can generate metric scores for various input data types ( e.g., work scores, talent scores, collaboration scores, remote work scores, location scores, etc.), and the machine learning models can generate a numerical score or a classification regarding whether a job can be performed remotely or not … model service 109 generates and trains machine learning models for classifying job descriptions (e.g., or information derived therefrom) into one or more categories or bins”; Page 12, [0092], “A training dataset (also referred to as a "teaching" dataset) can include labeled or unlabeled data … to identify particular job positions that are remote or non-remote based positions, a first training dataset and second training dataset are generated that includes a first portion including job position data describing known on-site job positions, and a second portion is generated that includes job position data describing known remote job positions”. The second training set contains categorical groups/bins (interpreted as pairing of similar job listings) with known job position data from the job listings which are labeled with various input data types to train the model including remote work scores (which are interpreted as labels indicative of similarity between each job listing within a bin/category)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the explicit disclosure of job company/title/location/description, workplace type including on-site/remote/hybrid types, and pairs of job listings in a second training set taught by Hanna, in the System taught by Wang and Tran for specific features to determine specific feature associations for prediction (see Hanna, Page 5, [0047], “… in response to a request for information on a particular job position, the data service 107 analyzes historical location data 117 and position data 119 and determines position titles, types, and tasks that were associated with the particular job position …”).
Regarding Claim 11:
Wang/Tran/Liu teach the system of Claim 1 but does not disclose explicitly disclose ingesting a job listing and predicting based off the ingested job listing.
However, Hanna teaches:
wherein the operations further comprise: obtain the ingested job listing from a third-party data source,
(see Hanna, Fig. 1: 103; Page 4, [0043], “The data service 107 can be configured to request, retrieve, and/or process data from data sources 103. In one example, the data service 107 is configured to automatically and periodically (e.g., every 6 hours, 3 days, 2 weeks, etc.) collect job descriptions from a database of a recruitment agency or from a company website”. The data service taught by Hanna retrieves data from multiple data sources (Fig. 1: 103) including 3rd party data sources such as a public website).
the ingested job listing lacking a value of the informational category; and predict the value of the informational category for the ingested job listing using the neural network.
(see Hanna, Page 13, [0102], “… the second training dataset may be unlabeled (e.g., absent information that identifies job positions therein as remote or on-site) … machine learning models may be trained to predict, from the second training dataset, how remotely a job can be performed”. Ingested job listings from a 3rd party data source may not include the informational category (job workplace type: on-site/remote/hybrid). Machine learning models are used to train and predict the informational category for the ingested job listing).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the explicit disclosure of job company/title/location/description, workplace type including on-site/remote/hybrid types, pairs of job listings in a second training set taught by Hanna, in the System taught by Wang and Tran for specific features to determine specific feature associations for prediction based off the ingested job listings (see Hanna, Page 5, [0047], “… in response to a request for information on a particular job position, the data service 107 analyzes historical location data 117 and position data 119 and determines position titles, types, and tasks that were associated with the particular job position …”).
Regarding Claim 12:
Wang/Tran/Liu/Hanna teach the system of Claim 11 and further teaches:
wherein the second portion comprises …
Wang teaches the second portion but does not explicitly disclose a sequence of text from the job listing.
However, Tran explicitly discloses:
a sequence of text from the job listing, the sequence of text comprising words in a language from a plurality of languages,
(Tran, Page 3, Table 1. Table 1: Job description column shows a sequence of text for a plurality of languages)
and the multilingual pre-trained text encoder is trained to predict a multilingual text embedding for the sequence of text.
(see Tran, Page 3, Fig. 3; Page 4, Column 1, Paragraph 5, “For the transformers pre-trained models, we use bertmultilingual- cased for m-BERT …”. m-BERT is a pre-trained model for multiple languages (multilingual), including Vietnamese and English, which is implemented within embedding layer to encode the input to multiple languages via the transformer model for the sequences of texts to find the optimal job titles).
The motivation of Claim 11’s combination of Wang/Tran/Liu/Hanna is still maintained.
Regarding Claims 14-19:
Claims 14-19 incorporate substantively all the limitations of Claims 2, 4, 7-8, and 11-12 in a method (see Wang, Page 2, [2.2 Recommendation Method]. The Recommendation System comprises trained models that acquire data from memory utilizing TensorFlow (machine learning software library used for training neural networks on multiple operating systems within a computer); thus, the Recommendation System of Wang utilizes a method for recommendation); and further recites no new limitations; thus, Claims 14-19 are rejected for reasons set forth in the rejections of Claims 2, 4, 7-8, and 11-12, respectively.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Wang et al., “Research of Recommendation System Based on Deep Interest Network”, in view of Tran et al., “Predicting Job Titles from Job Descriptions with Multi-label Text Classification”, in view of Hanna et al., US PG PUB 2021/0383308 A1, in view of Liu et al., “Neural Matrix Factorization Recommendation for User Preference Prediction Based on Explicit and Implicit Feedback”.
Regarding Claim 20:
A system comprising:
(see Wang, Page 5, [5. Summary], Paragraph 1, “In this paper, a recommendation system on deep interest network is proposed …”).
circuitry; and a memory storing instructions that causes the circuitry to:
(see Wang, Page 2, [2.2 Recommendation Method]. The Recommendation System comprises trained models that acquire data from memory utilizing TensorFlow (machine learning software library used for training neural networks on multiple operating systems within a computer); thus, the system utilizing TensorFlow implies that the software library is implemented on a computing device, in which a processor and memory are inherent within the circuitry);
…
obtain a dense feature … ;
(see Wang, Page 1, [Abstract], “First, the user's purchased goods and the user's search are embedded coded …”; Page 2, Table 1. Table 1 shows the dense feature within the category User Behavior which is embedded with the sparse features of purchased goods (multi-hot type). User Behavior is interpreted as a dense feature as it is embedded with a list of vectors (purchase items/goods and search)).
obtain a plurality of sparse features from … ;
(see Wang, Page 1, [Abstract], “First, the user's purchased goods and the user's search are embedded coded, and the sparse features are transformed into low-dimensional dense features”; Page 2, Table 1; Page 3, Figure 1. Figure 1 shows the structure of the neural network (MLP) model which performs the operations of obtaining features to process and predict an output. Table 1 shows the sparse features within the categories: User Features, Purchase Items, and Context. They are interpreted as sparse features as they are one-hot/multi-hot types denoted in the table and not the embedded dense features such as User Behavior which is an embedded feature (dense feature) containing Purchase Items (sparse features)).
convert a first portion of the plurality of sparse features into a first vector using an encoding layer of a neural network;
(see Wang, Page 2, [3.1 Feature Representation], Paragraph 2, “We transform these features into high-dimensional sparse features by encoding”; Page 3, [3.2 Baseline Model], Paragraph 2, “According to the different input vectors, the following operations are performed: 1) If ti is the j-th element ti[j] = 1 of one-hot vector, then the embedded vector of ti is represented by a single vector …”; Table 1; Figure 1 & 2. User Profile Features and Context Features are interpreted as a first portion of the plurality of sparse features (one-hot type sparse features not including the multi-hot type sparse features) are encoded into single vectors which is shown in Figures 1 & 2 and considered a first vector).
convert a second portion of the plurality of sparse features into an embedding vector using an embedding layer of the neural network, the embedding layer comprising … ;
(see Wang, Page 3, [3.2 Baseline Model], Paragraph 2, “According to the different input vectors, the following operations are performed: … 2) If ti is multi-hot vector and ti[j] = 1 … then the embedded vector of ti is represented by a list of vectors …”; Table 1; Figure 1 & 2. Purchase Items are interpreted as a second portion of the plurality of sparse features (a multi-hot type sparse feature not including the one-hot type sparse features) into an embedding layer as they are embedded within the User Behavior embedding vectors which is shown in Figures 1 & 2 and considered an embedding vector).
concatenate the first vector and the embedding vector into a concatenated vector using a concatenated dense layer of the neural network;
(see Wang, Page 3, [3.2 Baseline Model], “Finally, all dense vectors are connected together and fed into MLP”. Figure 1 & 2. The concatenated vector is connected together in Figure 1 & 2 within the Concat & Flatten layer which connects all dense vectors; thus, a concatenated dense layer of the neural network).
pass the concatenated vector as an input to a rectifier linear unit (ReLU) to produce an output;
(see Wang, Page 3, [3.2 Baseline Model], “Finally, all dense vectors are connected together and fed into MLP”. Figure 1 & 2. Once the dense vectors are combined, they are then fed into the MLP within a plurality of PReLUs (interpreted as a ReLUs as they are type of ReLUs named Parametric-ReLUs) which is shown in Figure 1 & 2 to produce an output).
predict a value for the informational category … by the neural network;
(see Wang, Figure 1 & 2; Tables 2-4. The Deep Internest Network (shown in Figure 2) is utilized for the recommendation system of Wang. As the predictions are for items within the informational categories (ex. items within mother and infant; items within sports goods; items within books …). Thus, the neural network (utilizing the Deep Internest Network) is used to make predictions based on user interest within an e-commerce dataset for a value of the informational categories).
While Wang teaches all limitations of Claim 20 within an e-commerce dataset. Wang fails to teach a … multilingual pre-trained text encoder within the embedding layer and:
… ingested job listing …
receive implicit prediction feedback regarding the predicted value of the neural network; and
retrain the neural network using the machine-learning algorithm to change a value for a parameter of the neural network based on the feedback.
However Tran explicitly discloses:
… ingested job listing …
(see Tran, Page 3, Fig. 3, Column 1, Paragraph 1, “… our task is defined as below, … Input: Job descriptions as texts … ”; Page 1, Column 1, Paragraph 2, “This task takes the input as a post of job descriptions, then predicts the suitable job titles for job description post as the output. … output for a job description post are not only one job title but also several of related job titles corresponding to the job description content …”. Fig. 3 indicates the architecture of the model where the Input layer is where the job descriptions are ingested as text values for predicting job titles based off job descriptions and several other titles associated with each job listing description; thus, interpreted by the examiner as ingested job listing).
… a multilingual pre-trained text encoder
(see Tran, Page 3, Fig. 3; Page 4, Column 1, Paragraph 5, “For the transformers pre-trained models, we use bertmultilingual- cased for m-BERT …”. m-BERT is a pre-trained model for multiple languages (multilingual), including Vietnamese and English, which is implemented within embedding layer to encode the input to multiple languages via the transformer model).
Wang in combination with Tran does explicitly disclose an ingested job listing but does not disclose missing a value for an informational category.
However, Hanna explicitly discloses:
retrieve an ingested job listing from a third-party data source, the ingested job listing missing a value for an informational category;
(see Hanna, Fig. 1: 103; Page 4, [0043], “The data service 107 can be configured to request, retrieve, and/or process data from data sources 103. In one example, the data service 107 is configured to automatically and periodically (e.g., every 6 hours, 3 days, 2 weeks, etc.) collect job descriptions from a database of a recruitment agency or from a company website”; Page 13, [0102], “… the second training dataset may be unlabeled (e.g., absent information that identifies job positions therein as remote or on-site) … machine learning models may be trained to predict, from the second training dataset, how remotely a job can be performed”. The data service taught by Hanna retrieves data from multiple data sources (Fig. 1: 103) including 3rd party data sources such as a public website. Ingested job listings from a 3rd party data source (such as a public website) may not include the informational category (job workplace type: on-site/remote/hybrid). Machine learning models are used to train and predict the informational category for the ingested job listing).
Wang/Tran/Hanna does not explicitly disclose:
receive implicit prediction feedback regarding the predicted value of the neural network; and
retrain the neural network using the machine-learning algorithm to change a value for a parameter of the neural network based on the feedback
However, Liu teaches:
receive implicit prediction feedback regarding the predicted value of the neural network; and
(see Liu, Page 5, Column 1, Paragraph 1, “… to predict the degree of user preferences for non-interacted items and utilizes a new loss function based on explicit and implicit feedback proposed in this paper and the forward and backward propagation of the neural network model to update the relevant parameters of the EINMF model”; Page 5, Column 2, Paragraph 2, “ … the preference prediction value of the interaction between user u and item i based on explicit and implicit feedback …”. Liu teaches receiving both explicit AND implicit feedback regarding the neural network for the predicted values; thus, the examiner interprets the structure of the EINMF model to receive implicit prediction feedback regarding the predicted value of the neural network).
retrain the neural network using the machine-learning algorithm to change a value for a parameter of the neural network based on the feedback.
(see Liu, Page 5, Column 1, Paragraph 1, “… to predict the degree of user preferences for non-interacted items and utilizes a new loss function based on explicit and implicit feedback proposed in this paper and the forward and backward propagation of the neural network model to update the relevant parameters of the EINMF model”; Page 6, Column 2, Paragraph 2, “… we use both explicit feedback and implicit feedback information together in the objective optimization function to optimize and update the recommendation model parameters”. Based on the implicit feedback, the EINMF model taught by Liu updates the recommendation model’s parameters (change a value for a parameter of the neural network); thus, interpreted by the examiner as retraining the neural network as the model parameters are now updated based on the implicit feedback).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the system taught by Wang to utilize the ingested job listings of Hanna, in the system taught by Wang which utilizes Tran’s multilingual job listings/pre-trained text encoder to reduce error. The combination of Wang/Tran/Hanna with Liu’s architecture incorporates receiving feedback and incorporates retraining based on the feedback received to improve the model for predictions, improve accuracy, integration of different kinds of feedback, and preference prediction (see Liu, Page 10, Column 2, Paragraphs 3 “… To improve the accuracy of recommendation systems and enhance user satisfaction, this paper proposes a user preference prediction neural matrix factorization algorithm integrating explicit feedback and implicit feedback … and solve the defects of the current deep learning algorithm in training the model using only one feedback data … which improves the accuracy of the recommendation system for user preference prediction … implicit feedback data fusion method to further alleviate the sparse data …”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM RAHMAN whose telephone number is (703)756-1646. The examiner can normally be reached M-F 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.R./ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights