DETAILED ACTION
The Response filed on 10/23/2025 has been correspondingly accepted and considered in the office action. Claims 1-46 are pending and Claims 1, 12, 27, 34 and 41 are independent and amended.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The claim limitation interpretation of Claims 1, 7, 34, 41, 43 and 44 under 35 U.S.C. 112(f) has been withdrawn in view of Applicant’s amendments.
The rejections to Claims 1-46 under 35 U.S.C. § 101 as being directed to Abstract Idea have been withdrawn in view of Applicant’s amendments to the claims and persuasive arguments.
Response to Arguments
Claims 1-46 stand rejected under 35 U.S.C. § 103. Applicant’s arguments with respect to Claims 1-46 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In order to expedite prosecution, and as to the material from the Specifications that are not in the Claim and are argued by the Applicant, please note Formal et al. ("SPLADE: Sparse lexical and expansion model for first stage ranking." Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, hereinafter, Formal).
For at least the supra provided reasons, Applicant's arguments have been fully considered but they are not persuasive.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-8, 11-22, 27-38 and 41-46 are rejected under 35 U.S.C. 103 as being unpatentable over Boytsov et al. (U.S. Pub. No. 2022/0253447, hereinafter, Boytsov) in view of Formal.
Regarding Claim 1,
Boytsov discloses a computer-implemented neural information retrieval model for performing an information retrieval task in response to a query, the neural information retrieval model comprising (Boytsov, Fig.1, par [094], "…a search retrieval system 100…"; par [095], "…When a user submits a query (i.e., a search request to find indexed documents) using the user device (e.g., desktop, laptop, smartphone, etc.) 112..."): a query encoder implemented by the processor (Boytsov, Fig.5, par [129], "…the machine 500 includes at least one processor 502...") configured to receive the query and generate a representation of the query over the vocabulary (Boytsov, Fig.1, par [095], "…When a user submits a query (i.e., a search request to find indexed documents) using the user device…"; Fig.2, paras [098-099], "…context-independent embeddings 206 for query tokens…", "…a Transformer model (e.g., BERT) produces contextualized query token embeddings 212…");
a comparator block implemented by the processor and configured to compare the generated representation of the query to the generated sparse representations of the one or more documents to generate a set of respective document scores and output a rank for the one or more documents based on the generated set of document scores, wherein the output rank identifies a set of documents in the information retrieval task (Boytsov, Fig.2, par [102], "…embeddings 212, 214, and 216 are fed into an interaction neural network, which produces a final query-document ranking score 220. The interaction neural network 218 may be a multi-layer interaction neural network 218.");
wherein the document encoder and the query encoder are respectively separate encoders (Boytsov, Fig.2, par [099], "…an optional neural network such as a Transformer model (e .g., BERT) produces contextualized query token embeddings 212 and contextualized document token embedding 216..."; note the separate query and document token embedding path).
Boytsov discloses a document/query encoders comprising a pretrained language model layer (par [099], "…an optional neural network such as a Transformer model (e .g., BERT)..."), but does not explicitly discloses the limitation "the document encoder being configured to receive one or more documents and generate a sparse representation for each of the documents predicting term importance of the document over a vocabulary."
However, Formal, in the analogous field of endeavor, discloses the document encoder being configured to receive one or more documents and generate a sparse representation for each of the documents predicting term importance of the document over a vocabulary (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...We consider the importance 𝑤𝑖i𝑗 of the token 𝑗 (vocabulary) for a token 𝑖 (of the input sequence)"; 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation...");
Therefore, It would have been obvious to one of ordinary skill in the art, before effective filing date of the claimed invention, to have modified the interaction layer of a neural network in information retrieval systems of Boytsov with explicit sparsity regularization and a log-saturation effect on term weight for the neural ranker of Formal with a reasonable expectation of success to boost efficiency of the information retrieval and reduce the computation cost while maintaining the desirable properties of dense retrieval models based on BERT Siamese models or bag-of-word models such as the exact matching of terms and the efficiency of inverted indexes (Formal, Abstract, 2 Related Works).
Regarding Claim 2,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Formal further discloses wherein the document encoder and the query encoder are differentiated from one another by one or more of model architecture, model size, model weights, model training, model regularization, model hyperparameters, or model location within the neural information retrieval model (Formal, 3 SPARSE LEXICAL REPRESENTATIONS FOR FIRST-STAGE RANKING, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…We propose to combine the best of both worlds for end-to-end training of sparse, expansion-aware representations of documents and queries...Where Lreg is a sparse regularization (l1 or lFLOPS).We use two distinct regularization weights for queries and documents – allowing to put more pressure on the sparsity for queries, which is critical for fast retrieval...").
Regarding Claim 3,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Formal further discloses wherein the document encoder and the query encoder are trained using a different regularizer; and wherein the query encoder is regularized using L1 regularization, and the document encoder is regularized using FLOPS regularization (Formal, 3 SPARSE LEXICAL REPRESENTATIONS FOR FIRST-STAGE RANKING, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation...To obtain a well-balanced index, Paria et al. [20] introduce the FLOPS regularizer, a smooth relaxation of the average number of floating-point operations necessary to compute the score of a document, and hence directly related to the retrieval time…Overall loss. We propose to combine the best of both worlds for end-to-end training of sparse, expansion-aware representations of documents and queries...Where Lreg is a sparse regularization (l1 or lFLOPS).We use two distinct regularization weights for queries and documents – allowing to put more pressure on the sparsity for queries, which is critical for fast retrieval...").
Regarding Claim 4,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Formal further discloses wherein an architecture of the query encoder is smaller than an architecture of the document encoder (Boytsov, paras [098-099], "…network first produces context independent embeddings 206 for query tokens...", "...an optional neural network such as a Transformer model (e .g., BERT) produces contextualized document token embedding 216..."); and
Formal further discloses wherein the document encoder is configured for document expansion within the vocabulary, and the query encoder is not configured for query expansion within the vocabulary (Formal, "…by modeling implicit or explicit (latent, contextualized) expansion mechanisms – similarly to standard expansion models in IR – these models can reduce the vocabulary mismatch...we propose the SParse Lexical AnD Expansion (SPLADE) model, based on a logarithmic activation and sparse regularization. SPLADE performs an efficient document expansion [1, 16], with competitive results with respect to complex training pipelines for dense models...").
Regarding Claim 5,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Formal further discloses wherein the query encoder is more efficient than the document encoder; and wherein efficiency is gained by one of (i) reducing how many layers form part of the query encoder, (ii) reducing the query encoder to a tokenizer, and (iii) regularizing query representation (2 Related Works, "…Our SPLADE model relies on such regularization, as well as other key changes, that boost both the efficiency and the effectiveness of this type of models..."; 3.2 SPLADE: Sparse Lexical AnD Expansion model, "...Where Lreg is a sparse regularization (l1 or lFLOPS).We use two distinct regularization weights for queries and documents – allowing to put more pressure on the sparsity for queries, which is critical for fast retrieval...").
Regarding Claim 6,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Boytsov further discloses wherein the query encoder comprises a pretrained language model implemented by the processor that is more efficient than the pretrained language model of the document encoder (Boytsov, par [092], "…In a retrieval system, a neural ranking model can be applied to queries and documents to produce estimates of relevance scores", "…a neural lexical translation model (e.g., IBM Model 1), which can be applied to state-of-the-art neural networks such as BERT…"); and
Formal further discloses wherein the efficiency is gained by using FLOPS regularization during pretraining or middle training (Formal, 2 Related Works, "…Our SPLADE model relies on such regularization, as well as other key changes, that boost both the efficiency and the effectiveness of this type of models..."; 3.2 SPLADE: Sparse Lexical AnD Expansion model, "...Where Lreg is a sparse regularization (l1 or lFLOPS).We use two distinct regularization weights for queries and documents – allowing to put more pressure on the sparsity for queries, which is critical for fast retrieval...").
Regarding Claim 7,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Boytsov further discloses wherein the document encoder receives a document as a tokenized input sequence, wherein the tokenized input sequence is tokenized using the vocabulary (Boytsov, par [005], "…The operations include receive a query and documents, tokenize the query into a sequence of query tokens and tokenize, for each document, the documents into a sequence of document tokens...";
wherein the pretrained language model layer is configured to embed each token in the tokenized input sequence with contextual features (Boytsov, Fig.3 step 314, par [0105], "…optionally passed through a contextualizing neural network 310. Contextualized Document Embeddings 314") and
Formal further discloses to predict an importance with respect to each token of the embedded input sequence over the vocabulary by transforming the context embedded tokens using one or more linear layers (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...We consider the importance 𝑤𝑖i𝑗 of the token 𝑗 (vocabulary) for a token 𝑖 (of the input sequence)"); and
wherein the document encoder further comprises: a representation layer implemented by the processor and configured to receive the predicted importance with respect to each token over the vocabulary and obtain the predicted term importance of the input sequence over the vocabulary (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...We consider the importance 𝑤𝑖i𝑗 of the token 𝑗 (vocabulary) for a token 𝑖 (of the input sequence)"; 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation..."),
said representation layer comprising a concave activation layer configured to perform a concave activation of the predicted importance over the embedded input sequence (Formal, 1 Introduction, 5 Conclusion, "…the SParse Lexical AnD Expansion (SPLADE) model, based on a logarithmic activation (i.e., concave activation) and sparse regularization...");
wherein the representation layer outputs the predicted term importance of the input sequence as the representation of the input sequence over the vocabulary (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...We consider the importance 𝑤𝑖i𝑗 of the token 𝑗 (vocabulary) for a token 𝑖 (of the input sequence)… The final representation is then obtained by summing importance predictors over the input sequence tokens, after applying ReLU to ensure the positivity of term weights...").
Regarding Claim 8,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1.
Formal further discloses wherein the pretrained language model of the document encoder is trained by middle training before the language model is fine-tuned for information retrieval; wherein the middle training occurs subsequent to pretraining the pretrained language model for predicting, or the middle training occurs concurrently with pretraining the pretrained language model for predicting to provide enhanced pretraining; and wherein the middle training or enhanced pretraining comprises training the LM using masked language model (MLM) training combined with FLOPS regularization (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...Note that Eq. 1 is equivalent to the MLM prediction, thus it can be also be initialized from a pre-trained MLM model..."; 3 SPARSE LEXICAL REPRESENTATIONS FOR FIRST-STAGE RANKING, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation...To obtain a well-balanced index, Paria et al. [20] introduce the FLOPS regularizer, a smooth relaxation of the average number of floating-point operations necessary to compute the score of a document, and hence directly related to the retrieval time...using ℓFLOPS thus pushes down high average term weight values, giving rise to a more balanced index...").
Regarding Claim 11,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1, wherein the sparse representation for each of the documents predicting term importance of the document over the vocabulary is a high-dimensional vector with more than half of its elements having a zero-value (Formal, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation...").
Claim 12 is a method claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale. Rationale for combination is similar to that provided for Claim 1.
Claim 13 is a method claim with limitations similar to the limitations of Claim 2 and is rejected under similar rationale.
Claim 14 is a method claim with limitations similar to the limitations of Claim 3 and is rejected under similar rationale.
Claim 15 is a method claim with limitations similar to the limitations of Claim 4 and is rejected under similar rationale
Claim 16 is a method claim with limitations similar to the limitations of Claim 4 and is rejected under similar rationale
Regarding Claim 17,
Boytsov in view of Formal discloses the method of claim 12.
Formal further discloses wherein said generating generates the sparse representation using concave activation functions combined with regularization (Formal, 1 Introduction, 5 Conclusion, "…the SParse Lexical AnD Expansion (SPLADE) model, based on a logarithmic activation (i.e., concave activation) and sparse regularization...").
Claim 18 is a method claim with limitations similar to the limitations of Claim 7 and is rejected under similar rationale.
Claim 19 is a method claim with limitations similar to the limitations of Claim 7 and is rejected under similar rationale.
Claim 20 is a method claim with limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Regarding Claim 21,
Boytsov in view of Formal discloses the method of claim 20, wherein the pretrained language model is pretrained for predicting; and wherein the middle training or enhanced pretraining comprises training the pretrained LM using masked language model (MLM) training combined with FLOPS regularization (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...Note that Eq. 1 is equivalent to the MLM prediction, thus it can be also be initialized from a pre-trained MLM model..."; 3 SPARSE LEXICAL REPRESENTATIONS FOR FIRST-STAGE RANKING, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation...To obtain a well-balanced index, Paria et al. [20] introduce the FLOPS regularizer, a smooth relaxation of the average number of floating-point operations necessary to compute the score of a document, and hence directly related to the retrieval time...using ℓFLOPS thus pushes down high average term weight values, giving rise to a more balanced index...").
Regarding Claim 22,
Boytsov in view of Formal discloses the method of claim 21, wherein the middle training or enhanced pretraining is based on a loss comprising: a standard MLM loss; an MLM loss over a sparse set of logits; and a FLOPS regularization loss (Formal, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Overall loss. We propose to combine the best of both worlds for end-to-end training of sparse, expansion-aware representations of documents and queries..."; see Eq (6) for overall loss).
Claim 27 is a computer-implemented method claim reciting all limitations of Claim 12 and is rejected under similar rationale. Additionally,
Formal further discloses wherein the pretrained language model of the document encoder is trained by middle training before the language model is fine-tuned for information retrieval (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...We consider the importance 𝑤𝑖i𝑗 of the token 𝑗 (vocabulary) for a token 𝑖 (of the input sequence)"; 3.2 SPLADE: Sparse Lexical AnD Expansion model, "...Where Lreg is a sparse regularization (l1 or lFLOPS).We use two distinct regularization weights for queries and documents – allowing to put more pressure on the sparsity for queries, which is critical for fast retrieval...").
Rationale for combination is similar to that provided for Claim 1.
Claim 28 is a method claim with limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Regarding Claim 29,
Boytsov in view of Formal discloses the method of claim 27, wherein the query encoder comprises an additional pretrained language model implemented by the processor, and where the additional pretrained language model of the query encoder is trained by middle training or enhanced pretraining before the language model is fine-tuned for information retrieval (Formal, SparTerm, "…SparTerm predicts term importance – in BERT WordPiece vocabulary (|V|=30522) – based on the logits of the Masked Language Model (MLM) layer...Note that Eq. 1 is equivalent to the MLM prediction, thus it can be also be initialized from a pre-trained MLM model..."; 3 SPARSE LEXICAL REPRESENTATIONS FOR FIRST-STAGE RANKING, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Learning Sparse representation...To obtain a well-balanced index, Paria et al. [20] introduce the FLOPS regularizer, a smooth relaxation of the average number of floating-point operations necessary to compute the score of a document, and hence directly related to the retrieval time...using ℓFLOPS thus pushes down high average term weight values, giving rise to a more balanced index...").
Claim 30 is a method claim with limitations similar to the limitations of Claim 22 and is rejected under similar rationale.
Claim 31 is a method claim with limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Claim 32 is a method claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale.
Claim 33 is a method claim with limitations similar to the limitations of Claim 3 and is rejected under similar rationale.
Claim 34 is a method claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale. Additionally,
Formal further discloses optimizing, using the processor, a loss including a ranking loss based on the generated representations of the one or more documents and queries and at least one regularization loss; wherein the ranking loss and/or the at least one regularization loss is weighted by a weighting parameter Formal, 3.2 SPLADE: Sparse Lexical AnD Expansion model, "…Overall loss. We propose to combine the best of both worlds for end-to-end training of sparse, expansion-aware representations of documents and queries..."; see Eq (6) for overall loss).
...
Rationale for combination is similar to that provided for Claim 1.
Claim 35 is a method claim with limitations similar to the limitations of Claim 2 and is rejected under similar rationale.
Regarding Claim 36,
Boytsov in view of Formal discloses the method of claim 34, wherein the at least one regularization loss is determined based on different regularizers for the document encoder and the query encoder; wherein the query encoder is regularized using L1 regularization, and the document encoder is regularized using FLOPS regularization; and wherein the query encoder comprises a pretrained language model that is more efficient than the pretrained language model of the document encoder (Boytsov, paras [098-099], "…network first produces context independent embeddings 206 for query tokens...", "...an optional neural network such as a Transformer model (e .g., BERT) produces contextualized document token embedding 216..."; Formal, 2 Related Works, "…Our SPLADE model relies on such regularization, as well as other key changes, that boost both the efficiency and the effectiveness of this type of models..."; 3.2 SPLADE: Sparse Lexical AnD Expansion model, "...Where Lreg is a sparse regularization (l1 or lFLOPS).We use two distinct regularization weights for queries and documents – allowing to put more pressure on the sparsity for queries, which is critical for fast retrieval...").
Claim 37 is a method claim with limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Claim 38 is a method claim with limitations similar to the limitations of Claim 21 and is rejected under similar rationale.
Claim 41 is a method claim with limitations similar to the limitations of Claim 1 and is rejected under similar rationale. Additionally,
Boytsov discloses a computer-implemented method for training a neural encoder for an information retrieval task (paras [071, 103], training mode and training), the method comprising:
…
Rationale for combination is similar to that provided for Claim 1.
Claim 42 is a method claim with limitations similar to the limitations of Claim 8 and is rejected under similar rationale.
Claim 43 is a method claim with limitations similar to the limitations of Claim 7 and is rejected under similar rationale.
Claim 44 is a method claim with limitations similar to the limitations of Claim 7 and is rejected under similar rationale.
Regarding Claim 45,
Boytsov in view of Formal discloses the method of claim 44, wherein the encoder comprises a document encoder implemented by the processor, and the document encoder is incorporated into the neural information retrieval model for information retrieval (Boytsov, Fig.2, par [099], "…an optional neural network such as a Transformer model (e .g., BERT) produces contextualized query token embeddings 212 and contextualized document token embedding 216..."; note the separate query and document token embedding path).
Regarding Claim 46,
Boytsov in view of Formal discloses the method of claim 44, wherein the encoder comprises a query encoder, and the query encoder is incorporated into the neural information retrieval model for information retrieval (Boytsov, Fig.2, par [099], "…an optional neural network such as a Transformer model (e .g., BERT) produces contextualized query token embeddings 212 and contextualized document token embedding 216..."; note the separate query and document token embedding path).
Claims 9, 23-24 and 39-40 are rejected under 35 U.S.C. 103 as being unpatentable over Boytsov in view of Formal further in view of Hofstatter et al. ("Improving efficient neural ranking models with cross-architecture knowledge." arXiv preprint arXiv:2010.02666, 2020, hereinafter, Hofstatter).
Regarding Claim 9,
Boytsov in view of Formal discloses the neural information retrieval model of claim 1, wherein the neural information retrieval model is trained using optimization including one or more hyperparameters (Formal, 1 Introduction, "…(1) we build upon SparTerm [1], and show that a mild tuning of hyperparameters brings improvements...");
Boytsov in view of Formal discloses optimization including hyperparameters, but does not explicitly discloses the limitations, "wherein the hyperparameters are selected based on predetermined query and document sizes; and wherein the neural information retrieval model is trained using distillation."
Hofstatter, in the analogous field of endeavor, discloses wherein the hyperparameters are selected based on predetermined query and document sizes (Hofstatter, 3 Cross-Architecture Knowledge Distillation, pg.4, "…We train ranking models on batches containing triples of queries Q, relevant passages P+, and non-relevant passages P-. We utilize the output margin of the teacher model Mt as label to optimize the weights of the student model Ms (i.e., hyperparameter)… "; Fig.2: Knowledge Distillation Process, "…we utilize the same training triples for every step…"); and
wherein the neural information retrieval model is trained using distillation (Hofstatter, 1 Introduction, pg.1, "… we propose a model-agnostic training procedure using cross-architecture knowledge distillation from BERTCAT with the goal to improve the effectiveness of efficient passage ranking models without compromising their query latency benefits…").
Therefore, it would have been obvious to one of ordinary skill in the art, before effective filing date of the claimed invention, to have modified the multi-layered neural ranker with term-importance measures taught by Boytsov in view of Formal with a cross-architecture training of neural rankers adapting knowledge distillation of Hofstatter with a reasonable expectation of success to achieve low query latency and significantly improve re-ranking effectiveness without compromising their efficiency. (Hofstatter, Abstract).
Claim 23 is a method claim with limitations similar to the limitations of Claim 9 and is rejected under similar rationale.
Claim 24 is a method claim with limitations similar to the limitations of Claim 9 and is rejected under similar rationale.
Claim 39 is a method claim with limitations similar to the limitations of Claim 9 and is rejected under similar rationale.
Claim 40 is a method claim with limitations similar to the limitations of Claim 9 and is rejected under similar rationale.
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Boytsov in view of Formal further in view of Khattab et al. ("Colbert: Efficient and effective passage search via contextualized late interaction over BERT." Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 2020, hereinafter, Khattab).
Regarding Claim 26,
Boytsov in view of Formal discloses the method of claim 12, but does not discloses the limitations of the claim. However, Khattab, in the analogous field of endeavor, discloses wherein the document encoder generates the sparse representations for at least a subset of the one or more received documents while offline (Khattab, Fig.3: The general architecture of ColBERT given a query q and a document d, 3.4 Offline Indexing: Computing & Storing Document Embeddings, "…ColBERT isolates almost all of the computations between queries and documents to enable pre-computing document representations offline..."); and
wherein the query encoder generates the representation of the received query while online (Fig.3, 3.1 Architecture, (a) a query encoder fQ).
Therefore, It would have been obvious to one of ordinary skill in the art, before effective filing date of the claimed invention, to have modified the multi-layered neural ranker with term-importance measures taught by Boytsov in view of Formal with a late interaction architecture of ColBERT that independently encodes the query and document of Khattab with a reasonable expectation of success to fine-tune deep language model and pre-compute document representations offline, considerably speeding up query processing for efficient information retrieval (Khattab, Abstract).
Allowable Subject Matter
Claims 10 and 25 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Bai et al. ("Sparterm: Learning term-based sparse representation for fast text retrieval." arXiv preprint arXiv:2010.00768 (2020) discloses the problem of transferring the deep knowledge of the pre-trained language model (PLM) to Term-based Sparse representations, aiming to improve the representation capacity of bag-of-words(BoW) method for semantic-level matching and a novel framework SparTerm to directly learn sparse text representations in the full vocabulary space.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JANGWOEN LEE whose telephone number is (703)756-5597. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BHAVESH MEHTA can be reached at (571)272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JANGWOEN LEE/Examiner, Art Unit 2656
/BHAVESH M MEHTA/Supervisory Patent Examiner, Art Unit 2656