DETAILED ACTION
Summary and Status of Claims
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is in response to Application No. 19/485,246 filed 11/17/2025 with a preliminary amendment filed therewith.
Claims 1-11 are pending.
Claim 8 is rejected under 35 U.S.C. 112(b).
Claims 1-3, 6, 8, 10, and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Menghani (US Patent Pub 2022/0222235).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Menghani (US Patent Pub 2022/0222235) in view of Tepper et al. (US Patent Pub 2020/0327365).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Menghani (US Patent Pub 2022/0222235) in view of Tristan et al. (US Patent Pub 2019/0095805).
Claims 7 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Menghani (US Patent Pub 2022/0222235) in view of Dema et al. (US Patent Pub 2022/0383157).
Priority
This application is a national stage of PCT/US2024/029948 filed 5/17/2024. The priority claim to US Provisional Application 63/502,842 filed 5/17/2023 is acknowledged.
Information Disclosure Statement
The information disclosure statement filed 11/17/2025 has been fully considered, initialed, and signed by the Examiner. A copy is attached to this Office action.
The information disclosure statement filed 12/1/2025 has been fully considered, initialed, and signed by the Examiner. A copy is attached to this Office action.
The information disclosure statement filed 12/9/2025 has been fully considered, initialed, and signed by the Examiner. A copy is attached to this Office action.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: In Fig. 2, reference 200 mentioned at para. 0071. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 8 is rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claim 8 recites “the embedding values” in line 4. There is lack of antecedent basis for this limitation in the claim.
Note on Prior Art Rejections
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Applicant may rely on the exception under 35 U.S.C. 102(b)(1)(A) to overcome this rejection under 35 U.S.C. 102(a)(1) by a showing under 37 CFR 1.130(a) that the subject matter disclosed in the reference was obtained directly or indirectly from the inventor or a joint inventor of this application, and is therefore not prior art under 35 U.S.C. 102(a)(1).
Claims 1-3, 6, 8, 10, and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Menghani (US Patent Pub 2022/0222235).
In regards to claim 1, Menghani discloses a computer-implemented method to generate embedding tables with improved computational efficiency, the method comprising:
obtaining, by a computing system comprising one or more computing devices, a set of feature data that corresponds to one or more items, wherein the set of feature data comprises a plurality of values for each of a plurality of features (Menghani at para. 0066-68, 0071)1;
establishing, by the computing system, a multiplexed embedding table in which at least two of the values that are respectively associated with at least two different features of the plurality of features are mapped to a shared vector in the multiplexed embedding table (Menghani at para. 0069-71, 0074)2; and
storing, by the computing system, a plurality of embeddings associated with the plurality of values in the multiplexed embedding table. Menghani at para. 0075.3
In regards to claim 2, Menghani discloses the computer-implemented method of claim 1, wherein the shared vector comprises a shared row. Menghani at para. 0044.4
In regards to claim 3, Menghani discloses the computer-implemented method of claim 1, wherein establishing, by the computing system, the multiplexed embedding table comprises using a hash function to map the plurality of values for each feature to vectors of the multiplexed embedding table, and wherein the hash function is permitted to generate inter- feature collisions. Menghani at para. 0050-51, 0058.5
In regards to claim 6, Menghani discloses the computer-implemented method of claim 3, wherein a same hash function is used for at least two of the features. Menghani at para. 0050-51.6
In regards to claim 8, Menghani discloses the computer-implemented method of claim 1, wherein a first value for a first feature of the plurality of features is mapped to multiple vectors in the multiplexed embedding table, and wherein retrieving a first embedding associated with the first value comprises concatenating the embedding values stored in the multiple vectors. Menghani at para. 0098-99.7
In regards to claim 10, Menghani discloses one or more non-transitory computer-readable media that store a multiplexed embedding table (Menghani at para. 0116), wherein the multiplexed embedding table was generated by the performance of operations comprising:
obtaining, by a computing system comprising one or more computing devices, a set of feature data that corresponds to one or more items, wherein the set of feature data comprises a plurality of values for each of a plurality of features (Menghani at para. 0066-68, 0071)8;
establishing, by the computing system, the multiplexed embedding table in which at least two of the values that are respectively associated with at least two different features of the plurality of features are mapped to a shared vector in the multiplexed embedding table (Menghani at para. 0069-71, 0074)9; and
storing, by the computing system, a plurality of embeddings associated with the plurality of values in the multiplexed embedding table. Menghani at para. 0075.10
In regards to claim 11, Menghani discloses a computing system comprising:
one or more processors (Menghani at para. 0103); and
one or more non-transitory computer-readable media (Menghani at para. 0116) that collectively store:
a machine-learned model (Menghani at para. 0026); and
instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
retrieving, by the computing system, an embedding from a multiplexed embedding table in which at least two values that are respectively associated with at least two different features of a plurality of features are mapped to a shared vector in the multiplexed embedding table (Menghani at para. 0069-71, 0074, 0096-99)11;
processing, by the computing system, the embedding with the machine- learned model to generate, as an output of the machine-learned model, a prediction (Menghani at para. 0099-100)12; and
providing, by the computing system, the prediction as an output. Menghani at para. 0100.13
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Menghani (US Patent Pub 2022/0222235) in view of Tepper et al. (US Patent Pub 2020/0327365) (Tepper).
In regards to claim 4, Menghani discloses the computer-implemented method of claim 3, but does not expressly disclose wherein the hash function comprises a 2-universal hash function.
Tepper discloses a system and method for storing data classes that represent a plurality of clusters. The method includes receiving input data and decomposing the data into feature vectors. Tepper at paras. 0018-19. The feature vectors are hashed for similarity calculations. Hashing techniques include 2-universal hashing schemes. Tepper at para. 0031.
Menghani and Tepper are analogous art because they are directed to the same field of endeavor of machine learning models and feature vectors.
At the time before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to modify Menghani by adding the features of wherein the hash function comprises a 2-universal hash function, as disclosed by Tepper.
The motivation for doing so would have been because 2 universal hash functions have predictable collision probabilities based on the table size.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Menghani (US Patent Pub 2022/0222235) in view of Tristan et al. (US Patent Pub 2019/0095805) (Tristan).
In regards to claim 5, Menghani discloses the computer-implemented method of claim 3, but does not expressly disclose wherein a different hash function is used for each feature.
Tristan discloses a decision system based on machine learned decision models using feature hashing models. The method includes taking an input data and generating feature vectors using hash functions. Some techniques utilize different hash functions to generate different feature vectors. Tristan at para. 0031.
Menghani and Tristan are analogous art because they are directed to the same field of endeavor of machine learning models and feature vectors.
At the time before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to modify Menghani by adding the features of wherein a different hash function is used for each feature, as disclosed by Tristan.
The motivation for doing so would have been to allow different models to look at different subsets of features and improve accuracy overall. Tristan at para. 0048.
Claims 7 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Menghani (US Patent Pub 2022/0222235) in view of Dema et al. (US Patent Pub 2022/0383157) (Dema).
In regards to claim 7, Menghani discloses the computer-implemented method of claim 1, but does not expressly disclose wherein a first value for a first feature of the plurality of features is mapped to a different number of vectors in the multiplexed embedding table than a second value for a second feature of the plurality of features.
Dema discloses a system and method for machine learning for data at scale. The method includes receiving raw input of a variety of different types for a variety of different purposes. For example, the raw data may be legal documents, radiology images, loan applications, or voice recordings (i.e., multiple different production systems). The raw data is used to extract feature data, which are indexed using multiple hash functions, which create overlapping hash values (i.e., indexed vectors). Dema at paras. 0013, 0020. Since features are hashed using multiple hash functions, which have one or more overlaps, the result would be a feature (i.e., first feature) having a different number of indexes (i.e., mapped to a different number of vectors) than another feature (i.e., second feature).
Menghani and Dema are analogous art because they are directed to the same field of endeavor of indexing features to embeddings.
At the time before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to modify Menghani by adding the features of wherein a first value for a first feature of the plurality of features is mapped to a different number of vectors in the multiplexed embedding table than a second value for a second feature of the plurality of features, as disclosed by Dema.
The motivation for doing so would have been to allow for sorting of features into buckets based on similarity and their hash values. Dema at para. 0021.
In regards to claim 9, Menghani discloses the computer-implemented method of claim 1, but does not expressly disclose wherein the feature data comprises multiple subsets of feature data associated with multiple different production systems.
Dema discloses a system and method for machine learning for data at scale. The method includes receiving raw input of a variety of different types for a variety of different purposes. For example, the raw data may be legal documents, radiology images, loan applications, or voice recordings (i.e., multiple different production systems).
At the time before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to modify Menghani by adding the features of wherein the feature data comprises multiple subsets of feature data associated with multiple different production systems, as disclosed by Dema.
The motivation for doing so would have been to provide support for different types of contexts. Dema at para. 0013.
Additional Prior Art
Additional relevant prior art are listed on the attached PTO-892 form. Some examples are:
Joglekar et al. (US Patent Pub 2021/0004693) discloses a system and method for real time generation of feature based label embeddings.
Barkan et al. (US Patent Pub 2021/0192000) discloses a system and method for searching for items using associated embedding vectors.
Wei et al. (US Patent Pub 2021/0264220) discloses a system and method for updating embedding tables for machine learning models.
Jha et al. (US Patent Pub 2022/0114644) discloses a system and method for a recommendation system using sparse feature encoding.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Examiner Michael Le whose telephone number is 571-272-7970 and fax number is 571-273-7970. The examiner can normally be reached Mon-Fri 9:30 AM – 6 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tony Mahmoudi can be reached on 571-272-4078. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL LE/Examiner, Art Unit 2163
/TONY MAHMOUDI/Supervisory Patent Examiner, Art Unit 2163
1 Features are obtained that correspond to items, such as text or speech. The features can be used to generate values for each of the plurality of features.
2 An embedding table is generated based on the feature values. The embedding table is generated by mapping feature values to index entries of the embedding table. Multiple different values can be mapped to the same index and associated embedding (i.e., multiplexed embedding table …).
3 The generated embedding table is stored.
4 The embedding table comprises entries, such as rows, to store an embedding (i.e., vector).
5 A hash function is used on the input values to generate an index where multiple values can map to the same index.
6 All the values are hashed in the same manner (i.e., same hash function).
7 Feature data, such as a bigram, can be embedded to multiple embedding indexes and each one can be individually decompressed (i.e., first value of a first feature … is mapped to multiple vectors …). The decompressed embeddings can be concatenated and provided as output (i.e., concatenating the embedding values stored in the multiple vectors).
8 Features are obtained that correspond to items, such as text or speech. The features can be used to generate values for each of the plurality of features.
9 An embedding table is generated based on the feature values. The embedding table is generated by mapping feature values to index entries of the embedding table. Multiple different values can be mapped to the same index and associated embedding (i.e., multiplexed embedding table …).
10 The generated embedding table is stored.
11 Embeddings are retrieved from the embedding table, which allows for multiple different features to be mapped to a shared vector.
12 An output, in the form of a prediction, is generated based on processing the retrieved embeddings.
13 Then generated output (i.e., prediction) is provided.
