Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Procedural Summary
This is responsive to the claims filed 6/20/2024.
Claims 1-21 are pending.
Signed copies of the IDS’ are attached.
The Drawings filed 6/20/2024 are noted.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 7, 8, 16 & 21 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Pedersen et al. (U.S. Pub. No.: 2024/0382854 A1).
Regarding Claims 1 & 16: Pederson discloses a system, (Abstract) comprising: one or more processors executing one or more neural networks, (¶ 33); and one or more memories to store multimodal input data comprising a plurality of input modalities, (¶¶ 28, 219, 220), and to store a corresponding plurality of latent representations of the input modalities, (e.g., ¶ 113, 177); wherein the one or more neural networks, (¶ 122) are configured to: process a combined representation of the multimodal input data and the latent representations to generate an intermediate representation having a feature set that is larger than a feature set of the combined representation, (¶¶ 111, 220); and apply one or more fuser layers to the intermediate representation to generate a fused multimodal representation of the multimodal input data and the latent representations as output for downstream processing, (¶¶ 111, 220).
Regarding Claims 7 & 21: Pederson discloses wherein the plurality of latent representations comprises a plurality of substantially disentangled latent representations of the input modalities, (¶¶ 177, 182-186).
Regarding Claim 8: Pederson discloses applying to the intermediate representation one or more of a group that includes a normalization layer, (¶¶ 116, 191), a cross-attention layer, or a multilayer perceptron (MLP).
Allowable Subject Matter
Claims 9-15 are allowed.
Claims 2-6 & 17-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding 35 USC § 101: The claimed invention is drawn to processes and systems for fusing multimodal data used to generate NPCs in virtual simulations. This is not an abstract idea. Additionally, the claimed invention is rooted in computing technology and improvements thereto: For example, the claimed invention is directed to processing combined representations of multimodal input data and applying fused layers to an intermediate representation, to generate content for downstream processing.
Regarding the prior art: The closest prior art, Pedersen et al. (U.S. Pub. No.: 2024/0382854 A1) fails to disclose, suggest or render obvious, in combination with the other claimed limitations:
Per Claims 2, 3, 6, 17 & 18: wherein processing the combined representation comprises processing the combined representation via one or more Mixture of Experts (MoE) encoders that each comprises a router layer and a plurality of heterogeneous expert layers.
Per Claims 4, 5, 19 & 20: tokenizing the multimodal input data to generate mode-specific tokenized representations of the multimodal input data; and processing the tokenized representation of the multimodal input data by one or more processing layers.
Per Claims 9-15: One or more processors executing one or more neural networks; and one or more memories to store multimodal input data comprising a plurality of input modalities, wherein the one or more neural networks are configured to process the combined representation via one or more Mixture of Experts (MoE) encoder blocks to generate an intermediate representation; apply one or more fuser layers to the intermediate representation to generate a fused multimodal representation of the multimodal input data.
Conclusion
Additional Relevant References: See 892
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OMKAR A DEODHAR whose telephone number is (571)272-1647. The examiner can normally be reached on M-F, generally 9am-5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Lewis can be reached on 571-272-7673. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/OMKAR A DEODHAR/Primary Examiner, Art Unit 3715