DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 5 and 17 are objected to because of the following informalities: “one or more of: an accelerometer, a gyroscope, a magnetometer” recited in claim 5, ln. 1-2 and claim 17, ln. 1-2 should likely read “one or more of: an accelerometer, a gyroscope, and a magnetometer”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 9, analyzed as representative claim:
[Step 1] Claim 9 recites in part “A method”, which falls within the “process” statutory category of invention.
[Step 2A – Prong 1] The claim recites a series of steps which can practically be performed by one or more humans through mental process (i.e., observation, evaluation, judgment, and/or opinion) (see MPEP 2106.04(a)(2)(III)).
Claim 9 recites: A method, comprising:
accessing image data, motion data, and audio data;
providing the image data, the motion data, and the audio data to a machine learning (ML) model as input;
receiving, from the ML model in response to the input, an output indicating a player input to a computer game; and
in response to receiving the output, providing the player input to the computer game.
The underlined limitations, under their broadest reasonable interpretation, encompasses a mental process (i.e., visually observing image, motion, and audio data and mentally determining a player input based on the visually observed data), and thus the claim recites an abstract idea.
[Step 2A – Prong 2] The claim fails to recite additional limitations to integrate the abstract idea into a practical application. The recitation of a ML model for receiving input data (the image, motion, and audio data) and providing an output is recited at a high level of generality such that it amounts to no more than mere instructions to implement a generic computing component to perform the abstract idea and/or generally link the abstract idea to a particular technological environment (i.e., machine learning environment) (see MPEP 2106.05(f) & (h)).
Additionally, and/or alternatively, the limitations of providing the accessed data to a machine learning (“ML”) model, receiving an output from the ML model in the form of a player input to a computer game, and providing the output (player input) to the computer game are directed to insignificant extra-solution activity (data gathering and/or data transmission) which do not integrate the abstract idea into a practical application (see MPEP 2106.05(g)).
There is no indication that the combination of elements improves the functioning of a computer or other technology (see MPEP 2106.05(a)), recites a “particular machine” to apply or use the abstract idea (see MPEP 2106.05(b)), recites a particular transformation of an article to a different thing or state (see MPEP 2106.05(c)), or recites any other meaningful limitation (see MPEP 2106.05(e)). Accordingly, the claim is directed to the abstract idea.
[Step 2B] As discussed above with respect to integration of the abstract idea into a practical application, the claim does not further include additional elements that are sufficient to amount to significantly more than the judicial exception. Instead, the additional elements recite instructions to implement the abstract idea using a generic computing component as a tool to perform the abstract idea, generally link the abstract idea to a particular technological environment, and/or insignificant extra-solution activity.
Furthermore, the Specification demonstrates that the additional element of a machine learning model is recited for its well-understood, routine, and conventional functionality, wherein the Specification refers to the element in a manner that indicates that it is sufficiently well-known that the Specification does not need to describe the particulars of such additional element to satisfy enablement (see Specification, p. 12, “Present principles may employ various machine learning models, including deep learning models. Machine learning models consistent with present principles may use various algorithms trained in ways that include supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms, which can be implemented by computer circuitry, include one or more neural networks, such as a convolutional neural network (CNN), a recurrent neural network (RNN), and a type of RNN known as a long short-term memory (LSTM) network. Generative pre-trained transformers (GPTT) also may be used. Support vector machines (SVM) and Bayesian networks also may be considered to be examples of machine learning models. In addition to the types of networks set forth above, models herein may be implemented by classifiers.”).
Therefore, the claim is not patent eligible.
Independent claim 1 recites an apparatus with the additional limitations of at least one processor system configured to perform the limitations of claim 9 discussed above, while independent claim 14 recites an apparatus comprising the additional limitations of at least one computer readable storage medium that is not a transitory signal and comprising instructions executable by a processor system to perform the limitations of claim 9, analyzed above. These additional limitations are recited at a high level of generality such that they do not amount to a particular machine or technical improvement thereof, nor do they represent an improvement in other technology. Rather, the generic manner in which the additional elements are claimed amount to mere instructions to implement the abstract idea in a computer environment and/or to utilize generic computer components as tools to perform the abstract idea (see MPEP 2106.05(f) & (h)). The Specification further demonstrates that the additional elements are recited for their well-understood, routine, and conventional functionality, wherein the Specification refers to the elements in a manner that indicates that they are sufficiently well-known that the Specification does not need to describe the particulars of such additional elements to satisfy enablement (see Specification, pp. 5-6, “A processor may be a single- or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, and control lines and registers and shift registers. A processor including a digital signal processor (DSP) may be an embodiment of circuitry. A processor system may include one or more processors acting independently or in concert with each other to execute an algorithm, whether those processors are in one device or more than one device.”; p. 11, “at least one tangible computer readable storage medium 56 such as disk-based or solid-state storage”). Thereby, claims 1 and 14 are also not patent eligible.
Claims 2-8, 9-13, and 15-20 are dependent on claims 1, 9, and 14, respectively, and therefore recite the same abstract idea noted above. While the dependent claims may have a narrower scope than the independent claims, the claims fail to recite additional limitations that would integrate the abstract idea into a practical application or provide significantly more. For example:
Claims 2, 10, and 15 recite additional insignificant extra-solution activity (i.e., data display) and/or an additional abstract idea (e.g., certain methods of organizing human activity – specifically managing personal behavior or relationships or interactions between people – including social activities, teaching, and following rules or instructions (i.e., instructions to run a computer game consistent with the mentally determined player input)).
Claims 3-8, 11-13, and 16-20 further define how the accessed (visually observed) data is originally gathered (e.g., via camera, inertial measurement unit (i.e., accelerometer, gyroscope, and/or magnetometer), and microphone), thus describing the insignificant extra-solution activity of data gathering and/or implementation of the abstract idea of visually observing data using generic components performing their routine functions, as well as further defining the ML model.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-6, 9-11, and 14-18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Crabtree et al. (U.S. Pub. 2025/0352907 A1) (hereinafter “Crabtree”).
Regarding claim 1, Crabtree discloses an apparatus (Fig. 2; [0012]; [0117]; [0744]; [0747]; [0757], platform implemented using one or more computing devices comprising a processor and a memory), comprising:
at least one processor system (Fig. 2; [0012]; [0117]; [0744]; [0747]; [0757]) configured to:
access image data, motion data, and audio data ([0079]; [0121-0122]; [0125]; [0702], visual inputs (status images and real-time video streams), speech recognition, and motion and positional data);
provide the image data, the motion data, and the audio data to a machine learning (ML) model as input ([0121-0122]; [0125]; [0140]; [0224]; [0231]; [0241], the multi-modal inputs are provided to ML models);
receive, from the ML model in response to the input, an output indicating a player input to a computer game ([0079]; [0121-0122]; [0125]; [0702-0706], wherein the multi-modal data is fused to create a coherent and rich sensory experience that is updated in the game world (e.g., a virtual reality cooking simulation)); and
in response to receiving the output, provide the player input to the computer game (Fig. 32; [0079]; [0121-0122]; [0125]; [0702-0706], the processed multi-modal input is used to coordinate updates to the game world).
Regarding claim 2, Crabtree further discloses wherein the at least one processor system is configured to execute the computer game consistent with the player input (Fig. 32; [0079]; [0121-0122]; [0125]; [0702-0706], render output to the player).
Regarding claim 3, Crabtree further discloses wherein the image data is received from a camera, and wherein the image data comprises video ([0121], the visual input may be real-time video streams captured by a camera).
Regarding claim 4, Crabtree further discloses wherein the motion data is received from an inertial measurement unit (IMU) ([0125], the motion data captured from motion tracking systems, such as accelerometers and gyroscopes).
Regarding claim 5, Crabtree further discloses wherein the IMU comprises one or more of: an accelerometer, gyroscope, a magnetometer ([0125], e.g., accelerometers and gyroscopes).
Regarding claim 6, Crabtree further discloses wherein the audio data is received from a microphone ([0122]; [0665]).
Regarding claims 9-11 and 14-18, these claims are rejected for similar reasoning as noted above regarding claims 1-6.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-6, 9-11, and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Shah et al. (U.S. Pub. 2020/0384362 A1) (hereinafter “Shah”) in view of Bleasdale-Shepherd et al. (U.S. Pub. 2021/0146241 A1) (hereinafter “Bleasdale-Shepherd”).
Regarding claim 1, Shah discloses an apparatus (Fig. 5; [0004-0006]; [0042]; [0082]), comprising:
at least one processor system (Figs. 1 & 5; [0041-0042]; [0082]; [0084]; [0093]; [0095]) configured to:
access audio data ([0005]; [0019-0020], e.g., voice utterances);
provide the audio data to a machine learning (ML) model as input (Fig. 4; [0019-0020]);
receive, from the ML model in response to the input, an output indicating a player input to a computer game (Fig. 4; [0019-0020]); and
in response to receiving the output, provide the player input to the computer game ([0019-0020], perform in-game action based on the output of the ML model).
Shah further discloses where other input mechanisms, such as gestures (motion data) captured via motion sensing input devices and/or cameras can be used in place of speech utterances ([0023]). However, Shah may not disclose using audio data in combination with image and motion data. Nevertheless, Bleasdale-Shepherd teaches the use of sensor data (multimodal data) as input to a machine learning model to generate an output used to generate game control data (Fig. 2; [0015]; [0043]; [0045]; [0055], where the sensor data may comprise audio input received via a microphone and/or camera input (image inputs and/or motion/gestural inputs)). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to utilize additional input data in conjunction with the audio data, as taught by Bleasdale-Shepherd, to better interpret user intentions and corresponding in-game data/actions.
Regarding claim 2, Shah further discloses wherein the at least one processor system is configured to execute the computer game consistent with the player input ([0019-0020], perform in-game action based on the output of the ML model).
Regarding claim 3, Bleasdale-Shepherd further teaches wherein the image data is received from a camera, wherein the image data comprises video (Fig. 2; [0015]; [0043]; [0045]; [0055], e.g., camera input representing the motion data).
Regarding claim 4, Bleasdale-Shepherd further teaches wherein the motion data is received from an inertial measurement unit (IMU) (Fig. 2; [0012]; [0015]; [0028]; [0045]; [0052], the motion data input from a camera or other types of sensors, such as accelerometers and/or gyroscopes).
Regarding claim 5, Bleasdale-Shepherd further teaches wherein IMU comprises one or more of: an accelerometer, a gyroscope, a magnetometer (Fig. 2; [0012]; [0015]; [0028]; [0045]; [0052], the motion data input from a camera or other types of sensors, such as accelerometers and/or gyroscopes).
Regarding claim 6, Shah further discloses wherein the audio data is received from a microphone ([0041]; [0049]).
Regarding claims 9-11 and 14-18, these claims are rejected for similar reasoning as noted above regarding claims 1-6.
Claims 7-8, 12-13, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Crabtree in view of Kumar et al. (U.S. Pub. 2025/0111578 A1) (hereinafter “Kumar”).
Regarding claim 7, Crabtree may not further explicitly disclose wherein the ML model comprises a latent vector model. However, Kumar, directed to artificial intelligence and gaming technology, specifically the generation of non-player characters in a virtual environment based on multimodal inputs ([0001-0003]; [0016-0017]), teaches the use of a generative model to learn latent representations of input data ([0078]). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to utilize a latent vector model, as taught by Kumar, for the analyzation of the multi-modal inputs to reduce computational load by discarding information and working with lower-dimensional representations of data.
Regarding claim 8, Kumar further teaches wherein the ML model comprises a player-based encoder ([0078], where the generative model is a variational autoencoder).
Regarding claims 12-13 and 19-20, these claims are rejected for similar reasoning as noted above regarding claims 7-8.
Claims 7-8, 12-13, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Shah in view of Bleasdale-Shepherd, as applied to claims 1, 9, and 14, and in further view of Kumar.
Regarding claim 7, Shah may not further explicitly disclose wherein the ML model comprises a latent vector model. However, Kumar, directed to artificial intelligence and gaming technology, specifically the generation of non-player characters in a virtual environment based on multimodal inputs ([0001-0003]; [0016-0017]), teaches the use of a generative model to learn latent representations of input data ([0078]). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to utilize a latent vector model, as taught by Kumar, for the analyzation of the multi-modal inputs to reduce computational load by discarding information and working with lower-dimensional representations of data.
Regarding claim 8, Kumar further teaches wherein the ML model comprises a player-based encoder ([0078], where the generative model is a variational autoencoder).
Regarding claims 12-13 and 19-20, these claims are rejected for similar reasoning as noted above regarding claims 7-8.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Pub. 2025/0041731 A1 – This reference teaches generating immersive virtual environments based on real-life activities and game data utilizing machine learning algorithms.
U.S. Pub. 2024/0115947 A1 – This reference teaches integrating real-world multimodal data into video games.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALYSSA N BIANCAMANO whose telephone number is (571)272-4280. The examiner can normally be reached M-F: 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Dmitry Suhol, can be reached at (571)272-4430. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALYSSA N BIANCAMANO/Examiner, Art Unit 3715
Prosecution Insights