Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of copending Application No. 18/375,517 (reference application). Although the claims at issue are not identical, they are not patentably distinct from each other because: copending Application No. 18/375,517 disclose all the limitations in the pending claims of instant application.
This is a provisional nonstatutory double patenting rejection because the patentably indistinct claims have not in fact been patented.
18/430488 18/835517
1. A computer-implemented method comprising: monitoring, via a brain computer interface (BCI), brain data of a user associated with the user's interaction with GUI elements, the BCI being configured to detect brain activity of the user associated with the user's interaction with interface elements within an environment, the environment comprising GUI elements and at least one of a mixed reality environment or a 2D software environment; recording gaze information associated with a gaze location of the user during interaction with the environment, decoding brain data of the user in temporal conjunction with detecting, via the gaze information, the user's visual attention on a particular element of the environment; determining, based on decoding the brain data of the user, an imagined action indicating that the user wishes to interact with the particular element of the environment; and generating, in response to determining the imagined action indicating that the user wishes to interact with the particular element, an interaction event with the particular element of the environment.
2. (original) The computer-implemented method of claim 1, wherein decoding the brain data comprises generating a saliency map of the user's visual attention from brain data from the brain computer interface.
3. (original) The computer-implemented method of claim 1, wherein recording the gaze information associated with the gaze location of the user comprises receiving eye-tracking data from an eye-tracker component.
4. (original) The computer-implemented method of claim 1, wherein determining the imagined action comprises determining, based on decoding the brain data of the user, an imagined motor movement of the user's body.
5. (original) The computer-implemented method of claim 1, wherein determining the imagined action comprises determining an imagined spoken language command from the brain data.
6. The computer-implemented method of claim 1, wherein decoding the brain data of the user comprises: creating, based on the brain data of the user, a brain-data saliency map by: creating a plurality of brain data images that each represent at least one feature from the brain data mapped to at least one scalp location; representing the brain data images in latent space by processing through a variational autoencoder; generating, using a generator network, the brain-data saliency map based on the brain data images represented in latent space; creating, based at least in part on the recorded gaze information, an eye-tracking saliency map by representing the recorded gaze information in latent space by processing through the variational autoencoder; distinguishing, using a discriminator network, differences between the eye-tracking saliency map and the brain-data saliency map; deriving, based on an output of the discriminator network, a loss function; and modifying, based on a feedback loop that uses the loss function to inform modifications to the generator network, the generator network.
7. The method of claim 1, wherein determining the imagined action comprises at least one of: detecting imagined speech, wherein the imagined speech comprises a word from a predetermined list of words that indicate intention to interact with user interface elements; or detecting a premotor brain signal.
8. (original) The method of claim 1, wherein determining the imagined action comprises detecting an imagined motor movement, wherein the imagined motor movement comprises an imagined physical interaction with the particular element of the environment.
9. (original) The method of claim 8, wherein detecting the imagined motor movement comprises: processing, by a decoder communicatively coupled to the BCI and configured to receive detected motor movement of motor areas, the brain data; and distinguishing, by the decoder, between imagined movement of the user's body and a rest state of the user.
10. (original) The method of claim 8, wherein detecting the imagined motor movement comprises detecting the imagined motor movement via electrodes placed on the user's scalp in proximity to a motor cortex of the user's brain.
11. The method of claim 1, recording the gaze information comprises processing the gaze of the user over a dwell time of 70-SOOms to determine a gaze direction.
12. The method of claim 1, wherein recording the gaze information comprises recording the gaze information by an augmented reality headset.
13. The method of claim 1, wherein monitoring the gaze and the brain activity of the user method further comprises: recording the gaze and the brain activity of the user during a training period; and training, based on data gathered during the training period, a classifier that detects imagined physical actions in the brain activity.
14. (original) The method of claim 13, wherein determining the imagined action comprises detecting the imagined physical action via the classifier.
15.The method of claim 1, wherein the BCI is integrated into a mixed reality headset, the method further comprising: monitoring the brain data of the user comprises monitoring the brain data of the user by the BCI that is integrated into the mixed reality headset; recording the gaze information comprises monitoring the user's gaze by the mixed reality headset; and displaying the environment to the user by the mixed reality headset.
16. The method of claim 1, wherein the BCI comprises a housing that is physically separate from a headset that displays the environment.
17. (original) The method of claim 1, wherein generating the interaction event comprises visually transforming the element to indicate the interaction event.
18. The method of claim 1, further comprising: detecting the user's visual attention on an additional element of the environment; detecting a lack of the imagined action in the brain activity of the user in temporal conjunction with the user's visual attention on the additional element; determining, based on the lack of the imagined action in temporal conjunction with the user's visual attention on the additional element, that the user does not intend to interact with the additional element; and refraining, in response to determining that the user does not intend to interact with the additional element, from generating an interaction event with the additional element.
19. system comprising: at least one physical computer processor; a memory coupled to the at least one computer processor; and one or more computing components comprising non-transitory computer readable media containing and/or configured to execute computer-readable instructions, the computer-readable instructions comprising instructions that, when executed by the at least one computer processor, cause the system to perform operations including: monitoring, via a brain computer interface (BCI), brain data of a user associated with the user's interaction with GUI elements, the BCI being configured to detect brain activity of the user associated with the user's interaction with interface elements within an environment, the environment comprising GUI elements and a mixed reality environment; recording the gaze and the brain activity of the user during a training period, wherein recording the gaze information comprises processing the gaze of the user over a dwell time to determine a gaze direction; training, based on data gathered during the training period, a classifier that detects imagined physical actions in the brain activity; recording gaze information associated with a gaze location of the user during interaction with the environment; decoding brain data of the user in temporal conjunction with detecting, via the gaze information, the user's visual attention on a particular element of the environment; determining, based on decoding the brain data of the user, an imagined action indicating that the user wishes to interact with the particular element of the environment, the determining the imagined action comprising detecting an imagined motor movement, which is comprised of an imagined physical interaction with the particular element of the environment, including: detecting the imagined motor movement via electrodes placed on the user's scalp; and distinguishing, by a decoder communicatively coupled to the BCI and configured to receive detected motor movement of motor areas, between imagined movement of the user's body and a rest state of the user; and generating, in response to determining the imagined action indicating that the user wishes to interact with the particular element, an interaction event with the particular element of the environment.
20. One or more non-transitory computer-readable media containing and/or configured to execute computer-readable instructions, the computer-readable instructions comprising instructions that, when executed by one or more processors, cause the one or more processors to: monitoring, via a brain computer interface (BCI), brain data of a user associated with the user's interaction with GUI elements, the BCI being configured to detect brain activity of the user associated with the user's interaction with interface elements within an environment, the environment comprising GUI elements and a mixed reality environment; recording gaze information associated with a gaze location of the user during interaction with the environment, decoding brain data of the user in temporal conjunction with detecting, via the gaze information, the user's visual attention on a particular element of the environment; determining,
based on decoding the brain data of the user, an imagined action indicating that the user wishes to interact with the particular element of the environment; and generating, in response to determining the imagined action indicating that the user wishes to interact with the particular element, an interaction event with the particular element of the environment; wherein the decoding the brain data of the user includes:
creating, based on the brain data of the user, a brain-data saliency map; wherein the method further comprises:
creating, based on the recorded gaze information, an eye-tracking saliency map by representing the recorded gaze information in latent space; distinguishing, using a discriminator network, differences between the eye-tracking saliency map and the brain-data saliency map; and modifying, based on a feedback loop that an output of the discriminator network to inform modifications to the generator network, the generator network.
1. A computer-implemented method comprising: monitoring, via a brain computer interface (BCI), brain data of a user associated with the user's interaction with GUI elements, the BCI being configured to detect brain activity of the user associated with the user's interaction with interface elements within an environment, the environment comprising GUI elements and at least one of a mixed reality environment or a 2D software environment; recording gaze information associated with a gaze location of the user during interaction with the environment, decoding brain data of the user in temporal conjunction with detecting, via the gaze information, the user's visual attention on a particular element of the environment; determining, based on decoding the brain data of the user, an imagined action indicating that the user wishes to interact with the particular element of the environment; and generating, in response to determining the imagined action indicating that the user wishes to interact with the particular element, an interaction event with the particular element of the environment.
2. (original) The computer-implemented method of claim 1, wherein decoding the brain data comprises generating a saliency map of the user's visual attention from brain data from the brain computer interface.
3. (original) The computer-implemented method of claim 1, wherein recording the gaze information associated with the gaze location of the user comprises receiving eye-tracking data from an eye-tracker component.
4. (original) The computer-implemented method of claim 1, wherein determining the imagined action comprises determining, based on decoding the brain data of the user, an imagined motor movement of the user's body.
5. (original) The computer-implemented method of claim 1, wherein determining the imagined action comprises determining an imagined spoken language command from the brain data.
6. The computer-implemented method of claim 1, wherein decoding the brain data of the user comprises: creating, based on the brain data of the user, a brain-data saliency map by: creating a plurality of brain data images that each represent at least one feature from the brain data mapped to at least one scalp location; representing the brain data images in latent space by processing through a variational autoencoder; generating, using a generator network, the brain-data saliency map based on the brain data images represented in latent space; creating, based at least in part on the recorded gaze information, an eye-tracking saliency map by representing the recorded gaze information in latent space by processing through the variational autoencoder; distinguishing, using a discriminator network, differences between the eye-tracking saliency map and the brain-data saliency map; deriving, based on an output of the discriminator network, a loss function; and modifying, based on a feedback loop that uses the loss function to inform modifications to the generator network, the generator network.
7. The method of claim 1, wherein determining the imagined action comprises at least one of: detecting imagined speech, wherein the imagined speech comprises a word from a predetermined list of words that indicate intention to interact with user interface elements; or detecting a premotor brain signal.
8. The method of claim 1, wherein determining the imagined action comprises detecting an imagined motor movement, wherein the imagined motor movement comprises an imagined physical interaction with the particular element of the environment.
9. (original) The method of claim 8, wherein detecting the imagined motor movement comprises: processing, by a decoder communicatively coupled to the BCI and configured to receive detected motor movement of motor areas, the brain data; and distinguishing, by the decoder, between imagined movement of the user's body and a rest state of the user.
10. (original) The method of claim 8, wherein detecting the imagined motor movement comprises detecting the imagined motor movement via electrodes placed on the user's scalp in proximity to a motor cortex of the user's brain.
11. The method of claim 1, recording the gaze information comprises processing the gaze of the user over a dwell time of 70-500ms to determine a gaze direction.
12. The method of claim 1, wherein recording the gaze information comprises recording the gaze information by an augmented reality headset.
13. The method of claim 1, wherein monitoring the gaze and the brain activity of the user method further comprises: recording the gaze and the brain activity of the user during a training period; and training, based on data gathered during the training period, a classifier that detects imagined physical actions in the brain activity.
14. (original) The method of claim 13, wherein determining the imagined action comprises detecting the imagined physical action via the classifier.
15. The method of claim 1, wherein the BCI is integrated into a mixed reality headset, the method further comprising: monitoring the brain data of the user comprises monitoring the brain data of the user by the BCI that is integrated into the mixed reality headset; recording the gaze information comprises monitoring the user's gaze by the mixed reality headset; and displaying the environment to the user by the mixed reality headset.
16. The method of claim 1, wherein the BCI comprises a housing that is physically separate from a headset that displays the environment.
17. (original) The method of claim 1, wherein generating the interaction event comprises visually transforming the element to indicate the interaction event.
18. The method of claim 1, further comprising: detecting the user's visual attention on an additional element of the environment; detecting a lack of the imagined action in the brain activity of the user in temporal conjunction with the user's visual attention on the additional element; determining, based on the lack of the imagined action in temporal conjunction with the user's visual attention on the additional element, that the user does not intend to interact with the additional element; and refraining, in response to determining that the user does not intend to interact with the additional element, from generating an interaction event with the additional element.
19. A system comprising: at least one physical computer processor; a memory coupled to the at least one computer processor; and one or more computing components comprising non-transitory computer readable media containing and/or configured to execute computer-readable instructions, the computer-readable instructions comprising instructions that, when executed by the at least one computer processor, cause the system to perform operations including: monitoring, via a brain computer interface (BCI), brain data of a user associated with the user's interaction with GUI elements, the BCI being configured to detect brain activity of the user associated with the user's interaction with interface elements within an environment, the environment comprising GUI elements and a mixed reality environment recording the gaze and the brain activity of the user during a training period, wherein recording the gaze information comprises processing the gaze of the user over a dwell time of 70-500ms to determine a gaze direction training, based on data gathered during the training period, a classifier that detects imagined physical actions in the brain activity; recording gaze information associated with a gaze location of the user during interaction with the environment; decoding brain data of the user in temporal conjunction with detecting, via the gaze information, the user's visual attention on a particular element of the environment; determining, based on decoding the brain data of the user, an imagined action indicating that the user wishes to interact with the particular element of the environment, the determining the imagined action comprising detecting an imagined motor movement, which is comprised of an imagined physical interaction with the particular element of the environment, including: detecting the imagined motor movement via electrodes placed on the user's scalp in proximity to a motor cortex of the user's brain; processing, by a decoder communicatively coupled to the BCI and configured to receive detected motor movement of motor areas, the brain data; and distinguishing, by the decoder, between imagined movement of the user's body and a rest state of the user; and generating, in response to determining the imagined action indicating that the user wishes to interact with the particular element, an interaction event with the particular element of the environment.
20. One or more non-transitory computer-readable media containing and/or configured to execute computer-readable instructions, the computer-readable instructions comprising instructions that, when executed by one or more processors, cause the one or more processors to: monitoring, via a brain computer interface (BCI), brain data of a user associated with the user's interaction with GUI elements, the BCI being configured to detect brain activity of the user associated with the user's interaction with interface elements within an environment, the environment comprising GUI elements and a mixed reality environment; recording gaze information associated with a gaze location of the user during interaction with the environment, decoding brain data of the user in temporal conjunction with detecting, via the gaze information, the user's visual attention on a particular element of the environment
determining,
based on decoding the brain data of the user, an imagined action indicating that the user wishes to interact with the particular element of the environment; and generating, in response to determining the imagined action indicating that the user wishes to interact with the particular element, an interaction event with the particular element of the environment; wherein the decoding the brain data of the user includes:
creating, based on the brain data of the user, a brain-data saliency map by: creating a plurality of brain data images that each represent at least one feature from the brain data mapped to at least one scalp location;
representing the brain data images in latent space; and generating the brain-data saliency map based on the brain data images represented in latent space; wherein the method further comprises: creating, based on the recorded gaze information, an eye-tracking saliency map by representing the recorded gaze information in latent space; distinguishing, using a discriminator network, differences between the eye-tracking saliency map and the brain-data saliency map; deriving, based on an output of the discriminator network, a loss function; and modifying, based on a feedback loop that uses the loss function to inform modifications to the generator network, the generator network.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Van N Chow whose telephone number is (571)272-7590. The examiner can normally be reached M-F 10-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Ke can be reached at 5712727776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/VAN N CHOW/ Primary Examiner, Art Unit 2627
Prosecution Insights