DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1, 9, and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In view of amendment, a new reference of Buford et al. (US Pub. 2016/0018888 A1) is applied to a new ground of rejection.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 2, 4, 5, 8-10, 12, 13, and 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Fajt et al. (US Pub. 2019/0155384 A1) in view of Yerli (US Pub. 2022/0070237 A1) and Buford et al. (US Pub. 2016/0018888 A1).
Regarding claim 1; Fajt teaches a method performed by one or more data processing apparatus (see Abstract, a system and a method of interacting between one or more users each using an endpoint system in a shared virtual environment), the method comprising:
identifying a respective objective for each user of two or more users in a group of users (Fig.1, para. [0029,0034], the method comprises determining an operation (e.g., throwing a virtual object, for example, a ball 112 in Fig.1) in a group of two or more users); wherein the respective objective for each user of two or more users to perform comprises a first objective for a first user of the two or more users (Figs.10A-10C, for example, a first objective of the user may be to throw an object 1022 to a target 1028), the first objective of the first user comprises a first subject object to which the first user should be looking (Figs.10A-10C, the target 1028 would be an object to which the user should be gazing), and the first subject object is specified by a leader user of the group of users or by a system (In a gaming system, the target 1028 would be specified by the system);
for each user of two or more users (Fig.1, para. [0034], the shared virtual environment may include two or more users which may interact with each other and with objects):
determining, based on inputs from multiple sensors having different input modalities (Figs.1,9A,9B; see Abstract, para. [0007,0008,0056,0073,0084], inputs may include a gazing input, a motion input, buttons, trackpads, directional pads, analog sticks, capacitive sensors…), a respective intent of the user (e.g., determining an interaction of a user with an object based on the gaze input and the motion input), wherein at least a portion of the multiple sensors are sensors of a device of the user that enables the user to participate in a shared virtual space (Figs.9A,9B; see Abstract, para. [0007,0084], the system receives different inputs (e.g., gaze input, motion input…) and determines a path adjustment for the object based at least in part on the gaze input and the motion input); the respective intent of the user comprises a first intent of the first user, and the first intent of the first user comprises a first target object with which the first user is likely to interact (Figs.10A-10C, the target 1028 would be a target object that the user is likely to interact with); and
determining, based on the respective intent of the user, whether the user is performing a respective objective for the user (see Abstract, para. [0007-0009], the system determines an operation of the virtual object based on the inputs (e.g., gaze vector and motion input). The object may be presented in the virtual environment as being thrown by an avatar associated with the endpoint system. For example, the method includes determining whether the gaze vector or a viewing frustum is aligned with a target and, if the gaze vector or viewing frustum is aligned with the target, a path adjustment may be calculated for the object such that the path of the object hits or comes nearer to the target); comprises determining whether the first subject object to which the first user should be looking matches the first target object with which the first user is likely to interact (Figs.10A-10C, para. [0009,0031], the system would be configured to determine whether the gaze vector 1006 is aligned with the target 1028);
generating, for the group of users, output data based on a respective objective for each of the two or more users and a respective intent for each user of the two or more users (Fig.9A, para. [0036,0059, 0060,0085-0093], a user may use a head-mounted display device HMDD to generate gaze input and use a controller device to generate motion input. The endpoint system generates a physical simulation of the object based at least in part of the gaze input and the motion input); and
providing, to a respective device of each of one or more users in the group of users, the output data for presentation at the respective device of each user of the one or more users (Fig.1, para. [0036,0049,0055,0059-0061,0063,0065], the endpoint system is configured to simulate the shared virtual environment for presentation).
Fajt does not teach identifying, for a common objective for a group of users in a shared virtual space, a respective object directed to the common objective for each user of two or more users in a group of users; a first subject object to which the first user should be looking instead of a leader user of the group of users.
Yerli teaches identifying, for a common objective for a group of users in a shared virtual space, a respective object directed to the common objective for each user of two or more users in a group of users (Figs.10A-10B, para. [0206, 0283-0284, and 0307], Yerli discloses a virtual shared environment. A plurality of students 1002 are looking at a virtual classroom whiteboard 1008. A teacher 1004 shares the same virtual space with the students 1002), a first subject object to which the first user should be looking instead of a leader user of the group of users (Fig.1A, para. [0307], the students should be looking at the whiteboard. The students may participate in the class through various interactions (e.g., hand-raising, content-projecting, presenting, expressing a doubt or contribution orally or in through text, etc.) and may provide the teacher with special administrative rights (e.g. giving someone the floor, muting one or more students during the lecture, sharing content through a digital whiteboard, etc.), see para. [0206]).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the system of Fajt to include the teaching of Yerli of providing a shared virtual environment to a group of students so that a student may interact with the teacher and other students in the classroom through a whiteboard. The motivation would have been in order to enable real-time multi-user collaboration and interactions in the virtual environment (Yerli, para. [0007]).
Fajt in view of Yerli does not teach determining that the first target object is a distraction for the first user when the first subject object does not match the first target object; wherein the output data comprises an alert to notify the leader user of the distraction.
PNG
media_image1.png
448
654
media_image1.png
Greyscale
(Fig.9 of Buford reproduced)
Buford teaches determining that the first target object is a distraction for the first user when the first subject object does not match the first target object; wherein the output data comprises an alert to notify the leader user of the distraction (Fig.9, para. [0064-0066], Buford discloses a real-time web conference system in which a presenter 901 is presenting visual content to a plurality of participants 902-905. The system is configured to detect whether a participant is distracted (e.g., not looking at the content); and notify the present the distraction. For example, a participant 905 may be looking at a different location within the content than is everyone else. A graphical indicator 5 may be displayed to inform the presenter 901 of the distraction of the participant 905).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the system of Fajt in view of Yerli to include the teaching of Buford of monitoring a distraction of a participant during a real-time web conferencing session; and informing a presenter of a distraction. The motivation would have been in order to improve the interaction efficiency.
Regarding claim 2; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt teaches a respective objective for at least one user of two or more users comprises a task to be performed by at least one user (Figs.1 and 10A-10C, para. [0006,0007], a task may include throwing a ball 112 performed by a first user based at least in part on the gaze input and the motion input).
Regarding claim 4; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt further teaches a respective device of each user comprises a wearable device (para. [0007,0034], a head-mounted display device); and determining the respective intent of the user comprises: receiving, from the wearable device of the user, gaze data specifying a gaze of the user (para. [0007], a gaze input), gesture data specifying a hand gesture of the user (para. [0031], a basic gesture such as moving a hand forward in a basic throwing motion), and direction data specifying a direction in which the user is moving (para. [0008], a motion input includes a direction); and determining, as the respective intent of the user, an intent of the user with respective to a target object based on the gaze data, the gesture data, and the direction data (para. [0007-0009,0029-0031,0034], the method determines an intent of a user (e.g., throwing a ball) based on the gaze input and the motion input).
Regarding claim 5; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt further teaches generating, for the group of users, output data based on the respective objective for each user of two or more users and the respective intent for each user of two or more users (Figs.1, 10A-10D; para. [0094-0099], two or more users are interacting with each other and with objects in the shared virtual environment. Each other is able to view what happening in the shared virtual environment. For example, each user may be able to see a first user manipulating a handheld controller device (HCD) and generating gaze input to manipulate a virtual object 1022), comprises determining that a particular user is not performing the respective objective for the particular user (e.g., Fig.10B, a user does not perform correct motion and gaze input to manipulate the object 1022 to hit a target 1028); and providing, to a respective device of each user of one or more users in the group of users, the output data for presentation at the respective device of each user of one or more users, comprises providing, to a device of the leader user, data indicating the particular user and data indicating that the particular user is not performing the respective objective for the particular user (Fig.10B, the endpoint system is configured to generate a simulation of the path of the object 1026 which does not hit the target 1028. It is understood that each user participating in the shared virtual environment would be able to see that the user in Fig.10B fails to manipulate the object 1022 to hit the target 1028 (e.g., para. [0030]). Generally, any user may be a leader user. For instance, the user who hits the target in Fig.10A may be a leader user).
Regarding claim 8; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt further teaches performing an action based on the output data (e.g., Fig.1, para. [0007], performing an action of throwing a ball 112); the action comprises reassigning one or more users to a different objective based on the output data (para. [0084], the user may be assigned an objective including throwing a ball by hand, kicking a ball by foot, hitting a ball by a racquet, or launching an arrow by a bow. Therefore, it would be understood that the user may be first assigned a first objective of throwing a ball in a first game. Then, the user may be later reassigned to a second objective of kicking a ball by foot in a second game. It is understood different interfaces would be outputted according to the first game and second game).
Regarding claim 9; Fajt in view of Yerli and Buford teaches a computer-implemented system (a computing device 600, Fig.6, para. [0069])), comprising: one or more computers (processor 602, Fig.6); and one or more computer memory devices (memory 604, Fig.6) interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions (Fig.6, para. [0069]) that, when executed by the one or more computers, perform operations comprising: identifying, for a common objective for a group of users in a shared virtual space, a respective objective directed to the common objective for each user of two or more users in the group of users, wherein the respective objective for each user of two or more users comprises a first objective for a first user of the two or more users, the first objective of the first user comprises a first subject object to which the first user should be looking instead of a leader user of the group of users, and the first subject object is specified by the leader user of the group of users or by the computer- implemented system; for each user of two or more users: determining, based on inputs from multiple sensors having different input modalities, a respective intent of the user, wherein at least a portion of the multiple sensors are sensors of a device of the user that enables the user to participate in the shared virtual space, the respective intent of the user comprises a first intent of the first user, and the first intent of the first user comprises a first target object with which the first user is likely to interact; determining that the first target object is a distraction for the first user when the first subject object does not match the first target object; and determining, based on the respective intent of the user, whether the user is performing a respective objective for the user, comprises determining whether the first subject object to which the first user should be looking matches the first target object with which the first user is likely to interact; generating, for the group of users, output data based on a respective objective for each user of the two or more users, wherein the output data comprises an alert to notify the leader user of the distraction; and a respective intent for each user of the two or more users; and providing, to a respective device of each user of one or more users in the group of users, the output data for presentation at the respective device of each user of the one or more users (similar to the analysis of claim 1).
Regarding claim 10; Fajt in view of Yerli and Buford teaches the system of claim 9 as discussed above. The limitation of claim 10 is substantially similar to claim 2. Thus, claim 10 is rejected based on the similar analysis as claim 2.
Regarding claim 12; Fajt in view of Yerli and Buford teaches the system of claim 9 as discussed above. The limitation of claim 12 is substantially similar to claim 4. Thus, claim 12 is rejected based on the similar analysis as claim 4.
Regarding claim 13; Fajt in view of Yerli and Buford teaches the system of claim 9 as discussed above. The limitation of claim 13 is substantially similar to claim 5. Thus, claim 10 is rejected based on the similar analysis as claim 5.
Regarding claim 15; Fajt in view of Yerli and Buford teaches the system of claim 9 as discussed above. The limitation of claim 15 is substantially similar to claim 7. Thus, claim 15 is rejected based on the similar analysis as claim 7.
Regarding claim 16; Fajt in view of Yerli and Buford teaches the system of claim 15 as discussed above. The limitation of claim 16 is substantially similar to claim 8. Thus, claim 16 is rejected based on the similar analysis as claim 8.
Regarding claim 17; Fajt in view of Yerli and Buford teaches a non-transitory, computer-readable medium (a system memory 604, Fig.6, para. [0069]) storing one or more instructions executable by a computer system to perform operations (para. [0069]) comprising: identifying, for a common objective for a group of users in a shared virtual space, a respective objective directed to the common objective for each user of two or more users in the group of users, wherein the respective objective for each user of two or more users comprises a first objective for a first user of the two or more users, the first objective of the first user comprises a first subject object to which the first user should be looking instead of a leader user of the group of users, and the first subject object is specified by the leader user of the group of users or by the computer system; for each user of two or more users: determining, based on inputs from multiple sensors having different input modalities, a respective intent of the user, wherein at least a portion of the multiple sensors are sensors of a device of the user that enables the user to participate in the shared virtual space, the respective intent of the user comprises a first intent of the first user, and the first intent of the first user comprises a first target object with which the first user is likely to interact; determining that the first target object is a distraction for the first user when the first subject object does not match the first target object; and determining, based on the respective intent of the user, whether the user is performing a respective objective for the user, comprises determining whether the first subject object to which the first user should be looking matches the first target object with which the first user is likely to interact; generating, for the group of users, output data based on a respective objective for each user of the two or more users, wherein the output data comprises an alert to notify the leader user of the distraction and a respective intent for each user of the two or more users; and providing, to a respective device of each of one or more users in the group of users, the output data for presentation at the respective device of each of the one or more users (similar to the analysis of claim 1).
Regarding claim 18; Fajt in view of Yerli and Buford teaches the non-transitory, computer-readable medium of claim 17 as discussed above. The limitation of claim 18 is substantially similar to claim 2. Thus, claim 18 is rejected based on the similar analysis as claim 2.
Claims 3, 11, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Fajt et al. (US Pub 2019/0155384 A1) in view of Yerli (US Pub. 2022/0070237 A1) and Buford et al. (US Pub. 2016/0018888 A1) as applied to claims 1, 10, and 18 above; further in view of Pusch et al. (US Patent No. 10,922,890).
Regarding claim 3; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt does not teach identifying the respective objective for each user of two or more users, comprises determining, as a subject object, a target to which at least a threshold amount of users in the group of users is looking.
Pusch teaches identifying the respective objective for each user of two or more users, comprises determining, as a subject object, a target to which at least a threshold amount of users in the group of users is looking (In the abstract, Fig.1, Pusch discloses a system and method for virtual and augmented reality which can be used in user training or gaming. For example, a trainer or an operator is providing instructions to a plurality of participants. Pusch discloses that “in a multi-participant simulation, an alarm/indicator/notifier may be triggered and/or displayed to a user/operator when a configurable threshold, such as a configurable number (e.g., a count of participants) or percentage of participant's in the simulation, trigger a condition (e.g., a count of 2 out of 4 participants in a presentation are not directing their gaze at the virtual slide screen or the presenter/operator/user)” (see column 9, lines 41-48). In other words, for example, if a count of participants not directing their gazes to the presenter/operator is greater than a threshold number, the alarm/indicator/notifier would be triggered to notify the presenter/operator).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the VR system of Fajt in view of Yerli to include the teaching of Pusch of notifying a presenter/operator if a count of participants not directing their gazes at the presenter/operator is greater than a threshold number. The motivation would have been in order to improve the interaction between the presenter/operator and the participants.
Regarding claim 11; Fajt in view of Yerli and Buford teaches the system of claim 10 as discussed above. The limitation of claim 11 is substantially similar to claim 3. Thus, claim 11 is rejected based on the similar analysis as claim 3.
Regarding claim 19; Fajt in view of Yerli and Buford teaches the non-transitory, computer-readable medium of claim 18 as discussed above. The limitation of claim 19 is substantially similar to claim 3. Thus, claim 19 is rejected based on the similar analysis as claim 3.
Regarding claim 20; Fajt in view of Yerli, Buford, and Pusch teaches the non-transitory, computer-readable medium of claim 19 as discussed above. Fajt does not teach the operations further comprise: determining that at least a second threshold amount of users are not looking to the subject object: and presenting a notification to the second threshold amount of users to pay attention to the subject object.
Pusch teaches the operations further comprise: determining that at least a second threshold amount of users are not looking to the subject object: and presenting a notification to the second threshold amount of users to pay attention to the subject object (see column 9, lines 41-48).
The motivation is the same as the rejection of claim 3.
Claims 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Fajt et al. (US Pub 2019/0155384 A1) in view of Yerli (US Pub. 2022/0070237 A1) and Buford et al. (US Pub. 2016/0018888 A1) as applied to claims 1 and 9 above; further in view of Golden et al. (US Pub 2020/0054939 A1).
Regarding claim 6; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt does not teach the output data comprises a heat map indicating an amount of users in the group of users performing a respective objective of a user.
Golden teaches the output data comprises a heat map indicating an amount of users in the group of users performing a respective objective of a user (para. [0003], an electronic gaming system provides a shared virtual environment for many players to interact in a virtual world. In particular, Figs.5 and 6, para. [0077-0080], the system may output a heat map which is indicative of a number of players).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the VR system of Fajt to include the teaching of Golden of outputting a heat map indicating a number of players in a shared virtual environment. The motivation would have been in order to improve a link between the virtual world and the real world (Golden, para. [0021]).
Regarding claim 14; Fajt in view of Yerli and Buford teaches the system of claim 9 as discussed above. The limitation of claim 14 is substantially similar to claim 6. Thus, claim 14 is rejected based on the similar analysis as claim 6.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Fajt et al. (US Pub 2019/0155384 A1) in view of Yerli (US Pub. 2022/0070237 A1) and Buford et al. (US Pub. 2016/0018888 A1) as applied to claim 1 above; further in view of Rangan et al. (US Pub. 2018/0122254 A1).
Regarding claim 7; Fajt in view of Yerli and Buford teaches the method of claim 1 as discussed above. Fajt does not teach the first subject object is a physical object.
Rangan teaches the first subject object is a physical object (Figs. 9 and 16, para. [0010, 0076, 0117, and 0223], Rangan discloses an e-learning system having a local classroom including a teacher 903, a whiteboard 905, and students 935. The teacher and each student are equipped with a Google Glass device such that each student can see the teacher directly. In other words, the teacher and the whiteboard would be physical objects that the students are interacting with).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the system of Fajt in view of Yerli to include the teaching of Rangan of providing a local classroom in which a student sees a teacher directly through a Google Glass device. The motivation would have been in order to improve the interaction efficiency.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NGUYEN H TRUONG whose telephone number is (571)270-1630. The examiner can normally be reached M-F: 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chanh Nguyen can be reached at 571-272-7772. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NGUYEN H TRUONG/Examiner, Art Unit 2623
/CHANH D NGUYEN/Supervisory Patent Examiner, Art Unit 2623
Prosecution Insights