DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 7, 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Silverstein et al. (US PGPub 2022/0100261) in view of Domae et al. (US PGPub 2024/0114181).
Regarding claim 1, Silverstein discloses an electronic device (fig. 1, VR system 100) comprising:
a camera (fig. 1, ceiling mounted camera 110);
a communication interface (fig. 1, VR controller 106);
a display (fig. 1, display device 108);
memory storing instructions (fig. 4, memory subsystem 14); and
at least one processor ([0033], “the computer system 11 may contain multiple processors typical of a relatively large system; however, in other embodiments the computer system 11 may alternatively be a single CPU system. Each CPU 12 may execute instructions stored in the memory subsystem 14 and may comprise one or more levels of on-board cache”),
wherein the instructions, when executed by the at least one processor, individually or
collectively, cause the electronic device to:
obtain, through the camera, a plurality of images comprising motion information of a
user ([0019], “If VR controller 106 determines that one or more of people 114 are spectators, then VR controller 106 can prompt them to opt-in to the VR experience. This can occur, for example, using display device 108 and/or cameras 104 and/or 110. If any of people 114 wish to join the experience, they can acknowledge, for example, using a gesture or verbal speech” additionally, see fig. 3, step 306 spectators opt-in);
transmit, through the communication interface, the plurality of images to an
electronic device ([0018]-[0019], “The information from cameras 104 and 110 can be sent to and analyzed by VR controller 106. VR controller 106 can detect and monitor people 114 in the spectator area. Furthermore, VR controller 106 can analyze people 114 to determine whether they are spectators (showing interest in the VR experience) or bystanders (present but uninvolved with the VR experience).
If VR controller 106 determines that one or more of people 114 are spectators, then VR controller 106 can prompt them to opt-in to the VR experience. This can occur, for example, using display device 108 and/or cameras 104 and/or 110. If any of people 114 wish to join the experience, they can acknowledge, for example, using a gesture or verbal speech. Once some or all of people 114 have opted-in, then they can be added to the VR space that is viewable by user 112. Thereby, any of people 114 who want to be spectators can be involved in the VR experience instead of merely observing it”);
in response to a group mission being achieved determined based on the motion
information, obtain, from the ([0024], “For examples of motions, in some embodiments, movements of the body and large gestures by people 114 are monitored by VR system 100 and reflected in the VR state of the respective avatars 214. This can include relative motion between user 112 and people 114, as well as movements in place such as jumping or arm waving. For examples of expressions, in some embodiments, subtle movements of extremities and/or static poses including facial expressions and hand gestures can be reflected in the VR state of the respective avatars 214” additionally see fig. 3, step 306 track spectators and step 312 monitor spectators); and
display the content through the display ([0021], “people 114 have been recognized as spectators in the spectator space and have opted-in to the VR experience. Therefore, avatar 214A has been rendered in VR view 200 to represent person 114A, and avatar 214B has been rendered in VR view 200 to represent person 114B. Because cameras 104 and/or 110 can determine the relative and/or absolute locations of people 114 with respect to user 112 (for example, using temporal parallax), avatars 214 are sized and positioned accordingly in the VR space. For example, because person 114A is closer than person 114B to user 112 in the real world, avatar 214A appears larger than avatar 214B in VR view 200” additionally see fig. 3, step 312, render avatars).
While Silverstein discloses VR controller 106 which can send information to and receive information from display device 108 and the display device is part of the electronic device (VR system 100), it has been known to send/receive information to/from external devices. Additionally, Silverstein teaches content related to a performance where the content is spectators who are presumably watching a performance and thus the spectator content is based on the performance, furthermore content that includes a performance has been known.
In a similar field of endeavor of VR systems, Domae discloses an external electronic device (fig. 1, distribution server 20); in response to a group mission being achieved determined based on the motion information, obtain, from the external device through the communication interface, content that includes a performance ([0055], “the performer display switching control unit 223 may perform the switching in a case where a state of at least one user avatar in the virtual space satisfies a predetermined condition. The predetermined condition is not particularly limited, but may be, for example, detection of a certain number of user avatars which have made the same motion. Furthermore, the predetermined condition may be detection of a certain number of user avatars which have simultaneously made a motion designated by the performer. Excitement (the amplitude of a motion of the user avatar, the number of jumps, applause, and the like) of all the event participants in the virtual space may be set as the predetermined condition. Furthermore, the predetermined condition may also be a text amount of cheering messages that exceeds a predetermined value, the cheering messages being sent from all the users to the performer during the event. The predetermined condition may also be the number of event participants (the number of user avatars in the audience seats of the event) that exceeds a predetermined value. In a case where the display of the performer avatar is switched depending on situations of a large number of user avatars, the performer display switching control unit 223 may perform control to uniformly perform switching between the character 3D model and the live-action 3D model for all users (a sense of unity of the participants is created)”).
In view of the teachings of Silverstein and Domae, it would have been obvious to one of ordinary skill in the art to implement the feature of having the electronic device responsively obtain from the external electronic device the performance content for display as taught by Domae within the system of Silverstein, for the purpose of improving a user’s experience since external electronic devices have known benefits including being easily replaceable and portable between different systems and the benefit of improving a user’s experience with performance content by switching an appearance of a performer in a virtual space (Domae: [0006]).
Regarding claim 2, the combination of Silverstein and Domae further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to based on the at least one image obtained by the camera, determine whether the user is responding to the group mission (Silverstein: [0027], “The determination at block 304 can be made for each person 114 separately based on, for example, tracking of eye movements of each person 114. More specifically, if a person 114 is watching display device 108 for a significant amount or percentage of time, then the person 114 is considered a spectator. On the other hand, if the person 114 is not looking at display device 108 a significant amount or percentage of the time (e.g., if they are doing another activity in the spectator area, such as reading), then the person 114 is considered a bystander, in some embodiments”).
Regarding claim 7, the combination of Silverstein and Domae further discloses wherein the electronic device further comprises at least one speaker, and
wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on receiving a second input signal from the external electronic device, output a sound signal, through the at least one speaker (Silverstein: [0025], “For examples of sounds, in some embodiments, utterances (e.g., spoken words or yelling) and noises (e.g., clapping) can be reflected in the VR states of avatars 214. … However, in an embodiment where there is a large stadium of VR fans (e.g., in a sports arena), the sounds of people 114 can be transformed in such a way to make it sound like there is a whole stadium of people making those sounds”).
Claims 11 and 12 are method claims drawn to the device of claims 1 and 2 and are therefore interpreted and rejected based on similar reasoning.
Claims 3-6, 10 and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Silverstein and Domae in view of Burns et al. (US PGPub 2024/0056492).
Regarding claim 3, the combination of Silverstein and Domae further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to display, in a virtual reality environment, the content and response information indicating a degree of response to the group mission (Silverstein: [0031], “For example, if people 114 are quietly watching display device 108, then avatars 214 may be absent from VR view 200. But when people 114 begin jumping and cheering (e.g., because of an exciting event in the VR experience), then avatars 214 can be rendered. Furthermore, VR controller 106 can record or annotate (a.k.a., tag) a recording of the VR experience in response to a dramatic and/or loud expression by people 114. Such a recording or annotation can allow user 112 to review the most exciting portions of the VR experience, for example, to make a highlight reel. In some embodiments, audience participation can be factored into the VR experience in other ways than just avatars 214. For example, dramatic and/or loud expressions by people 114 can boost the abilities, score, and/or experience points of user 112, which further closes the gap between a player and their spectators (e.g., in an e-sport VR experience)”).
While the combination of Silverstein and Domae teaches a degree of response, it has been known to provide a percentage indicating a degree of response. In a similar field of endeavor of VR/XR/AR systems, Burns discloses response information that includes a percentage indicating a degree of response to the group mission ([0117], “The slider may represent the audience attention on a sliding scale from low to high. The number may represent the percentage of the audience that is indicated to have a high attention level, an average relative attention level for the audience members, etc. As another example, a given percentage of the individual avatars in the audience (matching the percentage of audience members with a high attention level) may be provided with an attentive appearance”).
In view of the teachings of Silverstein, Domae and Burn it would have been obvious to one of ordinary skill in the art to include the percentage of Burn within the system of Silverstein and Domae for the purpose of providing a known representation of a degree of information which a user intuitively understands.
Regarding claim 4, the combination of Silverstein, Domae and Burns further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on identifying that the motion information corresponds to a first motion state, display the virtual reality environment in a first view mode, and
wherein, in the first view mode, the content, at least one first avatar corresponding to the user, at least one second avatar corresponding to at least one second user participating in the performance, and the response information, are displayed in the virtual reality environment (Silverstein: [0021], “people 114 have been recognized as spectators in the spectator space and have opted-in to the VR experience. Therefore, avatar 214A has been rendered in VR view 200 to represent person 114A, and avatar 214B has been rendered in VR view 200 to represent person 114B. Because cameras 104 and/or 110 can determine the relative and/or absolute locations of people 114 with respect to user 112 (for example, using temporal parallax), avatars 214 are sized and positioned accordingly in the VR space. For example, because person 114A is closer than person 114B to user 112 in the real world, avatar 214A appears larger than avatar 214B in VR view 200”).
Regarding claim 5, the combination of Silverstein, Domae and Burns further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on identifying the motion information corresponds to a second motion state, display the virtual reality environment in a second view mode, and
wherein, in the second view mode, the content is displayed over a full screen of the display (Silverstein: [0017], “display device 108 can show various portions of the VR experience (e.g., the view that user 112 is seeing) to allow persons 114A and 114B (collectively, “people 114”) to see and participate in the VR experience”).
Regarding claim 6, the combination of Silverstein, Domae and Burns further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on receiving a first input signal from the external electronic device:
based the virtual reality environment being displayed in the first view mode, switch from the first view mode to the second view mode (Silverstein: [0020], “FIG. 2 shows VR view 200 of the VR space, for example, from the inside of VR headset 102 as user 112 would see it. VR view 200 includes avatars 214A and 214B (collectively, “avatars 214”), desk 216, book 218, pen 220, ink 222, painting 224, and spiderweb 226. Because VR view 200 takes place in the greater context of the real world (shown in FIG. 1), references may be made to the features of FIG. 1”); and
based on the virtual reality environment being displayed in the second view mode, switch from the second view mode to the first view mode (Silverstein: [0027], “FIG. 3 shows a flowchart of method 300 of generating a VR experience. During the discussion of method 300, references may be made to the features of FIGS. 1 and 2. In the illustrated embodiment, at block 302, user 112 initializes VR system 100, and opts-in to including avatars 214 of spectators in the VR experience. At block 304, VR system 100 recognizes people 114 in the spectator area and determines whether they are spectators or bystanders. The determination at block 304 can be made for each person 114 separately based on, for example, tracking of eye movements of each person 114. More specifically, if a person 114 is watching display device 108 for a significant amount or percentage of time, then the person 114 is considered a spectator. On the other hand, if the person 114 is not looking at display device 108 a significant amount or percentage of the time (e.g., if they are doing another activity in the spectator area, such as reading), then the person 114 is considered a bystander, in some embodiments”).
Regarding claim 10, the combination of Silverstein, Domae and Burns further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on the at least one first avatar and the at least one second avatar approaching within a predetermined distance in the virtual reality environment, perform predetermined interaction (Silverstein: [0021], “In the illustrated embodiment, people 114 have been recognized as spectators in the spectator space and have opted-in to the VR experience. Therefore, avatar 214A has been rendered in VR view 200 to represent person 114A, and avatar 214B has been rendered in VR view 200 to represent person 114B. Because cameras 104 and/or 110 can determine the relative and/or absolute locations of people 114 with respect to user 112 (for example, using temporal parallax), avatars 214 are sized and positioned accordingly in the VR space. For example, because person 114A is closer than person 114B to user 112 in the real world, avatar 214A appears larger than avatar 214B in VR view 200”).
Claims 13-15 are method claims drawn to the device of claims 3-5 and are therefore interpreted and rejected based on similar reasoning.
Claims 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Silverstein, Domae and Burns in view of Cui (US PGPub 2025/0021164).
Regarding claim 8, the combination of Silverstein, Domae and Burns further discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on receiving a third input signal from the external electronic device, send an (Silverstein: [0026], “Because avatars 214 can reflect the physical states of people 114, avatars 214 reflect the emotional states of people 114. This can allow user 112 to see the reactions of people 114 to the VR experience such that people 114 are both witnessing the VR experience and adding to the VR experience. This can provide a more satisfying experience for both people 114 and user 112 and can also allow people 114 to give non-verbal feedback to user 112. For example, if the VR experience is a tour of a proposed building designed by user 112, people 114 (e.g., the customers) can follow the tour and engage with user 112. This can allow user 112 to gauge the enthusiasm of people 114 for the design, as well as allow people 114 to point to places that they want the tour to go to without having to verbally explain them in detail” where Monti discloses the external electronic device as discussed in claim 1 above).
While the combination of Silverstein, Domae and Burns discloses an emotional state, an emoticon is a known type of emotional representation of a user. In a similar field of endeavor of virtual reality interaction, Cui discloses an emoticon ([0077], “For example, the first object controls the first virtual object to transmit a non-contact interactive message to the second virtual object. For example, the non-contact interactive message is a “love” message, which may be expressed by transmitting a heart emoticon or transmitting a kiss emoticon. Based on this, when the second virtual object receives the “love” message, a head display device worn by the second object triggers vibration feedback”).
In view of the teachings of Silverstein, Domae, Burns and Cui, it would have been obvious to one of ordinary skill in the art to use the emoticon of Cui to describe the emotional state of the user in Silverstein, Domae and Burns, where emoticons are known to represent an emotional state of a user which user’s typically intuitively know the emotional meaning.
Regarding claim 9, while the combination of Silverstein, Domae and Burns discloses a first and second avatar and objects (Silverstein: fig. 2, elements 214A and 214B and 216, 218, 220, 222, 224, 226), it has been known to have the avatars interact using a virtual object. In a similar field of endeavor of virtual reality interaction, Cui discloses wherein the instructions, when executed by the at least one processor, individually or collectively, cause the electronic device to, based on obtaining a user input for performing interaction between the at least one first avatar and the at least one second avatar, display a visual object having a predetermined shape in a vicinity of the at least one second avatar in the virtual reality environment ([0050], “For example, a virtual object A controlled by a user A and a virtual object B controlled by a user B are in a same VR scene. The user A wears the VR kit 120, the user B wears the VR kit 130, and the VR kit 120 establishes a communication connection with the VR kit 130 through the server 110. When the user A controls the virtual object A to trigger an action to the virtual object B, the user B receives corresponding somatosensory feedback through the VR kit 130 worn by the user B”).
In view of the teachings of Silverstein, Domae, Burns and Cui, it would have been obvious to one of ordinary skill in the art, to have avatars interact by using virtual objects as taught by Cui, within the system of Silverstein, Domae and Burns, which improves a user’s experience by allowing the avatars of a plurality of users to interact with each other.
Response to Arguments
Applicant’s arguments with respect to claims 1, 3, 11, and 13 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mitchell (US PGPub 2014/0198096) discloses “The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider)” ([0058]).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY J FRANK whose telephone number is (571)270-7255. The examiner can normally be reached Monday-Thursday 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Benjamin C Lee can be reached at (571)272-2963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EJF/
/BENJAMIN C LEE/Supervisory Patent Examiner, Art Unit 2629
