DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
1. This action is responsive to communications: Amendment, filed on 0/10/2026.
This action is made FINAL.
2. Claims 1, 4-5 and 7-23 are pending in the case. Claims 1, 19 and 20 are independent claims. Claims 1, 4, 5, 7-10, 12-17, 19, and 20 have been amended. Claims 2-3 and 6 are cancelled. Claims 21-23 are newly added.
Response to Arguments
Applicant's arguments filed July 10, 2026 have been fully considered but they are not persuasive.
Applicant argues claim 1 includes “match a coordinate system set in a first space in which a first user is present with a coordinate system set in a second space in which a second user is present such that ... positional information of the first user in the first space corresponds to a first position in the second space and positional information of the second user in the second space corresponds to a second position in the first space ... the second user being different from the first user”. Nakajima does not disclose that positional information of a user in one space corresponds to a position in the other space (in either direction).
In response, Nakajima discloses a server providing virtual reality to each of a plurality of users to enable each user to see the same image and enjoy a common scene (Fig. 10; p. 4, Para 10-11). Nakajima additionally discloses (Fig. 3, p. 4) setting an HMD uvw view coordinate system parallel to the real coordinate system (xyz) to maintain a constant relationship regardless of change in position/inclination. The virtual space includes partial meshes defined in an XYZ coordinate system that is a global coordinate system defined in the virtual space (p. 5, Para 3). An HMD set includes HMDs respectively associated with the different users to individually present a virtual space, where the virtual spaces are configured by the same data to provide a shared virtual space (Fig. 12A) that shares a positional representation of each user/device (Fig. 15). An HMD in the set may provide an augmented reality space (p. 19, Para 2). Thus, as the same data is used to provide a shared environment amongst multiple user devices viewing a same scene including users simultaneously viewed at differing positions, Nakajima discloses “match a coordinate system set in a first space in which a first user is present with a coordinate system set in a second space in which a second user is present such that ... positional information of the first user in the first space corresponds to a first position in the second space and positional information of the second user in the second space corresponds to a second position in the first space ... the second user being different from the first user”.
Applicant argues (Claim 1) Nakajima fails to disclose "present the first space to the first user based on three-dimensional map data of the second space at the first position, the three-dimensional map data being read from a three-dimensional map data server."
In response, Nakajima HMD users immersed in a 3D virtual space (Fig. 1; p. 3, Para 9). The virtual space includes partial meshes defined in an XYZ coordinate system that is a global coordinate system defined in the virtual space (p. 5, Para 3). A server communicates with a plurality of HMDs to provide the same virtual space for a common game/environment (Fig. 1 “600”). The uvw view coordinate system of the virtual camera 14 in the virtual space 11 is defined so as to be linked to the uvw view coordinate system of the HMD 120 in the real space (real coordinate system) (p. 6, Para 4). The view area 15 corresponds to an area in the virtual space 11 that the user 5 wearing the HMD 120 visually recognizes (p. 6, Para 5). A memory module 530 holds space information, object information, and user information. The space information holds one or more templates defined for providing the virtual space 11 (p. 8, Para 14-15). A server stores, integrates and transmits avatar information to sync shared display among the HMD set (p. 10, Para 6). As player information/updates are received from an HMD, processor 210A updates information (position, orientation, and the like) of the avatar object 6B included in the object information stored in the memory module 530 (p. 10, Para 7; Fig. 13 “S1330A”). Thus, Nakajima discloses "present the first space to the first user based on three-dimensional map data of the second space at the first position, the three-dimensional map data being read from a three-dimensional map data server."
Applicant argues (Claim 1) Nakajima fails to disclose "displaying a first avatar corresponding to the first user at the first position in the second space and a second avatar corresponding to the second user at the second position in the first space."
In response, Nakajima discloses a first avatar (Fig. 12A, 15A “6A”) in a virtual space is visible to a second user HMD device that provides a second user avatar (Fig. 12A “6B”) to the first device. Each user’s avatar is represented at a respective position in the space viewed by the other user. Thus, Nakajima discloses "displaying a first avatar corresponding to the first user at the first position in the second space and a second avatar corresponding to the second user at the second position in the first space."
Applicant argues (claim 1) Nakajima fails to disclose "in response to motion of the first user in the first space update the first position in the second space ... re-read the three-dimensional map data ... at the updated first position ... display the first avatar at the updated first position ... and present the first space ... based on the re-read three-dimensional map data."
In response, Nakajima discloses user motion updates the image in the virtual space on the first device (Fig. 7), wherein an update of avatar information is transmitted from a server to each of the HMD sets, resulting in changes in avatar objects being presented to other HMD users (Fig. 13 “S1330A/B). The server stores, integrates and transmits avatar information to sync shared display among the HMD set (p. 10, Para 6). Thus, Nakajima discloses “in response to motion of the first user in the first space update the first position in the second space ... re-read the three-dimensional map data ... at the updated first position ... display the first avatar at the updated first position ... and present the first space ... based on the re-read three-dimensional map data."
Applicant argues independent claims 1, 19 and 20 are patentable as Nakajima fails to disclose combination of features recited by the independent claims.
In response, independent claims 1, 19 and 20 are not patentable based on the above response to Applicant’s Remarks. Additionally, the dependent claims are not patentable at least by virtue of their respective dependencies from rejected base claims.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 4-5, 7-21 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kento Nakakima, JP 2020004388 A.
Independent claim 1, Nakajima discloses an information processing device comprising:
Processing circuitry (Fig. 12A “200”) configured to match a coordinate system set in a first space in which a first user is present with a coordinate system set in a second space in which a second user is present (i.e. In one embodiment, in the HMD system 100, a real coordinate system that is a coordinate system in the real space is set in advance…the real coordinate system are defined as x-axis, y-axis, and z-axis, respectively - computer hardware configuration, Fig. 2; The HMD sensor 410 sets the uvw visual field coordinate system to the HMD 120 based on the inclination of the HMD 120 in the real coordinate system. The uvw visual field coordinate system set in the HMD 120 corresponds to a viewpoint coordinate system when the user 5 wearing the HMD 120 views an object in a virtual space - computer hardware configuration, Fig. 2) such that, in the matched coordinate system, positional information of the first user in the first space corresponds to a first position in the second space and positional information of the second user in the second space corresponds to a second position in the first space (i.e. setting an HMD uvw view coordinate system parallel to the real coordinate system (xyz) to maintain a constant relationship regardless of change in position/inclination. – Fig. 3; p. 4; The virtual space includes partial meshes defined in an XYZ coordinate system that is a global coordinate system defined in the virtual space - p. 5, Para 3; An HMD set includes HMDs respectively associated with the different users to individually present a virtual space, where the virtual spaces are configured by the same data to provide a shared virtual space - Fig. 12A – and shares a positional representation of each user/device - Fig. 15; An HMD in the set may provide an augmented reality space - p. 19, Para 2; e.g. the same data is used to provide a shared environment amongst multiple user devices viewing a same scene including users simultaneously viewed at differing positions), the first space being a VR space (Fig. 15A) and the second space being a real space (i.e. a second terminal corresponding to a second user HMD in a real space – Fig. 35 “3594”; The system 1700 may include a device that reflects information in the virtual space 11 to the real space. The device provides a virtual experience to an audience 1636 in an Augmented Reality (AR) space, for example. For example, the apparatus may include a display unit that displays a real space, and may display a first image corresponding to information in the virtual space 11 on the display unit together with the real space. Note that “displaying the real space” may be, for example, displaying an image (second image) of the real space, or displaying the real space as it is. - Example 1 of reflecting virtual space information to real space, Fig. 17);
present the first space to the first user based on three-dimensional map data of the
second space at the first position, the three-dimensional map data being read from a three-dimensional map data server (i.e. HMD users immersed in a 3D virtual space - Fig. 1; p. 3, Para 9; The virtual space includes partial meshes defined in an XYZ coordinate system that is a global coordinate system defined in the virtual space - p. 5, Para 3; A server communicates with a plurality of HMDs to provide the same virtual space for a common game/environment - Fig. 1 “600”; The uvw view coordinate system of the virtual camera 14 in the virtual space 11 is defined so as to be linked to the uvw view coordinate system of the HMD 120 in the real space (real coordinate system) - p. 6, Para 4; The view area 15 corresponds to an area in the virtual space 11 that the user 5 wearing the HMD 120 visually recognizes - p. 6, Para 5; A memory module 530 holds space information, object information, and user information. The space information holds one or more templates defined for providing the virtual space 11 (p. 8, Para 14-15). A server stores, integrates and transmits avatar information to sync shared display among the HMD set - p. 10, Para 6; As player information/updates are received from an HMD, processor 210A updates information (position, orientation, and the like) of the avatar object 6B included in the object information stored in the memory module 530 - p. 10, Para 7; Fig. 13 “S1330A”);
control presentation of an interaction with one user in accordance with a motion of the other user on a basis of the positional information and motion information of the first user and positional information and motion information of the second user in the matched coordinate system (i.e. control presentation to display a VR experience of each user, e.g. a performer and a spectator, where position and motion of the performer is viewed by the spectator(s) and the spectator position and motion (including movement of a light stick) is updated to all users in the experience – Fig. 12A, 15, 16, 27), including displaying a first avatar corresponding to the first user at the first position in the second space and a second avatar corresponding to the second user at the second position in the first space (Figs. 12A, 15); and
in response to motion of the first user in the first space update the first position in the second space in accordance with the motion (i.e. user motion updates the image in the virtual space on the first device – Fig. 7; an update of avatar information is transmitted from a server to each of the HMD sets, resulting in changes in avatar objects being presented to other HMD users – Fig. 13 “S1330A/B), re-read the three-dimensional map data of the second space at the updated first position from the three-dimensional map data server, display the first avatar at the updated first position in the second space, and present the first space to the first user based on the re-read three-dimensional map data (i.e. The view area 15 corresponds to an area in the virtual space 11 that the user 5 wearing the HMD 120 visually recognizes - p. 6, Para 5; A memory module 530 holds space information, object information, and user information. The space information holds one or more templates defined for providing the virtual space 11 (p. 8, Para 14-15). A server stores, integrates and transmits avatar information to sync shared display among the HMD set - p. 10, Para 6; As player information/updates are received from an HMD, processor 210A updates information (position, orientation, and the like) of the avatar object 6B included in the object information stored in the memory module 530 - p. 10, Para 7; Fig. 13 “S1330A”; user motion updates the image in the virtual space on the first device – Fig. 7; an update of avatar information is transmitted from a server to each of the HMD sets, resulting in changes in avatar objects being presented to other HMD users – Fig. 13 “S1330A/B).
Claim 4, Nakajima discloses the information processing device according to claim 2, the information processing device according to wherein the presentation control unit controls presentation of the interaction using AR information to the first user or the second user present in the real space (i.e. FIG. 12A is a schematic diagram illustrating a situation where each HMD 120 provides the user 5 with the virtual space 11 in the network 2. The computers 200A to 200D provide the virtual spaces 11A to 11D to the users 5A to 5D via the HMDs 120A to 120D, respectively. In the example shown in FIG. 12A, the virtual space 11A and the virtual space 11B are configured by the same data. – Avatar object, Fig. 12A).
Claim 5, Nakajima discloses the information processing device according to claim 1, the information processing device according to wherein the presentation control unit controls presentation of the interaction in a case where each of the first user and the second user has an account of the same service (i.e. FIG. 12A is a schematic diagram illustrating a situation where each HMD 120 provides the user 5 with the virtual space 11 in the network 2. The computers 200A to 200D provide the virtual spaces 11A to 11D to the users 5A to 5D via the HMDs 120A to 120D, respectively. In the example shown in FIG. 12A, the virtual space 11A and the virtual space 11B are configured by the same data. – Avatar object, Fig. 12A).
Claim 7, Nakajima discloses the information processing device according to claim 6, the information processing device according to wherein the presentation control unit controls real-time presentation of the interaction to the second user in the second space in accordance with a direct motion of the first user in the first space with respect to the second avatar (i.e. The virtual space 11A shown in FIG. 15A is constructed by playing live content on the computer 200A. In the virtual space 11A, the avatar object 1533 performs a performance as a live performer, and other avatar objects including the avatar object 6A watch the performance as a live viewer. In virtual space 11A, avatar objects 6B and 6C are individually associated with users 5B and 5C, respectively. Although details will be described later, the avatar object 1533 is associated with a performer performing a live performance in a real space (hereinafter, simply referred to as a “performer”). The avatar object 1534 is an avatar object that is not associated with the user 5 and the performer (e.g. display based on position and motion of each user) – Audience virtual space – Fig. 15A).
Claim 8, Nakajima discloses the information processing device according to claim 6, to wherein the presentation control unit controls presentation of the interaction to the second user in the second space with a time delay in accordance with an indirect motion of the first user in the first space with respect to the second avatar (i.e. More specifically, the HMD sensor 410 can detect a temporal change in the position and the inclination of the HMD 120 using each value detected over time – computer hardware configuration, Fig. 2; In one aspect, a motion sensor attached to the performer 1635 detects the arrival time and angle of a signal (for example, an infrared laser) emitted from the base station 1659. The processor of the computer 1791 detects the position of the motion sensor with respect to the base station 1659 based on the detection result of the motion sensor. The processor may further normalize the position of the motion sensor with respect to the base station 1659 on the basis of a predetermined point (for example, the position of the motion sensor 1655 mounted on the head). The processor detects the movement of the performer 1635 based on the detected position of the motion sensor; The processor generates information including the current position information and the rotation direction, that is, information on the detected motion (hereinafter, referred to as “first motion information”), and transmits the information to the computer 200 via the server 600. The processor 210 executes a performance in which an avatar object 1533 arranged in the virtual space 11 for providing the user 5 with a virtual experience reflects the movement of the performer 1635 in a real space live according to the first motion information - Live in real space, Fig. 16).
Claim 9, Nakajima discloses the information processing device according to claim 8, the information processing device according to wherein the presentation control unit generates a virtual object as the interaction with the second user, and changes a presentation mode of the virtual object in accordance with an operation of the virtual object by the second user (i.e. In the example of FIG. 16, the spectator 1636 holds a chemical light 1658 (first terminal device). In the present embodiment, the chemical light 1658 includes a sensor (for example, an acceleration sensor) that detects the movement of the chemical light 1658. That is, the sensor can detect an operation (first operation) in which the audience 1636 shakes the chemical light 1658. Information indicating that the sensor has detected movement (hereinafter, referred to as “second movement information”) is transmitted to the server 600 via the gateway 1793 (not shown in FIG. 16) – Live in real space, Fig. 16).
Claim 10, Nakajima discloses the information processing device according to claim 6, The information processing device according to wherein the presentation control unit changes a presentation mode of the first avatar in the second space in accordance with a state change of the first user in the first space (i.e. The operation of the user 5B (the operation of the HMD 120B and the operation of the controller 300B) is reflected on the avatar object 6B arranged in the virtual space 11A by the processor 210A (e.g. presentation of a notification). Thereby, the user 5A can recognize the action of the user 5B through the avatar object 6B. – Fig. 12B).
Claim 11, Nakajima discloses the information processing device according to claim 10, The information processing device according to wherein the state change includes wearing of clothes, wearing of an article, and changing of a physical state (i.e. As shown in FIGS. 15 and 16, it is preferable that the avatar object 1533 associated with the performer 1635 is an avatar object very similar to the performer 1635. In other words, the avatar object 1533 is preferably a highly accurate 3D model of the performer 1635. Thereby, it is possible to give the user 5 who watches the live in the virtual space 11 a virtual experience as if he / she were watching the live of the performer 1635 – live in real space, Fig. 15, 16; continual update of the position of a swinging light held by user’s avatar – Fig. 27).
Claim 12, Nakajima discloses the information processing device according to claim 6, wherein the presentation control unit controls presentation of the object in the second space on a basis of a recognition result of the object designated by the first user in the first space (i.e. continual update of the position of a swinging light held by user’s avatar – Fig. 27).
Claim 13, Nakajima discloses the information processing device according to claim 1, further comprising a space management unit configured to manage a plurality of the coexisting first spaces, wherein the matching processing unit matches the coordinate system set in the first space with the coordinate system set in the second space for each of a plurality of the first spaces (i.e. FIG. 12A is a schematic diagram illustrating a situation where each HMD 120 provides the user 5 with the virtual space 11 in the network 2. The computers 200A to 200D provide the virtual spaces 11A to 11D to the users 5A to 5D via the HMDs 120A to 120D, respectively. In the example shown in FIG. 12A, the virtual space 11A and the virtual space 11B are configured by the same data. In other words, the computer 200A and the computer 200B share the same virtual space – Avatar object, Fig. 12A).
Claim 14, Nakajima discloses the information processing device according to claim 13, wherein the presentation control unit controls presentation of a notification regarding the first space in which the first user is present to the second user present in the second space in accordance with a motion of the first user present in any one of the first spaces (i.e. FIG. 12A is a schematic diagram illustrating a situation where each HMD 120 provides the user 5 with the virtual space 11 in the network 2. The computers 200A to 200D provide the virtual spaces 11A to 11D to the users 5A to 5D via the HMDs 120A to 120D, respectively. In the example shown in FIG. 12A, the virtual space 11A and the virtual space 11B are configured by the same data. – Avatar object, Fig. 12A; The operation of the user 5B (the operation of the HMD 120B and the operation of the controller 300B) is reflected on the avatar object 6B arranged in the virtual space 11A by the processor 210A (e.g. presentation of a notification). Thereby, the user 5A can recognize the action of the user 5B through the avatar object 6B. – Fig. 12B; The virtual space 11A shown in FIG. 15A is constructed by playing live content on the computer 200A. In the virtual space 11A, the avatar object 1533 performs a performance as a live performer, and other avatar objects including the avatar object 6A watch the performance as a live viewer. In virtual space 11A, avatar objects 6B and 6C are individually associated with users 5B and 5C, respectively. Although details will be described later, the avatar object 1533 is associated with a performer performing a live performance in a real space (hereinafter, simply referred to as a “performer”). The avatar object 1534 is an avatar object that is not associated with the user 5 and the performer (e.g. notification of motion of user present in any space) – Audience virtual space; Fig. 15A).
Claim 15, Nakajima discloses the information processing device according to claim 13, the information processing device according to wherein the presentation control unit controls presentation of a notification regarding one of the first spaces to the first user present in another one of the first spaces (i.e. The virtual space 11A shown in FIG. 15A is constructed by playing live content on the computer 200A. In the virtual space 11A, the avatar object 1533 performs a performance as a live performer, and other avatar objects including the avatar object 6A watch the performance as a live viewer. In virtual space 11A, avatar objects 6B and 6C are individually associated with users 5B and 5C, respectively. Although details will be described later, the avatar object 1533 is associated with a performer performing a live performance in a real space (hereinafter, simply referred to as a “performer”). The avatar object 1534 is an avatar object that is not associated with the user 5 and the performer (e.g. notification of motion of user present in any space) – Audience virtual space; Fig. 15A).
Claim 16, Nakajima discloses the information processing device according to claim 13, wherein in a case where the first user is present in one of the first spaces, when the interaction with the first user occurs in another one of the first spaces, the presentation control unit controls presentation of a movement confirmation to another one of the first spaces (i.e. The virtual space 11A shown in FIG. 15A is constructed by playing live content on the computer 200A. In the virtual space 11A, the avatar object 1533 performs a performance as a live performer, and other avatar objects including the avatar object 6A watch the performance as a live viewer. In virtual space 11A, avatar objects 6B and 6C are individually associated with users 5B and 5C, respectively. Although details will be described later, the avatar object 1533 is associated with a performer performing a live performance in a real space (hereinafter, simply referred to as a “performer”). The avatar object 1534 is an avatar object that is not associated with the user 5 and the performer (e.g. notification of motion of user present in any space) – Audience virtual space; Fig. 15A).
Claim 17, Nakajima discloses the information processing device according to claim 1, the information processing device according to wherein the interactions are presented by devices held or worn by the first user and the second user (i.e. In another aspect, server 600 may communicate with other computers 200 for providing virtual reality to HMD 120 used by other users. For example, when a plurality of users play a participatory game in an amusement facility, each computer 200 communicates a signal based on the operation of each user with another computer 200 via the server 600, and transmits a plurality of signals in the same virtual space. Of users can enjoy a common game. – Configuration of HMD system, Fig. 1; Fig. 12A “120A, 120B”; Fig. 15 “120A”).
Claim 18, Nakajima discloses the information processing device according to claim 17, the information processing device according to wherein each of the devices is at least one of an HMD, goggles, glasses (i.e. user/viewers wear goggles/HMD - Fig. 15 “120A”), and a mobile terminal.
Independent claim 19, the claim is similar in scope to claim 1. Therefore, similar rationale as applied in the rejection of claim 1 applies herein.
Independent claim 20, the claim is similar in scope to claim 1. Therefore, similar rationale as applied in the rejection of claim 1 applies herein.
Claim 21, Nakajima discloses the information processing device according to claim 1, wherein the processing circuitry is configured to adjust a movement amount of the first user in the first space in accordance with an input from a controller connected to a device of the first user, the movement amount corresponding to one step of the first user being set to one of a scale equal to a scale of the real space (i.e. The virtual camera 14 similarly moves in the virtual space 11 in conjunction with the movement of the HMD 120 in the real space. Thereby, the change of the position and the inclination of the HMD 120 in the real space can be similarly reproduced in the virtual space 11. – p. 5, Para 3) and a unit of one kilometer.
Claims 22 and 23 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHANTE HARRISON whose telephone number is (571)272-7659. The examiner can normally be reached Monday - Friday 8:00 am to 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHANTE E HARRISON/Primary Examiner, Art Unit 2615
