DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
2. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
4. Claims 1, 7, and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Pollefeys et al. (US Patent Application Publication No. 2020/0311396 A1) in view of Seibert et al. (US Patent Application Publication No. 2018/0261166 A1).
5. Regarding Claim 1, Pollefeys discloses An information processing apparatus comprising: (paragraph [0004] reciting “FIG. 3 shows an example head-mounted display (HMD) device.”) at least one processor which functions as: (paragraph [0116] reciting “The logic subsystem may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic subsystem may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. …”) an image acquisition unit configured to acquire an image of a real space; (paragraph [0039] reciting “… In other implementations, display 302 is at least partially transparent, allowing a user to view presented virtual imagery along with a real-world background viewable through the display to form an augmented-reality experience, such as a mixed-reality experience. …”;
paragraph [0046] reciting “… Alternatively or additionally, imaging system 312 may include one or more color cameras configured to collect image data in one or more color channels (e.g., RGB) in the visible spectrum. Alternatively or additionally, imaging system 312 may include one or more depth cameras configured to collect depth data. In one example, the depth data may take the form of a two-dimensional depth map having a plurality of depth pixels that each indicate the depth from a corresponding depth camera (or other part of HMD device 300) to a corresponding surface in the surrounding physical space. …”)
a control unit configured to control whether or not to link each of the plurality of virtual objects with movement of the virtual viewpoint; (paragraph [0040] reciting “In augmented-reality implementations, display 302 may present augmented-reality objects that appear display-locked and/or world-locked. A display-locked augmented-reality object may appear to move along with a perspective of the user as a pose (e.g., six degrees of freedom (DOF): x/y/z/yaw/pitch/roll) of HMD device 300 changes. As such, a display-locked, augmented-reality object may appear to occupy the same portion of display 302 and may appear to be at the same distance from the user, even as the user moves in the surrounding physical space. A world-locked, augmented-reality object may appear to remain in a fixed location in the physical space, even as the pose of HMD device 300 changes. In some examples, a world-locked object may appear to move in correspondence with movement of a real, physical object. In yet other examples, a virtual object may be displayed as body-locked, in which the object is located to an estimated pose of a user's head or other body part.” Therefore, objects can be either display-locked or world-locked. Display-locked objects are linked to the movement/change in HMD viewpoint (virtual viewpoint) whereas world-locked virtual objects are not linked to movement/change of HMD viewpoint.) and an image generation unit configured to generate a mixed reality image by combining the image of the real space with images of the plurality of virtual objects, (paragraph [0041] reciting “… In some examples, a display device including a non-transparent display may be used to present virtual imagery. Such a display device may overlay virtual imagery (e.g., representations of hand motion and/or objects) on a real-world background presented on the display device as sensed by an imaging system.” Virtual objects are overlaid upon the real-world background.)
While Pollefeys does not explicitly disclose, Seibert discloses an acquisition unit configured to acquire a position of a virtual viewpoint and positions of a plurality of virtual objects; (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.” The viewpoint is determined for the user to view physical objects/virtual entities from any position or angle.) a movement unit configured to move the virtual viewpoint based on a movement instruction from a user; (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.” The movement is based on user walkthrough which causes the viewpoint of the HMD (virtual camera) to change.)
wherein the movement instruction is an instruction for performing movement using a walkthrough function. (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.”
The movement is based on user walkthrough which causes the viewpoint of the HMD (virtual camera) to change.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys with Seibert. Pollefeys and Seibert are both concerned with displaying virtual objects in a mixed reality environment. Seibert discloses allowing the user to walkthrough the physical environment with the HMD in order to view physical or virtual objects from all degrees and angles. This is a beneficial modification for Pollefeys since Pollefeys also discloses displaying virtual objects in world or display locked positions. Therefore, Pollefeys modified by Seibert allows the HMD wear to walk around and view the mixed reality world casually and intuitively.6. Regarding Claim 7, Pollefeys further discloses The information processing apparatus according to claim 1, wherein the plurality of virtual objects are associated with a plurality of real objects. (paragraph [0040] reciting “… In some examples, a world-locked object may appear to move in correspondence with movement of a real, physical object. In yet other examples, a virtual object may be displayed as body-locked, in which the object is located to an estimated pose of a user's head or other body part.”)
7. Regarding Claim 14, Pollefeys discloses An information processing method comprising: (paragraph [0054] reciting “At 402, method 400 includes, at an HMD device, three-dimensionally scanning an environment including a first instance of a designated object…”)
acquiring image of a real space; (paragraph [0039] reciting “… In other implementations, display 302 is at least partially transparent, allowing a user to view presented virtual imagery along with a real-world background viewable through the display to form an augmented-reality experience, such as a mixed-reality experience. …”;
paragraph [0046] reciting “… Alternatively or additionally, imaging system 312 may include one or more color cameras configured to collect image data in one or more color channels (e.g., RGB) in the visible spectrum. Alternatively or additionally, imaging system 312 may include one or more depth cameras configured to collect depth data. In one example, the depth data may take the form of a two-dimensional depth map having a plurality of depth pixels that each indicate the depth from a corresponding depth camera (or other part of HMD device 300) to a corresponding surface in the surrounding physical space. …”)
controlling whether or not to link each of the plurality of virtual objects with movement of the virtual viewpoint; (paragraph [0040] reciting “In augmented-reality implementations, display 302 may present augmented-reality objects that appear display-locked and/or world-locked. A display-locked augmented-reality object may appear to move along with a perspective of the user as a pose (e.g., six degrees of freedom (DOF): x/y/z/yaw/pitch/roll) of HMD device 300 changes. As such, a display-locked, augmented-reality object may appear to occupy the same portion of display 302 and may appear to be at the same distance from the user, even as the user moves in the surrounding physical space. A world-locked, augmented-reality object may appear to remain in a fixed location in the physical space, even as the pose of HMD device 300 changes. In some examples, a world-locked object may appear to move in correspondence with movement of a real, physical object. In yet other examples, a virtual object may be displayed as body-locked, in which the object is located to an estimated pose of a user's head or other body part.” Therefore, objects can be either display-locked or world-locked. Display-locked objects are linked to the movement/change in HMD viewpoint (virtual viewpoint) whereas world-locked virtual objects are not linked to movement/change of HMD viewpoint.) and generating a mixed reality image by combining the image of the real space with images of the plurality of virtual objects, (paragraph [0041] reciting “… In some examples, a display device including a non-transparent display may be used to present virtual imagery. Such a display device may overlay virtual imagery (e.g., representations of hand motion and/or objects) on a real-world background presented on the display device as sensed by an imaging system.” Virtual objects are overlaid upon the real-world background.)
While Pollefeys does not explicitly disclose, Seibert discloses acquiring position of a virtual viewpoint and positions of a plurality of virtual objects; (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.” The viewpoint is determined for the user to view physical objects/virtual entities from any position or angle.) moving the virtual viewpoint based on a movement instruction from a user; (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.” The movement is based on user walkthrough which causes the viewpoint of the HMD (virtual camera) to change.)
wherein the movement instruction is an instruction for performing movement using a walkthrough function. (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.”
The movement is based on user walkthrough which causes the viewpoint of the HMD (virtual camera) to change.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys with Seibert. Pollefeys and Seibert are both concerned with displaying virtual objects in a mixed reality environment. Seibert discloses allowing the user to walkthrough the physical environment with the HMD in order to view physical or virtual objects from all degrees and angles. This is a beneficial modification for Pollefeys since Pollefeys also discloses displaying virtual objects in world or display locked positions. Therefore, Pollefeys modified by Seibert allows the HMD wear to walk around and view the mixed reality world casually and intuitively.8. Regarding Claim 15, Pollefeys discloses A non-transitory computer-readable storage medium storing a program for causing a computer to execute an information processing method, the information processing method comprising: (paragraph [0117] reciting “Storage subsystem 1604 includes one or more physical devices configured to hold instructions executable by the logic subsystem to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage subsystem 1604 may be transformed—e.g., to hold different data.”;
paragraph [0118] reciting “Storage subsystem 1604 may include removable and/or built-in devices. Storage subsystem 1604 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage subsystem 1604 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.”)
acquiring image of a real space; (paragraph [0039] reciting “… In other implementations, display 302 is at least partially transparent, allowing a user to view presented virtual imagery along with a real-world background viewable through the display to form an augmented-reality experience, such as a mixed-reality experience. …”;
paragraph [0046] reciting “… Alternatively or additionally, imaging system 312 may include one or more color cameras configured to collect image data in one or more color channels (e.g., RGB) in the visible spectrum. Alternatively or additionally, imaging system 312 may include one or more depth cameras configured to collect depth data. In one example, the depth data may take the form of a two-dimensional depth map having a plurality of depth pixels that each indicate the depth from a corresponding depth camera (or other part of HMD device 300) to a corresponding surface in the surrounding physical space. …”)
controlling whether or not to link each of the plurality of virtual objects with movement of the virtual viewpoint; (paragraph [0040] reciting “In augmented-reality implementations, display 302 may present augmented-reality objects that appear display-locked and/or world-locked. A display-locked augmented-reality object may appear to move along with a perspective of the user as a pose (e.g., six degrees of freedom (DOF): x/y/z/yaw/pitch/roll) of HMD device 300 changes. As such, a display-locked, augmented-reality object may appear to occupy the same portion of display 302 and may appear to be at the same distance from the user, even as the user moves in the surrounding physical space. A world-locked, augmented-reality object may appear to remain in a fixed location in the physical space, even as the pose of HMD device 300 changes. In some examples, a world-locked object may appear to move in correspondence with movement of a real, physical object. In yet other examples, a virtual object may be displayed as body-locked, in which the object is located to an estimated pose of a user's head or other body part.” Therefore, objects can be either display-locked or world-locked. Display-locked objects are linked to the movement/change in HMD viewpoint (virtual viewpoint) whereas world-locked virtual objects are not linked to movement/change of HMD viewpoint.)
and generating a mixed reality image by combining the image of the real space with images of the plurality of virtual objects, (paragraph [0041] reciting “… In some examples, a display device including a non-transparent display may be used to present virtual imagery. Such a display device may overlay virtual imagery (e.g., representations of hand motion and/or objects) on a real-world background presented on the display device as sensed by an imaging system.” Virtual objects are overlaid upon the real-world background.)
While Pollefeys does not explicitly disclose, Seibert discloses acquiring position of a virtual viewpoint and positions of a plurality of virtual objects; (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.” The viewpoint is determined for the user to view physical objects/virtual entities from any position or angle.) moving the virtual viewpoint based on a movement instruction from a user; (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.” The movement is based on user walkthrough which causes the viewpoint of the HMD (virtual camera) to change.) wherein the movement instruction is a movement instruction using a walkthrough function. (paragraph [0051] reciting “It should be understood that various operations described herein may be implemented in software executed by logic or processing circuitry, hardware, or a combination thereof. …”;
paragraph [0048] reciting “Consider yet another use case where an HMD is used for a work activity such as, for example, building renovation. The HMD may present the user with any from of display ranging from an entirely real physical reality (that is, a transparent display) all the way to a complete, immersive walkthrough of completed renovation (e.g., a VR experience), or any intermediate in-between (MR/AR). In either situation, the user may walk around the building and observe physical objects and/or virtual entities from any position or angle.”
The movement is based on user walkthrough which causes the viewpoint of the HMD (virtual camera) to change.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys with Seibert. Pollefeys and Seibert are both concerned with displaying virtual objects in a mixed reality environment. Seibert discloses allowing the user to walkthrough the physical environment with the HMD in order to view physical or virtual objects from all degrees and angles. This is a beneficial modification for Pollefeys since Pollefeys also discloses displaying virtual objects in world or display locked positions. Therefore, Pollefeys modified by Seibert allows the HMD wear to walk around and view the mixed reality world casually and intuitively.
9. Claims 5-6 and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Pollefeys in view of Seibert and further in view of Li et al. (US Patent Application Publication No. 2022/0139041 A1).
10. Regarding Claim 5, while the combination of Pollefeys and Seibert does not explicitly disclose, Li discloses The information processing apparatus according to claim 1, wherein the control unit controls whether or not to link each of the plurality of virtual objects with the movement of the virtual viewpoint based on a distance between the user and each of the plurality of virtual objects. (paragraph [0047] reciting “In some implementations, the follow-me mode can include three different sub-modes: (1) a world-locked follow mode, (2) a body-locked mode, and (3) a world-locked no-follow mode. A virtual object in the world-locked follow mode can have defined world-locked anchor points. When a user is within a threshold distance or in the same room as such an anchor, the virtual object can be positioned at that anchor. However, when the user is not within the threshold distance or is not within the same room as the anchor, the virtual object can become body locked. A virtual object in the body-locked mode says body locked to the user as the user moves around. A virtual object in the world-locked no-follow mode can determine which anchor defined for a virtual object the user is closest to, and have the virtual object appear there. In some cases, if there is no anchor for the virtual object in the same room as the user or not within a threshold distance, the virtual object can be hidden.” The linking or non-linking of the virtual object in world-locked follow mode corresponds to distance between the user and each individual virtual object that is under world-locked follow mode.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys and Seibert with Li. Pollefeys already discloses world/display locked virtual objects and Li merely expands upon that with more variety. Therefore, it would have been obviously beneficial to modify Pollefeys with Li so that greater variety of locking virtual objects can be generated for the mixed reality world.
11. Regarding Claim 6, while the combination of Pollefeys and Seibert does not explicitly disclose, Li discloses The information processing apparatus according to claim1 , wherein the plurality of virtual objects include a first virtual object and a second virtual object, and when a distance between the user and the first virtual object is shorter than a distance between the user and the second virtual object, the control unit controls such that the first virtual object is not linked with the movement of the virtual viewpoint, and the second virtual object is linked with movement of the virtual viewpoint. (paragraph [0047] reciting “In some implementations, the follow-me mode can include three different sub-modes: (1) a world-locked follow mode, (2) a body-locked mode, and (3) a world-locked no-follow mode. A virtual object in the world-locked follow mode can have defined world-locked anchor points. When a user is within a threshold distance or in the same room as such an anchor, the virtual object can be positioned at that anchor. However, when the user is not within the threshold distance or is not within the same room as the anchor, the virtual object can become body locked. A virtual object in the body-locked mode says body locked to the user as the user moves around. A virtual object in the world-locked no-follow mode can determine which anchor defined for a virtual object the user is closest to, and have the virtual object appear there. In some cases, if there is no anchor for the virtual object in the same room as the user or not within a threshold distance, the virtual object can be hidden.” The linking or non-linking of the virtual object in world-locked follow mode corresponds to distance between the user and each individual virtual object that is under world-locked follow mode. For example, if the user is within threshold distance of a first virtual object but outside the threshold distance of a second virtual object, the second virtual object will become body-locked to the user.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys and Seibert with Li. Pollefeys already discloses world/display locked virtual objects and Li merely expands upon that with more variety. Therefore, it would have been obviously beneficial to modify Pollefeys with Li so that greater variety of locking virtual objects can be generated for the mixed reality world.
12. Regarding Claim 10, while the combination of Pollefeys and Seibert does not explicitly disclose, Li discloses The information processing apparatus according to claim 7, wherein the control unit controls whether or not to link each of the plurality of virtual objects with the movement of the virtual viewpoint based on a distance between the user and each of the plurality of real objects. (paragraph [0047] reciting “In some implementations, the follow-me mode can include three different sub-modes: (1) a world-locked follow mode, (2) a body-locked mode, and (3) a world-locked no-follow mode. A virtual object in the world-locked follow mode can have defined world-locked anchor points. When a user is within a threshold distance or in the same room as such an anchor, the virtual object can be positioned at that anchor. However, when the user is not within the threshold distance or is not within the same room as the anchor, the virtual object can become body locked. A virtual object in the body-locked mode says body locked to the user as the user moves around. A virtual object in the world-locked no-follow mode can determine which anchor defined for a virtual object the user is closest to, and have the virtual object appear there. In some cases, if there is no anchor for the virtual object in the same room as the user or not within a threshold distance, the virtual object can be hidden.” The linking or non-linking of the virtual object in world-locked follow mode corresponds to distance between the user and each individual virtual object that is under world-locked follow mode.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys and Seibert with Li. Pollefeys already discloses world/display locked virtual objects and Li merely expands upon that with more variety. Therefore, it would have been obviously beneficial to modify Pollefeys with Li so that greater variety of locking virtual objects can be generated for the mixed reality world.
13. Regarding Claim 11, Pollefeys further discloses The information processing apparatus according to claim 1, wherein the plurality of virtual objects include a first virtual object and a second virtual object, the first virtual object is associated with a first real object, the second virtual object is associated with a second real object, (paragraph [0040] reciting “… In some examples, a world-locked object may appear to move in correspondence with movement of a real, physical object. In yet other examples, a virtual object may be displayed as body-locked, in which the object is located to an estimated pose of a user's head or other body part.”)
Wwhile the combination of Pollefeys and Seibert does not explicitly disclose, Li discloses and when a distance between the user and the first real object is shorter than a distance between the user and the second real object, the control unit controlssuch that the first virtual object is not linked with movement of the virtual viewpoint, and the second virtual object is linked with movement of the virtual viewpoint. (paragraph [0047] reciting “In some implementations, the follow-me mode can include three different sub-modes: (1) a world-locked follow mode, (2) a body-locked mode, and (3) a world-locked no-follow mode. A virtual object in the world-locked follow mode can have defined world-locked anchor points. When a user is within a threshold distance or in the same room as such an anchor, the virtual object can be positioned at that anchor. However, when the user is not within the threshold distance or is not within the same room as the anchor, the virtual object can become body locked. A virtual object in the body-locked mode says body locked to the user as the user moves around. A virtual object in the world-locked no-follow mode can determine which anchor defined for a virtual object the user is closest to, and have the virtual object appear there. In some cases, if there is no anchor for the virtual object in the same room as the user or not within a threshold distance, the virtual object can be hidden.” The linking or non-linking of the virtual object in world-locked follow mode corresponds to distance between the user and each individual virtual object that is under world-locked follow mode. For example, if the user is within threshold distance of a first virtual object but outside the threshold distance of a second virtual object, the second virtual object will become body-locked to the user.)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys and Seibert with Li. Pollefeys already discloses world/display locked virtual objects and Li merely expands upon that with more variety. Therefore, it would have been obviously beneficial to modify Pollefeys with Li so that greater variety of locking virtual objects can be generated for the mixed reality world.
14. Claims 8-9 are rejected under 35 U.S.C. 103 as being unpatentable over Pollefeys in view of Seibert and further in view of Yousuke Koyanaka et al. (US Patent Application Publication No. 2018/0158247 A1).
15. Regarding Claim 8, Pollefeys further discloses The information processing apparatus according to claim 7, wherein the acquisition unit acquires the position of the virtual viewpoint in a world coordinate system, (paragraph [0030] reciting “In addition to animating representation 208 in accordance with the time-varying pose of hand 104 recorded in instructor environment 108, the representation may be consistent with other attributes of the recorded hand motion. With respect to the time instances depicted in FIGS. 1A and 2A, the three-dimensional position (e.g., x/y/z), three-dimensional orientation (e.g., yaw/pitch/roll), and scale of representation 208 relative to light switch 210 are substantially equal to the three-dimensional position, three-dimensional orientation, and scale of hand 104 relative to light switch 106. …” The 3D position x/y/z corresponds to a world coordinate system for the HMD viewpoint.)
While the combination of Pollefeys and Seibert does not explicitly disclose, Koyanaka discloses and for each of the plurality of real objects, an individual coordinate system different from the world coordinate system is set. (paragraph [0170] reciting “Subsequently, as illustrated at the bottom of FIG. 13, the acquired distance is set to the center position Pn of each marker on the world coordinate system. Here, since the local coordinate system of each virtual 3D object is correlated with the coordinate system of each marker, by setting the distance in this manner, it is possible to represent the position of each virtual 3D object can show the position on the same coordinate system and to check interference between virtual 3D objects.”)
It would have been obvious to a person of ordinary skills in the art before the effective filing date of the claimed invention to modify Pollefeys and Seibert with Koyanaka. Pollefeys virtual objects that are either body-locked or world-locked and Koyanaka discloses a coordinate system for each virtual object to ensure that each virtual object does not interfere with another virtual object. Therefore, it is obviously beneficial to modify Pollefeys with Koyanaka to ensure the virtual objects, either body or world locked, are not interfering with each other.
16. Regarding Claim 9, Koyanaka further discloses The information processing apparatus according to claim 8, wherein each of the plurality of real objects is provided with a marker associated with the individual coordinate system. paragraph [0170] reciting “Subsequently, as illustrated at the bottom of FIG. 13, the acquired distance is set to the center position Pn of each marker on the world coordinate system. Here, since the local coordinate system of each virtual 3D object is correlated with the coordinate system of each marker, by setting the distance in this manner, it is possible to represent the position of each virtual 3D object can show the position on the same coordinate system and to check interference between virtual 3D objects.”)
Allowable Subject Matter
17. Claims 2-4 and 12-13 are objected to as being dependent upon a rejected base claims, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
18. The following is a statement of reasons for the indication of allowable subject matter: Claim 2 recites the limitation wherein the walkthrough function is a function of moving the virtual viewpoint within a virtual space without involving physical movement of the user which is neither disclosed nor suggested by the cited references, either singly or in combination. Claims 3-4 depend on claim 2.
19. The closest art of record is Pollefeys. While Pollefeys discloses HMD for experiencing the mixed-reality environment and for viewing virtual objects, nothing in Pollefeys discloses the above limitation of moving the virtual viewpoint within the virtual space without involving actual physical movement of the user.
20. Claim 12 recites the limitation wherein the control unit controls whether or not to link each of the plurality of virtual objects with the movement of the virtual viewpoint based on whether the user is in contact with each of the plurality of real objects which is neither disclosed nor suggested by the cited references, either singly or in combination. While the closest art of record Li discloses body-locked and world-locked virtual objects, nothing in Li discloses or suggests using the actual contact with virtual objects to determine whether to link the object to the body. While Li discloses grabbing a virtual object, there is a lack of teaching or suggesting that the virtual object’s body-lock or world-lock mode will change as a result of said grabbing.
21. Claim 13 recites the limitation and when the user is in contact with the first real object and not in contact with the second real object, the control unit controls such that the first virtual object is not linked with the movement of the virtual viewpoint, and the second virtual object is linked with the movement of the virtual viewpoint which is neither disclosed nor suggested by the cited references, either singly or in combination. While the closest art of record Li discloses body-locked and world-locked virtual objects, nothing in Li discloses or suggests using the actual contact with virtual objects to determine whether to link the object to the body. While Li discloses grabbing a virtual object, there is a lack of teaching or suggesting that the virtual object’s body-lock or world-lock mode will change as a result of said grabbing.
CONTACT
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FRANK S CHEN whose telephone number is (571)270-7993. The examiner can normally be reached Mon - Fri 8-11:30 and 1:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at 5712727794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/FRANK S CHEN/Primary Examiner, Art Unit 2611
Prosecution Insights