DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Jentz et al. (USPN 10,845,895 B1) in view of Gullicksen et al. (USPN 2022/0308342 A1).
As to claim 1, Jentz teaches one or more non-transitory computer-readable storage media having stored therein instructions that, when executed, cause one or more processors to perform operations (see col. 4 line 65 – col. 5 line 3 “the electronic interface 134 may include a wireless communication module that may be configured to provide data from the mutli-DOF sensor module 108 and/or from the mouse module 120 to at least one processor that is configured for controlling the 3D (e.g., artificial-reality) computing environment and/or the 2D computing environment.”; col. 6 lines 40-55 “the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes that may generate data for sensing the position and/or rotation of the handheld controller. Such data may be transmitted to at least one processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 3D computing environment (e.g., an artificial-reality environment)”; col. 7 lines 9-22 “the mouse module may generate data regarding a position and/or movement of the handheld controller relative to the physical surface. The data may be transmitted to a processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 2D computing environment (e.g., a productivity environment).”) comprising:
in a first operation mode, operating a first object based on mouse sensor data outputted from a mouse sensor of an input device (see at least col. 3 lines 37-40 “a mouse sensor 122 (e.g., an optical mouse sensor, a roller sensor, etc.) for tracking movement of the mouse module 120 relative to the physical surface.”; col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, such as in a 2D (e.g., productivity) computing environment to move a cursor or other object.”; col. 7 lines 9-11 “data from a mouse sensor of the mouse module may be received to sense movement of the handheld controller relative to the physical surface.”);
in a second operation mode, operating a second object based on gyro sensor data outputted from a gyro sensor of the input device (see at least col. 3 lines 20-22 “the multi-DOF sensor module 108 may include one or more accelerometers and one or more gyroscopes.”; col. 4 lines 22-24 “In the multi-DOF mode, data from the multi-DOF sensor module 108 may be utilized, such as in a 3D (e.g., artificial-reality) computing environment.”; col. 6 lines 13-18 “At least one multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment, such as to manipulate digital objects, make virtual selections, etc.”; col. 6 lines 48-55 “the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes that may generate data for sensing the position and/or rotation of the handheld controller. Such data may be transmitted to at least one processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 3D computing environment”); and
based on a first condition being satisfied, shifting from the first operation mode to the second operation mode (see at least col. 4 lines 5-9 “The switch 132 may also be configured to deactivate the mouse sensor 122 and activate the multi-DOF sensor module 108 when the proximity sensor 124 indicates that the mouse sensor 122 is not proximate to (e.g., removed from) the physical surface.”; col. 4 lines 10-15 “the handheld controller 100 may be switched between a mouse mode in which the mouse sensor 122 is active and a multi-DOF mode in which the multi-DOF sensor module 108 is active by simply placing the mouse module 120 against a physical surface or lifting the mouse module 120 away from the physical surface.”; col. 6 lines 30-35 “When the user 502 desires to use the handheld controller 504 in a 3D computer environment, the user 502 may lift the handheld controller 504 off of the physical surface 506, and the handheld controller 504 may automatically switch to a multi-DOF mode.”; col. 7 lines 23-28 “another signal from the proximity sensor, indicating that the handheld controller has been removed from its position proximate to the physical surface, may be received. In response to receiving this other signal, the mouse module may be deactivated and the multi-DOF sensor module may be activated.” – note removal or separation of the input device from physical surface shifts mouse mode to multi-DOF mode).
Jentz does not directly teach performing gyro correction for a direction perpendicular to a gravity direction.
Gullicksen teaches one or more non-transitory computer-readable storage media having stored therein instructions that, when executed, cause one or more processors to perform operations (see Gullicksen at least [0503] “the improvement or portions thereof may be embodied as a computer program product including one or more non-transient, computer-readable storage media, … The media may be encoded with instructions which, when executed on one or more computers or other processors, perform the process or processes described herein.”), comprising:
performing gyro correction for a direction perpendicular to a gravity direction (see at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.” - note a horizontal plane is perpendicular to the gravity direction).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Jentz so that, when removal from the physical surface causes activation of the gyro based multi-DOF mode, the controller performs Gullicksen’s gyro/yaw correction in the horizontal plane. Performing the correction when entering the gyro based mode would predictably initialize or update the pointing direction thereby reducing drift and improving pointing accuracy.
As to claim 10, Jentz teaches an information processing system comprising one or more processors and one or more non-transitory computer-readable storage media having stored therein instructions that, when executed, cause the one or more processors to perform operations (see col. 4 line 65 – col. 5 line 3 “the electronic interface 134 may include a wireless communication module that may be configured to provide data from the mutli-DOF sensor module 108 and/or from the mouse module 120 to at least one processor that is configured for controlling the 3D (e.g., artificial-reality) computing environment and/or the 2D computing environment.”; col. 6 lines 40-55 “the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes that may generate data for sensing the position and/or rotation of the handheld controller. Such data may be transmitted to at least one processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 3D computing environment (e.g., an artificial-reality environment)”; col. 7 lines 9-22 “the mouse module may generate data regarding a position and/or movement of the handheld controller relative to the physical surface. The data may be transmitted to a processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 2D computing environment (e.g., a productivity environment).”) comprising:
in a first operation mode, operating a first object based on mouse sensor data outputted from a mouse sensor of an input device (see at least col. 3 lines 37-40 “a mouse sensor 122 (e.g., an optical mouse sensor, a roller sensor, etc.) for tracking movement of the mouse module 120 relative to the physical surface.”; col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, such as in a 2D (e.g., productivity) computing environment to move a cursor or other object.”; col. 7 lines 9-11 “data from a mouse sensor of the mouse module may be received to sense movement of the handheld controller relative to the physical surface.”);
in a second operation mode, operating a second object based on gyro sensor data outputted from a gyro sensor of the input device (see at least col. 3 lines 20-22 “the multi-DOF sensor module 108 may include one or more accelerometers and one or more gyroscopes.”; col. 4 lines 22-24 “In the multi-DOF mode, data from the multi-DOF sensor module 108 may be utilized, such as in a 3D (e.g., artificial-reality) computing environment.”; col. 6 lines 13-18 “At least one multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment, such as to manipulate digital objects, make virtual selections, etc.”; col. 6 lines 48-55 “the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes that may generate data for sensing the position and/or rotation of the handheld controller. Such data may be transmitted to at least one processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 3D computing environment”); and
based on a first condition being satisfied, shifting from the first operation mode to the second operation mode (see at least col. 4 lines 5-9 “The switch 132 may also be configured to deactivate the mouse sensor 122 and activate the multi-DOF sensor module 108 when the proximity sensor 124 indicates that the mouse sensor 122 is not proximate to (e.g., removed from) the physical surface.”; col. 4 lines 10-15 “the handheld controller 100 may be switched between a mouse mode in which the mouse sensor 122 is active and a multi-DOF mode in which the multi-DOF sensor module 108 is active by simply placing the mouse module 120 against a physical surface or lifting the mouse module 120 away from the physical surface.”; col. 6 lines 30-35 “When the user 502 desires to use the handheld controller 504 in a 3D computer environment, the user 502 may lift the handheld controller 504 off of the physical surface 506, and the handheld controller 504 may automatically switch to a multi-DOF mode.”; col. 7 lines 23-28 “another signal from the proximity sensor, indicating that the handheld controller has been removed from its position proximate to the physical surface, may be received. In response to receiving this other signal, the mouse module may be deactivated and the multi-DOF sensor module may be activated.” – note removal or separation of the input device from physical surface shifts mouse mode to multi-DOF mode).
Jentz does not directly teach performing gyro correction for a direction perpendicular to a gravity direction.
Gullicksen teaches an information processing system comprising one or more processors and one or more non-transitory computer-readable storage media having stored therein instructions that, when executed, cause the one or more processors to perform operations (see Gullicksen at least [0503] “the improvement or portions thereof may be embodied as a computer program product including one or more non-transient, computer-readable storage media, … The media may be encoded with instructions which, when executed on one or more computers or other processors, perform the process or processes described herein.”), comprising:
performing gyro correction for a direction perpendicular to a gravity direction (see at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.” - note a horizontal plane is perpendicular to the gravity direction).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Jentz so that, when removal from the physical surface causes activation of the gyro based multi-DOF mode, the controller performs Gullicksen’s gyro/yaw correction in the horizontal plane. Performing the correction when entering the gyro based mode would predictably initialize or update the pointing direction thereby reducing drift and improving pointing accuracy.
As to claim 15, Jentz teaches a computer-implemented method comprising:
in a first operation mode, operating a first object based on mouse sensor data outputted from a mouse sensor of an input device (see at least col. 3 lines 37-40 “a mouse sensor 122 (e.g., an optical mouse sensor, a roller sensor, etc.) for tracking movement of the mouse module 120 relative to the physical surface.”; col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, such as in a 2D (e.g., productivity) computing environment to move a cursor or other object.”; col. 7 lines 9-11 “data from a mouse sensor of the mouse module may be received to sense movement of the handheld controller relative to the physical surface.”);
in a second operation mode, operating a second object based on gyro sensor data outputted from a gyro sensor of the input device (see at least col. 3 lines 20-22 “the multi-DOF sensor module 108 may include one or more accelerometers and one or more gyroscopes.”; col. 4 lines 22-24 “In the multi-DOF mode, data from the multi-DOF sensor module 108 may be utilized, such as in a 3D (e.g., artificial-reality) computing environment.”; col. 6 lines 13-18 “At least one multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment, such as to manipulate digital objects, make virtual selections, etc.”; col. 6 lines 48-55 “the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes that may generate data for sensing the position and/or rotation of the handheld controller. Such data may be transmitted to at least one processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 3D computing environment”); and
based on a first condition being satisfied, shifting from the first operation mode to the second operation mode (see at least col. 4 lines 5-9 “The switch 132 may also be configured to deactivate the mouse sensor 122 and activate the multi-DOF sensor module 108 when the proximity sensor 124 indicates that the mouse sensor 122 is not proximate to (e.g., removed from) the physical surface.”; col. 4 lines 10-15 “the handheld controller 100 may be switched between a mouse mode in which the mouse sensor 122 is active and a multi-DOF mode in which the multi-DOF sensor module 108 is active by simply placing the mouse module 120 against a physical surface or lifting the mouse module 120 away from the physical surface.”; col. 6 lines 30-35 “When the user 502 desires to use the handheld controller 504 in a 3D computer environment, the user 502 may lift the handheld controller 504 off of the physical surface 506, and the handheld controller 504 may automatically switch to a multi-DOF mode.”; col. 7 lines 23-28 “another signal from the proximity sensor, indicating that the handheld controller has been removed from its position proximate to the physical surface, may be received. In response to receiving this other signal, the mouse module may be deactivated and the multi-DOF sensor module may be activated.” – note removal or separation of the input device from physical surface shifts mouse mode to multi-DOF mode).
Jentz does not directly teach performing gyro correction for a direction perpendicular to a gravity direction.
Gullicksen teaches performing gyro correction for a direction perpendicular to a gravity direction (see at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.” - note a horizontal plane is perpendicular to the gravity direction).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Jentz so that, when removal from the physical surface causes activation of the gyro based multi-DOF mode, the controller performs Gullicksen’s gyro/yaw correction in the horizontal plane. Performing the correction when entering the gyro based mode would predictably initialize or update the pointing direction thereby reducing drift and improving pointing accuracy.
As to claim 2, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein the gyro correction comprises setting, as a front direction, a direction of a component in a longitudinal direction of the input device, the component being perpendicular to the gravity direction (see Gullicksen at least [0046] “The device 130 includes an IMU (Inertial Measurement Unit) configured to measure the device's orientation in space, e.g., using a gravity sensor . .. Ideally, the virtual ray 132 aligns with a long axis 138 of the device 130.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse” – note the pointing/front direction is aligned with the longitudinal/long axis of the device, and that the direction used for yaw is the projection of the detected device direction onto the horizontal plane. That projection is the component of the longitudinal direction perpendicular to gravity).
As to claim 3, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein the operations further comprise acquiring the mouse sensor data and the gyro sensor data, and in the first operation mode, operating the first object based on the acquired mouse sensor data (see Jentz at least col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, .. to move a cursor or other object.”; col. 6 lines 44-51 “data from a multi-DOF sensor module of a handheld controller may be received, .. the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes”; col. 7 lines 9-11 “data from a mouse sensor of the mouse module may be received to sense movement of the handheld controller relative to the physical surface.”).
As to claim 4, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein the first object and the second object are different objects (see Jentz at least col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, such as in a 2D (e.g., productivity) computing environment to move a cursor or other object.”; col. 6 lines 13-18 “At least one multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment, such as to manipulate digital objects, make virtual selections, etc.”).
As to claim 5, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition that is satisfied when the light‑receiving opening is not blocked (see Gullicksen at least [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”).
As to claim 6, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition that a longitudinal axis of the input device that is parallel to the bottom surface and extends from a near side to a far side when the input device is held by a user is inclined by a specified angle or more with respect to a direction perpendicular to the gravity direction (see Gullicksen at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.”).
As to claim 7, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 6 (see above rejection), wherein the first condition does not include a condition regarding inclination, with respect to a direction perpendicular to the gravity direction, of a short axis of the input device that is parallel to the bottom surface and is perpendicular to the longitudinal axis (see Gullicksen at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.”).
As to claim 8, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition indicating that a bottom surface of the input device is separated from a work surface (see Jentz at least col. 4 lines 5-9 “The switch 132 may also be configured to deactivate the mouse sensor 122 and activate the multi-DOF sensor module 108 when the proximity sensor 124 indicates that the mouse sensor 122 is not proximate to (e.g., removed from) the physical surface.”; col. 4 lines 10-15 “the handheld controller 100 may be switched between a mouse mode in which the mouse sensor 122 is active and a multi-DOF mode in which the multi-DOF sensor module 108 is active by simply placing the mouse module 120 against a physical surface or lifting the mouse module 120 away from the physical surface.”; col. 6 lines 30-35 “When the user 502 desires to use the handheld controller 504 in a 3D computer environment, the user 502 may lift the handheld controller 504 off of the physical surface 506, and the handheld controller 504 may automatically switch to a multi-DOF mode.”; col. 7 lines 23-28 “another signal from the proximity sensor, indicating that the handheld controller has been removed from its position proximate to the physical surface, may be received. In response to receiving this other signal, the mouse module may be deactivated and the multi-DOF sensor module may be activated.”).
As to claim 9, the combination of Jentz and Gullicksen teach the one or more non-transitory computer-readable storage media according to claim 1 (see above rejection), wherein the operations further comprise controlling the second object based on a movement in a rotational direction around a longitudinal axis directed in a longitudinal direction of the input device (see Jentz at least col. 3 lines 23-28 “the multi-DOF sensor module 108 may be a 6DOF sensor module configured to provide information (e.g., position, orientation, and/or movement) related to six degrees of freedom, namely: pitch; roll; yaw; forward and backward; upward and downward; and leftward and rightward.”; col. 6 lines 14-18 “multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment”; col. 6 lines 48-55 “the multi-DOF sensor module may include one or more accelerometers and/or gyroscopes that may generate data for sensing the position and/or rotation of the handheld controller. Such data may be transmitted to at least one processor, such as a processor configured to make computations for generating, manipulating, and/or interacting with a 3D computing environment”; and Gullicksen [0046] “the virtual ray 132 aligns with a long axis 138 of the device 130. The pointing device 130 may construct the virtual ray 132 based on knowledge of its own location and knowledge of the angle in which it is pointing.”; [0497] “A fine gesture such as twisting the mouse in a specific location could mean to unlock a lock.”).
As to claim 11, the combination of Jentz and Gullicksen teaches the information processing system according to claim 10 (see above rejection), wherein the first object and the second object are different objects (see Jentz at least col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, such as in a 2D (e.g., productivity) computing environment to move a cursor or other object.”; col. 6 lines 13-18 “At least one multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment, such as to manipulate digital objects, make virtual selections, etc.”).
As to claim 12, the combination of Jentz and Gullicksen teaches the information processing system according to claim 10 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition that a longitudinal axis of the input device that is parallel to the bottom surface and extends from a near side to a far side when the input device is held by a user is inclined by a specified angle or more with respect to a direction perpendicular to the gravity direction (see Gullicksen at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.”).
As to claim 13, the combination of Jentz and Gullicksen teaches the information processing system according to claim 12 (see above rejection), wherein the first condition does not include a condition regarding inclination, with respect to a direction perpendicular to the gravity direction, of a short axis of the input device that is parallel to the bottom surface and is perpendicular to the longitudinal axis (see Gullicksen at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.”).
As to claim 14, the combination of Jentz and Gullicksen teaches the information processing system according to claim 10 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition indicating that a bottom surface of the input device is separated from a work surface (see Jentz at least col. 4 lines 5-9 “The switch 132 may also be configured to deactivate the mouse sensor 122 and activate the multi-DOF sensor module 108 when the proximity sensor 124 indicates that the mouse sensor 122 is not proximate to (e.g., removed from) the physical surface.”; col. 4 lines 10-15 “the handheld controller 100 may be switched between a mouse mode in which the mouse sensor 122 is active and a multi-DOF mode in which the multi-DOF sensor module 108 is active by simply placing the mouse module 120 against a physical surface or lifting the mouse module 120 away from the physical surface.”; col. 6 lines 30-35 “When the user 502 desires to use the handheld controller 504 in a 3D computer environment, the user 502 may lift the handheld controller 504 off of the physical surface 506, and the handheld controller 504 may automatically switch to a multi-DOF mode.”; col. 7 lines 23-28 “another signal from the proximity sensor, indicating that the handheld controller has been removed from its position proximate to the physical surface, may be received. In response to receiving this other signal, the mouse module may be deactivated and the multi-DOF sensor module may be activated.”).
As to claim 16, the combination of Jentz and Gullicksen teach the computer-implemented method according to claim 15 (see above rejection), wherein the first object and the second object are different objects (see Jentz at least col. 4 lines 15-18 “In the mouse mode, data from the mouse sensor 122 may be utilized, such as in a 2D (e.g., productivity) computing environment to move a cursor or other object.”; col. 6 lines 13-18 “At least one multi-DOF sensor module of the handheld controllers 404, 406 may enable the user 402 to use the handheld controllers 404, 406 in space to interact with the 3D computer environment, such as to manipulate digital objects, make virtual selections, etc.”).
As to claim 17, the combination of Jentz and Gullicksen teach the computer-implemented method according to claim 15 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition that a longitudinal axis of the input device that is parallel to the bottom surface and extends from a near side to a far side when the input device is held by a user is inclined by a specified angle or more with respect to a direction perpendicular to the gravity direction (see Gullicksen at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.”).
As to claim 18, the combination of Jentz and Gullicksen teach the computer-implemented method according to claim 17 (see above rejection), wherein the first condition does not include a condition regarding inclination, with respect to a direction perpendicular to the gravity direction, of a short axis of the input device that is parallel to the bottom surface and is perpendicular to the longitudinal axis (see Gullicksen at least [0381] “The estimate of the device orientation will typically be quite good because we have gravity and the corrected magnetic field to control drift on the gyroscope.”; [0383] “A pointing device such as the Holographic Mouse or a game controller needs to have very accurate yaw relative to the user’s view.”; [0387] “The goal is to have yaw correction performed in the background without user participation being required.”; [0390] “The Holographic Mouse must be held at a specific angle for the depth camera to see the sheen of the material on the mouse. We ensure by design that the specific angle needed for the depth camera to successfully view the sheen of the material will depend on a Holographic Mouse orientation not otherwise utilized for game play or training. In other words, the system is designed to minimize the likelihood that the user would ever view the Holographic Mouse with an orientation that might inadvertently interrupt the LPS system's depth camera method for yaw correction.”; [0391] “A pattern (of 2 or more integrated patches) of the material is used on the surface of the mouse. The material is visible to the depth camera at the specific angle in the manner described above.”; [0393] “The line defined by these two points is then projected onto a horizontal plane. The projected line then represents the yaw of the Holographic Mouse in the local coordinate system of the AR glasses.”; [0394] “Detection of the yaw of the Holographic Mouse is lightweight enough to be run as a background task, enabling the yaw of the mouse to remain stable even as the magnetic field direction changes.”).
As to claim 19, the combination of Jentz and Gullicksen teach the computer-implemented method according to claim 15 (see above rejection), wherein a light-receiving opening for the mouse sensor is provided in a bottom surface of the input device, and the first condition includes a condition indicating that a bottom surface of the input device is separated from a work surface (see Jentz at least col. 4 lines 5-9 “The switch 132 may also be configured to deactivate the mouse sensor 122 and activate the multi-DOF sensor module 108 when the proximity sensor 124 indicates that the mouse sensor 122 is not proximate to (e.g., removed from) the physical surface.”; col. 4 lines 10-15 “the handheld controller 100 may be switched between a mouse mode in which the mouse sensor 122 is active and a multi-DOF mode in which the multi-DOF sensor module 108 is active by simply placing the mouse module 120 against a physical surface or lifting the mouse module 120 away from the physical surface.”; col. 6 lines 30-35 “When the user 502 desires to use the handheld controller 504 in a 3D computer environment, the user 502 may lift the handheld controller 504 off of the physical surface 506, and the handheld controller 504 may automatically switch to a multi-DOF mode.”; col. 7 lines 23-28 “another signal from the proximity sensor, indicating that the handheld controller has been removed from its position proximate to the physical surface, may be received. In response to receiving this other signal, the mouse module may be deactivated and the multi-DOF sensor module may be activated.”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNIFER L ZUBAJLO whose telephone number is (571)270-1551. The examiner can normally be reached Monday - Thursday 10 am - 8 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KE XIAO can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNIFER L ZUBAJLO/ Examiner, Art Unit 2627 9/24/2026
/KE XIAO/ Supervisory Patent Examiner, Art Unit 2627
Prosecution Insights