DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-18 are rejected under 35 U.S.C. 103 as being unpatentable over Aubauer et al. (USPN 2014/0055396 A1) in view of Wang et al. (USPN 2011/0107270 A1).
As to claim 1, Aubauer teaches a coordinate calculation system comprising:
a tablet that, in operation, detects a position of a pen tip of an electronic pen (see at least [0032] “The electronic device may be a tablet computer ... The user input object 500 (FIG. 2) is shown as user finger 500, but may be anything like a stylus (e.g., a small pen-shaped instrument), or a digital pen.”; [0033] “The touch-sensitive module 200 not only detects that a user input object 500 touches its active area 202, it also detects where the user input object 500 makes contact with the active area, i.e., the touch-sensitive display extracts the x- and y-coordinates of the contact area .. Thus, the touch-sensitive surface 200 determines first positional information, the first positional information solely depending on where the user input object 500 contacts the touch-sensitive module 200.”; [0047] “Any touch event is described by two coordinates representing a position on the touch-sensitive surface.”);
detecting a position of the electronic pen for inputting a position in a three dimensional space extending outward from the tablet (see at least [0035] “The electronic device 100 may further include a contact-free detection system 300. In general, contact-free detection system 300 in this sense is any system suitable for determining second positional information depending on the spatial configuration of the user input object 500 with regard to the touch-sensitive surface 200. The spatial configuration of the user input object 500 is to be understood as the three-dimensional arrangement, location, and orientation of the user input object 500 with regard to the touch-sensitive module 200.”; [0043] “the same media (space in front of the touch-sensitive surface ..) may be used by both the touch-sensitive module 200 and the contact-free detection means 300.”; [0044] “the second positional information comprises a second position that depends on the spatial configuration of a portion of the user input object that is within a predefined distance D (FIG. 2) of the touch-sensitive surface. The space defined by the touch-sensitive surface and the predefined distance is shown as cuboid 310.”; [0048] “the second positional information is determined and represented in three dimensions because the second positional information depends on the spatial configuration of the user input object in three-dimensional space. Any spatial configuration of an object can be described by three coordinates .. , the second positional information comprises the second position in the form of a 3-tuple.”; [0112] “other hand posture detection methods (cameras, infrared, ultrasound).”), and
a computer that, in operation:
in a first mode, calculates a first coordinate in a first coordinate system of the tablet based on the position detected by the tablet (see at least [0039] “The controller 120 is adapted for simultaneously and/or alternately determining the first positional information via the touch-sensitive surface and the second positional information via the contact free detection system 300.”; [0047] “Any touch event is described by two coordinates representing a position on the touch-sensitive surface.”; [0049] “the first positional information is transformed into the coordinate system of the second positional information. Likewise the second positional information could be transformed into the coordinate system of the first positional information, or both first and second positional information could be transformed into another coordinate system.”); and
in a second mode that is switched from the first mode in response to a predetermined operation and in which the position of the pen tip of the electronic pen is not detected by the tablet (see at least [0042] “the contact free detection means 300 is continuously active until it detects a touch-event. On touch event detection, the contact free detection means 300 is deactivated and the touch-sensitive module 200 activated and detects the touch position. When no touch is detected on the touch-sensitive module 200, the contact free detection means 300 is activated again.” – note in the touch mode, the touch-sensitive module detects the contact position, when the user performs the operation of removing/lifting the stylus from the tablet such that touch is no longer detected, the contact-free detector is activated, therefore, Aubauer teaches switching from a first tablet-coordinate mode to a second spatial-position mode in response to the predetermined pen-removal/no-touch operation, and in that second mode the position of the pen tip is not being detected by the tablet touch-sensitive module),
calculates a second coordinate in a second coordinate system that is different from the first coordinate system based on the position detected (see at least [0035] “contact-free detection system 300 in this sense is any system suitable for determining second positional information depending on the spatial configuration of the user input object 500 with regard to the touch-sensitive surface 200. The spatial configuration of the user input object 500 is to be understood as the three-dimensional arrangement, location, and orientation of the user input object 500 with regard to the touch-sensitive module 200.”; [0048] “the second positional information is determined and represented in three dimensions …, the second positional information comprises the second position in the form of a 3-tuple.”; [0049] “the first positional information is transformed into the coordinate system of the second positional information. Likewise the second positional information could be transformed into the coordinate system of the first positional information, or both first and second positional information could be transformed into another coordinate system.”).
Aubauer does not directly teach a tracking device positioned above the tablet, wherein the tracking device, in operation, detects a position of a tracker of the electronic pen for inputting a position in a virtual reality (VR) space; calculates a second coordinate in a second coordinate system of the VR space that is different from the first coordinate system based on the position of the tracker detected by the tracking device.
Wang teaches a tracking device, wherein the tracking device, in operation, detects a position of a tracker of the electronic pen for inputting a position in a virtual reality (VR) space; calculates a second coordinate in a second coordinate system of the VR space that is different from the first coordinate system based on the position of the tracker detected by the tracking device (see at least figs. 5A-5C, [0060] “A virtual environment (also known as an immersive environment, a virtual reality, or an immersive digital environment) is an artificial, computer-created three-dimensional (3D) virtual space in which a user interacts with virtual artifacts”; [0063] “the user 30 can move an electronic pointer 35 .. from a starting location 74 to an ending location 76 along a motion path 70. .. As the user moves the electronic pointer 35 along the motion path 70, the motion of the electronic pointer 35 is tracked by a positional sensor 5, and the contour 40 is generated within the 3D virtual space 15 of the virtual environment 2.”; [0092] “a positional sensor 422 and a tracked object 424. The positional sensor 422 may detect changes in position and orientation of the tracked object 424 as a user moves the tracked object within a field of view 426 of the positional sensor 422.”; [0093] “The positional sensor 422 may be a standard video camera, a Z-camera, a stereo camera, an infrared tracker or other optical sensor.”; [0094] “The tracked object 424 may be a powered electronic device, such as an electronic pointer, electronic pen, data glove, etc.”; [0095] “the pointers include active light emitters and/or passive light emitters (e.g., passive optical spheres). ... For an electromagnetic tracking system, the electronic pointer may include active electromagnetic markers (e.g., transponders) and/or passive electromagnetic markers (e.g., passive sensor coils)”; [0096] “The LEDs 504 enable an orientation of the electronic pointer to be accurately tracked, regardless of how the electronic pointer is rotated or where in the field of view of a positional sensor the electronic pointer is positioned.”; [0122] “Once the matrix M.sub.1 is computed, any position measured in the user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv) … by the transformation matrix M1”; [0154] “the computing device determines a position and orientation of the tracked object in the virtual space …, the position and orientation is determined based on coordinates of at least two points of the tracked object.”; [0155] “Calculating the position of the tracked object in one embodiment includes calculating a position of a tip of the tracked object in the virtual space …, the absolute position of the tip can be computed by multiplying the vector that represents the position of the tracked object's tip in the user space by the calibration matrix M1.”).
Note with respect to “a tracking device positioned above the tablet”, Aubauer and Wang do not expressly use the phrase “positioned above the tablet.” However, Aubauer expressly identifies the region in which its contact-free detection occurs as the space extending outward from the tablet’s touch-sensitive surface. Aubauer locates its contact free detection region in the “space in front of the touch-sensitive surface” (see [0043]) and describes the relevant space as extending a predefined distance from the touch-sensitive surface (see [0044]). Wang teaches that its positional sensor detects the tracked object while the object is “within a field of view 426 of the positional sensor 422” (see [0092]), that the sensor may be “a standard video camera, a Z-camera, a stereo camera, an infrared tracker or other optical sensor” (see [0093]), and that the tracked object may be an “electronic pen” (see [0094]). When Wang’s optical positional sensor is used to track Aubauer’s electronic pen operating on and in the space extending outward from the tablet surface, positioning the tracking device above the tablet with its field of view directed toward the tablet and pen interaction region would have been a predictable arrangement that permits the sensor to perform its disclosed tracking function. The tracking sensor continues to operate for the same disclosed purpose—observing the tracked electronic pen within its field of view.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Aubauer’s contact-free spatial-position mode to incorporate Wang’s tracked electronic pen, positional sensor, and VR coordinate transformation in order to extend the functionality of Aubauer’s stylus from two-dimensional tablet-surface interaction to three-dimensional interaction with a virtual environment while retaining Aubauer’s existing automatic switching between surface and contact-free input modes. The combination would predictably permit the same electronic pen to provide tablet coordinates while the pen is detected by the tablet and VR coordinates after the pen is lifted from the tablet and spatially tracked. The modification does not require changing Aubauer’s principle of operation – a digital pen, first and second positional-information sources, alternating those sources, switching back to contact-free sensing when touch is no longer detected, and transformation among different coordinate systems. Wang provides a known VR-specific implementation and use for Aubauer’s second, contact-free positional-information source.
As to claim 10, Aubauer teaches a coordinate calculation method comprising:
in a first mode, calculating a first coordinate in a first coordinate system of a tablet based on a position of a pen tip of an electronic pen detected by the tablet (see at least [0032] “The electronic device may be a tablet computer ... The user input object 500 (FIG. 2) is shown as user finger 500, but may be anything like a stylus (e.g., a small pen-shaped instrument), or a digital pen.”; [0033] “The touch-sensitive module 200 not only detects that a user input object 500 touches its active area 202, it also detects where the user input object 500 makes contact with the active area, i.e., the touch-sensitive display extracts the x- and y-coordinates of the contact area .. Thus, the touch-sensitive surface 200 determines first positional information, the first positional information solely depending on where the user input object 500 contacts the touch-sensitive module 200.”; [0039] “The controller 120 is adapted for simultaneously and/or alternately determining the first positional information via the touch-sensitive surface and the second positional information via the contact free detection system 300.”; [0047] “Any touch event is described by two coordinates representing a position on the touch-sensitive surface.”; [0049] “the first positional information is transformed into the coordinate system of the second positional information. Likewise the second positional information could be transformed into the coordinate system of the first positional information, or both first and second positional information could be transformed into another coordinate system.”); and
in a second mode that is switched from the first mode in response to a predetermined operation and in which the position of the pen tip of the electronic pen is not detected by the tablet (see at least [0042] “the contact free detection means 300 is continuously active until it detects a touch-event. On touch event detection, the contact free detection means 300 is deactivated and the touch-sensitive module 200 activated and detects the touch position. When no touch is detected on the touch-sensitive module 200, the contact free detection means 300 is activated again.” – note in the touch mode, the touch-sensitive module detects the contact position, when the user performs the operation of removing/lifting the stylus from the tablet such that touch is no longer detected, the contact-free detector is activated, therefore, Aubauer teaches switching from a first tablet-coordinate mode to a second spatial-position mode in response to the predetermined pen-removal/no-touch operation, and in that second mode the position of the pen tip is not being detected by the tablet touch-sensitive module),
calculating a second coordinate in a second coordinate system that is different from the first coordinate system based on a position of the electronic pen detected for inputting a position in a three dimensional space extending outward from the tablet (see at least [0035] “contact-free detection system 300 in this sense is any system suitable for determining second positional information depending on the spatial configuration of the user input object 500 with regard to the touch-sensitive surface 200. The spatial configuration of the user input object 500 is to be understood as the three-dimensional arrangement, location, and orientation of the user input object 500 with regard to the touch-sensitive module 200.”; [0043] “the same media (space in front of the touch-sensitive surface ..) may be used by both the touch-sensitive module 200 and the contact-free detection means 300.”; [0044] “the second positional information comprises a second position that depends on the spatial configuration of a portion of the user input object that is within a predefined distance D (FIG. 2) of the touch-sensitive surface. The space defined by the touch-sensitive surface and the predefined distance is shown as cuboid 310.”; [0048] “the second positional information is determined and represented in three dimensions because the second positional information depends on the spatial configuration of the user input object in three-dimensional space. Any spatial configuration of an object can be described by three coordinates .. , the second positional information comprises the second position in the form of a 3-tuple.”; [0049] “the first positional information is transformed into the coordinate system of the second positional information. Likewise the second positional information could be transformed into the coordinate system of the first positional information, or both first and second positional information could be transformed into another coordinate system.”; [0112] “other hand posture detection methods (cameras, infrared, ultrasound).”).
Aubauer does not directly teach calculating a second coordinate in a second coordinate system of a virtual reality (VR) space that is different from the first coordinate system based on a position of a tracker of the electronic pen detected by a tracking device positioned above the tablet for inputting a position in the VR space.
Wang teaches calculating a second coordinate in a second coordinate system of a virtual reality (VR) space that is different from the first coordinate system based on a position of a tracker of the electronic pen detected by a tracking device for inputting a position in the VR space (see at least figs. 5A-5C, [0060] “A virtual environment (also known as an immersive environment, a virtual reality, or an immersive digital environment) is an artificial, computer-created three-dimensional (3D) virtual space in which a user interacts with virtual artifacts”; [0063] “the user 30 can move an electronic pointer 35 .. from a starting location 74 to an ending location 76 along a motion path 70. .. As the user moves the electronic pointer 35 along the motion path 70, the motion of the electronic pointer 35 is tracked by a positional sensor 5, and the contour 40 is generated within the 3D virtual space 15 of the virtual environment 2.”; [0092] “a positional sensor 422 and a tracked object 424. The positional sensor 422 may detect changes in position and orientation of the tracked object 424 as a user moves the tracked object within a field of view 426 of the positional sensor 422.”; [0093] “The positional sensor 422 may be a standard video camera, a Z-camera, a stereo camera, an infrared tracker or other optical sensor.”; [0094] “The tracked object 424 may be a powered electronic device, such as an electronic pointer, electronic pen, data glove, etc.”; [0095] “the pointers include active light emitters and/or passive light emitters (e.g., passive optical spheres). ... For an electromagnetic tracking system, the electronic pointer may include active electromagnetic markers (e.g., transponders) and/or passive electromagnetic markers (e.g., passive sensor coils)”; [0096] “The LEDs 504 enable an orientation of the electronic pointer to be accurately tracked, regardless of how the electronic pointer is rotated or where in the field of view of a positional sensor the electronic pointer is positioned.”; [0122] “Once the matrix M.sub.1 is computed, any position measured in the user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv) … by the transformation matrix M1”; [0154] “the computing device determines a position and orientation of the tracked object in the virtual space …, the position and orientation is determined based on coordinates of at least two points of the tracked object.”; [0155] “Calculating the position of the tracked object in one embodiment includes calculating a position of a tip of the tracked object in the virtual space …, the absolute position of the tip can be computed by multiplying the vector that represents the position of the tracked object's tip in the user space by the calibration matrix M1.”).
Note with respect to “a tracking device positioned above the tablet”, Aubauer and Wang do not expressly use the phrase “positioned above the tablet.” However, Aubauer expressly identifies the region in which its contact-free detection occurs as the space extending outward from the tablet’s touch-sensitive surface. Aubauer locates its contact free detection region in the “space in front of the touch-sensitive surface” (see [0043]) and describes the relevant space as extending a predefined distance from the touch-sensitive surface (see [0044]). Wang teaches that its positional sensor detects the tracked object while the object is “within a field of view 426 of the positional sensor 422” (see [0092]), that the sensor may be “a standard video camera, a Z-camera, a stereo camera, an infrared tracker or other optical sensor” (see [0093]), and that the tracked object may be an “electronic pen” (see [0094]). When Wang’s optical positional sensor is used to track Aubauer’s electronic pen operating on and in the space extending outward from the tablet surface, positioning the tracking device above the tablet with its field of view directed toward the tablet and pen interaction region would have been a predictable arrangement that permits the sensor to perform its disclosed tracking function. The tracking sensor continues to operate for the same disclosed purpose—observing the tracked electronic pen within its field of view.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Aubauer’s contact-free spatial-position mode to incorporate Wang’s tracked electronic pen, positional sensor, and VR coordinate transformation in order to extend the functionality of Aubauer’s stylus from two-dimensional tablet-surface interaction to three-dimensional interaction with a virtual environment while retaining Aubauer’s existing automatic switching between surface and contact-free input modes. The combination would predictably permit the same electronic pen to provide tablet coordinates while the pen is detected by the tablet and VR coordinates after the pen is lifted from the tablet and spatially tracked. The modification does not require changing Aubauer’s principle of operation – a digital pen, first and second positional-information sources, alternating those sources, switching back to contact-free sensing when touch is no longer detected, and transformation among different coordinate systems. Wang provides a known VR-specific implementation and use for Aubauer’s second, contact-free positional-information source.
As to claim 2, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 1 (see above rejection), wherein the first coordinate system of the tablet is a coordinate system related to a position detection area of the tablet (see Aubauer at least [0047] “Any touch event is described by two coordinates representing a position on the touch-sensitive surface.”), and the second coordinate system of the VR space is a coordinate system related to a position detection area of the tracking device (see Wang at least [0092] “The tracking system 420 includes a positional sensor 422 and a tracked object 424. The positional sensor 422 may detect changes in position and orientation of the tracked object 424 as a user moves the tracked object within a field of view 426 of the positional sensor 422; see also [0122] “user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv)”).
As to claim 3, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 1 (see above rejection), wherein the computer, in operation, displays a virtual image in the VR space on a display device (see Wang at least [0063] “user interaction with a virtual environment… rendered by a stereo monitor”; [0067] “A stereoscopic display… such as a stereo monitor or stereoscopic goggles”).
As to claim 4, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 1 (see above rejection), wherein the computer, in operation, calculates the first coordinate in the first coordinate system of the tablet when the pen tip of the electronic pen is located within a predetermined distance from an input surface of the tablet (see Aubauer at least [0044] “the second positional information comprises a second position that depends on the spatial configuration of a portion of the user input object that is within a predefined distance D (FIG. 2) of the touch-sensitive surface”).
As to claim 5, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 1 (see above rejection), wherein the tracker of the electronic pen includes a light-emitting diode (LED) that, in operation, emits light (see Wang at least [0098] “The electronic pointer 540 includes multiple protrusions… At the end of each protrusion is an LED 548… Thus, fewer LEDs may be used to accurately track a position and orientation of the electronic pointer”).
As to claim 6, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 1 (see above rejection), wherein the VR space includes a mixed reality (MR) space and an augmented reality (AR) space (note Wang teaches interaction with a virtual environment rendered within a spatial tracking system (see Wang at least [0063]). Augmented reality and mixed reality environments represent known implementations of spatial computing environments that use the same positional tracking and coordinate mapping techniques as virtual reality systems. It would therefore have been obvious to implement the virtual environment of Wang as an augmented or mixed reality environment, since doing so would represent the use of a known alternative environment for presenting spatially tracked virtual content and would yield predictable results.).
As to claim 7, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 1 (see above rejection), wherein the computer, in operation, calculates a third coordinate in the second coordinate system of the VR space based on the first coordinate in the first coordinate system of the tablet (see Aubauer at least [0049] “the first positional information is transformed into the coordinate system of the second positional information… or both first and second positional information could be transformed into another coordinate system”; see also Wang at least [0122] “user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv) by multiplying a vector from the origin to the measured coordinates in the user space coordinate system by the transformation matrix”).
As to claim 8, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 7 (see above rejection), wherein the computer, in operation, corrects the first coordinate in the first coordinate system of the tablet and calculate the third coordinate in the second coordinate system of the VR space based on the corrected first coordinate (see Wang at least [0112] “calibrating a location in the user space to correspond to an origin of the virtual space(s) of the virtual environment (and possibly any standard workspaces) and/or otherwise calibrating a reference frame of the data glove to a reference frame of the virtual space(s) (and standard workspaces). ... That location may then be calibrated to the origin in the virtual space(s) of the virtual environment and/or to the origin in a standard workspace.”; [0122] “user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv)… by multiplying… by the transformation matrix M1.”).
As to claim 9, the combination of Aubauer and Wang teach the coordinate calculation system according to claim 7 (see above rejection), wherein the computer, in operation, displays a virtual image in the VR space on a display device based on the second coordinate and the third coordinate in the second coordinate system of the VR space (see Wang at least [0155] “Calculating the position of the tracked object… includes calculating a position of a tip of the tracked object in the virtual space…”; see also [0063] “user interaction with a virtual environment… rendered by a stereo monitor”).
As to claim 11, the combination of Aubauer and Wang teach the method according to claim 10 (see above rejection), wherein the first coordinate system of the tablet is a coordinate system related to a position detection area of the tablet, and the second coordinate system of the VR space is a coordinate system related to a position detection area of the tracking device (see Wang at least [0092] “The tracking system 420 includes a positional sensor 422 and a tracked object 424. The positional sensor 422 may detect changes in position and orientation of the tracked object 424 as a user moves the tracked object within a field of view 426 of the positional sensor 422; see also [0122] “user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv)”).
As to claim 12, the combination of Aubauer and Wang teach the method according to claim 10 (see above rejection), further comprising: displaying a virtual image corresponding to the tablet in the VR space on a display device (see Wang at least [0063] “user interaction with a virtual environment… rendered by a stereo monitor”; [0067] “A stereoscopic display… such as a stereo monitor or stereoscopic goggles”).
As to claim 13, the combination of Aubauer and Wang teach the method according to claim 10 (see above rejection), wherein the first coordinate is calculated when the pen tip of the electronic pen is located within a predetermined distance from an input surface of the tablet (see Aubauer at least [0044] “the second positional information comprises a second position that depends on the spatial configuration of a portion of the user input object that is within a predefined distance D (FIG. 2) of the touch-sensitive surface”).
As to claim 14, the combination of Aubauer and Wang teach the method according to claim 10 (see above rejection), wherein the tracker of the electronic pen includes a light-emitting diode (LED) that emits light (see Wang at least [0098] “The electronic pointer 540 includes multiple protrusions… At the end of each protrusion is an LED 548… Thus, fewer LEDs may be used to accurately track a position and orientation of the electronic pointer”).
As to claim 15, the combination of Aubauer and Wang teach the method according to claim 10 (see above rejection), wherein the VR space includes a mixed reality (MR) space and an augmented reality (AR) space (note Wang teaches interaction with a virtual environment rendered within a spatial tracking system (see Wang at least [0063]). Augmented reality and mixed reality environments represent known implementations of spatial computing environments that use the same positional tracking and coordinate mapping techniques as virtual reality systems. It would therefore have been obvious to implement the virtual environment of Wang as an augmented or mixed reality environment, since doing so would represent the use of a known alternative environment for presenting spatially tracked virtual content and would yield predictable results.).
As to claim 16, the combination of Aubauer and Wang teach the method according to claim 10 (see above rejection), further comprising: calculating a third coordinate in the second coordinate system of the VR space based on the first coordinate in the first coordinate system of the tablet (see Aubauer at least [0049] “the first positional information is transformed into the coordinate system of the second positional information… or both first and second positional information could be transformed into another coordinate system”; see also Wang at least [0122] “user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv) by multiplying a vector from the origin to the measured coordinates in the user space coordinate system by the transformation matrix”).
As to claim 17, the combination of Aubauer and Wang teach the method according to claim 16 (see above rejection), further comprising: correcting the first coordinate in the first coordinate system of the tablet and calculating the third coordinate in the second coordinate system of the VR space based on the corrected first coordinate (see Wang at least [0112] “calibrating a location in the user space to correspond to an origin of the virtual space(s) of the virtual environment (and possibly any standard workspaces) and/or otherwise calibrating a reference frame of the data glove to a reference frame of the virtual space(s) (and standard workspaces). ... That location may then be calibrated to the origin in the virtual space(s) of the virtual environment and/or to the origin in a standard workspace.”; [0122] “user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv)… by multiplying… by the transformation matrix M1.”).
As to claim 18, the combination of Aubauer and Wang teach the method according to claim 16 (see above rejection), further comprising: displaying a virtual image in the VR space on a display device based on the second coordinate and the third coordinate in the second coordinate system of the VR space (see Wang at least [0155] “Calculating the position of the tracked object… includes calculating a position of a tip of the tracked object in the virtual space…”; see also [0063] “user interaction with a virtual environment… rendered by a stereo monitor”;
Response to Arguments
Applicant's arguments filed 6/12/2026 have been fully considered but they are not persuasive.
Applicant argues –
“Aubauer relates to input mode control using a touch position detected by a touch- sensitive module and a three-dimensional spatial configuration of a user input object detected by a contact-free detection system. The contact-free detection system disclosed by Aubauer detects the posture, orientation, or spatial configuration of a user input object relative to a touch- sensitive surface, and merely supplements the touch-based input paradigm. Aubauer fails to teach or suggest that the posture, orientation, or spatial configuration includes a coordinate in a coordinate system of a VR space.
Notably, Aubauer fails to teach or suggest a virtual reality (VR) space. Thus, Applicant respectfully submits that Aubauer does not teach or suggest tracker-based coordinate calculation in a coordinate system of a VR space. More particularly, Aubauer does not teach or suggest calculating a second coordinate in a second coordinate system of a VR space that is different from a first coordinate system of a tablet based on a position of a tracker detected by a tracking device.”
Examiner disagrees –
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Examiner agrees that Aubauer does not expressly disclose a VR-space coordinate system. The rejection, however, does not rely upon Aubauer alone for that limitation. Aubauer is relied upon for the tablet and digital pen, tablet-based coordinate detection, contact-free three-dimensional positional detection, and alternate operation of the touch-based and contact-free detection mechanisms. Wang is relied upon for the additional teaching of tracking an electronic pen and calculating the spatially tracked position in a coordinate system of a virtual reality space.
Aubauer expressly discloses that its electronic device can be “a tablet computer” and that its input object can be “a stylus … or a digital pen” (see [0032]). Aubauer’s touch-sensitive module “extracts the x- and y-coordinates of the contact area” (see [0033]), and Aubauer expressly describes the first position as a 2-tuple representing a position on the touch-sensitive surface (see [0047]). Aubauer also determines second positional information concerning the input object’s “three-dimensional arrangement, location, and orientation” (see [0035]), including a second position represented by a 3-tuple (see [0048]).
Wang provides the VR-coordinate teaching that Applicant correctly observes is absent from Aubauer. Wang expressly defines its virtual environment as “a virtual reality” comprising a “three-dimensional (3D) virtual space” (see [0060]). Wang separately identifies the tracking system/user space coordinate system and the virtual space coordinate system (see [0104]–[0105]) and expressly teaches that: “any position measured in the user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zv)” (see [0122]). Wang further teaches during actual tracking that the computing device receives coordinates of an electronic pointer from an optical or electromagnetic positional sensor and determines the position of the tracked object in the virtual space (see [0151], [0153]–[0155]).
Therefore, the rejection does not equate Aubauer’s three-dimensional positional information with a VR-space coordinate. Wang is relied upon to supply the VR-space coordinate processing.
Applicant argues –
“Aubauer does not disclose a second mode in which a tablet does not detect a position of a pen tip of an electronic pen and in which a second coordinate in a coordinate system of a VR space is calculated based on a position of a tracker of the electronic pen detected by a tracking device.
Paragraph [0042] of Aubauer discloses that, for event triggered multiplexing, the contact free detection means 300 is continuously active until it detects a touch-event; on touch event detection, the contact free detection means 300 is deactivated and the touch-sensitive module 200 activated and detects the touch position wherein, when no touch is detected on the touch-sensitive module 200, the contact free detection means 300 is activated again. Such disclosure merely concerns which sensing mechanism of the touch-based input device is active. Aubauer does not teach or suggest a mode in which tablet-based pen-tip detection is absent and a VR-space coordinate is calculated based on a tracker detected by a tracking device.”
Examiner disagrees –
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Aubauer expressly teaches an embodiment in which its touch-sensitive and contact-free positional detection mechanisms are alternately activated: “For event triggered multiplexing, the contact free detection means 300 is continuously active until it detects a touch-event. On touch event detection, the contact free detection means 300 is deactivated and the touch-sensitive module 200 activated and detects the touch position. When no touch is detected on the touch-sensitive module 200, the contact free detection means 300 is activated again.” (see [0042]).
Thus, Aubauer teaches a tablet touch operating condition in which the touch sensitive module detects the position of the input object and a contact free operating condition in which no touch position is detected by the touch-sensitive module and the contact-free detection means is activated instead. Because Aubauer expressly identifies the user input object as including a stylus or digital pen (see [0032]), this disclosure teaches an operating condition in which the tablet does not detect the position of the pen tip through its touch-sensitive module.
Wang is relied upon for calculating the position supplied by the contact free tracking mechanism in a VR space coordinate system.
Applicant’s characterization that Aubauer paragraph [0042] “merely concerns which sensing mechanism … is active” does not distinguish the amended claim. The claim itself distinguishes the first and second modes in part according to the positional detection being used: the first coordinate is based on a position detected by the tablet, whereas in the second mode the pen-tip position is not detected by the tablet and the second coordinate is based on the position of the tracker detected by the tracking device. Aubauer’s alternate activation of tablet touch detection and contact-free positional detection is therefore directly relevant to the claimed distinction between the two modes.
Applicant argues –
“Paragraph [0035] of Aubauer discloses that the contact-free detection system 300 is any system suitable for determining second positional information depending on the spatial configuration of the user input object 500 with regard to the touch-sensitive surface 200, wherein the spatial configuration of the user input object 500 is to be understood as the three-dimensional arrangement, location, and orientation of the user input object 500 with regard to the touch- sensitive module 200. Nothing has been found, or pointed to, in Aubauer which teaches or suggests that the second positional information via the contact free detection system is a coordinate in a VR-space coordinate system.”
Examiner disagrees –
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Examiner agrees that Aubauer paragraph [0035], standing alone, does not disclose a VR-space coordinate system. The rejection does not rely on Aubauer for that limitation. Wang supplies the VR-space teaching, including the express transformation of a measured position in user/tracking space into a position in virtual space (see [0122]) and determination of the tracked object’s position in virtual space during ongoing tracking (see [0151], [0153]–[0155]).
Applicant argues –
“Although Wang may disclose tracking a tracked object or an electronic pointer in a virtual environment and transforming coordinates into a virtual space, Wang fails to teach or suggest a first mode in which a tablet detects a position of a pen tip of an electronic pen. Also, Wang fails to teach or suggest switching, in response to a predetermined operation, from such a tablet-based first mode to a second mode in which the tablet does not detect the pen-tip position.”
Examiner disagrees –
Wang is not relied upon for the tablet based first mode. Aubauer teaches that portion of the combination. Aubauer identifies a tablet and stylus/digital pen (see [0032]), extraction of x-y contact coordinates by the touch-sensitive module (see [0033]), and alternate operation of touch sensitive and contact free positional detection (see [0039], [0042]). Wang is relied upon for the known tracked pen and VR coordinate implementation of the contact free positional input.
Applicant argues –
“The tracking system disclosed by Wang is not used as a tracking device positioned above a tablet for switching between a tablet input mode and a VR-space input mode, as recited in amended claim 1. To the extent that Wang discloses two-dimensional and three-dimensional interaction modes within a virtual environment, such disclosure concerns how a tracked object is interpreted within a virtual environment.”
Examiner disagrees –
Examiner agrees that neither Aubauer nor Wang expressly uses the phrase “positioned above the tablet.” However, the rejection does not rely upon an express disclosure of that exact spatial relationship. Rather, that placement would have been an obvious physical arrangement when applying Wang’s optical tracking system to Aubauer’s tablet and pen-interaction region.
Aubauer locates its contact free detection region in the “space in front of the touch-sensitive surface” (see [0043]) and describes the relevant space as extending a predefined distance from the touch-sensitive surface (see [0044]). Wang teaches that its positional sensor detects the tracked object while the object is “within a field of view 426 of the positional sensor 422” (see [0092]), that the sensor may be “a standard video camera, a Z-camera, a stereo camera, an infrared tracker or other optical sensor” (see [0093]), and that the tracked object may be an “electronic pen” (see [0094]).
When Wang’s optical positional sensor is used to track Aubauer’s electronic pen operating on and in the space extending outward from the tablet surface, positioning the tracking device above the tablet with its field of view directed toward the tablet and pen interaction region would have been a predictable arrangement that permits the sensor to perform its disclosed tracking function. The tracking sensor continues to operate for the same disclosed purpose—observing the tracked electronic pen within its field of view.
Applicant argues –
“The tracking system disclosed by Wang does not involve switching from a tablet-based coordinate calculation mode to a VR-space coordinate calculation mode. More particularly, Wang fails to teach or suggest that, in one mode, a tablet detects a pen-tip position and, in another mode, the tablet does not detect the pen-tip position.
The Office asserts that paragraphs [0112] and [0122] of Wang teach calculating a second coordinate in a second coordinate system of the VR space that is different from the first coordinate system based on the position of the tracker detected by the tracking device. Notably, the Office has failed to point to anything in Aubauer and Wang that allegedly teaches the "predetermined operation" recited in claim 1.
Paragraph [0112] of Wang teaches that calibration of the glove may include calibrating a location in the user space to correspond to an origin of the virtual space(s) of the virtual environment and/or otherwise calibrating a reference frame of the data glove to a reference frame of the virtual space(s). Paragraph [0121] of Wang teaches that calibration of the tracking system to the virtual space of the stereo monitor can be represented by a 4x4 transformation matrix M1. Paragraph [0122] of Wang teaches that, once the matrix M1 is computed, any position measured in the user space (Xu, Yu, Zu) can be transformed into a position in the virtual space (Xv, Yv, Zy) by multiplying a vector from the origin to the measured coordinates in the user space coordinate system by the transformation matrix M1. Notably, Wang fails to teach or suggest that such calibrating is performed in a mode in which a position of the glove is not detected. Much less does Wang teach or suggest that such calibrating is performed in a mode in which a position of a pen tip of an electronic pen is not detected by a tablet.”
Examiner disagrees –
Aubauer teaches changing operational states in response to a predefined user operation: “The electronic device 100 is adapted for transitioning from a first state into a second state when or after a predefined gesture is identified by the gesture recognition module 138.” (see [0051]). Thus, Aubauer teaches a transition from a first state to a second state in response to a predefined gesture.
Wang teaches predetermined operations for expressly selecting among different operating modes: “there are multiple modes to interpret the 6D input information. When a first button is pressed, a subvolume visible clipping planes adjustment mode may be activated … When a second button is pressed, a 3D rotation mode may be activated.” (see [0085]). Wang also teaches gestures that “generates a command to enter a pan mode” (see [0089]), “generates a command to enter a rotate mode” (see [0090]), and “initiates a zoom mode” (see [0091]).
Aubauer and Wang independently establish that predefined gestures and button operations were known mechanisms for transitioning between operational states or selecting operating modes. In view of Aubauer’s alternative tablet based and contact free positional detection mechanisms, it would have been obvious to use such a predetermined user operation to select between those input modes.
Aubauer paragraph [0042] is separately relied upon for the newly recited condition that, in the contact free operating mode, the pen-tip position is not detected by the tablet. Aubauer expressly provides that “When no touch is detected on the touch-sensitive module 200, the contact free detection means 300 is activated again.” Thus, the “predetermined operation” and the “position of the pen tip … is not detected by the tablet” limitations are supported.
Wang is not relied upon for the absence of tablet pen-tip detection. Aubauer paragraph [0042] supplies that feature. Wang’s coordinate transformation teachings establish that a position detected by the tracking system can be calculated in a different VR space coordinate system.
Wang’s VR coordinate processing is not confined to initial calibration. Following calibration, Wang teaches ongoing tracking: “as a tracked object is moved within a field of view of the positional sensor, the computing device may track the object and determine a corresponding position and/or trajectory of the tracked object in the virtual environment.” (see [0151]). Wang further states that “the computing device receives coordinates of one or more points of a tracked object,” including an “electronic pointer,” from an electromagnetic sensor or an optical sensor (see [0153]), and thereafter “determines a position and orientation of the tracked object in the virtual space” (see [0154]). Wang further explains that “the absolute position of the tip can be computed by multiplying the vector that represents the position of the tracked object’s tip in the user space by the calibration matrix M1.” (see [0155]). Wang paragraph [0122] establishes the coordinate transformation between the tracking/user space coordinate system and virtual space coordinate system, while Wang paragraphs [0151]–[0155] establish application of the tracking and virtual-space determination to an electronic pointer during operation.
Applicant argues –
“For at least the reasons stated above, Applicant respectfully submits that Aubauer and Wang fail to teach or suggest a coordinate calculation system that includes "a computer that, in operation: in a first mode, calculates a first coordinate in a first coordinate system of the tablet based on the position detected by the tablet; and in a second mode that is switched from the first mode in response to a predetermined operation and in which the position of the pen tip of the electronic pen is not detected by the tablet, calculates a second coordinate in a second coordinate system of the VR space that is different from the first coordinate system based on the position of the tracker detected by the tracking device", in combination with the other elements recited in claim 1, as amended herein. Accordingly, withdrawal of the rejection of claim 1 is respectfully requested.
Claims 2-9 depend from claim 1 and are believed to be allowable for the same reasons as discussed above for claim 1, as well as for the specific limitations recited in those claims. Accordingly, withdrawal of the rejection of claims 2-9 is respectfully requested.
While the language and scope of independent claim 10 are not identical to the language and scope of claim 1, the allowability of claim 10 is apparent in view of the above discussion of claim 1 and the cited references. Claims 11-18 depend from claim 10 and are allowable at least by virtue of their dependencies, as well as for the specific limitations recited in those claims. Accordingly, withdrawal of the rejection of claims 10-18 is respectfully requested.”
Examiner disagrees –
The references are not being relied upon individually to disclose the entire claimed system. Aubauer teaches the tablet/digital pen architecture, tablet coordinate detection, contact free three-dimensional positional detection, alternate operation of the two positional detection mechanisms, and predefined state transitions. Wang teaches a positional tracking device tracking an electronic pen and associated tracking markers, VR-space interaction, transformation between tracking-space and virtual-space coordinate systems, and predetermined user operations for selecting operating modes. The claimed above tablet placement of Wang’s optical tracking device would have been an obvious and predictable physical arrangement when the sensor is applied to track Aubauer’s electronic pen operating on and above the tablet.
Therefore, Applicant’s arguments do not overcome the combination of Aubauer and Wang as discussed above, and the rejections of claims 1-18 under 35 U.S.C. § 103 are maintained for at least the reasons above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JENNIFER L ZUBAJLO whose telephone number is (571)270-1551. The examiner can normally be reached Monday - Thursday 10 am - 8 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, KE XIAO can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JENNIFER L ZUBAJLO/Examiner, Art Unit 2627 8/16/2026
/KE XIAO/Supervisory Patent Examiner, Art Unit 2627
Prosecution Insights