DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Newly added claims 2-20 are pending.
The Double Patenting rejection for claim 1 has been considered and the rejection is withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9, 11 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Subramanian et al. (US Publication Number 2015/0192995 A1, hereinafter “Subramanian”) in view of Long et al. (NPL, “Rendering Volumetric Haptic Shapes in Mid-Air using Ultrasound”, 2014).
(1) regarding claim 1:
As shown in fig. 1, Subramanian disclosed a method (fig. 3, para. [0037], note that a method of generating a tactile sensation is disclosed) comprising:
receiving haptic representation for a set of virtual content registered to a virtual volume from a content generation system (para. [0015], note that haptic feedback is provided in association with an interactive surface i.e. first region and second region feedback points for a user of an interactive surface. Also see para. [0035], note that the control points are defined in the 3D volume above the transducer array such that the control points take on the required distribution);
determining a set of user positions relative to a mid-air haptic interaction volume registered with the virtual volume (para. [0053], note that tactile sensations can be used with interactive displays, where mid-air gestures allow a user to interact with the display);
defining a set of virtual contact regions between the haptic representations and the set of user positions in the virtual volume (para. [0046], note that the control points may define a shape to be felt by a user as part of a haptic feedback system of which only the part of the shape with which the user is interacting may need to be defined. The control points in the second group may each be arranged to be adjacent to a control point of the first group, such that a gradient in amplitude of the acoustic field is produced at the edge of a virtual shape).
Subramanian disclosed most of the subject matter as described above except for specifically teaching operating a set of mid-air haptic effectors to generate mid-air haptic feedback within the mid-air haptic interaction volume corresponding to the set of virtual contact points.
However, Long disclosed operating a set of mid-air haptic effectors to generate mid-air haptic feedback within the mid-air haptic interaction volume corresponding to the set of virtual contact points (para. [0053], note that for providing tactile sensations can be used with interactive displays, where mid-air gestures allow a user to interact with the display).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach operating a set of mid-air haptic effectors to generate mid-air haptic feedback within the mid-air haptic interaction volume corresponding to the set of virtual contact points. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 1.
(2) regarding claim 2:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the virtual content is generated from measurements of a physical object.
However, Long disclosed wherein the virtual content is generated from measurements of a physical object (see fig. 9, note Hand-object intersection sampling. a) The hand in the scene touching the virtual cube with the ultrasonic transducer at ray position shown below it. b) The hand is converted into sixteen planes, some of which intersect the object in the scene, cutting the hand. c) The hand-object intersections are found as line segments and processed into contiguous arcs, from which control points are derived).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the virtual content is generated from measurements of a physical object. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 2.
(3) regarding claim 3:
Subramanian disclosed most of the subject matter as described as above except for specifically teaching wherein the physical object corresponds to a second user, wherein the virtual content is generated by: sampling a set of points on the second user; and generating the haptic representation based on the set of points.
However, Long disclosed wherein the physical object corresponds to a second user, wherein the virtual content is generated by: sampling a set of points on the second user (see fig. 11, page 8, para. [0001], note that Then at the point where the mesh and hand interact, the polygon-polygon intersection occurs, and line segment that is generated has its local curvature indication); and generating the haptic representation based on the set of points (page 9, 7 Limitations and Future Work, para. [0001], note that a system for producing volumetric haptic shapes in mid-air is demonstrated. Users are capable of discriminating and identifying volumetric haptic shapes with high accuracy).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the physical object corresponds to a second user, wherein the virtual content is generated by: sampling a set of points on the second user; and generating the haptic representation based on the set of points. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 3.
(4) regarding claim 4:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the haptic representations for the set of virtual content comprise condensed representations of spatial meshes characterizing a surface of each of a set of objects.
However, Long disclosed wherein the haptic representations for the set of virtual content comprise condensed representations of spatial meshes characterizing a surface of each of a set of objects (page 8, para. [0001], note that the curvature along mesh edge then the mesh and hand interact).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the haptic representations for the set of virtual content comprise condensed representations of spatial meshes characterizing a surface of each of a set of objects. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 4.
(5) regarding claim 5:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the haptic representation comprises a closed-form expression generated from a surface of the spatial mesh.
However, Long disclosed wherein the haptic representation comprises a closed-form expression generated from a surface of the spatial mesh (page 8, fig. 11, para. [0001], note that then at the point where the mesh and hand interact, the polygon-polygon intersection occurs, and line segment that is generated has its local curvature indicator specified by: κi(t) = κ0ec(t0−t) + κ1ec(t−t1) (13) where t is the parametric coordinate of the line, t0 the point intersecting the mesh edge at the beginning of the line, t1 the point intersecting the mesh edge at the end, c a decay constant, κ0 the edge curvature at the beginning and κ1 the curvature at the end).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the haptic representation comprises a closed-form expression generated from a surface of the spatial mesh. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 5.
(6) regarding claim 6:
Subramanian further disclosed the method of Claim 1, further comprising adjusting the haptic representation before defining the set of virtual contact points, wherein adjusting the haptic representation comprises at least one of a translating the haptic representation, rotating the haptic representation, or scaling the haptic representation (para. [0046], note that the power output by the transducer array may be scaled such that the transducer outputting the highest of the initial amplitudes operates at substantially full power. Scaling the power output in this way has an advantage in that it results in the power output of the transducer array being as high as possible for a given set of initial amplitudes).
(7) regarding claim 7:
Subramanian further disclosed the method of Claim 1, wherein the set of user positions comprises locations corresponding to a plurality of landmarks on a hand of the user (para. [0017], note that tracking a user's hand during interaction with an interactive screen and modifying the pressure patterns transmitted by the plurality of acoustic transducers in response to the movement of the user's hand).
(8) regarding claim 8:
Subramanian further disclosed the method of Claim 1, further comprising detecting a gesture from the set of user positions and adjusting the set of virtual content based on the gesture (para. [0022], note that the tracker device may detect movement of a user's hand in the play/pause control region and provide control information to the music player accordingly. For example, a gesture such as "tapping" in the play/pause region may result in the music player playing or pausing musical output. The tracker device may detect movement of a user's hand in the volume control region and provide control information to the music player accordingly).
(9) regarding claim 9:
Subramanian further disclosed the method of Claim 1, wherein the set of mid-air haptic effectors comprise ultrasound transducers (para. [0030], note that the transducers are arranged to transmit ultrasound waves at a frequency of 40 kHz).
(10) regarding claim 11:
As shown in fig. 1, Subramanian disclosed a system for generating a tactile sensation (para. [0018], note that a system for providing tactile sensation is disclosed), the system comprising:
a processing system (para. [0031], note that the system 10 is controlled by a PC 22, which sends control data to the projector 1);
a non-transitory computer readable medium storing instructions executable by the processing system to perform a method (para. [0031], note that the PC 22 controls the drive unit 24 such that a pressure pattern is created in the region above the transducer array 12), comprising:
determining haptic representations for a set of virtual content (para. [0015], note that haptic feedback is provided in association with an interactive surface i.e. first region and second region feedback points for a user of an interactive surface. Also see para. [0035], note that the control points are defined in the 3D volume above the transducer array such that the control points take on the required distribution);
determining a set of user positions relative to a mid-air haptic interaction volume (para. [0053], note that tactile sensations can be used with interactive displays, where mid-air gestures allow a user to interact with the display);
determining a set of virtual contact points based on the haptic representation and the set of user positions (para. [0046], note that the control points may define a shape to be felt by a user as part of a haptic feedback system of which only the part of the shape with which the user is interacting may need to be defined. The control points in the second group may each be arranged to be adjacent to a control point of the first group, such that a gradient in amplitude of the acoustic field is produced at the edge of a virtual shape); and
Subramanian disclosed most of the subject matter as described above except for specifically teaching controlling a mid-air haptic system to generate mid-air pressure points corresponding to the set of virtual contact points in a mid-air interaction volume.
However, Long disclosed controlling a mid-air haptic system to generate mid-air pressure points corresponding to the set of virtual contact points in a mid-air interaction volume (para. [0053], note that for providing tactile sensations can be used with interactive displays, where mid-air gestures allow a user to interact with the display).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach controlling a mid-air haptic system to generate mid-air pressure points corresponding to the set of virtual contact points in a mid-air interaction volume. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 11.
(11) regarding claim 19:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the haptic representation comprises a closed-form expression generated from a mesh of the virtual content.
However, Long disclosed wherein the haptic representation comprises a closed-form expression generated from a mesh of the virtual content (page 8, fig. 11, para. [0001], note that then at the point where the mesh and hand interact, the polygon-polygon intersection occurs, and line segment that is generated has its local curvature indicator specified by: κi(t) = κ0ec(t0−t) + κ1ec(t−t1) (13) where t is the parametric coordinate of the line, t0 the point intersecting the mesh edge at the beginning of the line, t1 the point intersecting the mesh edge at the end, c a decay constant, κ0 the edge curvature at the beginning and κ1 the curvature at the end).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the haptic representation comprises a closed-form expression generated from a mesh of the virtual content. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 19.
(13) regarding claim 20:
Subramanian further disclosed the system of Claim 11, further comprising the mid-air haptic system comprising a transducer array and a tracking system that determines the set of user positions (para. [0017], note that the acoustic transducers may transmit pressure patterns through the interactive screen. The method may comprise providing an object detection or tracking device and track a user’s hand during interaction with the interactive screen).
Claim(s) 10 and 16-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Subramanian and Long, and further in view of Beattie et al. (US Publication Number 2020/0218354 A1, hereinafter “Beattie”).
(1) regarding claim 10:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the set of virtual contact points is predicted by a machine learning model based on the haptic representations and the set of user positions.
However, Beattie disclosed wherein the set of virtual contact points is predicted by a machine learning model based on the haptic representations and the set of user positions (para. [0143], note that rendering roughness in visuo-haptic mid-air textures, comprising: (1) forming a haptic prediction model by: (a) training a visual texture dimension machine learning prediction and classification model).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the set of virtual contact points is predicted by a machine learning model based on the haptic representations and the set of user positions. The suggestion/motivation for doing so would have been in order to improve techniques for mid-air haptic-related interactions for textures (para. [0004]). Therefore, it would have been obvious to combine Subramanian, Long with Beattie to obtain the invention as specified in claim 10.
(2) regarding claim 16:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the set of virtual contact points is predicted by a machine learning model.
However, Beattie disclosed wherein the set of virtual contact points is predicted by a machine learning model (para. [0143], note that rendering roughness in visuo-haptic mid-air textures, comprising: (1) forming a haptic prediction model by: (a) training a visual texture dimension machine learning prediction and classification model).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the set of virtual contact points is predicted by a machine learning model. The suggestion/motivation for doing so would have been in order to improve techniques for mid-air haptic-related interactions for textures (para. [0004]). Therefore, it would have been obvious to combine Subramanian, Long with Beattie to obtain the invention as specified in claim 16.
(3) regarding claim 17:
Subramanian further disclosed the system of Claim 16, wherein the virtual content is generated by a game engine (para. [0056], note that the method and system may be used in a virtual reality gaming environment to provide tactile feedback to one or more players of a game).
(4) regarding claim 18:
Subramanian disclosed most of the subject matter as described above except for specifically wherein the virtual content comprises a virtual representation of a physical object that is generated by another instance of the system.
However, Long disclosed wherein the virtual content comprises a virtual representation of a physical object that is generated by another instance of the system (see fig. 9, note Hand-object intersection sampling. a) The hand in the scene touching the virtual cube with the ultrasonic transducer at ray position shown below it. b) The hand is converted into sixteen planes, some of which intersect the object in the scene, cutting the hand. c) The hand-object intersections are found as line segments and processed into contiguous arcs, from which control points are derived).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the virtual content comprises a virtual representation of a physical object that is generated by another instance of the system. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 18.
Claim(s) 12-13 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Subramanian and Long, and further in view of Urzhumov (US Publication Number 2021/0035420 A1).
(1) regarding claim 12:
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein the mid-air haptic system generates the mid-air pressure points by beamforming ultrasound.
However, Urzhumov disclosed wherein the mid-air haptic system generates the mid-air pressure points by beamforming ultrasound (para. [0022], note that a system 200 may include a source or an acoustic beamformer 202 for generating an ultrasound carrier wave or ultrasound beam 208 toward a nonlinear medium 206, such as human skin).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein the mid-air haptic system generates the mid-air pressure points by beamforming ultrasound. The suggestion/motivation for doing so would have been in order to provide a signal modulation that enhances, subject to the total radiated power constraint, a perceived strength of somatosensory response of at least one type of cutaneous mechanoreceptor cells in the skin of the human subject (para. [0004]). Therefore, it would have been obvious to combine Subramanian, Long with Urzhumov to obtain the invention as specified in claim 12.
(2) regarding claim 13:
Subramanian further disclosed the system of Claim 13, wherein the mid-air haptic system comprises a transducer array (para. [0045], note that the transducers of the transducer array are operated such that the transducer array outputs acoustic waves having the initial amplitudes).
Subramanian disclosed most of the subject matter as described above except for specifically teaching wherein beamforming ultrasound comprises determining a phase value based on a distance between a transducer and a virtual contact point of the set of virtual contact points.
However, Urzhumov disclosed wherein beamforming ultrasound comprises determining a phase value (para. [0078], note that acoustic metamaterials are elements specially designed to adjust the phase and amplitude of the incoming wave and manipulate the sound field (e.g. direct the focal point)).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein beamforming ultrasound comprises determining a phase value. The suggestion/motivation for doing so would have been in order to provide a signal modulation that enhances, subject to the total radiated power constraint, a perceived strength of somatosensory response of at least one type of cutaneous mechanoreceptor cells in the skin of the human subject (para. [0004]). Therefore, it would have been obvious to combine Subramanian, Long with Urzhumov to obtain the invention as specified in claim 13.
Also see, Long which disclosed wherein beamforming ultrasound comprises determining a phase value based on a distance between a transducer and a virtual contact point of the set of virtual contact points (page 4, 3.2, para. [0001], note that control points at the same amplitude will amplify each other as they move closer if the phase difference is zero, or can alternatively dampen and cancel as the distance narrows if the phase difference is half a period. This is also true for the local residual field surrounding each control point, asisshowninFigure3).
At the time of filing for the invention, it would have been obvious to a person of ordinary skilled in the art to teach wherein beamforming ultrasound comprises determining a phase value based on a distance between a transducer and a virtual contact point of the set of virtual contact points. The suggestion/motivation for doing so would have been in order to create three-dimensional haptic shapes in mid-air using focused ultrasound. This approach applies to the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations. This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical device (abs.). Therefore, it would have been obvious to combine Subramanian with Long to obtain the invention as specified in claim 13.
(3) regarding claim 15:
Subramanian further disclosed the system of Claim 13, wherein the transducer array comprises an ultrasonic transducer array (para. [0012], note that the acoustic transducers may be ultrasound transducers. The ultrasound transducers may be arranged to emit ultrasound at a frequency).
Allowable Subject Matter
Claim 14 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter: the prior arts made of record do not specifically teaching “wherein controlling a mid-air haptic system to generate the mid-air pressure points comprises transforming the set of virtual contact points into a set of coordinates in the mid-air interaction volume, and wherein the control subsystem determines the phase value for each transducer based on the set of coordinates”, as claimed in claim 14.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Hilina K Demeter whose telephone number is (571) 270-1676.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Y. Poon could be reached at (571) 270- 0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about PAIR system, see http://pari-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HILINA K DEMETER/Primary Examiner, Art Unit 2617
