DETAILED ACTION
This FINAL action is in response to Application No. 18/650,120 filed 4/30/2024. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The amendment presented on 6/4/2026 which provides amendments to claim 1, 3, 5-11, 13, 15-20, and cancelation of claims 2, 4, 12, and 14, is hereby acknowledged. Claims 1, 3, 5-11, 13, and 15-20 are currently pending.
Claim Rejections – Withdrawn
The previous Double Patenting rejections in view of Application No. 18/617,629 (now U.S. Patent 12,461,615) and U.S. Patent No. 11,068,058 are withdrawn as necessitated by amendment.
The previous 35 U.S.C §112 rejection of claims 1-20 is withdrawn as necessitated by amendment.
The previous 35 U.S.C §102 rejection of claims 1-3, 5, 7-9, 11-13, 15, 17, 18, and 20 is withdrawn as necessitated by amendment.
The previous 35 U.S.C §103 rejection of claims 6 and 16 is withdrawn as necessitated by amendment.
Response to Arguments
The amendments to the claims changed the scope necessitating an updated search and consideration. However, pertaining to the independent claims, previously cited Lee, in combination with Vaughn, is applicable to the claims.
Applicant’s discussion of Vaughn focuses on a dataflow pertaining to the action of “throwing” a virtual object (Vaughn, Figure 15). However, the Examiner is relying on the dataflow for “catching” a thrown virtual object (Vaughn, Figure 16). As the laws of physics tell us, a person exerts a stopping force on a moving object when catching it. Additionally, a vector represents a direction and an origin. Vector 632 in Figures 7, 9, and 11, is construable as this stopping force direction AND represents the location (origin) of the body part (hand) relative to the virtual object (based on trajectory vector 618) and, is therefore, identifying the claimed force applying direction and relative position of a body part to the target object. It is additionally noted that the claims only address a force direction, and do not address a force amount.
Applicant additionally contends that Vaughn’s generation of a haptic alert to guide a user to an appropriate position to catch a ball is a completely different objective than the application that focuses on generating contact feedback between the user and a virtual object. However, while the process of catching a thrown virtual object informs the user whether they are lined up with haptic alerts (Vaughn, Figure 16, steps 1308, 1310, and 1312), the process also informs the user whether they make contact / catch with the virtual object with haptic alerts (Vaughn, Figure 16, steps 1314 and 1316). The Examiner submits that the haptic alert when catching a thrown virtual object is equivalent to generating contact feedback between the user and a virtual object.
Applicant’s discussion of Lee contends that Lee does not teach of suggest determining the specific quantity of at least one finger based on a relative position. However, as the updated citations below show, Lee identifies the number of fingers that contact a virtual object based on the physics particles that contact the virtual object and executes vibration according to the number (Lee, [0043]). The crux of Lee pertains to varying the intensity of vibration based on how many physics particles contact a virtual object therefore, varying the vibration intensity according to the number of fingers contacting the virtual object would be obvious as shown below.
Therefore, the combination of Vaughn and Lee is maintained for the amended claims.
Claim Objections
Claims 1, 6, 11, and 16 are objected to because of the following informalities: “quantity of fingers included in the at least one finger”, for example, would be better grammatically. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 3, 5, 7-11, 13, 15, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Vaughn (US 2018/0001192 A1), and further in view of Lee et al. (US 2020/0103971 A1, hereinafter “Lee”).
Regarding claim 1, Vaughn teaches a somatosensory feedback method, applicable to a somatosensory feedback system. More specifically, a system for receiving haptic (vibration) feedback when a physical body part interacts with virtual objects (Vaughn, (abstract), [0023], [0029]-[0031]).
and comprising: obtaining information of interaction between a target object in an immersive environment and [body parts] of a user in a real-world environment according to sense data. More specifically, sensors within haptic devices worn or held by users are used to determine motion of users (Vaughn, [0023]). A process for receiving haptic feedback indicating whether a user virtually “catches” a “thrown” virtual object is described at least in Figure 16. Trajectory information of a virtual projectile is received in step 1302 and physical body part position information is determined in step 1304 (Vaughn, [0026], [0068]).
wherein the information of interaction comprises at least one of a relative position of the [body parts] with respect to the target object, a force applying direction of the [body parts], a moving direction of the target object…. More specifically, trajectory information (moving direction) includes a direction, speed, and end-of-path of a virtual projectile (target object) (Vaughn, [0068]). At step 1306, the trajectory of the virtual object is compared to the position of the body parts (Vaughn, [0026], [0068], relative position/moving direction, whether the virtual object will be caught by the physical hands).
and providing a vibration feedback for the [body parts] according to the information of interaction. More specifically, at steps 1308-1316, variable strength haptic feedback is received to indicate to the user how well the virtual object is lined up with the user’s hands as well as when the virtual object is caught (Vaughn, [0069]-[0070]).
wherein obtaining the information of interaction between the target object and the [body parts] comprises: determining the relative position of the [body parts] with respect to the target object by using the sense data and determining the force applying direction of the [body parts], the moving direction of the target object. More specifically, the process of catching a virtual projectile includes determining the trajectory of the virtual projectile as well as determining the catching body part vector (Vaughn, [0069]). A person catching an object is naturally applying force in generally the opposite direction of the objects trajectory. The trajectory of the virtual projectile is construed as the moving direction of the target object. The body part vector is construed as representing both the relative position of the [body part] with respect to the target object and the force applying direction a body part exerts on a virtual projectile when virtually catching it. Vector 632 represents the catcher’s body part’s relative position to the virtual projectile and force applying direction in Figures 7, 9, and 11.
However, Vaughn may not explicitly teach every aspect of
[the body parts are] at least one finger;
[the information of interaction comprises] a quantity of the at least one finger interacting with the target object;
[determining] and the quantity of the at least one finger interacting with the target object according to the relative position.
Lee discloses providing realistic feedback during contact with a virtual object. The method includes forming a plurality of physics particles to be distributed and arranged in a virtual hand model, detecting whether a physics particle of the virtual hand model contacts the virtual object and, recognizing the position of the physics particle that contacts the virtual object and transmitting vibration to a finger corresponding to the position when determining that the physics particle of the virtual hand model contacts the virtual object, wherein an intensity of the vibration is determined depending on the number of the physics particles that contact the virtual object and a penetration depth when the physics particle and the virtual object contact each other (Lee, abstract). Directions of force can be applied to physics particles (Lee, [0034]). The physics particles are associated with specific fingers, and therefore, are used to identify the number of fingers that contact the virtual object, resulting in different vibrations for different numbers of fingers (e.g., one finger vs five fingers) (Lee, [0043]). Therefore, if each finger is associated with a number of physics particles and applied directions of force, and the number of physics particles determines the vibration intensity, it is construable that number of fingers in contact with the virtual object ultimately determines the vibration intensities.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Vaughn and Lee that a method for providing variable strength vibration feedback in an immersive environment reflecting the interaction of a body part with a virtual target object would include [the body parts are] at least one finger; [the information of interaction comprises] a quantity of the at least one finger interacting with the target object; and [determining] the quantity of the at least one finger interacting with the target object according to the relative position. With Vaughn and Lee disclosing a user interacting with virtual objects and receiving variable vibration feedback accordingly, and with Lee additionally suggesting determining a number of fingers on a hand in contact with a virtual object and the direction of force each of the fingers apply to the virtual object in order to determine vibration feedback intensity, one of ordinary skill in the art of implementing a method for providing variable strength vibration feedback in an immersive environment reflecting the interaction of a body part with a virtual target object would include [the body parts are] at least one finger; [the information of interaction comprises] a quantity of the at least one finger interacting with the target object; and [determining] the quantity of the at least one finger interacting with the target object according to the relative position in order to provide variable vibration in accordance with how much of a user’s body is making contact with a virtual object that can providing a sense of the quality of contact. One would therefore be motivated to combine these teachings as in doing so would create this method for providing variable strength vibration feedback in an immersive environment reflecting the interaction of a physical object with a virtual target object.
Regarding claim 3, Vaughn and Lee teach the somatosensory feedback method of claim 1, wherein determining the relative position of the at least one finger with respect to the target object by using the sense data comprises: calculating pose data of the at least one finger from the sense data; and using the pose data of the at least one finger and pose data of the target object to determine the relative position of the physical object with respect to the target object. More specifically, trajectory (pose) information of a virtual projectile is received in step 1302 and physical body part position (pose) information is determined in step 1304 (Vaughn, [0026], [0068]). The trajectory information includes direction, speed, and end-of-path of a virtual projectile (Vaughn, [0068]). At step 1306, the trajectory of the virtual object is compared to the position of the body parts (Vaughn, [0026], [0068], relative position/moving direction, whether the virtual object will be caught by the physical hands). Lee suggests the body parts can be fingers which are associated with a position and direction of force relative to a virtual object (Lee, abstract, [0034], [0043]).
Regarding claim 5, Vaughn and Lee teach the somatosensory feedback method of claim 2, wherein determining the relative position of the at least one finger with respect to the target object by using the sense data comprises: calculating pose data of a peripheral device of a multi-device system, which is arranged on the at least one finger, from the sense data; transforming the pose data of the peripheral device by preset transformation data, to generate pose data of the at least one finger; and using the pose data of the at least one finger and pose data of the target object to determine the relative position of the at least one finger with respect to the target object. More specifically, the haptic device, which, in addition to providing the haptic feedback, determines position and motion signals of body parts, and could be peripherals in the form of wrist bands, gloves, and wands (Vaughn, [0023]-[0024]). Offsets related to the peripherals size and shape can be used to calculate the position of the body part such as a hand (Vaughn, [0034]). Lee suggests the body parts can be fingers which are associated with a position and direction of force relative to a virtual object (Lee, abstract, [0034], [0043]).
Regarding claim 7, Vaughn and Lee teach the somatosensory feedback method of claim 1, wherein providing the vibration feedback for the at least one finger according to the information of interaction comprises: determining a strength of the vibration feedback according to an angle between the force applying direction of the at least one finger and the moving direction of the target object, wherein the smaller the angle is, the greater the strength of the vibration feedback is. More specifically, Figures 7-12 depict the how the strength of the haptic feedback is affected by the alignment of a “catching” user with the trajectory of the virtual object. Figure 7 shows the most misalignment at a specific angle with Figure 8 depicting the resultant lessor strength of haptic feedback 638 at the end of the response scale 630. Figures 9 and 10 show the angle getting closer to appropriate for catching the virtual object as well as resultant increase in the strength of the haptic feedback. Figures 11 and 12 show that the alignment between the trajectory of the virtual object and the “catching” user is sufficient to catch the virtual object and therefore, outputs the highest strength haptic feedback (Vaughn, Figures 7-12, [0051]-[0056]). Lee suggests the body parts can be fingers which are associated with a position and direction of force relative to a virtual object (Lee, abstract, [0034], [0043]).
Regarding claim 8, Vaughn and Lee teach the somatosensory feedback method of claim 1, further comprising: by a motion sensor of the somatosensory feedback system, generating motion data related to a peripheral device of a multi-device system, which is arranged on the at least one finger, as the sense data. More specifically, multiple haptic devices 106 and 108 for multiple users, which, in addition to providing the haptic feedback, determines position and motion signals of physical body parts, and could be peripherals in the form of wrist bands, gloves, and wands (Vaughn, [0023]-[0024]). The motion sensing of the haptic devices is done by gyroscopic sensors, accelerometers, etc. (Vaughn, 0027). Offsets related to the peripherals size and shape can be used to calculate the position of the body part such as a hand (Vaughn, [0034]). Lee suggests the body parts can be fingers which are associated with a position and direction of force relative to a virtual object (Lee, abstract, [0034], [0043]).
Regarding claim 9, Vaughn and Lee teach the somatosensory feedback method of claim 1, further comprising: by a motion sensor of the somatosensory feedback system, generating motion data related to the at least one finger as the sense data. More specifically, multiple haptic devices 106 and 108 for multiple users, which, in addition to providing the haptic feedback, determines position and motion signals of physical body parts, and could be peripherals in the form of wrist bands, gloves, and wands (Vaughn, [0023]-[0024]). The motion sensing of the haptic devices is done by gyroscopic sensors, accelerometers, etc. (Vaughn, 0027). Offsets related to the peripherals size and shape can be used to calculate the position of the body part such as a hand (Vaughn, [0034]).
Regarding claim 10, Vaughn and Lee teach the somatosensory feedback method of claim 1, further comprising: by a camera of the somatosensory feedback system, generating image data related to the at least one finger as the sense data. More specifically, the image of an actual hand of a user, and subsequently, at least movement, speed, direction, position, shape, and size of the hand is captured with a camera (Lee, [0024]-[0025]).
Regarding claims 11, 13, 15, 17, 18, and 19, these claims recite the somatosensory feedback system that performs the steps of the somatosensory feedback method of claims 1, 3, 5, 7, 8, and 10, respectively, therefore, the same rationale of rejection is applicable.
Regarding claim 20, this claim recites the non-transitory computer readable storage medium with a computer program to execute the somatosensory feedback method of claim 1, therefore, the same rationale of rejection is applicable.
Claim(s) 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Vaughn and Lee, and further in view of Rhee et al. (US 10,776,618 B2, hereinafter “Rhee”)
Regarding claim 6, Vaughn and Lee teach the somatosensory feedback method of claim 1, wherein providing the vibration feedback for the at least one finger according to the information of interaction comprises: determining a strength of the vibration feedback according to the quantity of the at least one finger interacting with the target object. More specifically, directions of force can be applied to physics particles (Lee, [0034]). The physics particles are associated with specific fingers, and therefore, are used to identify the number of fingers that contact the virtual object, resulting in different vibrations for different numbers of fingers (e.g., one finger vs five fingers) (Lee, [0043]). Therefore, if each finger is associated with a number of physics particles and applied directions of force, and the number of physics particles determines the vibration intensity, it is construable that number of fingers in contact with the virtual object ultimately determines the vibration intensities.
However, Vaughn and Lee may not explicitly teach every aspect of
wherein the less the quantity is, greater the strength of the vibration feedback is.
Rhee discloses a virtual environment such that when the user's hand contacts a virtual object included in the virtual space, detecting a particular part of the user's hand contacting the virtual object on the basis of a plurality of pieces of sensing information received from the plurality of cameras, and controlling the plurality of feedback signal units to cause feedback signals having different directions to reach the particular part of the user’s hand (Rhee, abstract). The feedback may be vibration (Rhee, col 12, lines 7-15). The amount of contact area of a user’s hand with a virtual object determines the intensity of feedback. A relatively smaller contact area between the user's hand and the virtual object results in a stronger feedback signal than a relatively larger contact area which results in a weaker feedback signal, thereby providing a more realistic virtual reality (Rhee, col 24, line 44 – col 25, line 31). This implies, as the number of fingers contacting the virtual object increases, the resulting contact area increases, which in turn results in lower intensity feedback than a lessor number of fingers.
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Vaughn and Lee with Rhee that a method for providing variable strength vibration feedback in an immersive environment reflecting a quantity of a fingers contacting a virtual object would include wherein the less the quantity is, greater the strength of the vibration feedback is. With Vaughn, Lee, and Rhee disclosing a user interacting with objects and receiving variable vibration feedback accordingly, with Lee and Nakagawa counting the number of fingers interacting with objects to determine the strength of the vibration, and with Nakagawa additionally suggesting a smaller number of fingers results in a greater strength of vibration, one of ordinary skill in the art of implementing a od for providing variable strength vibration feedback in an immersive environment reflecting a number of a body parts interacting with an object would include a greater strength of vibration being associated with a lesser number of physical objects interacting with the target object in order to provide variable vibration in a way that reflects real world interactions with objects of mass in a virtual way. One would therefore be motivated to combine these teachings as in doing so would create this method for providing variable strength vibration feedback in an immersive environment reflecting a number of a body parts interacting with an object.
Regarding claim 16, this claim recites the somatosensory feedback system that performs the steps of the somatosensory feedback method of claim 6, therefore, the same rationale of rejection is applicable.
Pertinent Prior Art
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
Xiong (US 2022/0179491 A1) – varied haptic feedback to simulate different forces on a finger and/or hand in an immersive environment.
Ghanchi (US 10,265,627 B2) - varied haptic feedback according to whether a user, holding a virtual accessory (e.g., “a bat”), makes contact with a virtual object (e.g., “a ball”) according to the determined swing motion of the virtual accessory and the trajectory of the virtual object. Haptic feedback is varied based on how well the virtual object is contacted and/or the determined location of contact on the virtual accessory.
Geisert (US 2024/0194040 A1) – varied haptic feedback according an angle of a user approaching a virtual object or vice versa.
Rhin (US 2019/0391647 A1) - varied haptic feedback according the number of fingers of a user and/or force of a user interacting with a virtual object.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK F RIEGLER whose telephone number is (571)270-3625. The examiner can normally be reached M-F 9:30am-6:00pm, ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK F RIEGLER/ Primary Examiner, Art Unit 2171
Prosecution Insights