Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 15 is objected to because of the following informalities: it appears claim 15 should depend on claim 14. Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 14, and 20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 21, and 23 of U.S. Patent No. 12443286 in view of Pinchon et al (11294475) hereinafter, Pinchon.
‘286 fails to expressly teach comparing the distance to a threshold.
However, Pinchon teaches comparing the distance to a threshold. (fig. 5 (506 and 512)).
It would have been obvious to one of ordinary skill in the art to modify the teachings of ‘’286 to further include comparing the distance to a threshold as taught by Pinchon in order to allow for more intuitive inputs and allow for interaction such that the user’s hand does not block the view (col. 2 lines 25-54).
19328334
12443286
1. A method comprising: an electronic device having a processor: obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space; obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; determining a distance between the user hand position and the position of the virtual object; selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein: the direct interaction mode is selected if the distance is below the threshold; and the indirect interaction mode is selected if the distance is above the threshold, and interpreting the hand gesture using an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
1. A method comprising: at an electronic device having a processor: obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space; obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; determining an interaction mode based on the user hand position and the position of the virtual object in the 3D space, wherein a direct interaction mode is selected when a relationship between the user hand position and the position of the virtual object in the 3D space satisfies a criterion and an indirect interaction mode is selected when the relationship satisfies a second criterion different than the first criterion; and interpreting the hand gesture using an interaction recognition process associated with the determined interaction mode, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
14. A system comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising: obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space; obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; determining a distance between the user hand position and the position of the virtual object; selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein: the direct interaction mode is selected if the distance is below the threshold; and the indirect interaction mode is selected if the distance is above the threshold, and interpreting the hand gesture using an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
21. A system comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising: obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space; obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; determining an interaction mode based on the user hand position and the position of the virtual object in the 3D space, wherein a direct interaction mode is selected when a relationship between the user hand position and the position of the virtual object in the 3D space satisfies a criterion and an indirect interaction mode is selected when the relationship satisfies a second criterion different than the first criterion; and interpreting the hand gesture using an interaction recognition process associated with the determined interaction mode, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
20. A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising: obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; determining a distance between the user hand position and the position of the virtual object; selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein: the direct interaction mode is selected if the distance is below the threshold; and the indirect interaction mode is selected if the distance is above the threshold, and interpreting the hand gesture using an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
23. A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising: obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space; obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; determining an interaction mode based on the user hand position and the position of the virtual object in the 3D space, wherein a direct interaction mode is selected when a relationship between the user hand position and the position of the virtual object in the 3D space satisfies a criterion and an indirect interaction mode is selected when the relationship satisfies a second criterion different than the first criterion; and interpreting the hand gesture using an interaction recognition process associated with the determined interaction mode, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process is used when the indirect interaction mode is selected.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5, 7-8, 10-18 and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Pinchon et al (11294475) hereinafter, Pinchon
In regards to claim 1, Pinchon teaches a method comprising: an electronic device having a processor (abstract):
Embodiments described herein disclose methods and systems directed to input mode selection in artificial reality. In some implementations, various input modes enable a user to perform precise interactions with a target object without occluding the target object. Some input modes can include rays that extend along a line that intersects an origin point, a control point, and an interaction point. An interaction model can specify when the system switches between input modes, such as modes based solely on gaze, using long or short ray input, or with direct interaction between the user's hand(s) and objects. These transitions can be performed by evaluating rules that take context factors such as whether a user's hands are in view of the user, what posture the hands are in, whether a target object is selected, and whether a target object is within a threshold distance from the user.
obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space (fig. 2a-2b)(col. 1, lines 40-50)col. 3-4, lines 54-30))
PNG
media_image1.png
582
464
media_image1.png
Greyscale
obtaining a user hand position space based on sensor data, wherein the hand position is associated with a hand gesture (col. 5, lines 5-32) (col. 4, lines 55-67);
PNG
media_image2.png
624
870
media_image2.png
Greyscale
determining a distance between the user hand position and the position of the virtual object (col. 2, lines 45-55, col. 3, lines 3-10) (fig. 4 (436)/434);
PNG
media_image3.png
602
810
media_image3.png
Greyscale
selecting between a direct interaction mode (col. 3, lines 3-10 (fig. 4 (440)) (fig. 5 (506) to (508)) and an indirect interaction (fig. 5 506 to 510 or fig. 5 512)) mode based (fig. 4 (444/446)) on comparing the distance to a threshold (fig. 4 (436)), wherein:
the direct interaction mode is selected if the distance is below the threshold (col. 10, lines 22-35); and
the indirect interaction mode is selected if the distance is above the threshold (col. 10, lines 52-67), and
interpreting (col. 8, lines 10-20) the hand gesture using an interaction recognition process (col. 12, lines 1-17), wherein a direct interaction recognition process is used when the direct interaction mode is selected (fig. 5 (508)) and an indirect interaction recognition process is used when the indirect interaction mode is selected (col. 12, lines 17-67) (fig. 5 (514/516)).
In regards to claim 14, Pinchon teaches system comprising: a non-transitory computer-readable storage medium (abtract); and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on handthe one or more processors, cause the system to perform operations comprising: obtaining a position of a virtual object in an extended reality (XR) environment corresponding to a three-dimensional (3D) space(fig. 2a-2b)(col. 1, lines 40-50)col. 3-4, lines 54-30)); obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture (col. 5, lines 5-32) (col. 4, lines 55-67);; determining a distance between the user hand position and the position of the virtual object; selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold(col. 2, lines 45-55, col. 3, lines 3-10) (fig. 4 (436)/434);, wherein: the direct interaction mode is selected if the distance is below the threshold(col. 3, lines 3-10 (fig. 4 (440)) (fig. 5 (506) to (508)); and the indirect interaction mode is selected if the distance is above the threshold, (col. 10, lines 52-67), and interpreting the hand gesture using an interaction recognition process(col. 8, lines 10-20) (col. 12, lines 1-17),, wherein a direct interaction recognition process is used when the direct interaction mode is selected (fig. 5 (508)) and an indirect interaction recognition process is used when the indirect interaction mode is selected. (col. 12, lines 17-67) (fig. 5 (514/516)).
In regards to claim 20, Pinchon teaches non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising(abstract):: obtaining a user hand position in the 3D space based on sensor data, wherein the hand position is associated with a hand gesture; (fig. 2a-2b)(col. 1, lines 40-50)col. 3-4, lines 54-30)) (col. 5, lines 5-32) (col. 4, lines 55-67); determining a distance between the user hand position and the position of the virtual object(col. 2, lines 45-55, col. 3, lines 3-10) (fig. 4 (436)/434);; selecting between a direct interaction mode and an indirect interaction mode based on comparing the distance to a threshold, wherein: the direct interaction mode is selected if the distance is below the threshold; (col. 3, lines 3-10 (fig. 4 (440)) (fig. 5 (506) to (508)) and the indirect interaction mode is selected if the distance is above the threshold(col. 10, lines 52-67),, and interpreting the hand gesture using (col. 8, lines 10-20) (col. 12, lines 1-17), an interaction recognition process, wherein a direct interaction recognition process is used when the direct interaction mode is selected and an indirect interaction recognition process (fig. 5 (508)) is used when the indirect interaction mode is selected. (col. 12, lines 17-67) (fig. 5 (514/516)).
In regards to claim 2, Pinchon teaches method of claim 1 further comprising identifying which virtual object to associate the hand gesture with based on the selecting between the direct interaction mode and the indirect interaction mode (col. 10, lines 22-33).
In regards to claim 3, Pinchon teaches method of claim 2, wherein, for the direct interaction mode, identifying which virtual object to associate the hand gesture with is based on hand proximity (col. 10, lines 22-33 “pressing a virtual” button).
In regards to claim 4, Pinchon teaches method of claim 2, wherein, for the indirect interaction mode, identifying which virtual object to associate the hand gesture with is based on gaze direction. (14) Eye gaze input mode can include tracking head position and/or eye gaze direction of the user to determine input direction and the target object that a ray will be cast at. In some cases, the gaze input action can be signaled using a gaze timer (e.g., a gaze held without moving above a time threshold amount, such as every threshold amount of time 0.5, 1, 2, or 3 seconds). In other cases, the gaze input action can be signaled with an eye blink. In other cases, the gaze input mode activation can be a hand gesture recognized with a wearable (e.g., bracelet can recognize tap gesture from inertial measurement unit (IMU) data, electromagnetic motion data, or analysis of wrist muscle contraction). The eye gaze selection can be a fast, efficient, and natural way to select target objects, requiring less user energy as compared to requiring the user to lift her hand into view to select an object. (col 2-3, lines 55-3).
In regards to claim 5, Pinchon teaches method of claim 1, wherein the direct interaction mode is selected based on the hand position intersecting a 3D area associated with the virtual object in the 3D space. (col. 10, lines 22-33 “pressing a virtual” button).
In regards to claim 7, Pinchon teaches method of claim 1, wherein the hand gesture is a direct touch with the virtual object (col. 10, lines 22-33 “pressing a virtual” button)..
In regards to claim 8, Pinchon teaches method of claim 1 further comprising initiating a hover of the virtual object based on interaction mode being a direct interaction mode and the hand position satisfying being within a second threshold of the position of the virtual object and within an attention zone.(fig. 5 “short ray” and attention zone is the virtual object area)col. 14, lines 47-67))
In regards to claim 10, Pinchon teaches method of claim 1, wherein the indirect interaction mode is selected based on a gaze direction intersecting an area associated with at least one virtual object (col. 10, lines 1-25).
In regards to claim 11, Pinchon teaches method of claim 1, wherein the indirect interaction mode is selected based on the hand position being outside of a 3D area associated with the virtual object in the 3D space (fig. 5 (502)).
In regards to claim 12, Pinchon teaches method of claim 1, wherein the hand gesture is a pinch recognized by the indirect interaction recognition process as a selection of a second virtual object identified based on a gaze direction occurring during the pinch. “In some cases, eye gaze is the default input mode for selecting target objects, when a target object is selected (e.g. with a pinch gesture) but the user's hands are not in-view or the target object is beyond a threshold distance from the user.” Col. 12, lines 10-17)
In regards to claim 13, Pinchon teaches method of claim 1, wherein the electronic device is a head-mounted device (fig. 2a and 2b HMD).
In regards to claim 15, Pinchon teaches system of claim 1, wherein the operations further comprise identifying which virtual object to associate the hand gesture with based on the selecting between the direct interaction mode and the indirect interaction mode (fig. 5 (502-510) (col. 10, lines 22-33)).
In regards to claim 16, Pinchon teaches system of claim 15, wherein, for the direct interaction mode, identifying which virtual object to associate the hand gesture with is based on hand proximity. ( col.3, lines 3-10, col. 10, lines 22-35, fig. 4 (440)));
In regards to claim 17, Pinchon teaches system of claim 15, wherein, for the indirect interaction mode, identifying which virtual object to associate the hand gesture with is based on gaze direction. (col. 10, lines 1-25).
In regards to claim 18, Pinchon teaches system of claim 14, wherein the direct interaction mode is selected based on the hand position intersecting a 3D area associated with the virtual object in the 3D space. (fig. 5 (502)).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pinchon in view of Kaplan et al (2014/0118244) hereinafter, Kaplan.
In regards to claim 6, Pinchon fails to teach the method of claim 1 further comprising discontinuing interaction detection for a predetermined period of time following a recognition of the hand gesture via the direct interaction recognition process.
However, discontinuing interaction detection for a predetermined period of time following a recognition of the hand gesture via the direct interaction recognition process.[0070].
It would have been obvious to one of ordinary skill in the art to modify the teachings of Pinchon to further include discontinuing interaction detection for a predetermined period of time following a recognition of the hand gesture via the direct interaction recognition process as taught by Kaplan in order to help reduce unintentional inputs [0070].
In regards to claim 19, Pinchon in view of Kaplan teaches, see rational of claim 6, the system of claim 14, wherein the operations further comprise discontinuing interaction detection for a predetermined period of time following a recognition of the hand gesture via the direct interaction recognition process.[0070] Kaplan.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Pinchon in view of Faaborg et al (2017/0060230) hereinafter, Faaborg.
In regards to claim 9, Pinchon fails to teach the method of claim 8 further comprising maintaining the hover of the virtual object when the position of the virtual object is no longer in the attention zone.
However, Faaborg teaches comprising maintaining the hover of the virtual object when the position of the virtual object is no longer in the attention zone.(fig. 4a-4d (4c 102 touch/hover [0033] attention zone of gaze is moved).
PNG
media_image4.png
756
546
media_image4.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the teachings of Pinchon to further include further comprising maintaining the hover of the virtual object when the position of the virtual object is no longer in the attention zone as taught by Faaborg in order to interact with multiple objects in order to better interact and personalize control in 3d environment [002-005].
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRANT SITTA whose telephone number is (571)270-1542. The examiner can normally be reached M-F 7:30-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick Edouard can be reached at 571-272-6084. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRANT SITTA/Primary Examiner, Art Unit 2622
Prosecution Insights