Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 10-14, and 20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Victor-Faichney et al. (U.S. PGPUB 20240272764).
With respect to claim 1, Victor-Faichney et al. disclose a head-mounted display (paragraph 177, the head-wearable device 1811 can present to the user 101 a user interface within the artificial-reality environment), comprising:
a display (paragraph 184, FIG. 9B describes additional details of a head-mounted display (HMD) 1814), being configured to display a virtual object corresponding to a three-dimensional space in an image window (paragraph 41, FIG. 1A shows the user 101 standing in a room in physical reality (on right side of FIG. 1A) while wearing the artificial-reality headset 102, which is presenting a panel user interface 104 (shown from the user's perspective view on left side of FIG. 1A) to the user 101 within an artificial-reality environment that is being presented to the user);
a user input tracking device (paragraph 171, the front body 1456 and/or the frame 1454 includes one or more electronic elements, including one or more electronic displays, one or more IMUs, one or more tracking emitters or detectors, and/or any other suitable device or sensor for creating an artificial-reality experience), being configured to track at least one indicator object operated by a user (paragraph 84, FIGS. 5E and 5F illustrate a sequence of providing a user with a visual indication of selection of a user interface element in response to a detection that a user's digit or other selecting device (e.g., a controller) is within a predefined distance of the user interface element); and
a processor (paragraph 185, one or more processors 1850), being electrically connected to the display and the user input tracking device (Fig. 9B, paragraph 185, the housing 1806 includes a communication interface 1815, circuitry 1846, a power source 1807 (e.g., a battery for powering one or more electronic components of the housing 1806 and/or providing usable power to the HMD 1814), one or more processors 1850, and memory 1860. In some embodiments, the housing 1806 includes one or more supplemental components that add to the functionality of the HMD 1814. For example, in some embodiments the housing 1806 includes one or more sensors 1825, an AR processing module 1845, one or more haptic generators 1821, one or more imaging devices 1855, one or more microphones 1813, one or more speakers 1817, etc.), and being configured to perform the operations comprising:
determining whether there is at least one contact action located in at least one control area corresponding to the virtual object, wherein the contact action is generated by the at least one indicator object (paragraph 47, an indication is provided at a portion of each finger of the user 101 that is in virtual contact with the grab handle affordance associated with the grab-handle user interface element 108), and the at least one control area is close to at least one edge position of the virtual object (108 in Fig. 1E, 1F, 1G);
in response to having the at least one contact action located in the at least one control area corresponding to the virtual object, calculating a displacement value of the at least one indicator object (paragraph 53, a user can grab a panel with two hands (e.g., one on each grab handle) to resize (e.g., virtually stretch) the virtual object. When grabbing with two hands simultaneously, instead of just passing a respective user interface (which can be in a tablet display mode) from hand to hand, some embodiments support the respective user interface being held, positioned, and scaled by both hands); and
adjusting an appearance shape of the virtual object based on the displacement value of the at least one indicator object (paragraph 53, using two hands to interact with the grab-handle user interface element 108 enables precision in one or more additional axes/degrees-of-freedom, allowing the user to provide additional input control to the panel user interface 104).
With respect to claim 2, Victor-Faichney et al. disclose the head-mounted display of claim 1, wherein the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object further comprises the following operations:
calculating a projection trajectory of a projection ray emitted by the at least one indicator object (paragraph 59, A focus selector 204 is projected onto a location of the panel user interface 104 based on a position of a representation 206 of a hand of the user, which corresponds to a location and an orientation of the hand of the user 101. An arrow indicator 208 (which can be the tear-drop shaped element described above in the brief description of the appendices section) is presented in conjunction with the user 101 performing ray casting);
determining whether the projection trajectory is located in the at least one control area corresponding to the virtual object (paragraph 69, FIG. 3A shows the user 101 ray casting a focus selector 306 onto a grabbable portion of the grab-handle user interface element 304. The grab-handle user interface element 304 is presented such that it substantially surrounds the panel user interface 302, in accordance with the user 101 interacting with the grab-handle user interface element 304 via the focus selector 306 being projected by the ray casting gesture being performed by the user); and
in response to the projection trajectory being located in the at least one control area corresponding to the virtual object, determining there is the at least one contact action located in the at least one control area corresponding to the virtual object (paragraph 70, FIG. 3B shows causing the panel user interface 302 to translate in three-dimensional space based on the user adjusting a projected location of the focus selector 306. Based on the user 101 performing the ray casting gesture to move the panel user interface 302, the directional arrow indicator 308 is presented with an adjusted physical appearance. In some embodiments, based on the user performing the operation associated with the grab-handle user interface element 304, a simulated grabbing hand representation is presented at the location of the focus selector 306, to indicate that the user is performing a grab handle operation).
With respect to claim 3, Victor-Faichney et al. disclose the head-mounted display of claim 1, wherein the at least one indicator object is at least one controller operated by the user (paragraph 35, Users may interact with the artificial-reality content using controllers (e.g., game controllers), paragraph 40, the panel user interface 104 can be grabbed via an input (e.g., a grip button) of a handheld controller in electronic communication with the artificial-reality headset 102).
With respect to claim 4, Victor-Faichney et al. disclose the head-mounted display of claim 1, wherein the at least one indicator object is at least one finger corresponding to a target gesture (paragraph 52, FIG. 1K shows the user 101 performing a pinch gesture at a corner of the grab-handle user interface element 108, which causes the panel user interface 104 to be resized).
With respect to claim 10, Victor-Faichney et al. disclose the head-mounted display of claim 1, wherein the virtual object is a display panel (paragraph 39, FIGS. 1A-1K show an example sequence of a user 101 interacting with a panel user interface 104 that is configured to be presented in conjunction with a grab-handle user interface element 108 (revealed in FIG. 1E)), the display panel is configured to display a multimedia content (Fig. 1F and 1G), and the processor further performs the following operations: generating, based on an adjusted appearance shape, the multimedia content corresponding to the adjusted appearance shape (paragraph 53, a user can grab a panel with two hands (e.g., one on each grab handle) to resize (e.g., virtually stretch) the virtual object).
With respect to claim 11, Victor-Faichney et al. disclose a virtual object adjustment method, being adapted for use in an electronic device (paragraph 177, the head-wearable device 1811 can present to the user 101 a user interface within the artificial-reality environment), wherein the electronic device comprises a display (paragraph 184, FIG. 9B describes additional details of a head-mounted display (HMD) 1814), a user input tracking device (paragraph 171, the front body 1456 and/or the frame 1454 includes one or more electronic elements, including one or more electronic displays, one or more IMUs, one or more tracking emitters or detectors, and/or any other suitable device or sensor for creating an artificial-reality experience), and a processor (paragraph 185, one or more processors 1850), the display is configured to display a virtual object corresponding to a three-dimensional space in an image window (paragraph 41, FIG. 1A shows the user 101 standing in a room in physical reality (on right side of FIG. 1A) while wearing the artificial-reality headset 102, which is presenting a panel user interface 104 (shown from the user's perspective view on left side of FIG. 1A) to the user 101 within an artificial-reality environment that is being presented to the user), the user input tracking device is configured to track at least one indicator object operated by a user (paragraph 84, FIGS. 5E and 5F illustrate a sequence of providing a user with a visual indication of selection of a user interface element in response to a detection that a user's digit or other selecting device (e.g., a controller) is within a predefined distance of the user interface element), and the virtual object adjustment method comprises executing the system of claim 1; see rationale for rejection of claim 1.
With respect to claim 12, Victor-Faichney et al. disclose the virtual object adjustment method of claim 11, as executed by the system of claim 2; see rationale for rejection of claim 2.
With respect to claim 13, Victor-Faichney et al. disclose the virtual object adjustment method of claim 11, as executed by the system of claim 3; see rationale for rejection of claim 3.
With respect to claim 14, Victor-Faichney et al. disclose the virtual object adjustment method of claim 11, as executed by the system of claim 4; see rationale for rejection of claim 4.
With respect to claim 20, Victor-Faichney et al. disclose a non-transitory computer readable storage medium, having a computer program stored therein, wherein the computer program comprises a plurality of codes (paragraph 126, a non-transitory computer readable storage medium includes instructions that, when executed by a computing device in communication with an artificial-reality headset, cause the computer device to perform operations corresponding to any of A1-E1), the computer program executes a virtual object adjustment method after being loaded into an electronic device (paragraph 177, the head-wearable device 1811 can present to the user 101 a user interface within the artificial-reality environment), the electronic device comprises a display (paragraph 184, FIG. 9B describes additional details of a head-mounted display (HMD) 1814), a user input tracking device (paragraph 171, the front body 1456 and/or the frame 1454 includes one or more electronic elements, including one or more electronic displays, one or more IMUs, one or more tracking emitters or detectors, and/or any other suitable device or sensor for creating an artificial-reality experience), and a processor (paragraph 185, one or more processors 1850), the display is configured to display a virtual object corresponding to a three-dimensional space in an image window (paragraph 41, FIG. 1A shows the user 101 standing in a room in physical reality (on right side of FIG. 1A) while wearing the artificial-reality headset 102, which is presenting a panel user interface 104 (shown from the user's perspective view on left side of FIG. 1A) to the user 101 within an artificial-reality environment that is being presented to the user), the user input tracking device is configured to track at least one indicator object operated by a user (paragraph 84, FIGS. 5E and 5F illustrate a sequence of providing a user with a visual indication of selection of a user interface element in response to a detection that a user's digit or other selecting device (e.g., a controller) is within a predefined distance of the user interface element), and the virtual object adjustment method comprises executing the system of claim 1; see rationale for rejection of claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Victor-Faichney et al. (U.S. PGPUB 20240272764) in view of Dimitrov et al. (U.S. PGPUB 20180033204).
With respect to claim 5, Victor-Faichney et al. disclose the head-mounted display of claim 1. However, Victor-Faichney et al. do not expressly disclose the operation of
adjusting the appearance shape of the virtual object further comprises the following operations: calculating a curvature corresponding to the virtual object based on
the displacement value of the at least one indicator object; and adjusting the appearance shape of the virtual object based on the curvature corresponding to the virtual object.
Dimitrov et al., who also deal with virtual spaces, disclose a method wherein the operation of adjusting the appearance shape of the virtual object further comprises the following operations:
calculating a curvature corresponding to the virtual object based on the displacement value of the at least one indicator object; and adjusting the appearance shape of the virtual object based on the curvature corresponding to the virtual object (paragraph 55, adjusting the curvature of virtual screen 510 may be made by left and/or right side taps of the touchpad; use of a dedicated slider bar, scroll wheel, or rocker switch; and/or the like included in input controls 530. In some examples, changing the distance d between representative point 540 and point 550 may be made by top and/or bottom edge taps of the touchpad; use of a dedicated slider bar, scroll wheel, or rocker switch; and/or the like included in input controls 530).
Victor-Faichney et al. and Dimitrov et al. are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein the operation of adjusting the appearance shape of the virtual object further comprises the following operations: calculating a curvature corresponding to the virtual object based on the displacement value of the at least one indicator object; and adjusting the appearance shape of the virtual object based on the curvature corresponding to the virtual object, as taught by Dimitrov et al., to the Victor-Faichney et al. system, because it would be advantageous to have a virtual and/or augmented reality system that allows users to select, display, and/or manipulate non-virtual content within their virtual and/or augmented environments (paragraph 3 of Dimitrov et al.), thus allow greater user control in virtual environments.
With respect to claim 15, Victor-Faichney et al. as modified by Dimitrov et al. disclose the virtual object adjustment method of claim 11, as executed by the system of claim 5; see rationale for rejection of claim 5.
Claim(s) 6-7 and 16-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Victor-Faichney et al. (U.S. PGPUB 20240272764) in view of Stellmach et al. (U.S. PGPUB 20190324529).
With respect to claim 6, Victor-Faichney et al. disclose the head-mounted display of claim 1. However, Victor-Faichney et al. do not expressly disclose the processor further performs the following operations: calculating, based on an eyeball trajectory of the user, an eye gaze position in the image window of the user at each of a plurality time points; and determining whether to perform the operation of determining whether
there is the at least one contact action located in the at least one control area corresponding to the virtual object based on the eye gaze positions.
Stellmach et al., who also deal with virtual environments, disclose a method wherein the processor further performs the following operations: calculating, based on an eyeball trajectory of the user, an eye gaze position in the image window of the user at each of a plurality time points (paragraph 36, measuring the first position of the gaze location may include measuring and retaining in memory all positions of the gaze location for a buffer time, such as 5 milliseconds (ms), 10 ms, 15 ms, 20 ms, 25 ms, 50 ms, 100 ms, 500 ms, or more time. For example, upon receiving a selection command, the system may select the virtual element the user was looking at 10 ms prior to receiving the selection command); and
determining whether to perform the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object based on the eye gaze positions (paragraph 42, The user may interact with the virtual element 222 and/or the virtual environment 220 by positioning their gaze at the virtual element 222 or other portion of the virtual environment 220. The gaze-tracking system in data communication with the processor of the HMD may measure a gaze location 224 at a first location on the virtual element 222). If the eye gaze position falls on the virtual element, this determines whether to perform an additional contact action.
Victor-Faichney et al. and Stellmach et al. are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein the processor further performs the following operations: calculating, based on an eyeball trajectory of the user, an eye gaze position in the image window of the user at each of a plurality time points; and determining whether to perform the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object based on the eye gaze positions, as taught by Stellmach et al., to the Victor-Faichney et al. system, because this would implement improving interaction with virtual elements using gaze-based selection and manipulation (paragraph 21 of Stellmach et al.), thus allow for better hands-free user interaction.
With respect to claim 7, Victor-Faichney et al. as modified by Stellmach et al. disclose the head-mounted display of claim 6, wherein the operation of determining whether to perform the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object comprises the following operations:
in response to the eye gaze position being located in the at least one control area, determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object (Stellmach et al: paragraph 43, Referring now to FIG. 5, the virtual element 222 may be selected while the gaze location 224 is positioned on the virtual element 222 by providing a selection command 226 with an input device 228. In the depicted embodiment, the selection command 226 may be a downward “click” of a user's finger and the input device 228 may be gesture recognition device that recognizes the click of the user's finger or other hand gesture).
With respect to claim 16, Victor-Faichney et al. as modified by Stellmach et al. disclose the virtual object adjustment method of claim 11, as executed by the system of claim 6; see rationale for rejection of claim 6.
With respect to claim 17, Victor-Faichney et al as modified by Stellmach et al. disclose the virtual object adjustment method of claim 16, as executed by the system of claim 7; see rationale for rejection of claim 7.
Claim(s) 8 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Victor-Faichney et al. (U.S. PGPUB 20240272764) in view of Stellmach et al. (U.S. PGPUB 20190324529) and further in view of Okabe et al. (U.S. Patent No. 10,506,913).
With respect to claim 8, Victor-Faichney et al. as modified by Stellmach et al. disclose the head-mounted display of claim 7. However, Victor-Faichney et al. as modified by Stellmach et al. do not expressly disclose the processor further performs the following operations: in response to the eye gaze position being located at a first edge position of the at least one edge position at a first time point, determining whether a first contact action is located in a first control area of the at least one control area, wherein the first control area is close to the first edge position; in response to the eye gaze position being located at a second edge position of the at least one edge position at a second time point, determining whether a second contact action is located in a second control area of the at least one control area, wherein the second control area is close to the second edge position; and in response to the first contact action being located in the first control area and the second contact action being located in the second control area, calculating the displacement value of the at least one indicator object.
Okabe et al., who also deal with virtual spaces, disclose a method wherein the processor further performs the following operations: in response to the eye gaze position being located at a first edge position of the at least one edge position at a first time point, determining whether a first contact action is located in a first control area of the at least one control area, wherein the first control area is close to the first edge position (column 9, lines 36-44, Specifically, a point of gaze P1 (FIG. 9A) is detected from an image of an eye of the user obtained by the line-of-sight detecting camera 520);
in response to the eye gaze position being located at a second edge position of the at least one edge position at a second time point, determining whether a second contact action is located in a second control area of the at least one control area, wherein the second control area is close to the second edge position (column 9, lines 36-44, and then a point of gaze P2 (FIG. 9B) is detected in accordance with a movement of a line of sight of the user. A rectangular region EA, in which the point of gaze P1 and the point of gaze P2 are at the lower left corner and the upper right corner, respectively, is regarded as an enlargement range); and
in response to the first contact action being located in the first control area and the second contact action being located in the second control area, calculating the displacement value of the at least one indicator object (column 10, lines 7-9, After the enlargement range is set in step S112, the process proceeds to step S114, where the process of detecting a line of sight is suspended, and the process proceeds to step S116). The displacement is based on the defined enlargement range. The eye gaze is located at a first edge position and at a second edge position (column 12, lines 41-42, determine an operation target region in which the first point of gaze and second point of gaze are on a border).
Victor-Faichney et al., Stellmach et al., and Okabe et al. are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein the processor further performs the following operations: in response to the eye gaze position being located at a first edge position of the at least one edge position at a first time point, determining whether a first contact action is located in a first control area of the at least one control area, wherein the first control area is close to the first edge position; in response to the eye gaze position being located at a second edge position of the at least one edge position at a second time point, determining whether a second contact action is located in a second control area of the at least one control area, wherein the second control area is close to the second edge position; and in response to the first contact action being located in the first control area and the second contact action being located in the second control area, calculating the displacement value of the at least one indicator object, as taught by Okabe et al., to the Victor-Faichney et al. as modified by Stellmach et al. system, because accordingly, a necessary operation can be easily performed by using a line of sight and a gesture (for example, a gesture with the right hand) (column 10, lines 60-62 of Okabe et al.).
With respect to claim 18, Victor-Faichney et al. as modified by Stellmach et al. and Okabe et al. disclose the virtual object adjustment method of claim 17, as executed by the system of claim 8; see rationale for rejection of claim 8.
Claim(s) 9 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Victor-Faichney et al. (U.S. PGPUB 20240272764) in view of Stellmach et al. (U.S. PGPUB 20190324529) and further in view of Dessero et al. (U.S. PGPUB 20250078420).
With respect to claim 9, Victor-Faichney et al. as modified by Stellmach et al. disclose the head-mounted display of claim 6, wherein the operation of determining whether to perform the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object comprises the following operations: in response to the eye gaze position not being located in the at least one control area, stopping to determine whether there is the at least one contact action located in the at least one control area corresponding to the virtual object.
Dessero et al., who also deal with virtual spaces, disclose a method wherein the operation of determining whether to perform the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object comprises the following operations: in response to the eye gaze position not being located in the at least one control area, stopping to determine whether there is the at least one contact action located in the at least one control area corresponding to the virtual object (paragraph 216, As shown in FIG. 7C, user 712 ceases directing an input (e.g., ceases to move a hand for more than a threshold amount of time, depinching the user's fingers for an air pinch input, closing the user's eyes, or another input that indicates an end of the input) to first virtual object 704a and directs an input (e.g., an air pinch input, an air tap input, a pinch input, a tap input, an air pinch and drag input, an air drag input, a drag input, a click and drag input, a gaze input, and/or other input) to second virtual object 704b. Particularly, gaze 708 is directed to second virtual object 704b).
Victor-Faichney et al., Stellmach et al., and Dessero et al. are in the same field of endeavor, namely computer graphics.
Before the effective filing date of the claimed invention, it would have been obvious to apply the method wherein the operation of determining whether to perform the operation of determining whether there is the at least one contact action located in the at least one control area corresponding to the virtual object comprises the following operations: in response to the eye gaze position not being located in the at least one control area, stopping to determine whether there is the at least one contact action located in the at least one control area corresponding to the virtual object, as taught by Dessero et al., to the Victor-Faichney et al. as modified by Stellmach et al. system, because this would implement transitioning to interact with other portions of a virtual space.
With respect to claim 19, Victor-Faichney et al. as modified by Stellmach et al. and Dessero et al. disclose the virtual object adjustment method of claim 16, as executed by the system of claim 9; see rationale for rejection of claim 9.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. PGPUB 20250378666 to Li for a method of dragging the edge of a virtual object to enlarge or reduce its size
U.S. PGPUB 20240115934 to Ohashi for a method of changing the size of a virtual object according to the distance between two fingers.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW GUS YANG whose telephone number is (571)272-5514. The examiner can normally be reached M-F 9 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW G YANG/Primary Examiner, Art Unit 2614
6/18/26
Prosecution Insights