DETAILED ACTION
The action is responsive to the Application filed on 03/26/2024. Claims 1-22 are pending in the case. Claims 1, 21 and 22 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 6-8, 14 and 17-22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ravasz et al. (US 20200387228 A1, hereinafter Ravasz).
M
As to claim 1, Ravasz discloses a method, comprising:
at a computer system that is in communication with a display generation component (“In this example, HIVID 112 includes one or more processors 302 and memory 304… As discussed with respect to the example of FIG. 2, processors 302 are coupled to electronic display 203,” Ravasz paragraph 0049):
detecting a request to display a user interface overlay for a user interface region ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, activation request from user to display the menu); and
in response to detecting the request to display the user interface overlay, displaying, via the display generation component, the user interface overlay with respect to the user interface region based on a location of an anchor element within the user interface region ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, display menu prompt that is anchored to the user’s hand location (i.e., an anchor element anchored to the user’s hand) and then displaying menu based on a location of the hand and menu prompt), wherein:
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to a first location with respect to the user interface region and the user interface region is a first size, the user interface overlay has a first set of one or more visual characteristics that is based on the first location and the first size ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, x and y axis location of the user's hand and menu prompt to set the location of the overlay menu and z axis location of the user’s hand and menu prompt (which changes the displayed size of the user interface region) to set how close the overlay menu is to the user);
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to a second location, different from the first location, with respect to the user interface region and the user interface region is the first size, the user interface overlay has a second set of one or more visual characteristics that is based on the second location and the first size, wherein the second set of one or more visual characteristics is different from the first set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, overlay menu could be in a different x and y axis location while maintaining the z axis value (i.e., hand and menu prompt maintains the same size thus maintaining the displayed size of the UI region));
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the first location with respect to the user interface region and the user interface region is a second size, different from the first size, the user interface overlay has a third set of one or more visual characteristics that is based on the first location and the second size, wherein the third set of one or more visual characteristics is different from the first set of one or more visual characteristics and the second set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, can have a different z axis value (i.e., has the displayed UI has a different size) while maintaining the x and y axis location); and
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the second location with respect to the user interface region and the user interface region is the second size, the user interface overlay has a fourth set of one or more visual characteristics that is based on the second location and the second size, wherein the fourth set of one or more visual characteristics is different from the first set of one or more visual characteristics, the second set of one or more visual characteristics, and the third set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, overlay menu could have different x and y axis values thus changing both the location and the z-axis value for the hand and menu prompt can also be different thus changing the displayed size of the UI region).
As to claim 2, Ravasz further discloses the method of claim 1, further comprising:
prior to displaying the user interface overlay, displaying, via the display generation component, the anchor element concurrently with the user interface overlay ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072; Ravasz Figure 8 810; Ravasz Figure 7G 706 circle still being displayed).
As to claim 3, Ravasz further discloses the method of claim 1, wherein detecting the request to display the user interface overlay for the user interface region includes receiving an input that corresponds to selection of the anchor element ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, menu prompt needs to be displayed in order for menu activation gesture to have any effect (i.e., the effect corresponds to selection of the menu prompt anchor element)).
As to claim 4, Ravasz further discloses the method of claim 1, wherein:
in response to detecting the request to display the user interface overlay:
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the first location with respect to the user interface region, the user interface overlay is displayed over at least a subset of a first portion of the user interface region and is not displayed over a second portion of the user interface region ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close it is to the user the menu will overlap different parts of the underlying content); and
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the second location with respect to the user interface region, the user interface overlay is displayed over at least a subset of the second portion of the user interface region and is not displayed over the first portion of the user interface region ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close it is to the user the menu will overlap different parts of the underlying content).
As to claim 6, Ravasz further discloses the method of claim 1, wherein:
in response to detecting the request to display the user interface overlay:
a subset of a first portion of the user interface overlay is displayed at a first distance outside of the user interface region ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close it is to the user the menu will overlap different parts of the underlying content); and
no portion of the user interface overlay is displayed at a second distance outside of the user interface region that is different from the first distance ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close it is to the user the menu will overlap different parts of the underlying content).
As to claim 7, Ravasz further discloses the method of claim 6, wherein:
in accordance with a determination that the user interface region is at the first size, the first distance corresponds to a first value ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close the hand, menu prompt (i.e., changing the size of the displayed UI) and menu is to the user the menu will overlap different parts of the underlying content); and
in accordance with a determination that the user interface region is at the second size, the second distance corresponds to a second value different from the first value ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close the hand, menu prompt (i.e., changing the size of the displayed UI) and menu is to the user the menu will overlap different parts of the underlying content).
As to claim 8, Ravasz further discloses the method of claim 7, wherein the first size is greater than the second size, and wherein the first value is less than the second value ("Moreover, in accordance with the techniques of this disclosure, based on the sensed data, the artificial reality application detects gestures performed by user 110 and, in response to detecting one or more particular gestures, generates one or more user interface elements, e.g., UI menu 124 and UI element 126, which may be overlaid on underlying artificial reality content 122 being presented to the user," Ravasz paragraph 0026; "Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, depending on where the menu is placed and how close the hand, menu prompt (i.e., changing the size of the displayed UI) and menu is to the user the menu will overlap different parts of the underlying content).
As to claim 14, Ravasz further discloses the method of claim 1, further comprising:
while the user interface overlay is displayed at a fourth size, detecting a request to change the size of the user interface region ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, z axis location of the user’s hand and menu prompt (which changes the displayed size of the user interface region as more of the underlying user interface region is now obscured) to set how close the overlay menu is to the user); and
in response to detecting the request to change the size of the user interface region, changing the size of the user interface overlay from the fourth size to a fifth size different from the fourth size ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, z axis location of the user’s hand and menu prompt (which changes the displayed size of the user interface region as more of the underlying user interface region is now obscured) to set how close the overlay menu is to the user).
As to claim 17, Ravasz further discloses the method of claim 1, further comprising:
ceasing the display of the user interface overlay ("If the artificial reality system detects that the user's hand is no longer performing a menu prompt gesture or a menu activation gesture, then the artificial reality system can determine if a UI menu 124 or menu prompt 810 has been displayed. If so, the artificial reality system can remove the UI menu 124 or menu prompt 810 from the display (514) because the user's hand is no longer in the appropriate configuration to display the UI menu 124 or menu prompt 810," Ravasz paragraph 0079; "After removing the UI menu 124 or menu prompt 810, the flow returns to determine a new current configuration of the hand (502). There may be many other operations performed by the artificial reality system in between removing the UI menu 124 or menu prompt 810 and determining a new configuration of the hand," Ravasz paragraph 0080);
after ceasing to display the user interface overlay, detecting an input corresponding to selection of the anchor element ("If the artificial reality system detects that the user's hand is no longer performing a menu prompt gesture or a menu activation gesture, then the artificial reality system can determine if a UI menu 124 or menu prompt 810 has been displayed. If so, the artificial reality system can remove the UI menu 124 or menu prompt 810 from the display (514) because the user's hand is no longer in the appropriate configuration to display the UI menu 124 or menu prompt 810," Ravasz paragraph 0079; "After removing the UI menu 124 or menu prompt 810, the flow returns to determine a new current configuration of the hand (502). There may be many other operations performed by the artificial reality system in between removing the UI menu 124 or menu prompt 810 and determining a new configuration of the hand," Ravasz paragraph 0080; “The artificial reality system may determine a current configuration of a hand (502),” Ravasz paragraph 0066; “The artificial reality system may determine if the current configuration of the hand indicates that the user is performing a menu prompt gesture (504),” Ravasz paragraph 0067; "In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, display menu prompt that is anchored to the user’s hand location (i.e., an anchor element anchored to the user’s hand) and then displaying menu based on a location of the hand and menu prompt); and
in response to detecting the input corresponding to selection of the anchor element, displaying, via the display generation component, the user interface overlay ("In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072).
As to claim 18, Ravasz further discloses the method of claim 1, further comprising:
prior to detecting the request to display the user interface overlay for the user interface region, displaying the user interface region with a first amount of visual emphasis; and in response to detecting the request to display the user interface overlay, displaying the user interface region with a second amount of visual emphasis different from the first amount of visual emphasis ("In this example, a three-dimensional highlighting is used, where the menu item in proximity to the slidably engageable UI element 706 can be brought forward, thereby making the image appear larger to the user. In addition, the icon 722 corresponding to the menu item 724 can also be highlighted. In this example, the boarder of the icon 722 is highlighted," Ravasz paragraph 0090, background user interface region has a first visual emphasis and then a second visual emphasis based on three dimensional highlighting and how much of the background user interface region is obscured by the menu).
As to claim 19, Ravasz further discloses the method of claim 1, wherein the user interface overlay is a first distance from a viewpoint and the user interface region is a second distance from the viewpoint, and wherein the first distance is less than the second distance ("FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087; Ravasz Figure 7D; Ravasz Figure 7E 124 menu is now closer than the other UI elements to the user’s viewpoint).
As to claim 20, Ravasz further discloses the method of claim 1, further comprising:
while displaying the user interface overlay, concurrently displaying a user interface element and the user interface region (“As one example, artificial reality content 122 may be a consumer gaming application in which user 110 is rendered as avatar 120 with one or more virtual objects 128A, 128B,” Ravasz paragraph 0024; Ravasz Figure 7E 124 menu displayed concurrently with virtual object user interface elements 120, 128A and 128B), wherein:
the user interface overlay is displayed with a first distance in a z-depth from the user interface region ("FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087; Ravasz Figure 7D; Ravasz Figure 7E 124 menu is now closer than the other UI elements to the user’s viewpoint than the virtual objects 120, 128A and 128B); and
the user interface overlay is displayed with a second distance in the z-depth from the user interface element, wherein the first distance in the z-depth is different from the second distance in the z-depth ("FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087; Ravasz Figure 7D; Ravasz Figure 7E 124 menu is now closer than the other UI elements to the user’s viewpoint than the virtual objects 120, 128A and 128B).
As to claim 21, Ravasz discloses a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component (“In this example, HIVID 112 includes one or more processors 302 and memory 304… As discussed with respect to the example of FIG. 2, processors 302 are coupled to electronic display 203,” Ravasz paragraph 0049; “HIVID 112 includes an internal control unit 210, which may include an internal power source and one or more printed-circuit boards having one or more processors, memory, and hardware to provide an operating environment for executing programmable operations to process sensed data and present artificial reality content on display 203,” Ravasz paragraph 0045), the one or more programs including instructions for:
detecting a request to display a user interface overlay for a user interface region ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, activation request from user to display the menu); and
in response to detecting the request to display the user interface overlay, displaying, via the display generation component, the user interface overlay with respect to the user interface region based on a location of an anchor element within the user interface region ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, display menu prompt that is anchored to the user’s hand location (i.e., an anchor element anchored to the user’s hand) and then displaying menu based on a location of the hand and menu prompt), wherein:
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to a first location with respect to the user interface region and the user interface region is a first size, the user interface overlay has a first set of one or more visual characteristics that is based on the first location and the first size ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, x and y axis location of the user's hand and menu prompt to set the location of the overlay menu and z axis location of the user’s hand and menu prompt (which changes the displayed size of the user interface region) to set how close the overlay menu is to the user);
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to a second location, different from the first location, with respect to the user interface region and the user interface region is the first size, the user interface overlay has a second set of one or more visual characteristics that is based on the second location and the first size, wherein the second set of one or more visual characteristics is different from the first set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, overlay menu could be in a different x and y axis location while maintaining the z axis value (i.e., hand and menu prompt maintains the same size thus maintaining the displayed size of the UI region));
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the first location with respect to the user interface region and the user interface region is a second size, different from the first size, the user interface overlay has a third set of one or more visual characteristics that is based on the first location and the second size, wherein the third set of one or more visual characteristics is different from the first set of one or more visual characteristics and the second set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, can have a different z axis value (i.e., has the displayed UI has a different size) while maintaining the x and y axis location); and
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the second location with respect to the user interface region and the user interface region is the second size, the user interface overlay has a fourth set of one or more visual characteristics that is based on the second location and the second size, wherein the fourth set of one or more visual characteristics is different from the first set of one or more visual characteristics, the second set of one or more visual characteristics, and the third set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, overlay menu could have different x and y axis values thus changing both the location and the z-axis value for the hand and menu prompt can also be different thus changing the displayed size of the UI region).
As to claim 22, Ravasz discloses a computer system that is in communication with a display generation component, comprising:
one or more processors (“In this example, HIVID 112 includes one or more processors 302 and memory 304… As discussed with respect to the example of FIG. 2, processors 302 are coupled to electronic display 203,” Ravasz paragraph 0049; “HIVID 112 includes an internal control unit 210, which may include an internal power source and one or more printed-circuit boards having one or more processors, memory, and hardware to provide an operating environment for executing programmable operations to process sensed data and present artificial reality content on display 203,” Ravasz paragraph 0045); and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions (“In this example, HIVID 112 includes one or more processors 302 and memory 304… As discussed with respect to the example of FIG. 2, processors 302 are coupled to electronic display 203,” Ravasz paragraph 0049; “HIVID 112 includes an internal control unit 210, which may include an internal power source and one or more printed-circuit boards having one or more processors, memory, and hardware to provide an operating environment for executing programmable operations to process sensed data and present artificial reality content on display 203,” Ravasz paragraph 0045) for:
detecting a request to display a user interface overlay for a user interface region ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, activation request from user to display the menu); and
in response to detecting the request to display the user interface overlay, displaying, via the display generation component, the user interface overlay with respect to the user interface region based on a location of an anchor element within the user interface region ("In response to detecting the menu prompt gesture, the artificial reality system can render a menu prompt 810 between the index finger and thumb of the virtual hand. The menu prompt 810 can be a user interface element that serves as an indicator or reminder (i.e., a prompt) to the user that the user can perform an action with the thumb and index finger (e.g., a pinching action) to place user's hand in a menu activation gesture to cause the artificial reality system to provide a menu to the user," Ravasz paragraph 0069; "In response to determining that the user has performed the menu activation gesture, the artificial reality system can render a UI menu (510). In one or more aspects, the menu is rendered in proximity to a virtual hand representing the orientation of the user's hand. In some aspects, the artificial reality system may render the UI menu responsive to detecting a menu activation gesture only if the artificial reality system first detected a menu prompt gesture," Ravasz paragraph 0072, display menu prompt that is anchored to the user’s hand location (i.e., an anchor element anchored to the user’s hand) and then displaying menu based on a location of the hand and menu prompt), wherein:
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to a first location with respect to the user interface region and the user interface region is a first size, the user interface overlay has a first set of one or more visual characteristics that is based on the first location and the first size ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, x and y axis location of the user's hand and menu prompt to set the location of the overlay menu and z axis location of the user’s hand and menu prompt (which changes the displayed size of the user interface region) to set how close the overlay menu is to the user);
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to a second location, different from the first location, with respect to the user interface region and the user interface region is the first size, the user interface overlay has a second set of one or more visual characteristics that is based on the second location and the first size, wherein the second set of one or more visual characteristics is different from the first set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, overlay menu could be in a different x and y axis location while maintaining the z axis value (i.e., hand and menu prompt maintains the same size thus maintaining the displayed size of the UI region));
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the first location with respect to the user interface region and the user interface region is a second size, different from the first size, the user interface overlay has a third set of one or more visual characteristics that is based on the first location and the second size, wherein the third set of one or more visual characteristics is different from the first set of one or more visual characteristics and the second set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, can have a different z axis value (i.e., has the displayed UI has a different size) while maintaining the x and y axis location); and
in accordance with a determination that the location of the anchor element for the user interface overlay corresponds to the second location with respect to the user interface region and the user interface region is the second size, the user interface overlay has a fourth set of one or more visual characteristics that is based on the second location and the second size, wherein the fourth set of one or more visual characteristics is different from the first set of one or more visual characteristics, the second set of one or more visual characteristics, and the third set of one or more visual characteristics ("Returning to FIG. 6, after determining whether or not horizontal motion has been detected, the artificial reality system can determine if non-horizontal motion by the user hand has occurred (610). For example, the artificial reality system can determine if there has been motion in a vertical direction (i.e., motion parallel to the Y axis) and/or front-to-back or back-to-front motion (i.e., motion parallel to the Z axis). If non-horizontal motion of the user's hand is detected, the artificial reality system can translate the position of the virtual hand, slidably engageable UI element, and UI menu based with the non-horizontal motion," Ravasz paragraph 0085; "FIG. 7E is an example HIVID display 770 illustrating a UI menu after back-to-front motion has been detected (i.e., the user has moved their hand closer to themselves along the Z axis). The artificial reality system can detect the back-to-front motion and can translate the position of the UI menu 124, virtual hand 136 and slidably engageable UI element 706 based on the detected back-to-front motion. Thus, the position of the virtual hand, UI menu 124, and slidably engageable UI element 706 appear to be larger, and thus closer to the user," Ravasz paragraph 0087, overlay menu could have different x and y axis values thus changing both the location and the z-axis value for the hand and menu prompt can also be different thus changing the displayed size of the UI region).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5 and 9-12 are rejected under 35 U.S.C. 103 as being unpatentable over Ravasz et al. (US 20200387228 A1, hereinafter Ravasz) in view of Chmielewski et al. (US 20100192101 A1, hereinafter Chmielewski).
As to claim 5, Ravasz discloses the method of claim 1, however Ravasz does not appear to explicitly disclose a limitation wherein:
while displaying the user interface overlay with the first set of one or more visual characteristics:
in accordance with a determination that a distance between the location of the anchor element and a first boundary of the user interface region is greater than a distance threshold, the user interface overlay is displayed over a subset of a third portion of the user interface region that includes the first boundary and is not displayed over a fourth portion of the user interface region, different from the third portion, wherein the fourth portion does not include the first boundary;
in accordance with a determination that the distance between the anchor element and the first boundary of the user interface region is less than the distance threshold and a respective application expansion direction is assigned to the user interface region, the user interface is displayed over a subset of the fourth portion of the user interface region that does not include the first boundary and is not displayed over the third portion of the user interface region that includes the first boundary; and
in accordance with a determination that the distance between the anchor element and the first boundary of the user interface region is less than the distance threshold and the respective application expansion direction is not assigned to the user interface region, the user interface region is displayed over the subset of the third portion of the user interface region that includes the first boundary and is not displayed over the fourth portion of interface region that does not include the first boundary.
Chmielewski teaches a limitation wherein:
while displaying the user interface overlay with the first set of one or more visual characteristics:
in accordance with a determination that a distance between the location of the anchor element and a first boundary of the user interface region is greater than a distance threshold, the user interface overlay is displayed over a subset of a third portion of the user interface region that includes the first boundary and is not displayed over a fourth portion of the user interface region, different from the third portion, wherein the fourth portion does not include the first boundary ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103, if the distance to the UI edge is not too close then the menu is displayed normally where the cursor is and is not displayed aligned to or over the edge);
in accordance with a determination that the distance between the anchor element and the first boundary of the user interface region is less than the distance threshold and a respective application expansion direction is assigned to the user interface region, the user interface is displayed over a subset of the fourth portion of the user interface region that does not include the first boundary and is not displayed over the third portion of the user interface region that includes the first boundary (“in accordance with a determination” contingent limitation language in a method claim and thus does not need to occur since the distance could be above the first threshold); and
in accordance with a determination that the distance between the anchor element and the first boundary of the user interface region is less than the distance threshold and the respective application expansion direction is not assigned to the user interface region, the user interface region is displayed over the subset of the third portion of the user interface region that includes the first boundary and is not displayed over the fourth portion of interface region that does not include the first boundary (“in accordance with a determination” contingent limitation language in a method claim and thus does not need to occur since the distance could be above the first threshold).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to change a characteristic of the menu when it is placed too close the screen edge as taught by Chmielewski. One would have been motivated to make such a combination to ensure that all menu items are visible (Chmielewski paragraph 0103).
As to claim 9, Ravasz discloses the method of claim 1, however Ravasz does not appear to explicitly disclose a limitation wherein:
while the user interface overlay has the first set of one or more visual characteristics:
in accordance with a determination that the first location is a first distance from a third boundary corresponding to an edge of the user interface region, a first dimension of the user interface overlay has a first value; and
in accordance with a determination that the first location is a second distance from the third boundary corresponding to the edge of the user interface region, the first dimension of the user interface overlay has a second value, wherein the second value is different from the first value.
Chmielewski teaches a limitation wherein:
while the user interface overlay has the first set of one or more visual characteristics:
in accordance with a determination that the first location is a first distance from a third boundary corresponding to an edge of the user interface region, a first dimension of the user interface overlay has a first value ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103, if the distance to the UI edge is not too close then the menu is displayed normally where the cursor is); and
in accordance with a determination that the first location is a second distance from the third boundary corresponding to the edge of the user interface region, the first dimension of the user interface overlay has a second value, wherein the second value is different from the first value ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103, if the distance to the UI edge is too close then the menu is displayed offset from the cursor so that the entire menu is visible).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to change a characteristic of the menu when it is placed too close the screen edge as taught by Chmielewski. One would have been motivated to make such a combination to ensure that all menu items are visible (Chmielewski paragraph 0103).
As to claim 10, Ravasz discloses the method of claim 1, however Ravasz does not appear to explicitly disclose a limitation wherein:
while the user interface overlay has the first set of one or more visual characteristics:
in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied when the location of the anchor element for the user interface overlay corresponds to the first location with respect to the user interface region, the user interface overlay has a first alignment relative to the user interface region; and
in accordance with a determination that a second set of one or more criteria is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied when the location of the anchor element for the user interface overlay corresponds to the second location with respect to the user interface region, the user interface overlay has a second alignment, different from the first alignment, relative to the user interface region.
Chmielewski teaches a limitation wherein:
while the user interface overlay has the first set of one or more visual characteristics:
in accordance with a determination that a first set of one or more criteria is satisfied, wherein the first set of one or more criteria includes a criterion that is satisfied when the location of the cursor for the user interface overlay corresponds to the first location with respect to the user interface region, the user interface overlay has a first alignment relative to the user interface region ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103; Chmielewski Figure 11A 1010 pointer and menu is bottom aligned with the UI area, if the distance to the UI edge is too close to the bottom then the menu would be bottom aligned); and
in accordance with a determination that a second set of one or more criteria is satisfied, wherein the second set of one or more criteria includes a criterion that is satisfied when the location of the cursor for the user interface overlay corresponds to the second location with respect to the user interface region, the user interface overlay has a second alignment, different from the first alignment, relative to the user interface region ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103; Chmielewski Figure 11A 1010 pointer and menu is bottom aligned with the UI area, if the distance to the UI edge is too close to the top then the menu would be top aligned).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to change an alignment of the menu when it is placed too close the screen edge as taught by Chmielewski. One would have been motivated to make such a combination to ensure that all menu items are visible (Chmielewski paragraph 0103).
As to claim 11, Ravasz as modified by Chmielewski further discloses the method of claim 10, wherein the first set of one or more criteria includes a criterion that is satisfied when a second dimension of the user interface region has a third value, and wherein the second of one or more criteria includes a criterion that is satisfied when the second dimension of the user interface region has a fourth value different from the third value ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103, aligning the menu is based on the UI display area dimensions and thus would change if the UI display area had different dimensions).
As to claim 12, Ravasz as modified by Chmielewski further discloses the method of claim 10, wherein the first set of one or more criteria includes a criterion that is satisfied when a third dimension of the user interface overlay has a fifth value, and wherein the second set of one or more criteria includes a criterion that is satisfied when the third dimension of the user interface overlay has a sixth value different from the fifth value ("If it is determined that the screen pointer is not too close to an edge of the display area, in step 1530, the menu manager may display the radial menu in the display area such that a center of the radial menu is aligned with the screen pointer. On the other hand, if it is determined that the screen pointer is too close to an edge of the display area, then in step 1540, the menu manager 115 may display the radial menu in the display area such that all the radial menu items are visible in the display area. For example, as discussed above, the menu manager 115 may display the radial menu such that a center 310 of the radial menu is displaced at least a distance r (radial menu radius) from the screen edge," Chmielewski paragraph 0103; Chmielewski Figure 11A 1010 pointer and menu is bottom aligned with the UI area, if the distance to the UI edge is too close to any edge then the menu would be aligned with that edge and thus the menu could have a third dimension of different values that would trigger alignment).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Ravasz et al. (US 20200387228 A1, hereinafter Ravasz) in view of Mistry et al. (US 20140139422 A1, hereinafter Mistry) in further view of Eirinberg et al. (US 20220300081 A1, hereinafter Eirinberg).
As to claim 13, Ravasz discloses the method of claim 1, wherein the user interface overlay includes a set of one or more selectable user interface objects that includes a first selectable user interface object (“The UI menu 124 can include one or more UI elements 126 that are arrayed along a dimension of the UI menu 124. In one or more aspects, the one or more UI elements 126 can be menu items arrayed along a horizontal dimension of a coordinate space such as a viewing space or display space,” Ravasz paragraph 0073; “In one or more aspects, the menu item 708 in proximity to the slidably engageable UI element 706 can be highlighted or otherwise augmented or modified to indicate that the menu item 708 will be selected upon the user performing a selection gesture 708,” Ravasz paragraph 0076).
However Ravasz does not appear to explicitly disclose a limitation wherein:
in accordance with a determination that the number of selectable user interface objects included in the set of one or more selectable user interface objects is greater than a threshold number of selectable objects, the user interface overlay is displayed with a third size such that a first portion of the first selectable user interface object is displayed and a second portion of the first selectable user interface object is not displayed.
Mistry teaches a limitation wherein:
in accordance with a determination that the number of selectable user interface objects included in the set of one or more selectable user interface objects is greater than a threshold number of selectable objects, scrolling the user interface overlay (“In the case of a large number of menu items (e.g. beyond a particular threshold such as 40 captured images), the segments of the circular menu may disappear, and the visual indicator presented to the user may be a scroll bar 11020 that allows the user to circularly scroll through the various menu items,” Mistry paragraph 0190).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to scroll the menu overlay when the selectable number of objects exceeds a threshold as taught by Mistry. One would have been motivated to make such a combination to ensure that all menu items are visible with a sufficient size for selection, thus resulting in greater ease of use for the user.
However neither Ravasz nor Mistry appear to explicitly disclose a limitation wherein the user interface overlay is displayed with a third size such that a first portion of the first selectable user interface object is displayed and a second portion of the first selectable user interface object is not displayed.
Eirinberg teaches a limitation wherein the user interface overlay is displayed with a third size such that a first portion of the first selectable user interface object is displayed and a second portion of the first selectable user interface object is not displayed ("Specifically, the electronic mirroring device 130 can currently have a function corresponding to a first menu option 846 activated (e.g., a first filter 842 or augmented reality experience is presently displayed). In response to determining that the one or more fingers have been moved from pointing vertically up 854 to pointing horizontally to a first side 844 (e.g., a right side of the display), the hands-free control system 107 scrolls the menu of options to the next adjacent menu option 847 (e.g., to access a function corresponding to a second menu option)," Eirinberg paragraph 0141; Eirinberg Figure 8B menu items at the left and right are partially displayed to indicate scrolling is possible).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to partially display a menu item in the menu when the menu can be scrolled as taught by Eirinberg. One would have been motivated to make such a combination so that the user has a visual indication that the menu can be scrolled, thus enhancing ease of use for the user.
Claims 15 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Ravasz et al. (US 20200387228 A1, hereinafter Ravasz) in view of Murphy et al. (US 20190018567 A1, hereinafter Murphy) in view of Reese et al. (US 20190005030 A1, hereinafter Reese).
As to claim 15, Ravasz discloses the method of claim 1, however Ravasz does not appear to explicitly disclose a limitation further comprising:
while displaying the user interface overlay, detecting an input corresponding to a second request to change the size of the user interface region; and
in response to detecting the input corresponding to the second request to change the size of the user interface region, ceasing to display the user interface overlay.
Murphy teaches a limitation further comprising:
while displaying the user interface overlay, detecting an input corresponding to a second request to change the size of the user interface region (“For example, to translationally move a virtual object (e.g., application window, graphical icon, item (e.g., virtual table, ball, tool), menu, etc.), along an x-axis (using a typical Cartesian coordinate system in the VR/AR environment and a corresponding Cartesian coordinate system in the real world environment of the user operating input device 500), trigger 530 can be depressed to select (“grab”) the virtual object, and input device 500 can be rotated around a y-axis (see FIG. 7). To translationally move the virtual object along the y-axis, trigger 530 can be depressed to grab the virtual object, and input device 500 can be moved rotationally around an x-axis. To translationally move the virtual object along the z-axis, trigger 530 can be depressed to grab the virtual object, and touch pad 520 can be swiped (e.g., up/down) to move the virtual object toward and away (along the z-axis) from the user,” Murphy paragraph 0085, moving objects in the virtual environment including making them appear closer of farther away to change sizes).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to allow Ravasz’s virtual objects to be moved as taught by Murphy. One would have been motivated to make such a combination so that the user could have more control over the virtual environment thus allowing the virtual environment to be setup according to the needs and desire of the user.
However neither Ravasz nor Murphy appear to explicitly disclose in response to detecting the input corresponding to the second request to change the size of the user interface region, ceasing to display the user interface overlay.
Reese teaches in response to detecting the input corresponding to the second request to change the size of the user interface region, ceasing to display the user interface overlay ("Execution may proceed to step 620 wherein the text overlay is adjusted/removed if movement of the content being overlaid is detected. That is, if a page starts to move, the overlay image (content) will have to move accordingly or it will be removed and then replaced when page movement ceases," Reese paragraph 0082, removing overlay while moving).
Accordingly it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Ravasz to remove display of overlay menus when manipulating virtual objects as taught by Reese. One would have been motivated to make such a combination so that the entire area where the object can be repositioned to could be visible thus enhancing ease of use for the user.
As to claim 16, Ravasz as modified by Murphy and Reese further discloses the method of claim 15, further comprising:
while the user interface overlay is not displayed, detecting an end of the input corresponding to the second request to change the size of the user interface region; and in response to detecting the end of the input corresponding to the second request to change the size of the user interface region, displaying, via the display generation component, the user interface overlay ("Execution may proceed to step 620 wherein the text overlay is adjusted/removed if movement of the content being overlaid is detected. That is, if a page starts to move, the overlay image (content) will have to move accordingly or it will be removed and then replaced when page movement ceases," Reese paragraph 0082, overlay reappears after moving has ceased).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US 11175801 B2 to Dardailler et al. discloses an interactor for a graphical object where a graphical object is expanded in an expansion direction so that a context menu does not obscure it;
US 20150007114 A1 to Poulos et al. discloses a web-like hierarchical menu display configuration for a near-eye display where an overlay menu can be repositioned and resized; and
US 20190384460 A1 to Harnisch et al. discloses surfacing application functionality for an object where an overlay context menu associated with a virtual object in a VR environment can be displayed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL SAMWEL whose telephone number is (313) 446-6549. The examiner can normally be reached Monday through Thursday 8:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL SAMWEL/ Primary Examiner, Art Unit 2171
Prosecution Insights