Detailed Action
Notice of Pre-AIA or AIA status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This Final Office action is responsive to the communication filed under 37 C.F.R. § 1.111 on January 21, 2026 (hereafter “Response”). The amendments to the claims are acknowledged and have been entered.
Claims 1, 8, 13, 15, 18–20 are now amended.
Claims 3–5 and 10–17 are now canceled.
New claims 20–33 are now added.
Claims 1, 2, 6–9, and 18–33 are pending in the application.
Response to Arguments
Applicant’s arguments with respect to the claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 21, 2026 was filed after the mailing date of the Non-Final Office Action on October 21, 2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 20, 25, and 32 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections – 35 U.S.C. § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 6–9, 15, 18, 19, 21–24, 26, 28–31, and 33, are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by U.S. Patent Application Publication No. 2014/0347391 A1 (“Keane”).
Claim 1
Keane discloses:
A method, comprising: at an electronic device in communication with one or more displays and one or more input devices:
Reference is made to FIGS. 10–14A, which illustrate the operation of a computer system programmed to perform a single method (FIGS. 11–14A illustrate certain steps of FIG. 10 with greater detail). Keane ¶¶ 17–18.
The system, shown in FIGS. 3–7 with focuses on different components thereof, includes an electronic device that is in communication with one or more displays, and one or more input devices. See Keane FIGS. 3–7 and accompanying disclosure.
presenting, via the one or more displays, a three-dimensional environment,
During a first loop of the method shown in FIG. 10,1 a head mounted display displays a rendered image to a user in step 658. Keane ¶¶ 153 and 143. The image is a rendering of a three-dimensional scene. See Keane ¶¶ 111 and 141.
wherein the three-dimensional environment includes a first set of user interfaces that includes a first user interface and a second user interface,
“In embodiments, the scene map generated in a given frame may include the x, y and z positions of . . . virtual objects in the scene.” Keane ¶ 111. “These virtual objects may be facing any of various directions as a user moves around within the mixed reality environment.” Keane ¶ 123.
wherein the first set of user interfaces move together in response to movement inputs;
Due to the current claim language, there are a few different ways to interpret this limitation, which will now be discussed, alongside Keane’s disclosure of each interpretation of the limitation.
A first, narrow interpretation is that the “movement inputs” refer to the specific “request to move the first user interface” recited below. In that case, Keane discloses this limitation for the same reason it discloses the limitation below: the user can perform a gesture of either ceasing his motion in the physical environment, or resuming it, and the virtual objects will all rotate toward him, or back to their original positions, respectively. Keane ¶ 132.
A second, broader interpretation is that the “movement inputs” are not necessarily limited to the request to move the first user interface recited below, and instead, “wherein the first set of user interfaces move together in response to movement inputs” simply recites a capability (or set of capabilities) of the virtual objects. In that case, Keane discloses that “one or more of the virtual objects may rotate once the user starts moving again so as to continue to be body-locked on the user when the user is in motion,” Keane ¶ 89, and also discloses that the user may perform “grabbing and moving gestures” on one or more of the virtual objects in the scene, in order to reposition them. Keane ¶ 96.
Finally, it is also noted that if the second interpretation applies, there is a third interpretation that applies only to the method claims: “The broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met.” MPEP § 2111.04. If the second interpretation applies, then this entire limitation is optional, because the condition precedent of detecting or receiving such “movement inputs” is not a required step of the claim.
while presenting the three-dimensional environment, receiving, via the one or more input devices, a user input corresponding to a request to move the first user interface;
Keane anticipates this step in at least two different ways if “move the first user interface” is narrowly interpreted to require a change in the first user interface’s position in space, and anticipates this step in three different ways if “move the first user interface” is more broadly interpreted to include rotational motion of the first user interface.
(1) Keane’s first disclosure of the narrow interpretation. While displaying several virtual objects to the user (FIG. 8), the user inputs a gesture, via the sensors in the head mounted display 2, that includes (i) pointing, gazing, or otherwise selecting one or more of the virtual objects from a stationary position, and (ii) moving away from the stationary position. Keane ¶¶ 86 and 89. This gesture is treated as a request to “body-lock” the selected objects, such that they follow the user as he moves. Keane ¶ 89.
(2) Keane’s second disclosure of the narrow interpretation. While displaying several virtual objects to the user (FIG. 8), the user inputs a gesture, via the sensors in the head mounted display 2, that includes (i) sitting, standing, or walking within a small diameter of range, “with one or more portions of the body above the feet (knees, hips upper body, head, etc.) moving,” Keane ¶ 125, and (ii) the aforementioned moving of body parts above the feet comprise “grabbing and moving gestures” directed to one or more of the virtual objects in the scene, with the intent of repositioning the grabbed virtual objects within the space. Keane ¶ 96.
Both of these multi-part gestures fall within the broadest reasonable interpretation of “a request to move the first user interface,” because the specification explicitly contemplates “selection inputs” as comprising multiple stages of gestures, some of which even include “holding” a position for a threshold amount of time (Spec. ¶ 34), and additionally, the specification tells us that the singular forms of “a,” “an,” and “the” (in this case, “a user input” corresponding to “a request”) is “intended to include the plural forms,” (Spec. ¶ 140); see also Baldwin Graphic Systems, Inc. v. Siebert, 512 F.3d 1338, 1342 (Fed. Cir. 2008) (repeating the same meaning for those words as a general rule for patent claim construction). Given both of these pieces of evidence from the specification, it is reasonable to interpret “a user input corresponding to a request to move the first user interface” to include a plurality of user inputs corresponding to a plurality of requests to move the first user interface, and subsequent recitations of “the” user input or “the” request simply reinvoke that plural meaning.
(3) Keane’s disclosure of the broad interpretation. Under the broader interpretation, the “request to move the first user interface” can further include requests to rotate the first user interface, in addition to repositioning it. Thus, Keane discloses the broad interpretation in three ways: the first two are the same as those discussed above for actually repositioning the virtual objects, while the third is simply the gesture where the user either sits, stands, or walks within a small diameter of range, optionally “with one or more portions of the body above the feet (knees, hips upper body, head, etc.) moving,” which the system interprets as a request to rotate all of the virtual objects (or at least those within a certain “dynamic” category) to rotate to face the user. Keane ¶ 125; see also Keane ¶¶ 123 and 128.
and in response to receiving the user input corresponding to the request to move the first user interface: changing a first orientation of the first user interface about at least a first axis of rotation; and changing a second orientation of the second user interface about at least a second axis of rotation.
All three gestures discussed above, across both interpretations of “request to move the first user interface,” cause “the dynamic virtual objects [to] rotate about the x, y and/or z axes to face the user's body as indicated by the arrows in FIG. 9.” Keane ¶ 86. More specifically, “position vectors may be determined between each dynamic virtual object having a front and the user,” and then “the front of each dynamic virtual object may be rotated about x, y and/or z axes toward its position vector at a predetermined angular velocity. This rotation will continue each time through the loop of the steps shown in FIG. 10, until an object is x, y and z aligned with its position vector.” Keane ¶ 130.
Note that this constitutes at least two user interfaces simultaneously changing their orientations about first and second respective axes of rotation because all of the virtual objects are situated in different positions relative to the user. In other words, as shown in FIGS. 8–9, there is at least a first virtual object to the left of the user that rotates about a vertical axis that is to the left of the user 18, and likewise, there is at least a second virtual object to the right of the user 18 that rotates about a vertical axis that is to the right of the user 18.
Claim 2
Keane discloses the method of claim 1, further comprising:
in response to receiving the user input corresponding to the request to move the first user interface: moving the first user interface in accordance with the user input; and moving the second user interface in accordance with the user input.
All three disclosures from the rejection of claim 1 further disclose claim 2 for the following reasons.
Disclosure (1) anticipates moving both user interfaces in response to the user’s input, because the user’s input “body-locks” one or more virtual objects (and thus at least a first and second user interface), such that they “translate” in space “so as to move with the user and remain a fixed distance from the user as the user moves.” Keane ¶ 89.
Disclosure (2) anticipates moving both user interfaces in response to the user’s input, because the user’s input includes “grabbing and moving gestures” performed on one or more of the virtual objects in the scene, in order to reposition them. Keane ¶ 96.
Disclosure (3) anticipates moving both user interfaces in response to the user’s input, because the user’s input causes “the dynamic virtual objects [to] rotate about the x, y and/or z axes to face the user's body as indicated by the arrows in FIG. 9.” Keane ¶ 86.
Claim 6
Keane discloses the method of claim 1, wherein changing a first orientation of the first user interface includes:
in accordance with a determination that the request to move the first user interface includes a request to move the first user interface in a first direction, rotating the first user interface in a first orientation;
“Using the steps of FIG. 13, dynamic virtual objects . . . may rotate toward the user once a user remains motionless for a predetermined period of time as shown in FIG. 9.” Keane ¶ 132.
and in accordance with a determination that the request to move the first user interface includes a request to move the first user interface in a second direction, different from the first direction, rotating the first user interface in a second orientation, different from the first orientation.
Disclosures (2) and (3) both anticipate this element of claim 6, because in those disclosures, “[o]nce a user begins moving again, the rotated virtual objects . . . may rotate back to their positions prior to rotating toward the user.” Keane ¶ 132.
Claim 7
The additional language of claim 7 consists of limitations that are contingent upon unmet conditions precedent, but “[t]he broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met.” MPEP § 2111.04 (subsection II.).
Claim 7 requires a request to move the first user interface, but does not require the request to be a request to specify the first direction or second direction. Instead, claim 7 only recites what must happen if a determination reveals that the request specifies the first or second direction. Therefore, in order to reach a finding of anticipation, “the Examiner [does] not need to present evidence” of the method steps “that are not required to be performed under a broadest reasonable interpretation of the claim.” MPEP § 2111.04 (subsection II.) (quoting Ex parte Schulhauser, PTAB Appeal No. 2013-007847 (April 28, 2016) (precedential)).
Here, the only required elements of claim 7, according to its broadest reasonable interpretation, are the elements that claim 7 incorporates from parent claims 1 and 6 by reference. The prior art discloses those elements for the reasons given in the rejections of claims 1 and 6. Therefore, the prior art also anticipates claim 7.
Claim 8
The additional language of claim 8 consists of limitations that are contingent upon unmet conditions precedent, but “[t]he broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met.” MPEP § 2111.04 (subsection II.).
Claim 8 further specifies what happens if the request to move the first user interface in parent claim 6 is in a first direction, or if the request to move the first user interface in parent claim 6 is in a second direction. Neither claim 6 nor claim 8 require a determination that the request to move the first user interface was a request in the first direction or the second direction, so, neither of the rotations recited in claims 6 or 8, that occur in response to those non-required determinations are required by the claims. That is, claim 8 merely recites additional criteria for the request to move the user interface in the first or second directions, without actually reciting an affirmative step in which the user provides such a request. Therefore, in order to reach a finding of anticipation, “the Examiner [does] not need to present evidence” of the method steps “that are not required to be performed under a broadest reasonable interpretation of the claim.” MPEP § 2111.04 (subsection II.) (quoting Ex parte Schulhauser, PTAB Appeal No. 2013-007847 (April 28, 2016) (precedential)).
Here, the only required elements of claim 6, according to its broadest reasonable interpretation, are the elements that claim 6 incorporates from parent claims 1 and 6 by reference. The prior art discloses those elements for the reasons given in the rejections of claims 1 and 6. Therefore, the prior art also anticipates claim 8.
Claim 9
Keane discloses the method of claim 1,
wherein receiving the user input corresponding to the request to move the first user interface includes detecting a selection gesture from a hand of the user directed at a movement affordance and a movement of the hand of the user while maintaining the selection gesture.
“For example, the user may select a virtual display slate 460 for positioning at the center or near center of his FOV. Alternatively, a user may send a virtual display slate 460 onto a wall. These options may for example be carried out by the user performing grabbing and moving gestures with his or her hands, though it may be carried out in other ways in further embodiments.” Keane ¶ 96.
Claim 15
Keane discloses the method of claim 1, wherein:
before receiving the user input corresponding to the request to move the first user interface: a normal vector of the first user interface is directed at a location in the three-dimensional environment corresponding to a user of the electronic device and a normal vector of the second user interface is directed at the location in the three-dimensional environment corresponding to the user;
“[O]nce a user stands still or nearly still for a predetermined period of time, the dynamic virtual objects within the scene may change to being body-locked on the user 18. That is, the dynamic virtual objects may rotate about the x, y and/or z axes to face the user's body as indicated by the arrows in FIG. 9.” Keane ¶ 86. While Keane does not use the word “normal vector,” the description of “facing” the user, and the illustration of the arrows extending perpendicular from each “face” of each object in the figure, disclose that the normal vectors of each object indeed point to the user’s location. (See Spec. ¶ 55)
and after changing the first orientation of the first user interface and the second orientation of the second user interface in response to receiving the user input corresponding to the request to move the first user interface: the normal vector of the first user interface is directed at the location in the three- dimensional environment corresponding to the user, and the normal vector of the second user interface is directed at the location in the three-dimensional environment corresponding to the user.
“In a further embodiment, one or more of the virtual objects may rotate once the user starts moving again so as to continue to be body-locked on the user when the user is in motion. For example, when the user is stationary, the user may select one or more of the virtual objects, such as for example pointing, gazing or performing some other physical gesture with respect to the one or more virtual objects.” Keane ¶ 89.
Claim 21
Keane discloses the method of claim 1,
wherein the at least first axis of rotation and the at least second axis of rotation are different axes of rotation.
“As indicated in FIG. 8, a scene may include a number of virtual objects each facing at random directions, as indicated by the arrows in FIG. 8.” Keane ¶ 85. Thus, when they all face the user in FIG. 9, see Keane ¶ 86, they each must necessarily rotate about one or more (really several) different axes in order to face the user.
Claims 18, 22, 23, 26, and 28
Claims 18, 22, 23, 26, and 28 are directed to an electronic device, comprising: one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing exactly the same methods as set forth in corresponding claims 1, 2, 6, 8, 9, and 15.
Keane discloses the method(s) set forth in each of those claims, and further discloses an electronic device with the same general-purpose computer hardware and program instructions for performing the same. See Keane FIGS. 3–7 and accompanying disclosure. Claims 18, 22, 23, 26, and 28 are therefore rejected over the same findings and rationale as provided above for the method claims, taken together with Keane’s disclosure of the electronic device for performing the same.
Claim 24
Keane teaches the electronic device of claim 23, and further teaches instructions for:
in response to receiving the user input corresponding to the request to move the first user interface:
As discussed in the 35 U.S.C. § 102 rejection of claim 1 (which is incorporated by reference into the rejections of claims 18 and 23), the user may perform a multi-part gesture in order to provide the claimed “user input corresponding to the request to move the first user interface,” wherein the user first performs the stationary body gesture to rotate the user interfaces as shown in FIGS. 8 to 9, followed by (or together with) the “grabbing and moving gestures” on any one or more of the virtual objects in the scene, in order to translate them within the space. Keane ¶ 96.
in accordance with the determination that the request to move the first user interface includes the request to move the first user interface in the first direction, maintaining a distance between the first user interface and the second user interface;
By performing the body-locking gesture from Disclosure (1) discussed in the rejection of claim 1 on at least two of the virtual objects, each of the virtual objects move through space together as the user walks through the room, at the same fixed distance from the user. Keane ¶¶ 86 and 89.
and in accordance with the determination that the request to move the first user interface includes the request to move the first user interface in a second direction, changing a distance between a first part of the first user interface and a corresponding part of the second user interface.
On the other hand, if the user performs “grabbing and moving gestures” from Disclosure (2), directed to one of the virtual objects in the scene, then only that one virtual object will be moved in accordance with the user’s gesture. Keane ¶ 96. Since only that one object is moved, the distance between it and any second virtual object in the scene that was not moved necessarily changes—including the distance between the moved virtual object and the portion of the second virtual object that faces the moved virtual object.
Claims 19, 29, 31, 32, and 33
Claims 19, 29, 30, 31, and 33 are directed to a broader version of the memory portion of claims 18, 22, 23, 24, and 26, and are therefore rejected over the same findings and rationale as provided above for those claims.
Claim Rejections – 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 13 and 27 are rejected under 35 U.S.C. § 103 as being unpatentable over Keane as applied to claim 1 above, and further in view of Barrett Ens et al., The Personal Cockpit: A Spatial Interface for Effective Task Switching on Head-Worn Displays, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 3171–3180, https://doi.org/10.1145/2556288.2557058 (April 2014) (hereafter “Ens”).
Claim 13
Keane teaches the method of claim 1, and Disclosure (1) from Keane further teaches:
before receiving the user input corresponding to the request to move the first user interface: the first user interface has a first distance from a user of the electronic device, and the second user interface has [another] first distance from the user,
The gesture from Disclosure (1), where the user begins the gesture by standing in place, the effect in some embodiments is that “just those dynamic virtual objects within a predetermined radius of the user” are selected for rotating to face the user. Keane ¶ 129. Thus, in cases where such a gesture selects a plurality of virtual objects, all of those virtual objects were at least within the same “first distance” from the user (i.e., the radius).
While one could argue that it is at least possible for two virtual objects to be the same “first distance” under these circumstances (e.g., two virtual objects that were coincidentally positioned the same radius away from the user at different angles), this possibility is not sufficient to reach a finding of anticipation under 35 U.S.C. § 102.
and after receiving the user input corresponding to the request to move the first user interface: the first user interface has a second distance from a user of the electronic device, and the second user interface has the second distance from the user, wherein the first distance and the second distance are a same distance.
“Selected virtual objects may for example stay rotated and at a fixed distance from a user even after a user begins moving around again within the scene.” Keane ¶ 133. In other words, after the user performs the complete gesture of stopping, selecting, and leaving, the virtual objects that the user selected move within the environment, yet both of them maintain the same “fixed distance” relative to the user.
Accordingly, the only difference between the invention of claim 13 and Keane is that the first and second user interfaces do not necessarily have the same first distance from the user prior to receiving the user input, other than by happenstance.
However, this difference was obvious, because it involved nothing more than the application of the known technique of maintaining a fixed separation between the user and the virtual objects in a three-dimensional environment. For example, Ens teaches:
before receiving the user input corresponding to the request to move the first user interface: the first user interface has a first distance from a user of the electronic device, and the second user interface has the first distance from the user, and after receiving the user input corresponding to the request to move the first user interface: the first user interface has a second distance from a user of the electronic device, and the second user interface has the second distance from the user, wherein the first distance and the second distance are a same distance.
By way of background, Ens teaches a user interface called the “Personal Cockpit,” which “appropriate[s] empty space around the user to situate virtual windows for use with direct input,” Ens 3171 (Abstract), and much like Keane and the claimed invention, “we set each window’s orientation to face the user’s point of view,” in a “curved layout.” Ens 3177.
Crucially, the authors of the Personal Cockpit user interface require that their design “keep the working set of windows at a single depth,” Ens 3172, e.g., by placing each of the windows “50 cm from the user’s right shoulder.” Ens 3177. With this requirement in place, “the user can grab, move and resize windows at his leisure using in-air pinching gestures,” Ens 3179, but the single depth of windows is a requirement of the design.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to apply Ens’s technique of maintaining a single depth level of windows to Keane’s computer system, thereby maintaining the same distances between the user and each of Keane’s virtual objects. One would have been motivated to apply Ens’s technique of maintaining a single depth level because “performance is negatively impacted by mixed display distances.” Ens 3172.
Claim 27
The additional elements of claim 27 are substantially similar to the additional elements of claim 13, and therefore, claim 27 is rejected over the same findings and rationale as provided above for claim 13, taken together with the findings set forth for the parent claim of claim 27 in the 35 U.S.C. § 102 rejection earlier.
Other Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Patent Application Publication No. 2005/0125742 A1 teaches all of the additional elements recited in claims 7, 24, and 31.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 C.F.R. § 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 C.F.R. § 1.17(a)) pursuant to 37 C.F.R. § 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Justin R. Blaufeld whose telephone number is (571)272-4372. The examiner can normally be reached M-F 9:00am - 4:00pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James K Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Justin R. Blaufeld
Primary Examiner
Art Unit 2151
/Justin R. Blaufeld/Primary Examiner, Art Unit 2151
1 See Keane ¶ 154 (explaining “the processing unit may loop back” at the end of the steps, in order to update the display with new information); see also ¶ 91.
Prosecution Insights